[AV Media Matters] Answers to Some Technical Questions - WARNING - LONG AND MAY

[James Lindner <jim@vidipax.com>]

Answers to some technical questions asked

There have been some technical questions asked here lately and I thought
I would try to answer some of them together. For those of you who are
not technical ? I have tried to make the answers as ?non technical? as
possible ? hopefully you will be a bit more technically wise after these
not so brief answers. So with that disclaimer ? put on your diving suit
and prepare to dive ? dive - dive

There was a question regarding whether concealment was in fact done in
digital video formats. The answer is yes for several formats ? but not
necessarily all formats. I have what I call is the rule of 3 C?s as it
relates to digital video recording on digital videotape recorders.

The first C is error Correction. IF there is an error in the data then
the system tries to correct the error by examining the stream of digits
and trying to figure out which one is in error. This is sometimes not as
difficult as it sounds ? and sometimes is almost impossible to do,
depending on how much you have lost and where. How is it done ? well
there are several techniques, and those of you who want to read about it
in detail should not expect it here ? but there are many places on the
Web to look ? one of my favorite is www.ask.com. At this point you may
want to know that the code used most often is the Reed-Solomon code
which is a code that is particularly fast and robust for this type of
data.

For the rest of us - Here is a fast example using what is called a
checksum way of detecting an error. Let?s take a matrix of numbers and
add across and down

1 2 5  added across is 8
2 1 3  added across is 6
4 2 1  added across is 7
Now add down for the three columns and you get

7 5 9

OK ? with me so far ? good. Now imagine that when we make the recording
we keep track not only of the 9 numbers that are our data ? but also of
the extra digits, which are the sum of the rows and columns. When we
play back the storage ? we are going to play back not only the data
digits but also the extra digits and COMPARE the results of the playback
of the digits and the check digits. So ? when we play back the first row
we get

1 2 5 which is 8 and compare it to the one we stored which is 8 so this
row looks pretty good so far

The second row
2 2 3 and we add that and get 7 ? UH OH ? we have a different number
then the stored one. We now know that we have an error on this line. But
where? We know that the error is one higher then the original ? but is
it the first column the second or the third. We find out really easily.
By adding the columns down after checking out the third row (assuming
the third row is correct) we get 7 6 9. So the error had to have been in
the second column.

Now you are going to ask what about if the check digit is wrong, and any
number of other problems with this example ? well heck it is an example
so I kept it really simple ? but the principle is the same ? you add
digits and compare them and if you can correct it you do. That is the
first C.

The second C is concealment. Now this is not something really done in
the data world. Heck if there was an error in your pay check and the
system did not know if it should pay you $1000 or $100 you probably
don?t want it to pay you $550 (the average of the two)? but this is
VIDEO and if we really don?t know the number but the line above
particular area is black and the line below is black ? you have a pretty
good chance of guessing black ? at least for 1/60th of a second?. So
black it is. That is concealment. If you think that it is imperfect you
would be very correct ? but it works most of the time, which brings us
to the final C ?

Which is Crap? pardon the rough language ? but if correction does not
work you are in real trouble.

Correction and Concealment ARE used in most of the modern digital video
formats. It varies by format and implementation ?but for instance
Digital Betacam DOES do concealment.

The next question was whether data is recompressed in the Digital
Betacam format when making a copy. The answer is yes ? but you need to
understand how that works a bit because it is not as bad as it sounds.

To make sure I consulted the DigiBeta DVW A500 maintenance manual to
make sure I got it right. In digital betacam the compression, which they
refer to as the ?bit rate reduction encoder?, occurs AFTER the analog to
digital conversion process (if you are feeding in an analog signal) or
directly from the full bandwidth 601 serial data stream. The data is
field shuffled so that all of the same information is not in exactly the
same physical location on the resulting tape ? and then it is bit rate
reduced and then an error correction code is mixed in with the video and
sync data. So what is recorded on the actual tape is a mix of video
data, sync data, audio data, and what is known as outer parity and inner
parity data (which is error correction code data). When you play it back
the first thing that happens is the equalization of the signal and then
the error correction decoder figures out what is what ? the bit rate
reduction decoder ?inflates? the data back up and then there is a
concealment stage that may or may not be necessary.

When you re-record you go through the same stages ? so yes it is
re-compressed ? sort of ? but it is NOT the type of recompression that
will cause artifacts due to concatenation of different compression
algorithms.

There have been some comments about having a file-based system instead
of a tape-based system. This is an involved discussion ? but a few
comments. The jump between a tape based format and a file-based format
is not simple and NOT necessarily transparent. In the example above we
saw how the data goes through several stages ? this is not necessarily
the same with file based systems ? meaning that file based systems
generally do not have the concealment phase. One could argue that they
don?t have to ? but depending on a number of things concealment of
errors of picture information is important. Most file based systems at
this time use lossy compression which most in the archival circles feel
is a problem and most in the professional broadcasting circles have
accepted for many years now. I am not going to step in the middle of
this one at the moment ? let?s say that there is room for several points
of view ? but the important thing is that while file based systems
certainly have many advantages they also have several disadvantages on a
practical basis at this particular moment in technology ? so one can?t
unfortunately simply say ?not putting it on a file based system is a
mistake?. In fact most of the systems that I have seen that are in
actual use in the real world that ARE file based are really hybrid
systems. I personally feel that file based will be the way to go ? but
at this particular moment there are really no agreed upon standards
although there is movement in that direction. Even the metadata
standards are not quite set in ?stone? as of yet. You might say ? who
cares??

It is very important because you should NOT assume that one digital file
format would necessarily be perfect when migrated to another. This is a
very major point. Anyone who has migrated data over the years will have
war stories to tell you about trying to migrate word processing files
from some old relic to a new old relic?.. It is not necessarily a
perfect thing ? particularly when there is compression involved. So just
because video or audio information is saved as data on a file system
somewhere does NOT necessarily mean that it is safe or that it can
necessarily be migrated without any problems. Maybe it can ? maybe it
can?t.

I will go back to work now?

Your moderator?

Jim lindner

James Lindner
General Manager VidiPax Division
VidiPax - The Magnetic Media Restoration Company

Executive Vice-President
Loudeye Technologies
NASDAQ Symbol:LOUD

VidiPax
450 West 31 Street
New York, N.Y.  10001
212-563-1999 ext. 102
www.vidipax.com

Moderator: AV Media Matters Listserve
To subscribe to AV Media Matters Listserve send an email to:
AV-Media-Matters-subscribe@topica.com