RE: [AV Media Matters] Video Compression - a PLUS for archivists (but not too mu

[Jim Lindner <jim@vidipax.com>]

I wanted to thank Tony for an extremely well thought out and written
posting. I suspect that it will start a very interesting thread. I
personally wanted to comment on some of the points that he makes.

I agree that one must always evaluate and re-evaluate technology and the
"opportunities" that they represent. I think that it is fair to say that
while the archival community in general is not as "plugged in" as some
of the other areas from a technology point of view, in general they
certainly do evaluate and re-evaluate opportunities. It must be
understood, however, that archives and archivists often have a different
"mission" and certainly a different yardstick in the "time" department
then many engineers and those looking at things from a more technical
perspective.

I don't think that it is totally fair to say that all AV Archivists are
totally against compression an all forms. Indeed, if one were a purist
as Tony suggest, then one would have to acknowlege that television from
the mechanical era through today has ALWAYS used compression in one form
or another. Clearly interlace is a very significant form of compression
that causes all sorts of artifacts - speak to any computer animator or
"paintbox artist" and they will be able to discuss this at great length.
Clearly the color space that is transmitted for both NTSC and PAL is
another example of rather extreme compression, and one could certainly
discuss bandwidth compression on virtually all formats in the high
frequency range (heck it is not really compression - it is chainsaw). So
I don't think that all forms of compression are considered taboo by all
forms of the AV Archive community.  There is however a big difference in
the different TYPES of compression, and the associated tradeoffs - and I
think that the hesitancy to embrace all types of compression for
archival compression is well founded.

I think it helpful to draw a distinction between transmission (or
access) and preservation. While these two aspects of any archival
pursuit co-exist, the methods used to optimize either of these two areas
are not necessarily the same - nor should they be. This is not true for
just A/V materials - rather it applies to many different types of
materials of many different types and ages. Clearly the techniques used
to provide access to the contents of a very important document - say the
Bill of Rights - could be easily done in very compressed form - ascii
text. A more "true" representation could be a black and white 2 bit deep
scan - perhaps compressed with a lossLESS technique, a more exact
rendition could be a 36 bit deep scan at 1200 dpi (or higher) with lossy
compression and even higher with lossLESS compression. In this scenario
one has different levels of image quality which is associated with
different amounts of storage and bandwidth that may be required. But
these are access issues. Would one want to preserve the original
document as ANY of these formats? Clearly not.

Preservation of A/V materials seeks to pass along the original
information in as pristine form as possible - and the arbiter of
pristine is not necessarily your eyes. As a practical example of this I
suggest that you take a standard video DVD and upconvert that DVD to
HDTV. It will be instantly clear that this image is not acceptable in
many applications. And the irony in all of this is that as the quality
arbiter on one side of the industry keeps on getting higher (HDTV) on
one side - what one is supposed to accept in quality due for all sorts
of "technology issues" such as streaming media poses a clear dicotomy in
both thought and action.  Can one with a straight face say that MPEG
250MPS will be the FINAL format that will exist in the commercial
marketplace? I think that it is very clear that it is not. So if that is
the case - can we seriously consider PRESERVING material at anything
other then the most optimal technologieS (note the S - meaning that I do
not think that you should every preserve any A/V in one format)? I think
not.

Does that mean that you cannot make streaming media files of any
material you desire for use on the internet? No - this is an access
issue - NOT a preservation issue. I encourage people to do this all the
time. One could make a VERY strong arguement that material that is not
seen will die - so I think that all the different options available for
access are all ways to use the material - and this is all good stuff.
BUT do not confuse the two.

I hear the issue of money all the time - but the fact of the matter is
that Preservation and Access are part of the same thing - you  need
both, and the financial reality if you really sit down and do the
numbers is that the additional costs of doing the preservation work
WHILE you are doing the compression for access is a relatively low
number in the scope of the entire project.

I agree with the comment about the "chasm" but I am afraid that there
are two parties here. I have seen some amazingly - shall I say limited
thinking" speeches and press releases about archives being unnecessary
and that television stations should be "tapeless"... as if tape is some
sort of demon or something. The irony of all this is that of course the
material is still being stored on tape - it is just that it is digital
tape - and recorded on a high speed digital recorder which is really -
yes - a modified digital VIDEO recorder. And round we go. So I don't
think that any media type is the ENEMY - I think that the issue relates
more to ACCESS - being able to quickly locate material for broadcast,
research, and other purposes. But do not confuse this with preservation.

In the not too distant future - I predict less then 10 years - we will
no longer be talking about this type of lossy compression - it will be a
moot point - because the cost of strorage will be so low that it will
not be an issue, and bandwidth will not be an issue.  WE can learn from
our colleagues in the Audio Community. Does ANYONE in professional audio
talk about compression any more for archival work?  NO WAY?  Why would
one - the storage is available and cheap. Clearly if Moore's Law holds
and the costs of storage and processing power contine on their well worn
curves - there is EVERY reason to believe that lossLESS compression will
be the way that things are done - and those who have jumped on lossy
compression to early will be at a terriffic........loss.

jim lindner
Minister of Preservation and Access
VP and General Manager VidiPax Division
VidiPax - The Magnetic Media and Information Migration Full Services
Company
Telephone 212-563-1999
www.vidipax.com
Moderator of A/V Media Matters@topica.com