RE: [AV Media Matters] Video Compression - a PLUS for archivists (but not too

[lists.rlhess@mindspring.com]

Tony raises many good and important issues, but the real problem is that
whatever format(s) are chosen are going to become obsolete -- and sooner
rather than later.

Yes, compression is widely used in the major modern formats. D5 is still
being used extensively in the post production markets of the US, but not in
the mainstream broadcasters--at least some of whom are still Betacam SP!

Intellectually, I'm starting to think that mainstream data tape formats
(and LTO looks exciting) have more to offer than the niche videotape
formats for long-term archival storage.

The compression algorithms need to be easily understood and transportable
so they do not rely on specific, soon-to-be-obsoleted hardware...

Just my two cents.

I apologize for leaving too much of the original post, but there's a lot of
good stuff in there.

Cheers,

Richard

At 06:59 AM 10/02/2000 -0700, you wrote:
>
>> At this time, there is no ideal digital video format because
>> they either use compression or are very expensive.
>
>It seems to have become a conventional wisdom amongst a section of the
>'archive community' that video compression is undesirable. Like many fixed
>positions in any walk of life they need to be reviewed and questioned
>incessantly: not to do so will leave behind those that cling to it and those
>who don't make the effort to understand it in context, isolated from the
>main stream and real life.
>
>A further problem - and this is one that I have to deal with - is that a
>little knowledge is dangerous. Non technical but influential managers can
>quickly understand that compression does (in theory) introduce artefacts and
>conclude therefore that compression is bad under all circumstances. Where is
>the perspective in such a conclusion? Other factors are much more difficult
>to understand - and therefore don't enter into the decision making so easily
>- questions of types of compression, types of tape/binder, differences in
>DVD and CD recording technologies - the difference in durability between a
>pressed CD you buy in the CD shop and a CDR I make on my PC.
>
>I think there is a chasm opening between the professional broadcasters and
>the purest archivists. Archives equal money and many of the video archives
>are owned by private companies who are going to keep and exploit them for
>profit. The managing engineers have got to find cost effective business
>justifiable technical solutions. Are you saying that I have to store
>everything on D1 or D5 - old formats no longer actively marketed (but for a
>time supportable) (at least in Europe) - on machines which are going to be
>difficult and expensive to maintain. Why can't I use a mainstream format -
>say digibeta or one of the older or newer 50Mbps formats. If you now say to
>me I can't because they are compressed, out of principle, even though I can
>barely if ever see a difference between the original and the compressed
>picture, then I have to say to you that the machines are current,
>maintainable and reliable and that's a big part of the practical equation as
>well.
>
>Compression reduces the business costs. Good business. It makes it possible
>to have a coherent justifiable station archive preservation policy. The key
>is for the archivists to take the initiative and control of the compression
>to be used. Certainly not DVD-V, probably not systems with long GOPs and
>perhaps not the lower acquisition bit rate formats (20-25Mbps). But the
>various 50Mbps intraframe only systems are very very good for practically
>all TV pictures.
>
>What about the prospects for an archive format for video? This of course
>goes hand in hand with the EBU/SMPTE work on exchange formats - also
>compressed of course. They always will be, sorry. The compressed version of
>the same machine with the same tape and transport will be cheaper to run
>(per programme hour) than the same thing recording uncompressed. Who is
>going to buy the most machines - the broadcasters or the archivists? Where's
>the market research? If the archivists continue to stick out for an
>uncompressed format then that is going to be a format ignored by the
>majority of broadcast users more concerned with today than tomorrow. That's
>going to make the machine even more expensive and less likely to survive the
>decades required of it.
>
>Apply the same logic to robotic storage - compression equals more in the
>same machine.
>
>For most organisations the sums just don't add up - the revenue value of the
>archives does not cover the investment. You have to make it cheaper.
>So many organisations are still just ignoring the problems or dabbling
>without taking the plunge, hoping for the grail 'soon'.
>
>If the archivists make an alliance with today's broadcasters, their input
>will be taken seriously. Make a few compromises,(eg) accept and find the
>right type of compression, and you will have a format which is accepted and
>used by both the archivists and the broadcasters - and you will be much more
>sure that decades of survival are possible.
>
>The most significant advances now coming is where the data format of the
>video is support independent and available spearately on a connector on the
>output of the video recorder/player: up until now the compression and data
>system went hand in hand with the tape machine and tape type, at least in
>digital systems. If I can chose which type of support to use to put my video
>stream on, without changing that video stream as such, then I am better
>prepared for the future (particularly if the decoding is softaware based).
>
>Technology changes - if you miss a couple of IBCs or NABs you don't
>recognise the show anymore. If the archive community doesn't move hand in
>hand with the broadcasters and find some commonality of approach, then what
>each side means by archiving is going to be so different and so entrenched
>such that neither will have any empathy with the other. That would be a
>shame because it is the professional broadcasters who are producing the
>moving picture archives of the future and we are going to do it our way
>because we have got to make the decisions now and justify them in financial
>terms. The heads have got to come out of the sand and entrenched positions
>must become flexible and accountable.
>
>There is a window of opportunity now since integrated
>acquisition/storage/transmission/archive systems that work are being sold
>and interchange standards work is well advanced. Reassess these principled
>conventional wisdoms in the real world and grab the opportunity that won't
>be there for long.
>
>Tony Gardner
>European Commission A/V studios, Brussels
>These are my own opinions and do not necessarily represent those of my
>employer.

Richard L. Hess                              richard@richardhess.com
Glendale, CA USA                           http://www.richardhess.com/
Web page: folk and church music, photography,
                  broadcast engineering, home wiring, and more