[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[AV Media Matters] Compression discussion.
Dear Jim,
I sent this some time ago, but possibly to the wrong address. (the reply to
address does not always give this address). Do you accept such long
statements? I hope to provoke a good discussion. Many questions and aspects
are left in the air. Central themes are, (1) compression is an asset and not
always bad, and (2) what about data tape solutions.
Best wishes
Tony Gardner
Part of my work is for the video and sound archive services with
responsibilities for both long term preservation (10's of years) and medium
term storage (years) for convenient access and duplication of the archives
for our clients (mainly broadcasters). Our material is news, documentary,
and current affairs sound and vision.
Whilst we are not a commercial organisation, nevertheless the cost of our
policies plays an important role.
The list does a good job is airing the issues involved in archiving and
access. It is important that they are better and more widely understood.
Regarding compression, I do not agree that it is something to be avoided. I
think its properly considered use is a financial and technical asset.
Firstly the question of compression must be taken separately from the
support.
Secondly one must ask oneself questions about one's objectives - long term
preservation, access, both, transmission ...
I have heard it said by archive managers - off the list - that they will not
even consider a compressed format! Never mind all the other issues - NO
COMPRESSION!
Its plain daft.
OK, an archivist's agenda is different to a practical broadcaster, but to
exagerate one issue they think they understand over and above the other
technical, strategic and cost implications is not a recipe for responsible
management decisions.
Ignoring the recording media itself, for the purposes of long term
preservation can anyone realistically doubt that the light compression of -
for example - digital betacam or even the 50Mbps systems for practically all
the pictures we encounter in our professional lives is transparent, or that
any artefacts or modifications are insignificant and invisible?
I consider that the archive compression system should have a significant
'quality headroom' over and above the quality of the original material.
Remember that all TV scanning is compressed anyway, (now I'm being
pedantic), and most material is compressed in the camera even more - SP
(using analogue compression), DIGIBETA, DVCPRO or SX etc....
(We do not originate on film, in which case reasonable objections to the
principle of conserving film as video can be made).
For the purposes of access to the archive, I think that the degradation of
the 20-25Mbps systems (as used in DVCPRO and SX) is quite acceptable, and
also small enough to be substantially insignificant and nearly invisible on
everyday pictures. Depending on your type of material these compression
systems (or their 50Mbps partners) could also be seriously considered as
longer term formats (ignoring the support - see later). I believe some big
organisations in europe are already doing this de facto.
In terms of immediate access, we deliver by satellite digital channel at
8Mbps (as many others) and this is quite satisfactory to many of our clients
for access to pictures (but I do not suggest this for long term
preservation).
As far as the recording media goes, - whatever the compression - for long
term archiving you must choose a tape or disc format (or system) which has a
good chance of physically surviving tens of years, and which is well
established on the market (for the survival of replay machinery). Also
preferably multi-sourced, but that is wishful thinking.
I think that there are few video formats other than digibeta and beta SP
which anwser this requirement for the moment. Audio alone is less obvious.
The DV and SX formats fail because they are not sufficiently established in
the marketplace: will they still be sold in ten years? DV has the further
problem of tape size and stability to answer.
I doubt whether D3 (composite) should even be considered (surely it cannot
be considered current or have a long term future - it belongs in the past
with D2). Where is D5 going? D1 is surely the format for purists, but with
both they are pricey and long term maintainability must be in doubt.
DVD has too low a data rate for long term and DVD-R has what life time?
Nevertheless in cheap robots they make a good near on line retrieval system.
A good bet for long term conservation of any audiovisual data has to be
digital data tapes used by our ever closer friends in the computer world.
There are several well established formats - DTF, DLT, Eagle, - and many are
put to work in robots in data critcial industries like banking and
insurance. Consider this: well established technologies, large numbers of
machines in service, automated quality control, foreseeable strategies for
migration (automated), different data formats can be mixed ....
There should be much more talk about these formats and their potential for
preservation-archiving. I am not suggesting necessarily that 'someone'
should take one of these formats and make a video machine with it (what is
digital betacam anyway?) but to use them as they are in standard as-sold
form, do some systems engineering and add a server and codecs, to store
video as data. For digitisation, use main stream MPEG2 or DV-type profiles
and levels, preferably as recommended by the EBU/SPMTE report on
harmonization. Adapt you Mbps and GOPs and the rest to your application and
higher than the source quality (is retrieved material destined to be
re-edited?). The durability of the MPEG2 broadcast formats can be considered
as being long term.
I have been through a tender procedure for such a system and, although it
proved too expensive to fully implement (particularly with robots) in our
situation, I have no doubt that it is the future. You can archive
non-compressed or at any compression rate you like, and mix them depending
on the material - it can than becomes a technical management decision rather
than a principled debate. It follows that HDTV, EDTV, SDTV and internet TV
material can live happily together.
For video archives at the moment we decided to use digital betacam to
preserve, but not to rush to copy the SP's to digital betacam. We don't keep
other formats. We are looking again at datatape and disc technologies both
for access and long term storage.
Forget the grail of a single archive video format, but think about data tape
solutions - people are archiving data long term and effectively already out
there in other industries.
And end these interminable coffee time discussions about compression is bad:
concentrate on the archive strategies where significant quality improvements
are possible - should you be storing 2 or more 'master' copies in different
places (and different formats?); have you separated long term masters from
copies for access; how good is my database; metadata; database searching;
where does the material I am archiving come from and can I get better
quality copies than the ones 'they' send me before arhciving?; are the
procedures for selection and destruction of material satisfactory (or is
this also anathema to purists); are the quality of the archive tapes being
checked regularly - what is the strategy for this - what machines would we
like on the market to help (drop out counters, reconditioners ...) and how
should they be used.
These are my personal opinions, not necessarily the agreed policies of my
employer.
Tony Gardner
Press and Communications service,
(Archives - studios - europe by staellite)
European Commission
Brussels EUROPE
anthony.gardner@cec.eu.int
Tony Gardner
Press and Communication Service
Audiovisual Unit
TRE120 1/63
European Commission
Wetstraat 200 1049 Brussels
+32 2 299 9161 +32 2 299 9218(fax)
+32 75 828051(gsm)
anthony.gardner@cec.eu.int