Re: [ARSCLIST] FLAC?

[Tom Fine <tflists@xxxxxxxxxxxxxxxxxxx>]

Hi Kathleen/David:

A couple of points on this:

1. even if a format is truly lossless vis-a-vis audio quality (which is perfectly achievable through 
standard data-compression techniques), I would argue that it adds some risk and complexity to your 
digital archive. The reason is that compressed data, by its very nature, is more vulnerable to 
corruption and complete loss if a smaller area of a storage medium is damaged. Now, if you have 
full-time IT guys and they're screaming about compression, I'm assuming they have a very robust and 
redundant storage system and strategy, so this might be a phantom menace. However, the complexity 
issue is real. As David pointed out, what about when the format is abandoned for the newest flavor 
of the month? I would argue that WAV -- and at this point MP3 -- are so firmly embedded in the 
digital "culture" that they are formats to be supported for the long haul. I would say, less so for 
other formats. For archiving, I'd never advocate MP3 because it's both a data-compression (more 
information in less physical space, this more likely to be lost with physical damage to storage 
media) and it's quality-lossy, and it requires a CODEC to playback whereas WAV is long established 
PCM streaming data (long established = see Bell Labs work in the 60's and PCM digital audio dates 
from the late 70s).

2. note that FLAC is one of these "open source" projects. I think the idea of these sort of projects 
is very nice and idealistic, but realistically what happens is that since these projects rely on 
volunteer work, they usually don't flower to full fruition and many are abandoned as their leading 
people move on to other interests. One reason MP3 has flourished, I would argue, is that the Germans 
who invented it have a profit motive to make it flourish and have come up with licensing 
arrangements that let others who use it do so profitably. Yes, there is LAME and other workarounds, 
but most people do most MP3 work in German-licensed applications where there is a strong profit 
motive driving development. Funny thing, capitalism -- it usually drives innovation better than any 
other mechanism. Back on topic, the only lossless audio compression scheme that I know of made by a 
for-profit company is Apple Lossless Format (ALF). Interestingly, Apple has been slow to promote 
this excellent format (but its latest iPod and Nano are compatible, and iTunes has been 
ALF-compatible for several versions now). Since it has not had mass up-take, I'd argue it's not yet 
suitable for archives -- and I'd argue it's not an archive format because it crunches more 
information into less physical space.

3. as David and others have said, storage is cheap. I mean really cheap. If your IT guys don't think 
so, their supplier of storage media is ripping them off. I'd argue that it's much cheaper to just 
expand the storage farm than to go back and crunch hundreds of files to a new format, making sure 
things like metadata translate correctly. Human labor is the most expensive part of any archive, 
especially a digital one.

4. and finally there's metadata. My limited research indicates that there is no lossless compression 
format that keeps integrity to the robust metadata you can do with BWAV. I might be wrong on this 
but I think FLAC in particular either loses or truncates metadata. You'd want to do a lot of testing 
before committing to anything if you've put a lot of human labor into metadata.

-- Tom Fine

----- Original Message ----- 
From: "David Seubert" <seubert@xxxxxxxxxxxxxxxx>
To: <ARSCLIST@xxxxxxxxxxxxxxxx>
Sent: Friday, October 27, 2006 12:09 AM
Subject: Re: [ARSCLIST] FLAC?


> Kathleen,
>
> When UCSB chose to put wav files online I got a number of emails from people begging, threatening, 
> and insisting that we use FLAC because it would save on storage space and bandwidth. I consider 
> these to be unconvincing arguments. We distribute in other formats that don't consume the 
> bandwidth of wav and bandwidth is not an issue for us. While FLAC does reduce your file size by 
> half, storage is cheap and is not a reason to make a file format decision.
>
> I was familiar with FLAC from downloading live concerts from the Internet Archive and I always 
> thought it was a pain in the ass but I did some investigation to find out more. I came away 
> convinced that FLAC is inappropriate for archival use.
>
> First, as far as I could tell, FLAC's support for metadata is poor (If somebody knows otherwise, 
> please chime in). When you blow up a FLAC file to wav or convert it to MP3, you end up with a file 
> without metatdata. Metadata interchange is pretty sketchy in all audio formats that I've found, 
> but at least archivists have agreed that we'll use the BFW headers for wav files.
>
> Second, there is no guarantee that FLAC will be around in six months or two years. The "taping 
> community" is not large enough to set a standard for general use just like the archival community 
> is too small to set standards that will be broadly adopted except amongst ourselves. I think it is 
> very important that archival standards reflect broader consensus in the pro audio world, and not 
> just what archivists think is right.
>
> You only need to look at the previous "standard" for compressed, lossless audio files, Shorten 
> (shn) to see the problems. Shorten was also open source but has been abandoned and is no longer 
> being developed. I don't know the reasons behind this, but what I do know is that files stored as 
> shorten are now in a format where tools are no longer kept up-to-date. While they can still be 
> converted, who knows if the software still available will work with the next operating system 
> upgrades. Is FLAC here for good, or will the next format (ALE?) take over in a year or two?
>
> If the AES, the EBU, or major manufacturers get together and decide on a standard for the the 
> storage of losslessly compressed audio, I'll be right on board. Until then, FLAC is not a basket I 
> would put my archival eggs in.
>
> David Seubert
> UCSB
>
> McCormick, Kathleen wrote:
> > Hello,
> >
> > I am wondering if any one has experience with or comprehensive information on FLAC (free lossless 
> > audio codec) and its effect on audio. Here is the information page 
> > <http://flac.sourceforge.net/faq.html>.
> >
> > Our systems administrator is pushing hard for us to use FLAC to save and store audio files (as a 
> > replacement for wav) and seems impervious to my hesitations (and subsequent explanations) about 
> > any type of compression for digital audio (I'm speaking here of our preservation/archival 
> > copies), even so called lossless compression. If FLAC really does what it sells itself as doing 
> > and has no negative impact on audio, then it's something to look into but I need more information 
> > before I'm willing to believe that this is no effect.
> >
> > I appreciate the help and information.
> >
> >
> > Katie McCormick
> > Reference Archivist & Coordinator for the Oral History Program
> > Special Collections Dept., J. Murrey Atkins Library
> > UNC - Charlotte
> >
> > "Whatever we do we may fail, but if we do nothing, failure is guaranteed."