[Table of Contents]


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [ARSCLIST] long range file storage



Hello, Damien,

I'm sure we'll all be interested in hearing the solution for
Culpepper when it gets fully designed, implemented, debugged, and
running--it's a fascinating project.

Many (most?) archives don't grow 8PB per year. University of
Toronto's TSpace system is one facility I've discussed using as a
possibility for a campus project. This is modeled after work done at
MIT and Cambridge, as I understand it. It's a disc/tape hybrid.

An hour-long stereo audio CD, as you know, is about 0.6 GB. So, 1TB
can hold about 1600 one-hour audio CDs. Your growth sounds like
adding 12.8 million CDs to the archive annually. I realize that the
LoC archive includes video so that is what really adds up. Are you
planning on including high-resolution (4K?) film scans in this system?

I do think that many of the people struggling here (i.e. on this
list) have archives in the <10TB region (16,000 hours of stereo CD
quality recording). I could be wrong

Cheers,

Richard

At 05:11 PM 8/9/2005, you wrote:
But what would be the optimum system if you had an archive you
expect to grow at, say, 8 petabytes per year? Wouldn't spinning
disks be rather expensive or prohibitive in other ways?

Damien J. Moody
Information Technology Services
Library of Congress

>>> ArcLists@xxxxxxxxxxxxxxx 08/09/05 4:45 PM >>>

Hi, Russ,

I think there are some archives who are not ready to make this step.

Personally, I've made the step to spinning discs as my sole storage
medium. I have at least three copies of each file, soon to be in two
separate buildings, linked by fiber optic 100 Base FX. The two main
stores are 1TB each and then there is additional storage amounting to
more-or-less another 1TB on individual machines (that hold the third
copy). There is a fair amount of expansion space left in the systems
I have. I could probably go to 3TB each with the architecture I have.
I only retain client files for the short term.

The cloning software does NOT propagate deletes and, in the instance
of digital images, does not propagate updates to all copies (some
copies are marked "digital negative," essentially).

Long ago and far away, I made CD and then DVD copies of everything.
It took forever. Now, I check the backup logs a few times a week to
see if there are any abnormal error messages (I always get a few
error messages on email as files change during the compare/copy latency).

My Brother-In-Law has about 7,000 slides that he would like to
digitize. He's been photographing architecture to illustrate his
teaching of history. I just looked at the scans that he had done at
the college, and they ranged from 837x564 to a few at 1500x2242. I
suggested that these were probably not the best scans for
preservation. He wants CDs. He's not ready yet to move to spinning
disks. I suggested we could put the PSD files on disks and we could
burn high-rez JPEGs into gold CDs. I'd hate to put the raw PSDs on
CD! (I am anticipating PSDs > 20MB/image in the final archival
scanning and JPEGs~3MB per image).

Two mindsets/paradigms need to be brought into focus:
(1) It's all data
(2) Use data center management techniques to make sure you don't lose it

Cheers,

Richard

At 04:07 PM 8/9/2005, you wrote:
>I've been following the discussion on long-range file storage, and it seems
>that with all the complexities of burning and storing optical media as well
>as concerns about being able to play the media decades down the line
>(storing original player devices, etc.) it may not be impractical to
>consider the alternative of redundant arrays of independent hard disks and
>tape backups - along the business model of data storage?
>
>Yes, a plastic CD or DVD in itself is cheap (even at $1), but might it not
>be more efficient, even more economical to set up systems like this? Once
>the system is engineered and set up, the technicians just create and save
>the audio files, concerning themselves only with file management, naming,
>metadata, and so on. Any thoughts on this?
>
>Russ Hamm
>Ed Tech Specialist
>National School District
>San Diego, CA
>http://nsd.us
>

Richard L. Hess                           email: richard@xxxxxxxxxxxxxxx
Vignettes
Media                           web:   http://www.richardhess.com/tape/
Aurora, Ontario, Canada             (905) 713 6733     1-877-TAPE-FIX
Detailed contact information: http://www.richardhess.com/tape/contact.htm

Richard L. Hess email: richard@xxxxxxxxxxxxxxx Vignettes Media web: http://www.richardhess.com/tape/ Aurora, Ontario, Canada (905) 713 6733 1-877-TAPE-FIX Detailed contact information: http://www.richardhess.com/tape/contact.htm


[Subject index] [Index for current month] [Table of Contents]