Re: [ARSCLIST] Hard disk drives and DAT/ calculating the future

["Andes, Donald" <Donald.Andes@xxxxxxxxxx>]

Eric, let me post the obvious, to give scope of why not many attempt to
tackle these numbers.

Most of what you are trying to calculate is in itself very variable?
i.e. What are the specs on a common hard drive? (Size, MTTF, "Shelf"
life)

Most of what you are trying to calculate contains very speculative
products?
i.e. Blue Ray, LTO 4

There are MANY other concerns which most who haven't already delved into
the issue are completely unaware of?
i.e. File Format migration, Hard Drive Formating incompatibilities, Data
Corruption 

In probably 10 years we will be into new technology which is only in a
fringe state now such as Flash Hard Drives, speculating 50 or 100 years
out, is like trying to predict the Automobile, the Airplane, the
MicroChip, the PC, and the Internet, all while living in 1900.

Strangely, men at this time, armed with computers appear to believe that
we can do just that.

However, all things in perspective, the world is changing faster than it
ever has, and will be changing even more quickly as we race into the
future. I believe predicting the future is best left to salesmen.

Don Andes
EMI Music




-----Original Message-----
From: Association for Recorded Sound Discussion List
[mailto:ARSCLIST@xxxxxxx] On Behalf Of Eric Jacobs
Sent: Tuesday, March 27, 2007 5:27 PM
To: ARSCLIST@xxxxxxxxxxxxxxxx
Subject: Re: [ARSCLIST] Hard disk drives and DAT

I've been following this thread closely, and think a valuable exercise
would be to create detailed cost models for different storage scenarios.
Scenarios might include:

   - Hard disk on a shelf
   - Optical media (gold CD-R, DVD-R, Blue Ray)
   - Managed storage (mirrored on-line storage, tape back-up)

Common to each storage scenario:

   - Model three different size collections (1 TB, 10 TB, 50 TB)
   - Model a 50-year or 100-year life cycle
   - Make reasonable assumptions for media checking, refreshing,
     migration, and physical storage
   - Make assumptions about density and technology evolution 
     (Moore's law-ish)

Maybe there are other storage scenarios to be considered, and other
details to be considered, but you get the idea.  If I were an IT expert,
I'd crunch these cost models myself.

The ideal scenario will simply be the one with the lowest total cost,
since data integrity, migration, short-term and long-term costs will all
be accounted for, and the end goal (reliable long-term storage of data)
will be the same for each scenario.

I think all scenarios are viable - even the hard disk on the shelf (so
long as you have redundancy, check the integrity of the drive and the
data at some scheduled interval, and have a budget to deal with drive
failure and data recovery as needed, and refresh the hardware at some
reasonable interval).

I expect that different storage scenarios will be appropriate for
different collection sizes (1 vs 10 vs 50 TB).

To me, this seems like a pretty logical approach to the problem of
storage of digital assets for preservation.  As a collection grows,
different storage scenarios will make sense.  The question for many is
what storage scenario do you start with, and at what point do you
transition to the next level storage scenario.  In the end, it's all
cost driven (again, we're assuming that we drive each scenario to
equivalent levels of reliability through redundancy or other means).

I'd be surprised that no one has ever thought of working through these
different scenarios with a detailed cost model in order to determine
storage strategy and policy at some institution.

It's not rocket science, and the answer should be fairly straight-
forward I would think once you've captured all the costs and reliability
aspects (just because it's straight-forward doesn't mean that it's
easy!).

So, does anyone know of an existing cost-based study of long-term data
storage for several different scenarios AND several different size
collections?  If not, I may need to sharpen my pencil...

Eric Jacobs
Principal

The Audio Archive
tel: 408.221.2128
fax: 408.549.9867
mailto:EricJ@xxxxxxxxxxxxxxxxxxx

- --------------------------------------------------------------------




Music from EMI 

This e-mail including any attachments is confidential and may be legally privileged. If you have received it in error please advise the sender immediately by return email and then delete it from your system. The unauthorised use, distribution, copying or alteration of this email is strictly forbidden. If you need assistance please contact us on +44 20 7795 7000. 

This email is from a unit or subsidiary of EMI Group plc. 

Registered Office: 27 Wrights Lane, London W8 5SW 

Registered in England No 229231.


- --------------------------------------------------------------------