Re: [ARSCLIST] Clarification (was Re: A wiki-discopedia?)

[Jon Noring <jon@xxxxxxxxxxx>]

Steven C. Barr asked:

> 1) Define (if you would) the difference(s) between "data" and "text?!"

Well, data fields can comprise "text" (ASCII, UTF-8, etc.), but
basically what I meant by "text" was plain text which is visually
structured. The example is Brian Rust and his "typed by hand" books.

Human beings are pretty good at picking apart such visually structured
text to figure out what's what -- we have, after all, sentient-level
of intelligence. But it is much more difficult to train computers to
do that to an acceptable level of accuracy. (I know, this is my area
of expertise working in the e-book and text digitization area.)

Someday when we are able to create robots like Commander Data of Star
Trek, then all this discussion is academic -- just let one of them sift
through a collection of a million records and the label data will be
perfectly transcribed and structured. But until then...

Taking a data approach, such as XML, we will apply the right semantics
to each text field to describe exactly *what* it is for machine
recognition. For example:

   <recordlabel code="ARSC:123">Columbia</recordlabel>

However we exactly define what is <recordlabel> and the meaning of the
"code" attribute, we assign that meaning to the PCDATA string
"Columbia". So the actual data is "Columbia", and <recordlabel> is a
standardized way to describe what that data is.

(The 'code' attribute is added simply as a demo, but I know we may
need to provide some sort of normalization code to a standardized
list in order to handle unusual cases, so I included it there to
show that we can fine tune elements, which <recordlabel> is, with
attributes. Those who work with HTML might begin to see similarities
between HTML markup and markup as given above. After all, HTML is
simply a particular tag set or vocabulary to structure text for web
presentation -- we can create other tag sets or vocabularies suitable
for different purposes, such as discography. XML is the standardized
and generalized framework upon which we build our specialized
vocabulary. This allows the use of standardized XML-based toolsets to
create, validate, and process our XML documents.)


> 2) Virtually all "exceptions" can be covered if an additional "remarks"
> data field is included in each data record...or, anyway, that has
> always been my approach...

True, but we want to be able to have a machine be able to understand
and process such exceptions. This cannot be done just from a comment
field written by humans for humans.

Certainly, we probably can't build a 78 discography ontology that will
handle every conceivable exception we will ever encounter in 5+ million
or so records, but I think we can get the most common and important
ones handled. The really rare and odd stuff we'll simply have to apply
a flag with a human written comment explaining the exception.


> 3) The only items that ARE NOT (nor can they be) "text" are (a) label
> images and (b) sound files of the phonorecord's sonic content. When one
> considers that even a "limited to 78's" database would have to include
> about three million phonorecords...or SIX million side-oriented data
> records...one quickly realizes that the necessary funds to purchase
> the amount of computers and/or storage devices would be well over
> the practical limits...even without the above two rather large
> files included in the data records...!

Actually, today storing 6 million raw digitized sides (and in stereo
to capture more groove information) at 96/24 would not cost Fort Knox
to store. With terabyte drives coming down in price, and new technology
around the corner to increase capacities by as much as 10x for the
same price, petabyte storage is within the realm of affordability for
fairly small projects. A project starting today could have the goal of
digitizing 6 million sides, and by the time it is completed in a few
years (obviously have to use automation to some level), storage will
be dirt cheap, allowing redundancy as well as backup storage on fixed
media, which will also greatly improve in capacity and cost.

(I roughly calculate that 6 million raw transfers of 78s, in full
stereo and 96/24 will easily fit on a petabyte storage system, even
without lossless compression which will reduce the size by about 30%
or so -- my guesstimate of course. Some advanced technologies might
eventually allow a petabyte of storage to fit inside a shoe box.)


> 5) My personal 78-catalog database (which is sadly VERY incomplete)
> uses a three-level relational structure. The first table is labeled
> RECORDS (phono, NOT data!)...and includes the data which apply to
> BOTH sides of a phonorecord. The second is labeled SIDES...and
> includes the data which is specific to the side of a phonorecord
> (i.e. condition, usw.) The third table is labeled TRACKS...and 
> includes the data which applies to a track on a side (i.e. artist,
> title, usw.). Note that most phonorecords have only ONE track per
> side. Of course, improvements in hard drives mean that relational
> tables are no longer as necessary as they once were...?!

Yes, for the 78s, being able to handle multi-track discs, such as the
early 30's experiments we see with Harmony/etc. labels, is definitely
needed. In some ways this is a kind of exception when we talk about
78s. (The base paradigm is that there's only one recording per side of
a disc.)


> 6) I'm not totally conversant...nor experienced...in XML. I'd be
> interested in seeing anything you've "roughed out" along with an
> idea how you intend for it to work...!

All XML does is structure the data in a way which is both human and
machine readable. I hope one of these days to build a basic schema
(which doesn't handle exceptions) for containing the data concerning
vanilla 78 records. This would NOT include session related
information, which would be a separate XML schema -- linkage between
the two occurs at the master level via properly structured identifiers.

In essence, I hold a 78 record in my hand and it has information on
the label(s) and runout area(s): let's organize that data -- linking
that to session information (location, date, time, musicians, all
recordings known to have been made in the session, etc.)


> Remember my one ultimate goal is to live to see the final, ultimate
> 78rpm record database completed (or as nearly such as possible...?!)!

I do, too! We all want this!


Jon