The Digital Dark Age

Sydney Morning Herald

Friday September 23, 2005

John Huxley

The quicker we are to record and store our lives in digital form, the bleaker the future of information looks, writes John Huxley.

MATTHEW Connell, a curator at the Powerhouse Museum, recalls how he was approached several years ago by public relations officers for a successful Australian information technology company that was building new offices in Lane Cove.

"They planned to bury a time capsule containing, among other things, a laser disk that included a video of the current prime minister," Connell says. "They invited the museum to co-operate by undertaking to dig up the capsule after 50 years and put the objects on display."

At first it seemed an attractive proposition: a snapshot of Australian high-tech history, recorded for posterity. But slowly Connell, the museum's specialist in computing and mathematics, began to foresee problems.

"It's easy for curators to agree to such requests," he says. "They know they're not going to be around in 50 years. On behalf of my successors, though, I felt I had to ask a few questions. For instance, would there be a laser player included in the capsule? And a computer to run the display? And a back-up computer would be nice. And a full set of operating instructions? And so on."

Whether it was the time capsule or the idea that was eventually buried, Connell no longer recalls, but his story neatly illustrates a major challenge facing the "digital" generation: how can masses of machine-generated, machine-read material be stored in a form that is safe, secure from degradation and - potentially most calamitous in the long term - accessible to subsequent generations?

Connell is one of a growing number of computer experts worldwide who believe that, far from a panacea that provides increasingly efficient answers to problems of recording, storing and retrieving information, technology is deeply flawed.

They fear that rather than ushering mankind into a techno-utopia of paperless offices and clean, eco-friendly, endlessly flexible, virtual communication, it threatens to cast future generations into what Connell describes as a "digital dark age".

"It all seems very attractive - scanning documents, taking pictures, putting them into the computer for safekeeping, allowing us to throw away hard copies and to save space." Indeed, it is the most dramatic record-keeping revolution since the invention of printing.

"But what happens some time later," asks Connell, "when we discover that we no longer have the machines, the programs - the hardware, the software - the know-how, to access all that computer-based, digital material?"

Jeff Rothenberg, a Rand Corporation scientist specialising in digital longevity, provides a hypothetical, very human example of the problems that flow from accelerating obsolescence.

It is 2045, he suggests, and his grandchildren are exploring the attic of his old house when they come across a CD-ROM and a letter, which explains that the disk contains a document that provides directions to obtaining the family fortune. The children are excited. "But they've never seen a CD before - except in old movies - and, even if they found a suitable disk drive, how will they run the software necessary to interpret the information on the disk? How can they read my obsolete digital document?"

In the here and now, almost everyone, from the humblest to the highest and mightiest, is already confronting similar problems. Families contemplate how best to preserve old snapshots, which may have been copied and restored, then transferred to video and, more recently again,to disk.

Librarians and archivists - gloomily pondering the implications for future scholarship of the replacement of written letters by more easily killed, less easily opened emails - agonise over the digitation of irreplaceable hard-copy material.

Records of the entire present period of history are jeopardised by precisely the technology, and the pace of the technological change, that characterised it - a fact recognised by Rothenberg as long ago as 1999.

"The content and historical value of many governmental, organisational, legal, financial and technical records, scientific databases and personal documents may be irretrievably lost to future generations if we do not take steps to protect them," he said.

Irreplaceable material is already being rendered illegible, unintelligible, and in some cases lost, primarily in the US, which, of course, led the digital revolution.

Rothenberg says government files have been lost on subjects ranging from marijuana abuse to Agent Orange, public health to Vietnamese prisoners of war. Only belatedly was US census data rescued from digital tapes that became obsolete faster than expected.

"These cases exemplify all the modes of loss ... physical decay of media, loss of information about the format, encoding of compression of files,obsolescence of hardware and unavailability of software."

To those who believe in the immortality of new media files, Rothenberg offers a suitably new-age adage: "Digital information lasts forever - or five years, whichever comes first."

So what can be done to avert the digital dark age? The National Archives of Australia is working on a preservation project to create a standard process and infrastructure for the long-term storage of government digital records.

Meanwhile, State Records NSW has been wrestling with the problem for several years, as it tries to balance a commitment to deliver responsive government through improved use of IT with a legal requirement that machine-read records remain accessible over time.

The office has put together Future Proof, a 10-point strategy based on the best Australian standards. It stresses the need to plan long-term (even before records are created), build partnerships in and outside government, and devise record-keeping systems for managing and monitoring data.

Its options include:

* conservation, to ensure records do not degrade;

* conversion of records from one medium or format to another, such as paper records to microfilm or optical disk;

* migration of records from one system to another, such as from one software application to the next; and retention of original equipment.

But that's an enormous challenge - complicated by rapid obsolescence and the disappearance of spare parts - confronted by Connell almost daily.

Not only must he determine which pieces of computer hardware are worth saving, he is also trying to incorporate software into a museum whose culture has traditionally been "object-centred".

Though essentially a language, as a form of human endeavour it deserves its place, somewhere, somehow, alongside steam engines and space rockets.

But if software is to be saved, then what sort? Which version? How should it be kept to ensure it does not decay? And, the inevitable question: where will the machines be to guarantee it can be put on display?

Like Rothenberg, Connell is wary of the shortcomings of painstakingly converting and migrating digital data; "bit streams" can easily be polluted.

He's sceptical of the industry's ability, or willingness, to develop common standards: after all, most companies have a commercial interest in rapid obsolescence.

And, although he's fascinated by the possibility of retaining original machines to read data, he says the task is daunting.

"I thought at one point that the [Powerhouse] museum might get involved and offer a commercial service doing up old machines, but it's just not viable."

He points to the success - "a spectacular piece of collaboration" - of the British Computer Conservation Society and the London Science Museum in re-building a 1950s Ferranti Pegasus, of the type used on aspects of the design of the Sydney Opera House.

But he acknowledges such achievements are microscopic in the scheme of things, and temporary - as the Australian Computer Museum Society discovered when it tried to create a network of interactive museums displaying aspects of the industry.

It accumulated more than 100 tonnes of IT heritage before being forced to dispose of almost all of it after failing to win funding.

Connell believes the answer to the digital dilemma may lie in a combination of solutions, including the development of "emulators" - programs that effectively mimic and take the place of obsolete hardware.

Until this happens - until the "cavalry arrives", as a fellow curator once put it - his advice is simple: keep a hard copy. Or, assuming it's not already too late, make one.

Lessons of the Domesday project

There is probably no more dramatic example of the risk involved in relying on digital records than that of the BBC's Domesday Book project.

In 1986, on the 900th anniversary of the original archive book, the corporation spent ##2.5 million to create a computer-based, multimedia version - a snapshot of Britain in the mid-1980s stored on two interactive video discs.

It featured some 1 million people and contained TV video clips, 200,000 pictures and tens of thousands of maps.

The only problem was that within years the discs had become increasingly inaccessible, unreadable by subsequent generations of computers.

Fortunately, after three years' work, British and American teams have worked out new ways to access the material using software that emulated the obsolete Acorn Microcomputer system and the video discs.

The project manager, Paul Wheatley, says the experience was a classic example of the dangers facing digital heritage. "Time is of the essence. We must invest wisely in developing an infrastructure to preserve our digital records before it is too late.

"We mustn't make the mistake of thinking that recording on a long-lived medium gives us meaningful preservation." Like the original Domesday Book, which may be inspected in fine condition at the Public Record Office in London.

John Huxley

For safekeeping, go with 4000 years of the tried and true

Corporations and governments rush to save space and cut costs, to digitise data that represent the lives of companies, nations, people, little knowing that it may prove far more fragile than paper and far less accessible than ancient archaeological fragments.

The curator of technology at the Powerhouse Museum, Matthew Connell, refers to one of the museum's artefacts, a 4000-year-old Sumerian terracotta tablet featuring nine lines of cuneiform - that is, wedge-shaped - script. "People who could read, obviously could read the tablet. And because people have retained an interest in archaeology and cuneiform scholarship, and because the tablet has not been degraded, we can still read it today."

The tablet, he explains, dates back to the 11th month of the sixth year of Amar-Sin of the Third Dynasty of Ur, and is a receipt issued to Alulu for five sheep, one lamb and four grass-fed male kids to be used for a royal offering.

"We can look at it and read it. We do not need any machine. Which of our current digital formats can we expect to last 40 years, let alone 4000?" And, which of our current magnetic media can we expect to survive intact?"

John Huxley

© 2005 Sydney Morning Herald

Back to News Index | Back to Home

News Archive