A murder of emails
The Wall Street Journal published an article yesterday explaining some of the problems that the National Archives are having in storing email. Slashdot has commentary on the story here. The issue has come to the forefront as they prepare to receive the digital archives of the Bush administration in 2008. They expect 100 million emails, three times what they got from the Clinton people.
Expect to see more of this kind of story. Their problem isn't raw storage, it's two-fold. First, what mechanism do they use for long term storage, since they only want to do it once. Right now, they're transferring it to mag tape, clearly not a good forever solution.
The second problem, which is only lightly touched on in the article, is the problem of "format bending" or transformation of obsolete media standards, file formats and application data structures into some universal mechanism. There's a lot of obsolete standards out there and the Archives will eventually have to support all of them.
Some day there will be a profession of data archivists, who only deal with the obsolete and discarded, the midden heaps of the data world. We will need librarians trained in not only long dead spoken languages but also computer ones.
Posted on December 30, 2005