For Dweb Digest I wrote a short profile of the archivist Rick Prelinger, and his work to create decentralized copies of vast tranches of archival data alongside his friends and partners at the Internet Archive.

It was a lot of fun driving over to their East Bay warehouse and sitting down to talk to him and his team about how they find, clean, capture and share vast amounts of video taken from almost anywhere you can imagine. And particularly important given the Trump regime’s attempts to erase the past and rewrite the history books.

One of the issues both teams face is scale. The Prelinger Archives collection stands at maybe 40,000 videos (on top of 60,000 that were already donated to the Library of Congress), and it’s producing around 18 terabytes of video each week — too much for their current unoptimized decentralizing process. So right now, the team is producing smaller, more usable files for decentralization and keeping the higher-resolution originals elsewhere.
—Preserving the Past on a Decentralized Future

Fun fact: the same location was home to the Democracy’s Library attempt to scan in huge amounts of government information before it was disappeared, which you may have seen via a live video feed. It was interesting to see them in the flesh!