I pointed a crawler at eight Facebook posts from the July uprising and it came back with eight records. All eight were the same login wall. Not a partial capture, not a degraded one — the same page, saved eight times, containing none of what I had gone to get.
It is a small, stupid failure and I think about it constantly. The job had looked like a fetching problem. It was not. What I had actually built was a machine for confidently recording the outside of a door.
I run July Archive, a bilingual public record of the thirty-six days in 2024 that ended fifteen years of one government in Bangladesh, and of the roughly 1,400 people the UN's human-rights office counted as dead by the end of it. This piece is the short version of what building it taught me. The long one, with the standards and the receipts, is on the archive's own blog.
Uploading is not preserving
At least a dozen preservation sites appeared after July. Students scanning posters, families uploading photographs, someone maintaining the protest playlists. All of it worth doing, and almost none of it archiving, because the two words describe different jobs.
A website about a revolution stores things. An archive of one makes four promises: that it can say where a thing came from, prove the bytes have not changed, still resolve the link in ten years, and outlive the people currently paying for it. There is a whole ISO standard for this — 14721, written by the space-data community, who had the problem first and worst. None of the four promises is technically hard. They are just invisible, so they lose every argument with a deadline.
The one that gets skipped most often is the first, and it is the one the login wall broke. A screenshot is a picture of a page: croppable, editable, and evidence of nothing except that somebody once had a screenshot. A WARC is the transaction — the request, the response headers, the timestamp, the raw bytes, a cryptographic digest of all of it. One is a photograph of a document. The other is the document, with a chain of custody attached.
Why the distinction stops being pedantic
On 17 July 2014 a commander in eastern Ukraine posted about shooting down an aircraft. It was Malaysia Airlines Flight 17, and 298 people were on it. The post was deleted in about two hours. The Wayback Machine had already taken a copy, and that copy became evidence in an international investigation. A page that lived two hours has now outlived a decade.
That is not a curiosity, it is the job description. In 2017 the International Criminal Court issued an arrest warrant resting substantially on social-media video. The UN human-rights office and Berkeley have since published a protocol for it: hash on acquisition, keep the metadata, document every step. Read it as an engineer and it is a spec.
July needs that spec more than most events, because the numbers are contested. Estimates in circulation run from around 700 to over 8,000. The archive publishes the UN figure and says where it came from, because under the disagreement are names, and eventually a court or a historian will want material that survives cross-examination.
Things I got wrong
Two-thirds of what the crawler first gathered was duplicates. A domain-scoped crawl I had fenced badly once published 471 articles that had nothing to do with July. I have deleted more of my own output than I have published.
The failure that frightened me most, though, was one I had never considered. In 2007 physicists at CERN checked 8.7 terabytes across 33,700 files and found twenty-two of them silently corrupted — no error, no warning, files that open perfectly and contain the wrong bytes. About one in 1,500. Storage does not fail loudly; it fails quietly, and a backup of a corrupted file is a corrupted backup.
So every original and every WARC gets a SHA-256 on arrival, and a job runs each Monday at 05:00 UTC to re-download a sample, re-hash it, and shout if anything has drifted. The whole store copies nightly to a different bucket under a different account, and deletions are quarantined rather than propagated — because the failure I am actually defending against is not a disk dying, it is my own account being compromised and the deletion replicating perfectly.
What it changed about everything else I build
I did not expect an archive to rearrange how I think about ordinary software, but it did, and the rearrangement was mostly about time.
Pew found 38% of the web pages that existed in 2013 were unreachable by late 2023. A Harvard study found half the links cited in US Supreme Court opinions no longer resolve. Human Rights Watch found 21% of the Syrian Archive's 1.75 million YouTube videos already gone, because moderation software cannot tell propaganda from evidence and was never asked to. Yahoo bought GeoCities for about $3.6 billion and closed it ten years later; volunteers had six months to crawl what they could. Nobody plans to be GeoCities — which is precisely what the plan is for.
Four habits came out of that, and none of them are archive-specific:
- Checksum on arrival, verify on a schedule. If you cannot prove a file is the one you stored, you are not storing it, you are hoping.
- Identifiers must outlive hostnames. The archive's sources carry ARKs, issued under an institutional number rather than a domain, so a link survives us changing our minds about where things live. Registering a namespace costs nothing. Almost nobody does it.
- Build the export before you need it. The catalogue is harvestable over OAI-PMH in Dublin Core, so another institution could mirror the whole thing without asking. A system nobody else can copy is a single point of failure wearing a logo.
- Deletion has to be real, and it has to be recorded. A takedown needs written reasons, writes an audit entry before anything goes, and then genuinely removes the thing — the WARC and the replica included. A rights regime with no route to erasure is not one.
The uncomfortable part is how much of that is just retention policy with better manners. I have spent the last year writing about organisations that kept everything and lost it. An archive is the same discipline pointed the other way: keeping deliberately, provably, and with an exit.
Small and checkable
The archive currently publishes 68 sources. The timeline has fourteen entries; eight days are missing because the sourcing was too thin to stand behind. Of sixty-nine protest-rap tracks we have published eight. Those numbers are embarrassing next to the sites claiming thousands of items, and I have made my peace with it. I would rather be small and checkable than large and vague.
The promise that worries me is the fourth one — outliving ourselves — because it is the only one that is never finished. Brewster Kahle used to say the average web page lived about a hundred days. The figure was folklore even then. The direction was not.
The eight Facebook posts are still behind the login wall. They are still there, for now, which is the most any of this ever gets to say. But if you are running one of the other July collections: write WARCs instead of screenshots, hash your files, register an identifier namespace, and put your metadata somewhere a machine can fetch it. The tools are free and I would rather your material outlived us both.
And if the infrastructure is more than volunteers can carry, send it to us. We will preserve it with the provenance intact, under the name of whoever made it, and you can take it back whenever you like — because an archive you cannot leave is just another wall.