After eighteen-year-old Michael Brown was shot and killed by a police officer in Ferguson, Missouri, Bergis Jules found himself worrying not only over the horrors of the present, but also over how little of the present was likely to be preserved for the future. The best reporting on the aftermath in Ferguson was being produced by activists on Twitter, a notoriously ephemeral medium. Jules, then an archivist at the University of California, Riverside, had the impulse to start saving tweets, but wasn’t sure how. “That whole weekend, watching things unfold, I thought, ‘This is a really amazing historical moment; we should think about capturing it,’ but I was just talking to myself,” he says. The following week, attending a Society of American Archivists conference in Washington, D.C., he voiced his fears en route to drinks at the hotel bar. He caught the ear of Ed Summers, a developer who just so happened to be the author of a Twitter archiving tool—and who promptly programmed it to vacuum up #Ferguson tweets. Within two weeks, he had amassed more than 13 million.
Three weeks after the shooting, Summers blogged about the archive, which he and Jules were considering making public. Shortly thereafter, they received an inquiry from a data-mining company. When they pulled up the firm’s website, they read that its clients included the Department of Defense and, ominously, “the intelligence community.” What did the company want with the data? And what were the ethical implications of handing it over—perhaps indirectly to law enforcement—when the protesters’ tweets would otherwise evade collection? Using Twitter’s Application Programming Interface (API), the code that developers use to call up Twitter data, anyone can sift through tweets that were posted in the past week, but older posts disappear from the API’s search function, even if they still exist out on the web. The data-mining company was too late to nab a swath of the #Ferguson tweets. (Twitter has since unveiled a “premium” API that allows access to older data, for a substantial fee.) Newly mindful of the risks, Jules and Summers waited almost a year to publish their cache.