Logo Cover Cover Cover

Chapter 1: The opportunity for using open source information and user-generated content in investigative work

Craig Silverman is the founder of Emergent, a real-time rumor tracker and debunker. He was a fellow with the Tow Center for Digital Journalism at Columbia University, and is a leading expert on media errors, accuracy and verification. Craig is also the founder and editor of Regret the Error, a blog about media accuracy and the discipline of verification that is now a part of the Poynter Institute. He edited the Verification Handbook, previously served as director of content for Spundge, and helped launch OpenFile, an online local news startup that delivered community-driven reporting in six Canadian cities. Craig is also the former managing editor of PBS MediaShift and has been a columnist for The Globe And Mail, Toronto Star, and Columbia Journalism Review. He tweets at @craigsilverman.

Rina Tsubaki leads and manages the "Verification Handbook" and "Emergency Journalism" initiatives at the European Journalism Centre in the Netherlands. Emergency Journalism brings together resources for media professionals reporting in and about volatile situations in the digital age, and Tsubaki has frequently spoken on these topics at events, including a U.N. meeting and the International Journalism Festival. Earlier, she managed several projects focusing on the role of citizens in the changing media landscape, and in 2011 she was the lead contributor of the Internews Europe's report on the role of communication during the March 2011 Japan quake. She has also contributed to Hokkaido Shimbun, a regional daily newspaper in Japan. She tweets at @wildflyingpanda.

With close to 18,000 followers, the Twitter account @ShamiWitness has been a major source of pro-Islamic State propaganda. In their investigation of the account, British broadcaster Channel 4 reported that ShamiWitness’ tweets “were seen two million times each month, making him perhaps the most influential Islamic State Twitter account.” Channel 4 also reported that two-thirds of Islamic State foreign fighters on Twitter follow the account.

Channel 4 set out to investigate who was behind the account. All it had to go on was the account and its tweets — the person behind ShamiWitness had never shared personal information or anything that might indicate where they were based.

Simon Israel, the Channel 4 correspondent who led the investigation, said in the report that there were no known photos of ShamiWitness.

“But there are moments — and there are always moments — when the hidden trip up,” he said.

Israel said an analysis of the ShamiWitness account revealed that it used to go by a different handle on Twitter: @ElSaltador. At some point, the account owner changed it to @ShamiWitness.

Channel 4 investigators took that previous Twitter handle and searched other social networks to see if they could find anyone using it. That led them to a Google+ account, and then to a Facebook page. There they found photos and other details about a man living in Bangalore who worked as a marketing executive for an Indian company. Soon, they had him on the phone: He confirmed that he was behind the ShamiWitness account.

The result was an investigative story broadcast in December 2014. That report caused the man behind the Twitter account to stop tweeting.

Channel 4 used publicly available data and information to produce journalism that shut down a key source of propaganda and recruitment for the Islamic State.

Journalists, human rights workers and others are constantly making use of open data, user-generated content and other open source information to produce critically important investigations of everything from conflict zones to human rights abuse cases and international corruption.

“Open source information, which is information freely available to anyone through the Internet — think YouTube, Google Maps, Reddit — has made it possible for ANYONE to gather information and source others, through social media networks,” wrote Eliot Higgins on the Kickstarter campaign page for his open source investigations website, Bellingcat. “Think the Syrian Civil War. Think the Arab Spring.”

The abundance of open source information available online and in databases means that just about any investigation today should incorporate the search, gathering and verification of open source information. This has become inseparable from the work of cultivating sources, securing confidential information and other investigative tactics that rely on hidden or less-public information. Journalists and other who develop and maintain the ability to properly search, discover, analyze and verify this material will deliver better, more comprehensive investigations.

Higgins, who also goes by the pseudonym Brown Moses, is living proof of the power of open source information when combined with dedication and strong verification practices. He has become an internationally recognized expert in the Syrian conflict and the downing of Flight MH17 in Ukraine, to name but two examples. His website, Bellingcat, is where he and others now use open source materials to produce unique and credible investigate work.

In February 2015, Bellingcat launched a project to track the vehicles being used in the conflict in Ukraine. They invited the public to submit images or footage of military vehicles spotted in the conflict zone, and to help analyze images and footage that had been discovered from social networks and other sources. In its first week of operation, the project added 71 new entries to the vehicles database, almost doubling the amount of information they had previously collected. These were photos, videos and other pieces of evidence that were gathered from publicly available sources, and they told the story of the conflict in a way no one had before.

It’s all thanks to open source information and user-generated content. As chapters and case studies in this Handbook detail, this same material is being used by investigative journalists in Africa and by groups such as Amnesty International and WITNESS to expose fraud, document war crimes and to help the wrongly accused defend themselves in court.

This companion to the original Verification Handbook offers detailed guidance and illustrative case studies to help journalists, human rights workers and others verify and use open source information and user-generated content in service of investigative projects.

With so much information circulating and available on social networks, in public databases and via other open sources, it’s essential that journalists and others are equipped with the skills and knowledge to search, research and verify this information in order to use it in accurate and ethical ways.

This Handbook provides the fundamentals of online search and research techniques for investigations; details techniques for UGC investigations; offers best practices for evaluating and verifying open data; provides workflow advice for fact-checking investigative projects; and outlines ethical approaches to incorporating UGC in investigations.

The initial Verification Handbook focused on verification fundamentals and offered step-by-step guidance on how to verify user-generated content for breaking news coverage. This companion Handbook goes deeper into search, research, fact-checking and data journalism techniques and tools that can aid investigative projects. At the core of each chapter is a focus on enabling you to surface credible information from publicly available sources, while at the same time offering tips and techniques to help test and verify what you’ve found.

As with the verification of user-generated content in breaking news situations, some fundamentals of verification apply in an investigative context. Some of those fundamentals, which were detailed in the original Handbook, are:

  • Develop human sources.
  • Contact people, and talk to them.
  • Be skeptical when something looks, sounds or seems too good to be true.
  • Consult multiple, credible sources.
  • Familiarize yourself with search and research methods, and new tools.
  • Communicate and work together with other professionals — verification is a team sport.

Journalist Steve Buttry, who wrote the Verification Fundamentals chapter in the original Handbook, said that verification is a mix of three elements:

  • A person’s resourcefulness, persistence, skepticism and skill
  • Sources’ knowledge, reliability and honesty, and the number, variety and reliability of sources you can find and persuade to talk
  • Documentation

This Handbook has a particular focus on the third element: documentation. Whether it is using search engines more effectively to gather documentation, examining videos uploaded to YouTube for critical evidence, or evaluating data gathered from an entity or database, it’s essential that investigators have the necessary skills to acquire and verify documentation.

Just as we know that human memory is faulty and that sources lie, we must also remember that documents and data aren’t always what they appear. This Handbook offers some fundamental guidance and case studies to help anyone use open source information and user-generated content in investigations — and to verify that information so that it buttresses an investigation and helps achieve the goal of bringing light to hidden truths.


Published on: 15 April 2015
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.