The challenges of distributed investigative journalism

One of the clearest ideas to emerge from the excitement around the new media transformation of journalism is the notion that the audience should participate in the process. This two way street has been nicely described by Guardian editor Alan Rusbridger as the “mutualization of journalism.” But how to do it? What’s missing from what has been tried so far? Despite many experiments, the territory is still so unexplored that it’s almost impossible to say what will work without trying it. With that caveat, here are some more or less wild speculations about the sorts of tools that “open” investigative journalism might need to work.

There have been many collaborative journalism projects, from the Huffington Post’s landmark “Off The Bus” election campaign coverage to the BBC’s sophisticated “user-generated content hub” to CNN’s iReport. One lesson in all of this is that form matters. Take the lowly comment section. News site owners have long complained, often with good reason, that comments are a mess of trolls and flame wars. But the prompt is supremely important in asking for online collaboration. Do journalists really want “comments”? Or do they want error corrections, smart additions, leads, and evidence that furthers the story?

Which leads me to investigative reporting. It’s considered a specialty within professional journalism, dedicated to getting answers to difficult questions — often answers that are embarrassing to those in power. I don’t claim to be very good at journalistic investigations, but I’ve done enough reporting to understand the basics. Investigative reporting is as much about convincing a source to talk as it is about filing a FOIA request, or running a statistical analysis on a government data feed. At heart, it seems to be a process of assembling widely dispersed pieces of information — connecting the distributed dots. Sounds like a perfect opportunity for collaborative work. How could we support that?

A system for tracking what’s already known
Reporters keep notes. They have files. They write down what was said in conversations, or make recordings. They collect documents. All of this material is typically somewhere on or around a reporter’s desk or sitting on their computer. That means it’s not online, which means no one else can build on it. Even within the same newsroom, notes and source materials are seldom shared. We have long had customer relationship management systems that track every contact with a customer. Why not a “source relationship management” system that tracks every contact with every source by every reporter in the newsroom? Ideally, such a system would be integrated into the reporter’s communications tools: when I make a phone call and hit record (after getting the source’s permission of course) that recording could be automatically entered into system’s files, stamped by time, date, and source, then transcribed by machine to make it searchable. Primary documents would be also be filed in the system, along with notes and links and comments from everyone working on the story. The entire story of the story could be in one place.

There have been experiments in collaborative journalistic files, such as or even good local wikis. But I don’t believe there has yet been a major professional newsroom which operated with open files. For that matter, I am not aware of this type of information filing system in existence anywhere in journalism, though I suspect it’s what intelligence services do.

Public verification processes
Journalism aims to be “true,” a goal which requires elaborate verification processes. But in every newsroom I’ve worked with, essential parts of the verification standards are not codified. “At least two sources” is a common maxim, but are there any situations where one is enough? For that matter, who counts as a definitive source? When is a conflict of interest serious enough to disqualify what someone is telling you? The answers to these questions and many more are a matter of professional practice and culture. This is confusing enough for a new reporter joining staff, let alone outsiders who might want to help.

Verification is necessarily contextual. Both the costs of verification and the consequences of being in error vary widely with circumstance, so journalists must make situational choices. How sure do we have to be before we say something is true, how do we measure that certainty, and what would it take to be more sure? Until this sort of nuanced guidance is made public, and the public is provided with experienced support to encourage good calls in complex or borderline cases, it won’t be possible to bring enthusiastic outsiders fully into the reporting process. They simply won’t know what’s expected of them, to be able to participate in the the production of a product to certain standards. Those standards depend on what accuracy/cost/speed tradeoffs best serve the communities that a newsroom writes for, which means that there is audience input here too.

What is secret, or, who gets to participate?
Traditionally, a big investigative story is kept completely secret until it’s published. This is shifting, as some journalists begin to view investigation as more of a process than a product. However, you may not want the subject of an investigation to know what you already know. It might, for example, make your interview with a bank CEO tricky if they know you’ve already got the goods on them from a former employee. There are also off-the-record interviews, embargoed material, documents which cannot legally be published, and a multitude of concerns around the privacy rights of individuals. I agree with Jay Rosen when he says that “everything a journalist learns that he cannot tell the public alienates him from the public,” but that doesn’t mean that complete openness is the solution in all cases. There are complex tradeoffs here.

So access to at least some files must be controlled, for at least some period of time. Ok then — who gets to see what, when? Is there a private section that only staff can see and a public section for everyone else? Or, what about opening some files up to trusted outsiders? That might be a powerful way to extend investigations outside the boundaries of the newsroom, but it brings in all the classic problems of distributed trust, and more generally, all the issues of “membership” in online communities. I can’t say I know any good answers. But because the open flow of information can be so dramatically productive, I’d prefer to start open and close down only where needed. In other words, probably the fastest way to learn what truly needs to be secret is to blow a few investigations when someone says something they shouldn’t have, then design processes and policies to minimize those failure modes.

There is also a professional cultural shift required here, towards open collaboration. Newsrooms don’t like to get scooped. Fair enough, but my answer to this is to ask what’s more important: being first, or collectively getting as much journalism done as possible?

Safe places for dangerous hypotheses
Investigative journalism requires speculation. “What if?” the reporter must say, then go looking for evidence. (And equally, “what if not?” so as not to fall prey to confirmation bias.) Unfortunately, “what if the district attorney is a child molester?” is not a question that most news organizations can tolerate on their web site. In the worst case, the news organization could be sued for libel. How can we make a safe and civil space — both legally and culturally — for following speculative trains of thought about the wrongdoings of the powerful? One idea, which is probably a good idea for many reasons, is to have very explicit marking of what material is considered “confirmed,” “vetted,” “verified,” etc. and what material is not. For example, iReport has such an endorsement system. A report marked “verified” would of course have been vetted according to the public verification process. In the US, that marking plus CDA section 230 might solve the legal issues.

A proposed design goal: maximum amplification of staff effort
There are very many possible stories, and very few paid journalists. The massive amplification of staff effort that community involvement can provide may be our only hope for getting the quantity and quality of journalism that we want. Consider, for example, Wikipedia. With a paid staff of about 35 they produce millions of near-real time topic pages in dozens of languages.

But this is also about the usability of the social software designed to facilitate collaborative investigations. We’ll know we have the design right when lots of people want to use it. Also: just how much and what types of journalism could volunteers produce collaboratively? To find out, we could try to get the audience to scale faster than newsroom staff size. To make that happen, communities of all descriptions would need to find the newsroom’s public interface a useful tool for uncovering new information about themselves even when very little staff time is available to help them. Perhaps the best way to design a platform for collaborative investigation would be to imagine it as encouraging and coordinating as many people as possible in the production of journalism in the broader society, with as few full time staff as possible. These staff would be experts in community management and information curation. I don’t believe that all types of journalism can be produced this way or that anything like a majority of people will contribute to the process of journalism. Likely, only a few percent will. But helping the audience to inform itself on the topics of its choice on a mass scale sounds like civic empowerment to me, which I believe to be a fundamental goal of journalism.

102 thoughts on “The challenges of distributed investigative journalism”

  1. Not sure how we flew under your radar, but we’re addressing this very challenge with the Public Insight Network, a network of sources and newsrooms we’ve been building for 8+ years. I really like the term “Source Relationship Management” … a few weeks back I road-tested it ( as a general heading for what we’re creating: A relationship management tool for journalists.

    We have a network of 40+ newsrooms, including ProPublica, Ctr for Investigative Reporting, and the Center for Public Integrity. I won’t say we’ve *solved* this problem, but I do believe that given how systematic, well-funded, and technologically robust our platform is, we have a pretty darn good chance of figuring this out.

  2. What i don’t realize is if truth be told how you’re no longer really
    much more well-appreciated than you may be right now. You are so
    intelligent. You already know thus significantly
    in relation to this topic, produced me for my part imagine it from
    numerous various angles. Its like women and
    men aren’t fascinated except it is something to accomplish with Girl gaga!

    Your personal stuffs nice. All the time maintain it up!

  3. These are the compounds that initiate muscle growth and advertise formation of muscle fibers.
    Start running slowly and then increase the speed and intensity as time goes by.
    com is a discount prom dress company based in Darlington,

  4. Hi there to every one, the contents present at this web site are really remarkable for people
    knowledge, well, keep up the nice work fellows.

  5. Sometimes appoint escort girls for friendship and enjoy the companionship with a gorgeous and hot woman. A low profile
    tour company can work with little investment, if the entrepreneur is willing
    to publicize the business by word of mouth, scout the locations
    in advance to make the necessary arrangements, and make the necessary payments by collecting advance payments from the interested tour parties.
    Every single woman in India could give you pleasure just the way
    you want it.

  6. De inhoud kan ook worden verplaatst naar de Mac, gemanipuleerd
    en gezet op de HTC opslagkaart. De huidige situatie is
    dat, kabelinstallatie Sacramento heeft haar invloed niet alleen in de steden maar in en rond het
    (onafhankelijk van stedelijke en landelijke gebieden).

    Na de i – OS 5 update is gedownload, back-ups van uw volledige inrichting
    – applicaties, muziek, video’s en alle – en installeert de
    update, dan synchroniseert al uw gegevens weer.

  7. They provides Residential, Automotive and Commercial locksmith services.
    To solve to sore back issues consider purchasing a quality office chair with plenty of lumbar
    support and plenty of adjustability available at any office store
    as Staples and Office – Max. The old designs had
    expensive heavy, bulky, dark, carved or gilded oak or mahogany wooden pieces with cluttered embellishment.

  8. Foor new home buyers,taking out a fixed rate loan would probanly be a
    wise move. Maany companies offer services for anything
    to do with residences, from the porch, to the deck area, to the bathroom,
    to the windows. After you hqve invented the final figures,
    make suire that your present and future business prospects will not find it too difficult to service
    the home renovation loan.

  9. If some one needs expert view regarding blogging then i propose him/her to pay
    a quick visit this website, Keep up the fastidious work.

  10. It consists of five minutes of cardio exercise, usually
    with very moderate routines. Applied as a powder, white clay is a
    deodorant and an excellent natural alternative to talcum powder.
    Clinical results have shown reduced appearance of pores, fine lines and wrinkles, and improved skin tone.

  11. bilbolaget service sundsvall
    service elektro helios sundsvall
    ets service sundsvall
    jaguar service sundsvall

    It also pays for you to access amenities such as pools,
    fitness rooms, and party guest rooms. By you budgeting your heating bills yourself, you
    get the interest fee. Japan is in the long term 20 year long deflation.

    mtd service sundsvall
    service och teknik sundsvall
    service bil sundsvall
    kärcher service sundsvall

  12. En “Pascualino y los globos”, tenemos el relato dе un hombre a punto ԁe morir
    bɑjo el peso de unaa elefantisiaca mujer ϲon la que está
    teniendo sexo oral.

    Нere іs my web ρage … aplicaciones tablets android

  13. Thanks for the auspicious writeup. It actually used to be a
    amusement account it. Glance complex to more added agreeable from you!
    By the way, how can we keep up a correspondence?

  14. Can I simply just say what a comfort to uncover someone who truly understands what they’re discussing over the
    internet. You definitely realize how to bring a problem to light
    and make it important. More and more people really need to look at this and understand this side of your story.
    I can’t believe you aren’t more popular since you most certainly have the gift.

  15. My spouse and I stumbled over here by a different web address and thought I
    may as well check things out. I like what I see so i am just following you.
    Look forward to finding out about your web page yet again.

    Feel free to visit my webpage … spotify free code premium

  16. This is the right web site for anybody who wishes to find out about
    this topic. You realize so much its almost tough to argue with you (not that I actually would want to…HaHa).
    You definitely put a brand new spin on a subject that has been written about for
    ages. Great stuff, just excellent!

  17. Thank you for some other great post. Where else may just anyone get that type of information in such an ideal approach of writing?
    I have a presentation subsequent week, and I’m on the search for such information.

  18. Directed toward the restoration of vital body systems such as neuromuscular, musculoskeletal,
    cardiovascular and respiratory systems; all vital systems that
    contribute to the well being of an individual and his proper coping and recovery.
    Several verbalize that Bowflex Power Pro is simply a “gimmicky” for typical bodybuilding
    and muscle training weights. Additionally, the digestive system can be given with
    ample fluids with these laxatives by absorbing fluids from several other body parts.

  19. It’s actually a nice and helpful piece of info.

    I’m satisfied that you just shared this useful info
    with us. Please stay us informed like this. Thanks for sharing.

  20. Reminds me of the heart of the ocean from Titanic, doesn’t it.
    Foot orthotics boxing techinque presses and then forges, equally, can also
    be frequently utilised in the jewellery processing business nowadays,
    to aid guarantee the production of a consistent, highly money-making merchandise.
    Tiffany: Rare Yellow Diamonds The Tiffany yellow diamond,
    which was mined from South Africa, was 128.

  21. Wanted to drop a remark and let you know your Feed is not functioning today. I tried adding it to my Google reader account but got absolutely nothing

  22. Hi there, this weekend is fastidious designed for me, since this
    point in time i am reading this great informative post here at my house.

    My site is across at RWFlowers.Com,
    be sure to have a look sometime!

  23. It won’t necessarily cost less to get your site this way, but it does
    provide a way to spread the cost over time. This shows you
    how user friendly these templates are. Predicting i
    – Phone 6: Size, features and release date.

  24. Hey fantastic blog! Does running a blog like this require a massive amount work?
    I’ve very little understanding of programming but I had been hoping to start my own blog in the near future.
    Anyways, if you have any ideas or techniques for new blog owners please share.
    I know this is off subject nevertheless I just had to ask.


  25. If some one wishes expert view about running a blog then i propose him/her to pay a quick visit
    this weblog, Keep up the pleasant job.

Leave a Reply

Your email address will not be published.