Journalism, capital J, is supposed to be about ideals such as “democracy” and “the public interest.” It’s probably important to be an informed voter, but this is a very shallow theory of why journalism is desirable. Most of what we see around us isn’t built on votes. It’s built on people imagining that some part of the world should be some other way, and then doing what it takes to accomplish that. Democracy is fine, but a real civic culture is far more participatory and empowering than elections. This requires not just information, but information tools.

Newspaper stories online and streaming video on a tablet are not those tools. They are transplantations of what was possible with paper and television. Much more is now possible, and I’m going to try to sketch the outlines of how newsroom products might better support the people who are actually changing the world.

What’s a journalism “product”?

This is an essay about product design, so I want to be clear on what I mean by a journalism “product.” Since journalists don’t make tangible objects, the product is defined by the user’s experience. It’s whatever the user interaction with the news is. It’s picking up the paper at breakfast, or watching CNN in bed, or waiting for your mobile app to update the headlines on the bus. And yes, the product includes the stories delivered by the medium, but those stories alone are not the product; they never were. The stories were packaged into a newspaper or a television show, and that was the product. Or more precisely, the newspaper and the television show as the user chose to use it was the product.

Much of the ongoing future-of-journalism discussion focusses on how reporting needs to change, and rightly so. But that analysis stops short of the user, and how journalism is actually used — or could be used.

Digital news product design has so far mostly been about emulation of previous media. Newspaper web sites and apps look like newspapers. “Multimedia” journalism has mostly been about clicking somewhere to get slideshows and videos. This is a little like the dawn of TV news, when anchors read wire copy on air. Digital media gives us an explosion of product design possibilities, but the envisioned interaction modes have so far stayed mostly the same.

This is not to say that the stories themselves don’t need to change. In fact, I think they do. But the question can’t be “how can we make better stories?” It must be “who are our users, what would we like to help them to do, and how can we build a system that helps them with that?”

Maybe the product has to include advertising too, and maybe it must be subsidized with celebrity gossip and movie listings. There are clearly business considerations in defining journalistic products, and that’s fine, but I’m going to focus on the user experience here. Lots of other people are already talking about the business of journalism. I’m trying to make something that actually fills a need in people’s lives. If you can do that you’ll have an audience, and then you can advertise to them or sell them subscriptions, or syndicate the product, or whatever.

The first thing I ask of an improved product is this: I want people to use it. Or rather, I want people to want to use it.

Is the news boring?

By boring I mean unengaging. Not something you want to spend a lot of time with. As Joshua Benton of the Nieman Journalism Lab put it,

You never hear from people “Man, I just got lost on that news site!” We don’t create experiences that people just want to live in for a while.

He was referring to the following fact: Americans apparently spend 12 minutes a month on the average news site, versus seven hours per month on Facebook. (I’ll use American numbers as illustrative, because they’re well studied. Please let me know of data for other countries!)

It’s not that people are paying less attention to the news than they used to. American spend about 70 minutes a day with all news media, the same as in 1996. (Most of that is television news, but online consumption is climbing.) Meanwhile, total consumption of media is at an all time high. This discrepancy in growth means that people aren’t choosing to spend their increasingly connected time learning about the world — or at least they’re not learning about it through professional journalism.

I think that this is because existing journalism formats are not very good at engaging curiosity.

I’m not talking about making the writing “sharper”, adding “multimedia”, or doing better page layout. Those might be good ideas, but it’s all rearranging the furniture. There’s no fundamental change in such things. When editors give smug interviews about their publication’s “fresh new style” I want to throw things. It’s still exactly the same model of interacting with the user. There are better ways.

Design for curiosity

I live surrounded by infophiles, and for almost all of them Wikipedia is a better resource than a news site when they want to learn about the world. People spend hours roaming Wikipedia; they don’t spend hours on bbc.co.uk or cnn.com or nytimes.com or news.yahoo.com (which actualy has a far bigger audience than any traditional news outlet.) Wikipedia also tends to take the top spot in Google results, which means that more people link to it than they do to any news site.

Why is this? If professional news products are supposed to be such a vital resource, why are users overwhelmingly choosing to satisfy their curiosity elsewhere?

I think the answers to this are starting to be understood. Standard news coverage is written as a series of incremental updates, which are useless if you’re not already following the story closely. When I returned to coverage of the oil spill after not checking on it for a couple of weeks, Wikipedia was far, far better at bringing me up to date on the story. If I’d gone to CNN I would have been forced to wade through a series of daily updates to learn what I wanted to know. This has become known as the “context” problem within the future-of-journalism community, and there’s no better introduction to it than Matt Thompson’s “An Antidote for Web Overload.”

The professional news media also have a serious problem with linking, or rather, not linking. Linking is vital for creating immersive experiences online. Links create the web, the greatest time waster ever invented. They’re also neccesary for any site that wishes to be indispensable. I don’t want the product with the best content overall, I want the product that is going to serve me up the best content every single time, regardless of whether or not it was created in-house. That means links. (And content syndication, but no walled garden will ever match the wild richness of the whole web.)

Links are also vital for transparency and depth; they provide the option of more serious investigation. The uses of links in journalism, and their relative paucity in professional work, is a topic I covered in great detail in a three part series for the Nieman Journalism Lab this summer, and I also wrote about the use of links for deep storytelling here. But to make it short: linking is not yet part of professional journalism culture, and this creates a serious problem with the product.

There are many other features that afford the satisfaction of curiosity. Take the humble “search” box. Almost every news site has one, and search is an obvious way for a user to answer their question. But news web sites almost always implement search badly, and many news apps don’t even have a search function. Or, you can only search content from last few days or within the current “issue.” I don’t care about the metaphor of “issues” when I have a specific desire to learn something.

And then the river of data stemming from search logs is poorly analyzed. There is an ongoing debate about exactly how much news organizations should respond to user demand. If “Paris Hilton” is the number one search, I don’t think that the newsroom should make Paris Hilton stories their number one priority, but ignoring what users want is folly. And it’s no way to engage my curiosity, which is about what I think, not what the editor thinks.

Don’t fear fragmentation

Which leads us to another deep problem with existing journalism products: they’re not designed to be personal.

Perhaps this stems from the classic “we’ll tell them what they need to know” mindset among editors. That makes sense for a paper product, where everybody reads exactly the same copy, and you can actually “finish” consuming the content when you run out of pages to read. It doesn’t make sense when pages are effectively infinite, and there’s no obvious reason that my reading should overlap with yours. Grizzled hacks love the idea of everyone talking about the same set of stories, just like the good old days, but I don’t believe that, today, it is a reasonable thing to expect or want.

NYU journalism profession Jay Rosen recently introduced me to a 20th century social critic named Raymond Williams. “There are no masses, “ Williams wrote in 1958, “there are only ways of seeing people as masses.” Lofty discussions of journalism use words like “public” and “citizen,” but of course those are ideas. There is no homogenous audience. And at last it is possible for our media to reflect this truth, because the internet is not a broadcast medium. A newspaper or a TV report is the same experience for everyone, while my usage of a web site or an app can be extremely personal.

Then we have the problem of information overload. One person can only track so many stories. If everyone follows the same few stories, lots of important things are going to be ignored. (Which is exactly what’s happening now, according to internet scholar Ethan Zuckerman.) Delivering the same information to everyone and expecting that everyone engage it equivalently is not only an insult to diversity, it ignores that fact that our civilization is built on specialization. Different people have different capabilities, positions, and passions, and so they’re going to need different news agendas. Empathetic journalists have always been aware of this, but it wasn’t previously possible to target individual stories to specific people.

This targeting is something that users could do for themselves to a large extent, given the right interface. Today, the personalization features of almost every digital news product amount to 1) showing me stories that happen in my zip code and 2) letting me pick from a small set of coarsely defined “sections.” Why can’t I subscribe to updates on particular stories? Why can’t I set up alerts for particular terms? Why can’t I tell my news app “no more updates on Lindsay Lohan, ever”? Etc.

Who needs to know?

If we’re going to try to deliver specific stories to specific people, then who should we target? First of all: anyone who wants to know. The broadest possible access is important, and happily, the internet has been very good at extending access.

Beyond that, I go back to the idea that journalism should be useful by those who are acting to change and build our societies. Then the answer is “whoever is going to do something about it.” The product should serve the people who choose to act.

Some people will have shaping society as part of their job description, such as teachers and civil servants. Other’s won’t, and this is truly a great era to be an interested amateur. It’s not that every individual is a motivated genius just waiting for the right call to action. Rather, let’s say that there is tremendous knowledge and capability scattered throughout society, untapped. Some people are highly trained specialists with spare weekends. Other are merely informed and interested.  Some problems call for unskilled mass participation, but many more call for the dedication of a small number of the right sort of people.

My favorite example of this is the The Guardian’s successful attempt to find someone among their audience who could untangle the mystery of Tony Blair’s tax returns. Eventually, some crusading geek of an accountant stepped forward with the solution (and bravo to you, sir.)

In fact, we know that most people won’t work on anything at all. This is the nut of the 90-9-1 rule which seems to hold across many different types of online social activities: 90% of people are just audience, 9% are involved in some way, and only 1% are the real creators. The numbers vary somewhat from platform to platform, but the upshot is always that only a very small number of people do the core work. There’s no reason to believe that offline social initiatives — everything from campaigning for gay rights to  getting potholes repaired – would typically attract a much broader base of core agents. And this is fine. This is normal, and nothing to be upset about. One percent of a population is still a huge pool of talent and labor.

In this framework, the purpose of journalism is to deliver each story to the right 1%, at the point when they need it. Saying that most people won’t read a story about Madagascar doesn’t get us very far; that’s expected, and that’s the only way it can be in an era of spectacular information overload.

I don’t know whose job it is to facilitate the creation and nurturing of communities around issues; maybe it’s fine to let Facebook have this role. What I do know is that journalism needs to concern itself explicitly with figuring out who its audience is — for each story, down to the level of individual people and groups. Where in the world are those 1% who have something to say or do about the coup in Madagascar, and how do we connect to them, and connect them to each other?

This probably requires that journalists listen a lot better to what’s already happening out there on the tubes. As Felix Salmon of Reuters puts it, we need to teach journalists to read.

Journalism designed for agency

I come from a culture of makers; my personal life and my chosen communities are steeped in the ethic of hackers, entrepreneurs, crafters and artists. All of these activities are the expression of a participatory instinct, an instinct that extends to trying to shape the society we live in. I want information systems that facilitate self-selected civic action and social entrepreneurship in all its forms. If this is the goal, then journalism needs a better understanding of how its products could actually be used to effect change.

News can no longer be (only) about the mass update. Stories need to be targeted to those who might be able to improve the situation. And journalism’s products — which are more than its stories — must be designed to facilitate this.

News needs to be built to engage curiosity about the world and the problems in it — and their solutions. People need to get lost in the news like they now get lost in Wikipedia and Facebook. There must be comprehensive stories that get the interested but uninformed up to speed quickly. Search and navigation must be improved to the point where satisfaction of curiosity is so easy it becomes a reflex. Destination news sites need to be more extensively hyperlinked than almost anything else (and not just insincere internal links for SEO, but links that are actually useful for the user.) The news experience needs to become intensely personal. It must be easy for users to find and follow exactly their interests, no matter how arcane. Journalists need to get proficient at finding and engaging the audience for each story.

And all of this has to work across all modes of delivery, so it’s always with us. Marketers understand this; it’s amazing to me that the news industry has been so slow to catch on to multi-modal engagement.

It sounds like a tall order, but there’s nothing here that requires exotic technology. Just real product design, in pursuit of concrete journalistic goals.

(thanks to @chanders for feedback on an early draft of this post)

Here’s a list of things that professional journalists do:

• decide what should be more broadly known
• decide what should be more deeply investigated
• collect information from sources both public and private
• check that information for factual accuracy
• construct narratives to make sense of that information
• produce content to convey those narratives
• publish and market that content

This list is by no means definitive or exhaustive. It’s just illustrative, a starting point for a thought experiment. Who could do each of these things best? And what tools to do they need to do it?

Having a network of people producing journalism around a newsroom is not a new idea. Jeff Jarvis has been discussing networked journalism since at least 2006, and naturally I think he’s on to something. In this essay I want concentrate on process and roles. If cheap networks make new types of collaboration possible, they also set the stage for new types of specialization. I think one of the problems of the traditional, mainstream media newsroom is that it it tries to handle the entire journalistic process internally, even the parts that it’s not actually very good at.

An example

On November 25, a video appeared on YouTube which appears to be the testimonial of a young woman recently fired from the credit card collections division of Bank of America. She had been allowing the bank’s most desperate customers to enroll in fixed-payment debt recovery schemes. Many of these customers are currently paying 30% interest as a result of recent rate hikes, so this was a great kindness. It was also against company policy.

The video is powerful. It’s an amazing first-person testimonial of the greed and heartlessness of large corporations.

So is this journalism?

Not quite yet, according to the traditional definitions. It’s a powerful story only if it’s true — and even if it’s true it may not be the whole story. I haven’t checked the video out myself, such as by trying to contact this woman, or calling Bank of America to see if anyone by that name ever worked there. And even if her description of events is accurate, I wonder what fraction of deeply indebted Bank of America credit card customers are denied access to debt relief programs, and on exactly what grounds. What about at other banks? What could be done that isn’t being done to address this situation, if it is a situation, and by whom?

A professional journalist on the story would answer these questions. They would make the calls, keep a notebook of what they found, select the most relevant points, and publish a full account of what they discovered as quickly as possible, or at least as full as time and word limits permitted. Good stuff nonetheless. And here the conversation with most pro journalists ends. “If paid professionals don’t do this, who will?” A pro might say that this video isn’t journalism at all.

The obvious problem with the “this isn’t journalism” line of thought it that denies the value of what this woman has done. She may not be a “journalist”, but she’s certainly participating in a journalistic process.

Focus on the economics

I’m going to approach the question of how journalism “should” be done by asking how to produce it as cheaply as possible.

In general I’m not convinced that efficiency is the goal. Markets fail too often for that. But journalism is currently very inefficiently produced in a rather obvious way: lots of reporters covering the same story. Even if only one reporter covered any given story, I still suspect that a traditional newsroom would be wildly inefficient, compared to what is possible in the age of the internet.

If solid journalism in the public interest is cheap to produce, we as a civilization can afford a lot of it. And I think we do want a lot of it. Transparency from those in power is important (and this applies both to governments and corporations) but it will always be possible to adhere to the letter of the disclosure law, rather than the spirit. Keeping track of what the hell is actually going on is always going to be a required function in a free society.

(I write this from corruption-laden Indonesia, where it’s alarmingly difficult to keep tabs on the powers that be.)

My suspicion is that the same shift that has destroyed the traditional, publishing monopoly-based business model for news organizations can also drop the cost of production quite dramatically. How? Cheap communication networks and specialization — the same factors that have been increasing productivity in all fields for the last several centuries. Let’s look at who might perform each of the journalistic functions I listed above.

Decide what should be more broadly known

This was previously the role of the editor in collaboration with the reporter. The editor assigns stories, or the reporter comes up with them. The decision of what to report on relies on “news judgement”, which has been described to me by one old wire-service hack as “tribal.” Different publications have different ideas about what counts as news. And this is great — Wired should cover different stories than the Wall Street Journal.

The problem is that the news media are not very good at consulting their audiences when developing news judgement. Letters to the editor, unsolicited tips to reporters, and the occasional marketing survey are all very narrow and unreliable back-channels.

This is not about tailoring a product to audience demand. It’s about service to the community, which news organizations need to engage with every tool at their disposal. It’s also about trust and authority, which works differently in the social media era. When Wikipedia goes through convulsions of public discussion about every major editorial change, will audiences really trust organizations that claim to decide the public interest behind (mostly) closed doors?

The possibilities for improvement are many, but there are blatantly obvious and simple things that could be done immediately. Facebook and other social media sites have well-publicized and lively forums where users debate the future of the product. News organizations do not.

Decide what should be more deeply investigated

Reporters are tasked with coming up with interesting questions, then getting them answered. The seasoned reporters I’ve had the pleasure of engaging are typically very knowledgeable folks, very bright and very widely read (or the equivalent contemporary expression — widely surfed?) But they are not experts in every field. They will always lack the context and detailed knowledge that allows them to perceive certain key questions. Similarly, who gets to decide whether a story is followed up or dropped?

Spot.us is an interesting attempt to let the audience vote with its wallet. Readers submit ideas for investigative stories, which freelance journalists then write pitches for, describing the work they propose to undertake asking for a specific amount of money. Other readers donate money to those pitches they want to see executed. The site has so far mostly been used for political issues in California.

HelpMeInvestigate.com takes the process a step further. Readers submit questions — a current example is “How much of the rent charged for University of Birmingham halls is actually spent directly on related costs?” — and then readers work together to answer them. The site acts as a clearinghouse for facts uncovered so far.

Professional reporters need to be embedded in systems like this. While it is true that many of the questions that people ask are going to be uninteresting — or easily answered from existing sources — not every idea is going to be bad. There’s been lots of talk and many products designed to help organizations track and manage their collective knowledge. For journalism, such systems need to extend outside the newsroom into the audience.

Collect information from sources both public and private

This is where traditional journalists are both strongest and weakest. A huge part of the value of a career reporter is the network of contacts and sources they build up over a lifetime. Simply put, it’s their job to cultivate relationships with knowledgeable and powerful people. This means the pro reporter has an irreplaceable investigative role to perform.

But the professional journalist is hardly the only person who can publish otherwise unavailable information. This is precisely the role that the woman in the video was playing: we didn’t have any reports of what had happened at Bank of America, now we do.

The fact that her report is unconfirmed does not mean that it is not valuable. Information of all grades is valuable.

On the internet, filtering comes after publishing, as Clay Shirky has noted. This concept is an inversion of traditional journalistic practice, but it is necessary because the journalist cannot be the filter for the entire web. Filtering must be collaborative to scale. Remember that there is no such thing as “automated” filtering: Google Search results may be returned by an algorithm, but that algorithm uses the manually placed links on the web to determine what content is relevant.

Filtering is also the key to finding facts in thick documents, of which there are now many. That buried government report is critical, especially as governments practice increasing data transparency. A blog post saying “hey, page 283 of this document is interesting” may not seem like a story to a reporter. Another blog post referring back to the first one even less so. In fact, both posts are extrmely valuable, because they are filtering mechanisms.

Publish first, ask questions later is the rule of the web. This applies to journalists too: facts or reports that aren’t immediately usable in a story should be considered for rough-and-ready publication anyway, such as through micro-blogging — or, better, by making the newsrooms files open wherever possible. This not only increases transparency, it allows users to build on the reporter’s work.

Check information for factual accuracy

Some facts can only be checked by making calls to highly-placed sources. Other facts can be confirmed by anyone with a telephone. And a great many others can be confirmed online. Only the first category of facts can only be checked by a career journalist.

Bloggers draw attention to unconfirmed reports or documents all the time with remarks like, “this is interesting. Can anyone confirm it?” I’d like to see reporters distributing more of their workload in this way, especially for material that isn’t immediately needed for a deadline.

Asking for help is one way that the work gets to the people who can do it best.

Saying that something needs to be fact checked is almost as valuable as checking it. It draws attention. It puts a pencilled-in question mark above an item that everyone else can see. Wikipedia, the greatest collaborative fact checking system of all time, recognizes this point with its famous “citation needed” tag.

We are entering the era of transparency in fact checking. It’s no longer enough to be right; the audience has to be able to understand why you are right. Compare the links and footnotes on the Wikipedia and New York Times articles for “Global Warming.” Meanwhile, Associated Press stories are still entirely plain text — no reference links at all.

Construct narratives to make sense of information

Everybody in the world wants to be part of narrative construction, if the number of active blogs is any indication. Career journalists have an advantage in that they are (hopefully) intimately familiar with the facts and history of a particular topic. But if smart readers are given deep access to those same facts (transparency!) I don’t see why the reporter’s narrative/interpretation is going to be any better than anyone else’s.

The days when a single publishing organization can set the agenda are over, though God knows Fox tries. Audience members who are broadly connected to each other in real time for free will negotiate the narrative among themselves, thank you very much. Taking part in this negotiation is not something a newsroom can charge for.

Refereeing the negotiations, providing the forum, or filtering the conversation might add value, if done properly. This is different than yelling your own point of view, however nobly constructed.

Of course, opinion columnists are popular. But they don’t seem to be profitable, or at least something that can be charged for. Both the New York Times and The Economist put their opinion outside their paywalls.

Produce content to convey those narratives

Demand Media Inc. pays an average of US $15 for an article of a few hundred words, and $20 for an original short video clip. They currently produce thousands of items each month, and aim to be producing a million items per month by next year. Article topics are assigned automatically by an algorithm that computes the expected search-engine ad value of all future hits to that page.

We are witnessing the beginning of industrial content production. Yes, it’s cheap, poor-quality stuff. That doesn’t matter, in exactly the same way that most of us now wear mass-manufactured clothing. My mass-produced clothes are actually pretty good these days, and in just this way the quality of industrially produced content is going to come up as producers figure out the efficiency issues.

High-quality, artisanal content — sparkling writing, slick video production — will always have higher value, but the market for it is in the process of collapsing. Polished is good, but is it necessary to the journalistic mission? As one blogger put it, “it’s casual Friday on the web.”

Publish and market content

It’s no longer useful to think of a news organizations as publishers, because their irreplaceable role has nothing to do with making information available to the public; that’s just a necessary sub-product, handled mostly by the telecommunications industry.

Marketing is the more interesting role. If a story has no value if it has no impact, then people need to know about the hot stories. Obviously social media offers unique opportunities here, and most online news sites have a decent array of “share this” buttons below each item.

This is a start. It is not enough. The possibility for personalized news is huge. I have yet to see a Facebook application that delivers me useful news and social recommendations for news. Both The Huffington Post’s Social News and The New York Times’ TimesPeople applications seem to be DOA due to a combination of technical and marketing mistakes, and besides they only filter stories on their own sites — which strikes me as incredibly presumptuous. TwitterTime.es, which aggregates the links that my friends have tweeted from all sources, is a lot closer to what I have in mind.

Social Journalism

I imagine a system where the traditional journalistic functions of a newsroom are distributed throughout a community consisting of newsroom staff and audience. The audience is going to be unpaid, let’s assume. (Paying your audience doesn’t seem like a sustainable business model.)

The audience can report information. The audience can check information. The audience sometimes even creates good content. Above all, the audience filters information for each other and for the journalist — not just the smattering of stories produced in the newsroom, but every story and piece of a story that they can find online or in their lives. And every scrap of information that the newsroom can give them access to, in its files and archives.

Salience is mostly decided by audience, not editor. I find it quite startling that when web audiences assemble their own news using sites such as Digg and Reddit, their is very little overlap with mainstream media. It’s even more surprising to me that in a recent ten-country survey, not even journalists ranked stories in agreement with the prominence they were given in the media. The current news agenda-generating process serves no one at all, apparently. As we say on the internet, ur doin it rong.

Instead, the editor needs to ask the audience to filter the ongoing discussion around the best use of society’s very scarce and expensive reporting resources.

The audience performs all of these roles — surfacing facts and checking them, filtering, setting agendas — because it’s in their interest, and because the newsroom makes it really easy for them. This requires software.

Social Software

Journalism needs its own killer social media application to organize all of this, and it hasn’t been invented yet. Social software is architecture and environment: different types of software are conducive to different types of behavior in its users. The resulting social system is the combination of the software and the community that is nurtured on top of it.

Social systems can produce things. Flickr produces a tagged database of photographs. YouTube produces videos. Facebook has produced the personal information of 350,000,000 people. Twitter produces global, real-time conversation. Wikipedia produces the most extensive encyclopedia in history, and makes it available free.

Wikipedia is worth examining very closely, because it may be the closest live example of how social journalism software could work. Users create topics, then edit them collaboratively. Less well known are the conflict-resolution procedures that Wikipedians use to resolve editorial disputes. These progress from discussion on each article’s talk page to informal mediation to binding arbitration. All of these activities are organized and staffed by volunteers.

The Wikimedia Foundation, of which Wikipedia is the flagship product, does not edit content or (usually) engage in dispute resolution. They provide the infrastructure. This is both the software itself, and the rules the govern the community. For example, Wikimedia is ultimately charged with maintaining the editorial policies that they believe will produce the best possible product. These policies are set in consultation with the user community, of course.

Wikimedia employs about 30 full-time staff. This represents an astounding amplification of effort and money. Imagine what an engaged community of readers could do around a single professional newsroom. For maximum amplification, the journalists in the newsroom must perform only those functions that no one else can do.

To accomplish this, the software has to be designed so that it can be used by a community to produce news without any professional journalists. A neighborhood, an organization, or a town should be able to use a social journalism software tool to track and inform itself, supplying any combination of paid and volunteer labor that it deems appropriate. Only when newsrooms give as much capability as possible to their audiences will they understand where the crucial gaps lie that professional journalists must fill.

When won’t this work?

There are many cases where the work of journalism is still going to look a lot like it always has.

First there is the issue of access. Stories that are primarily about the actions of elites who restrict access will need to be covered by accredited professionals — such as the White House press pool. Similarly, professional journalists are often allowed across police lines, into conferences, and in other restricted situations. Accreditation is necessary in such circumstances, and the current system of career journalists working for recognized institutions is probably a reasonable way of deciding how to apportion limited access.

Reporting across language and cultural barriers or from dangerous places will also require professionals. I spoke last week with a reporter who covered the massacres in the Philippines for the BBC. This required contacting authorities in another country, working with translators, and most of all having worked in Asia long enough to have the address book and understand how to operate. It’s not the sort of story that a community of interested followers can generate from the other side of the world. For this reason, I suspect that professional foreign correspondents are likely to be irreplaceable for some time.

Finally, the whole notion of networked journalism rests on the availability of the network. Here in Indonesia, internet access is still slow, expensive, and not widely used. That makes traditional centralized journalism both necessary and profitable, for the time being.

And there are doubtless other cases. Again, the fundamental shift that needs to take place is for professional journalists to try to do only those things that absolutely require their services.

So what does the newsroom do?

I would like to see the newsroom at the center of a system of social news production. The newsroom provides experienced journalists who have fat address books and access to elites. The newsroom designs, produces, and evolves a specialized social media application that allows its audience to self-organize to perform all of the functions that do not strictly require newsroom staff.

Jay Rosen refers to newsrooms as “closed” journalism and blogging as “open” journalism, and sees them as complementary processes that produce different things. I think the lines need to be a little more blurred.

It is true that newsrooms sometimes need to protect sources and keep certain facts private until the broader story is clear — or sometimes keep things secret forever. This is part of the game of getting people to talk. But whenever something does not have to be secret, it should be public. As much as possible, the newsroom should not have access to bigger files or better tools than the audience. Rather than the simple “inside” and “outside” that exists today, I imagine a journalistic source/article/fact/notes tracking system that has fine-grained privacy controls.

Using this shared system, journalists make their data, notes, and tools public whenever possible, so that the audience can help them. The newsroom now owns a sophisticated information tracking and filtering system which acts as a focal point for the aggregation of journalistically interesting material. The audience provides reports and facts. The audience checks facts. The audience interprets facts. Sometimes the audience creates content. The audience provides expert guidance. The audience assembles itself into communities around an issue, identity or topic. The audience constructs narratives and decides on its questions and its goals.

The editors no longer get to decide what goes on the front page. This not only matches the reality of how people consume information online, it’s implied by personalization. Besides, editorial curation of content won’t scale. A journalistic system has to be designed so that the audience — each of many audiences served by the same huge steam of content — can bring the most relevant content to the front. The audience filters, and from this filtering the newsroom also learns what is important to the audience.

I believe that the correct goal of the editor is not the production of stories but the management of the ecosystem of journalistic production, including reporters, community, software, files, and processes.

No doubt I am wrong in the details of how all of this has to work. No doubt there are ways in which this scenario is too optimistic. But I’ve yet to hear of anyone seriously attempting it.

The edit network for “telephone tapping” shows a bipartite structure, indicating that the topic is controversial (image from Brandes et al.)

An interesting new paper defines the “edit network” of a Wikipedia article by drawing edges to indicate that one person has deleted or restored text written by another. While it’s always fun to look at pictures, the surprise here is that we can verify that the resulting graph structure really does tell us something useful about the article. In this study, articles with a more “bipolar” edit network — meaning that the authors split into basically two camps who routinely undid each other’s edits — were also much more likely to appear on a manually-maintained list of controversial pages.

Although there has been previous work on network mapping of Wikipedia in particular (and of course volumes of work on social networks in general) I find this paper interesting because it tries very carefully to understand whether the pictures mean anything. Like all science, what you find depends on where you look, and the practitioner of network analysis has an absurd amount of freedom to define what a “node” is, what an “edge” is, and how the resulting graph is visually laid out (since the point of a map is a visual representation, it’s very important that graphical properties such as distance, size, color, etc. have the right sort of metaphorical relationships to the more abstract properties we are trying to understand.)  

For example, one might try to identify people who hold similar opinions by analyzing who “interacts” with whom and looking for clusters. In this case the nodes are people, and the edges are “interactions.” But what is an “interaction?” A hallway conversation? A co-authored paper?  An appearance on the same talk show? If we draw an edge between two people if they’ve ever stood in the same room, we won’t necessarily get a good map of who “agrees” with whom. It is therefore very important to make sure that your definition of an “edge” properly embodies the question you are trying to ask — in this case a question about “similar opinions.”  (The most cogent critique of my COIN Policy Author Graph made exactly this point.)

This is a real and serious methodological problem in network analysis, and it’s made worse by our preconceptions. Suppose we suspect that republicans and democrats read different books. We might look for a definition of “edge” that we can apply to sales or reading data, and choose the one that gives the cleanest separation of people into two distinct groups. This makes pretty pictures, but it’s not clear that we can learn anything from such an exercise: all we’ve done is thrown out all the evidence that didn’t prove the notion we had already decided upon.

Back to Wikipedia: the paper is titled “Network Analysis of Collaboration Structure in Wikipedia,” and was written by Ulrik Brandes, Patrick Kenis, Jürgen Lerner and Denise van Raaij, who wanted to know if the structure of the edit network for a particular article could tell us something about where that topic fits into a broader discourse. In particular, they decided to see if they could produce a numerical measure of an article’s “controversiality” by measuring how close the graph is to being bipartite, that is, whether the authors tended to split into two camps, each of whom routinely deleted words written by the other. It is important to note that both the text processing that mines the data and the selection criteria which define an edge are very complex — meaning that other criteria which may not give “good results” are effectively excluded from this study.  For this reason, it’s very important to have something to compare their graph metrics against, and they do: the manually-maintained list of controversial pages.

To test whether high values of the bipolarity indicator  point to controversy in authors’ opinions, we computed the bipolarity of articles linked from the page Wikipedia:List of controversial issues, with our hypothesis being that bipolarity is high on those controversial articles and lower on non-controversial ones. … To compare controversial articles with non-controversial ones that did receive enough attention, we have chosen so-called featured articles which are listed on the page Wikipedia:Featured articles. 

…

The bipolarity index of controversial articles is [statistically] significantly higher than the bipolarity of featured articles. Thus, the controversy of topics is indeed reflected in the edit behavior on the associated Wikipedia article. 

Do you edit your subject of expertise, or the one you wish you were an expert on? Is Wikipedia so shallow it makes you cringe, or so technical that only experts can read it? Is editing an impulse for you, that fanboy urge to correct, or do you plan your essays? Do you learn by editing? Do you know what you’d like to learn?

Who is writing our collective present? Are they white and middle-class like me, geeks with too much education and even more bandwidth? Are they high-school kids in small towns with no one else to feed their minds? Are there any women on the internet? When will Wikipedia be on StuffWhitePeopleLike? What does the Arabic version talk about? Will British or American spellings prevail?

Wikipedia is a tool of the conservatives; wikipedia has a liberal bias. Food irradiation is perfectly safe, and you’re violating NPOV. Who made you an administrator, anyway? So, revert revert revert! Biodynamics is not psuedo-science, it’s anthroposophy. Why can’t the entry for The Beatles just explain that they were way better than Elvis?

If you wrote your own book, what would it be on? Will Google Books ever let us read every page? Will our parents ever be comfortable with instant messaging? Do you believe that touch is more important than words? Do we all need to get out more?

Who was that one person in your life who was once patient enough to give you their best answer, over and over, to your plaintive “why?”