Internet as information democracy, or new media news monopolies?

There was a dream that the internet would mean the end of the media gatekeeper; that anyone could get their message out without having to get the attention and approval of the media powers that be. This turns out to be not quite the case.

I took data from the Project form Excellence in Journalism’s State of the News Media 2010 report to create this chart showing the market share of the top 20 news web sites. In theory, the internet busts media monopolies by allowing anyone to publish for free. And there’s no doubt it’s been disruptive. But according to data from Nielsen, the top 7% of 4600 news and information sites get 80% of traffic (from American viewers.) We see a big concentration of power, as the rapid falloff in the chart above shows, and much of it still belongs to “old media.”

Organizations such as CNN, Fox, the New York Times and USA Today rank in the top 20. But so do new media giants AOL, Google News, The Huffington Post and Yahoo.com, which is the biggest news site of all.

(It’s also interesting to note that many of the top 20 new media news sites produce little or none of their own news; in the extreme case Google News produces no stories at all of its own. While some see aggregation as parasitic, I think it’s obvious that it delivers a tremendously valuable service to readers.)

For better or worse, the ability to publish anything nearly for free hasn’t meant the end of big media monopolies. It’s simply shifted the landscape and the power balance.

The limiting factor to getting your message out is no longer having access to an expensive printing press or a TV station. It’s attention: how many minutes of time can you get from how many people? In this game, brand still matters hugely. There are only so many URLs a person can remember, only so many sites they can check in a day.

You have an audience, or you don’t. Mindshare is now the barrier to entry in the media world. Perhaps it always was, though I daresay it was easier to get viewers to check out your new television network when there were only 13 channels. Online, the number of channels is infinite for all intents and purposes; a single person will never exhaust them all.

Which is not to say that the internet has changed nothing. We have seen over and over that bottom-up effects can propel something to mass attention, with no big company behind them. This is often called “going viral,” but that’s not quite a broad enough description of the effect. In many cases, what happens is that something becomes just popular enough to get picked up by mainstream media, who then propel it into the spotlight.

And what this PEJ top 20 list doesn’t take into account is that people now get online news from lots and lots of sources other than news websites.

Facebook is now the most widely used news reading program. It’s also now the #1 site on the internet. Should it top this chart of news sources? Meanwhile, Twitter has become a primary news source for very many people. And then there are mobile news apps, some of which belong to old media news organizations and some of which don’t. The richness of news distribution systems today is well captured in another PEJ report on the “participatory news consumer.”

So has the internet made it easier to get non-mainstream messages out? I think the answer can only be yes. But don’t expect that anyone will be reading your alternative narratives just because you’ve put them online. Your best bet to to be heard still lies with a small number of very large companies. And although the internet per se is relatively uncensored in many countries, commercial gatekeepers like Apple and Facebook own important dedicated channels, and both of them engage in censorship (1, 2).

Don’t throw that out! Editing like it’s paper destroys journalistic value

CementVegetables

The village of Kangzhuang, in Henan Province, China, was built in 2006 next to the Tianrui cement factory (in background, above), to house villagers relocated after the government bought their land. A huge grey cloud of dust hangs permanently over the village. The villagers told me that the factory is shut down for a day or two whenever air quality authorities come to visit, then started up again as soon as they leave.

I visited Kangzhuang on another story. But eating dinner that night, covered in a film of dust, I suggested to a Chinese colleague that this issue of cheated environmental regulations deserved to be investigated further.

“That’s not news,” she said. “That happens all over China, every day.”

And she’s right: it’s not news. All chronic problems fade from attention, and old news won’t interest (most) readers, or sell papers. But we’re leaving the paper era, and this changes things. In the web era, documenting old, marginal, or incomplete stories is much more valuable — and much more affordable — than it used to be.

There has been much discussion of how web writing style must differ from paper writing style, but the difference in medium also changes what stories can and should be covered. Despite its physicality, paper is a deeply transient medium. Yesterday’s newspaper lines birdcages, but yesterday’s web stories will be showing up on Google five years from now. An editor selecting stories needs to be thinking about not only tomorrow’s page views but next year’s as well, and also, crucially, how the story will function in combination with stories from other outlets. There are close ties here to the concept of stock and flow in journalism, and the new-media notions of topic pages and context.

I ran into this paper-web cultural divide discussing a pitch for a story about the number of civilian casualties in the Iraq war. I had done a little bit of new reporting on the topic, but mostly I just wanted to write a thorough summary of the various estimates and a careful, clear analysis of how reliable each might be. An editor friend said that, since the studies I would be referencing were several years old, this wasn’t news and would be hard to sell to the outlet I was proposing; I said that I was planning to write an authoratative article on a topic of great international concern for a general audience, a piece which doesn’t really exist yet. I felt that this would be exactly the sort of public service that journalism is supposed to be, and further, we’d have a good shot at getting in the top five results on Google.

We clearly had somewhat different conceptions of what stories count as publishable journalism, which seemed to be derived from our focus on different media.

Not only is the web a permanent medium, it’s a distributed, accessible-from-anywhere medium (unless your government doesn’t want you to know about certain things, but there are ways around that.) A single report of environmental cheating at a cement factory is not going to change anything for the people who will get sick from cement dust, but hundreds of such reports all over China might. If editors, used to selling papers, think of stories as throwaway consumables rather than careful additions to a permanent store, they miss opportunities for collective action.

But how to fund such long-term, speculative projects? After all, every story costs money to produce. I think part of the answer lies in another medium-driven difference: the web is more amenable to journalism of different levels of quality and completeness. The New York Times aims to be “the paper of record,” which means it hopes always to tell the full story, and to never get a fact wrong. On the web, this is ridiculously inefficient. As a social medium, the web draws power from collaboration and conversation — say, between different papers in different places — and that process is severely hindered if only “finished” work makes it online.

Yes, there will always be a need for the authoritative voice and the carefully edited, sub-edited, copy-edited and fact-checked article. But what about all that other good stuff that journalists produce in the course of their work? What about the juicy trimmings that had to be cut from the main story, the tantalizing leads that the reporter hasn’t had time to follow up, and the small incidents that have meaning only in aggregate? The web demands that we put more online than we would publish on paper, and provides a place for information of all grades. In this new medium, amateur journalists (such as bloggers and thoughtful commenters) are often much more adept at creating value from information by-products than their professional peers. News organizations will have to find forms for publishing unpolished information, such as the beat blog.

The report of environmental malfeasance in Kangzhuang is not yet a story. It hasn’t been checked against other sources, we haven’t heard from the local government ministry about whether they did a proper environmental study prior to putting 2,300 people next to a factory, and anyway the journalists who visited the town (myself among them) are currently busy writing up an entirely different story. But this tidbit deserves to be aired in a public place, so that others can build on it in the future. This is a valuable public service that could be provided for low cost, a service that is possible only on the web. Will an industry trained in the paper era see this possibility?

My suggestions for Dowser.org, who aim to report on global solutions

Dowser logo

Dowser.org is a new online news site that aims to cover solutions to social problems. It’s journalism, but focussed on what we are learning about what can be done.

A media theorist might say this is reporting from a “solution” frame. Whatever you call it, I support the idea.

They are actively soliciting feedback on their idea. This was mine:

Great idea.

I think solid information on who is solving which problems is really important. I have three broad suggestions.

1) Please consider including a set of “topic pages” as well as running news stories. Social problems are often complex and contextual, and we badly need continuously-updated primer articles that bring the newly curious up to speed on specific problems.

2) If you guys have any money at all, you should solicit contributions from freelance journalists for coverage and analysis of interesting global projects.

3) consider publishing in more than one language, or allowing hybrid computer-human translations using, e.g. the free World Wide Lexicon system.

I’m a freelance journalist and computer scientist living in Hong Kong, who specializes in applications of technology to social problems. @jonathanstray on twitter.

Would love to keep in touch.

There’s a great deal more I could say on why I made these specific recommendations. For now, I’ll leave it to the hyperlinks. For additional ideas on how to offer good translations of every page for near-free, see also Yeeyan.com, a successful volunteer news translation community with 100,000 users.

(hat tip: @NiemanLab for bringing them to my attention.)

Countries Seen Through Comments

The comments on the news are more revealing of a culture than the news itself. Journalism too often has a commitment to a sort of sanitized neutrality, and certainly tries for clarity, smoothing away complex disagreements. That has its uses, but the comments are a much messier, more divided, more personal look at a culture. They have the texture of life at street level.

First, America. On January 30 New York Times ran an article headlined “U.S. Suspends Haitian Airlift in Cost Dispute” which described how the US had stopped medical evacuations to Miami because of a state vs. federal arguments over who would pay for their medical care. The comments reveal a deeply divided country:

The richest country in the world bickering about who is going to pay before it treats patients who need critical care from the poorest country in the Western hemisphere! We Americans should be very proud of ourselves!

Another wave of third world, uneducated people of an alien culture is about to hit our shores, helped this time by the Obama administration’s desire to show compassion. Unfortunately, the tax-paying citizens of this country will have to pay, in more ways than one.

Meanwhile, Nigerians are talking about the Underwear Bomber. Global Voices has helpfully collected some of the blogger reactions. For America, the story was about fear and terrorism and security. For Nigeria, it was about reputation and identity:

Be honest, when you heard a Nigerian man tried to commit a terrorist act in America, how many of you immediately thought ‘Please don’t let him be [insert your ethnic group]?

There’s an Igbo proverb that says, “If one finger touches palm oil, it spreads to all the other fingers.” This is indicative of how Nigerians the world over felt when they heard the news of a young man who attempted to detonate a bomb on U.S. soil in the name of Al Qaeda. Many of us worried that the actions of this one finger would spread to cover the entire 150 million of us.

How does disowning him help Nigerians understand what role extreme Islamic ideology played in causing him to attempt detonating an explosive device on board a US-bound airliner? How does it help Nigerians understand the complex interplay of religious faith, access to extremist religious groups and ideological brainwashing?

Meanwile, the ever-wonderful ChinaSMACK took a break from pop-culture scandal (on the site right now: Hong Kong Girl Shows Off C Cup Breasts To Ex-Boyfriend) to translate online Chinese reactions to news of US sales of arms to Taiwan. And the Chinese, often portrayed as uniformly nationalist, are just as diverse and divided as any other country:

We don’t need to fear America selling arms to Taiwan, as soon as a war started these advanced weapons would be quickly consumed by our lower-quality but numerous weapons and many soldiers.

I only know that without America, the whole world would be chaotic.

As long as America exists, the world cannot be peaceful.

What I want to know is, where are the foreign voices in these conversations? Right now each culture is talking about the others like they’re not in the room. And they’re right. Our global conversation is fragmented and unmapped. It’s not a small world after all.

Comments on the New York Times’ Comments System

Here are some problems I see with the implementation of the commenting system on the New York Times web site. Assuming that they want the discussions about their content to be taking place on their site. The way things stand now, I suspect that they’re actively sending many readers to Facebook instead of keeping them on nytimes.com.

  • How come some articles allow comment and others don’t? Is the policy of which articles get comments explained anywhere? Arriving at Times content from a link, I’m confused about whether I can expect a good discussion or just broadcast.
  • Even for articles that we are allowed to comment on, the comments are hidden. Sometimes there’s a pull-out quote (which is cool!) but more often we see only this:

    NYTimes hidden comments

  • The number of comments on each article is not visible from the front page or the section pages. There’s no way for readers to see, at a glance, which discussions are hot.
  • The “recommend” button on each comment is welcome, and serves as a useful way to filter comments. The “highlight” button which seems to appear on the more recommended comments is a little more obscure — does it just put the comment in the “highlight” list, or is there editor moderation involved? The “what’s this” tip doesn’t clear this up (click image below for larger)

    Highlight what

  • There is no comment view that is sorted for both relevance and freshness, which is the most useful way to track a discussion. Digg and others get this right by adding a time-weighting to a comment’s position in the list.
  • There is no way to reply to someone else’s comment. This makes it impossible to have a real discussion on the site. Many other commenting systems organize comments into threads. By not supporting this, the Times is saying that we can talk to them, but not to each other.
  • The comments on an article close after a certain point. Although I can see that this might be due to moderator workload issues, it’s also a way to drive away future traffic — as when that link goes viral a week later, or the discussion lasts for months.
  • Speaking of which, the comments are moderated. This topic’s a little more complex: there are advantages and disadvantages to this. But I’d like to note that there are plenty of civil discussions happening on the internet in unmoderated places. The strategy of putting a “flag this” link on each comment that sends it for human review is relevant here.
  • From the comments FAQ: “We appreciate it when readers and people quoted in articles or blog posts point out errors of fact or emphasis and will investigate all assertions. But these suggestions should be sent by e-mail.” Really? Why? Wouldn’t claims of possible errors of fact be the among the first things you’d want readers to see? Your comments are moderated anyway. And I’d like to point out that your own journalists and editors read the site too — you need those corrections up quickly for internal communication (see: the hilarious Washington Post vs. Public Enemy “911” correction saga.)

In short, the comment system seems to be to have been designed by someone who has never responded to a message that says “there’s a great discussion about that going on at [link].”

Comments?

Newsroom vs. Web Culture Clash

“No one cares what you think,” said my reporting instructor at the beginning of term. He repeated it slowly. “No one cares what you think. That’s not your job.” I had to think about this for a while, but as I got the hang of writing classic wire service copy, I began to see the logic. Reporter as conduit. Objectivity, whatever that tricky word means. So in my story about Google Goggles, my editor obliged me to kill a paragraph stating something I knew to be true.

“Even if you know it’s true, someone else has to say it,” she said. “Your story mixes news and editorial.”

I’m not saying she made the wrong call. She’s a very nice woman and she knows a lot more about editing a newspaper than I do. I’m saying that after four days in the Jakarta Globe, I’m beginning to see how deep traditional newsroom culture runs. And I think it’s probably not the right culture for web media.

Classic newsroom values: don’t use material that your competitors came up with, and don’t speak with your own voice. Emerging web values: linking to other people’s work is honest, efficient, and valuable, and each of us is expected to add what we know to the conversation. No wonder there’s been a war brewing between “old” and “new” media.

Another example: yesterday Twitter.com was briefly hacked to point to a screen which claimed that the “Iranian Cyber Army” was behind the attack. In my self-appointed role as the interim technology editor on the Jakarta Globe web team, I felt that we should cover it. Our online readers (and Facebook and Twitter followers) are big into social media and they’d find it interesting. Our options were:

  1. Write an original story on the topic, perhaps making some phone calls to get original quotes.
  2. Run a wire service story on the topic (AP and AFP and probably others had already reported the event)
  3. Reblog or otherwise link to a report elsewhere.

Writing an original story is really time consuming, and duplicates effort — something that an industry facing deep revenue shortages from the loss of their advertising monopoly cannot afford. Web media thinking says 3 is the right answer: find out who has the best coverage and send readers there. But that violates classic newsy values, which say that you should avoid acknowledging the existence of any other publications if at all possible. We went with running the AFP story.

Again, I’m not saying that this was the wrong move, but it does illustrate a classic newsroom practice that doesn’t translate well to the web.

Non-journalists are probably not familiar with newswire services. These are organizations like the Associated Press (AP), Reuters, Bloomberg, Agence France-Presse (AFP) and so on that maintain huge international staffs of reporters spitting out stories as fast as possible. It’s really quite amazing to sit at a terminal and watch the news roll in over the wires, a new story every minute or so from all over the world. Newspapers subscribe to these feeds for thousands of dollars a month, both for the information and for the license to rerun stories.

It’s actually an efficient scheme — how many local papers can afford a Botswana correspondent? Running a wire story is sort of like reblogging for the old media world.

Where this scheme falls down is rewrites. “We spend a huge amount of time chasing wire stories and rewriting them so we can put our own name on it,” a reporter for the International Herald Tribune in Hong Kong told me earlier this month. “It’s stupid.”

And this is part of why wire service stories are dry as toast: the newswire company promises anonymous prose that is easy to rewrite and rebrand. Not only is this a waste of time, but the voicelessness of wire copy is a big problem for the web.

First of all, wire stories are plain text. No links at all. This is not only inconvenient for the user who wants to know more, it’s a transparency problem. The reader can’t tell what the reporter’s references are, or where they are coming from. There is no way to link this story with others by the same person, to get a sense of the author’s experience, knowledge base, and point of view. That is the difference between linking to a blog which has a public identity and history, and running a wire service story which is intentionally anonymous and therefore opaque.

Dan Froomkin of the Washington Post has summed this up masterfully by saying that the web “abhors lack of voice.”

So what’s the “right” approach? Exactly what the blog media have been doing: do what you do best and link to the rest. There are only two useful things a newsroom can do:

  1. Put new content online by doing original reporting
  2. Filter the internet to tell users where to find other good stuff.

I think a news organization has to do original reporting in some form to be worthy of the name. To develop authority and convince its audience to listen, it probably has to let its reporters expertise shine through. What newsrooms don’t seem to understand yet (and Google does) is that filtering is just as useful, if not more so. Running or rewriting wire copy does serve to inform the reader, but linking is far more efficient for the newsroom and far more useful to the reader. Any organization that wants readers to come to its site first can ill afford to pretend that the reader doesn’t want the rest of the web too.

Lifelogging + Machine Transcription = Public Reporters’ Notebooks

want reporters to make their notebooks public, because I want news organizations and audiences alike to enjoy the fruits of transparency that the web is bringing to us. There are at least two problems with this idea: these notes are indecipherable scribbles on paper, and reporters won’t want to do it. Recording audio instead of ink and using machine transcription will help with the first problem, while the cultural shifts brought by the slow but inevitable rise of lifelogging will help with the second.

ReportersNotebook

The traditional “voice of god” news writing style just doesn’t cut it on the web, where Wikipedia has made us expect hyperlinked footnotes to every statement and bloggers happily admit their uncertainties. I’m hardly alone in calling for professional journalists to be more upfront about the sausage-making that is news production.

Not only might this renew the public’s battered trust of pro media, but showing your work lets other people build on it. Bloggers and Tweeters don’t wait until they have the full story to publish a new fact or post an interesting interview, allowing that information to be used immediately by someone else. As I have argued extensively, working journalists need to get into the habit of collaborating with their communities.

It’s quite possible that everyone wins when professional journalists show their work.

First problem: the scribbles. It’s really hard to write fast when you’re taking down a quote. The result is chicken scratchings and personal shorthands. Each reporter’s notebook is written in its own code. Even if anyone could read the writing, they’d find only a battered mess of sentence fragments and unidentified quotes. Besides, the notebook is on paper. We could ask reporters to come back to the newsroom after each assignment and scan their notes, but that would be useless.

Reporters might consider recording their interviews instead, as they often do already, and posting those recordings online. Want to know more than the article says? Just watch the video or listen to the tape. The Nieman Journalism Lab already makes a habit of this.

But transcribing recordings is a huge time-sucking pain in the ass, and AV isn’t searchable so there’s no quick way for either the reporter or the audience to find that favorite quote. Enter machine transcription, automated speech-to-text systems that produce a transcript of any conversation. This sort of technology is still in its infancy, and it has accuracy problems — but it’s already used by Google Voice to convert voicemail to email,  by YouTube to make videos of political speeches searchable, and in Adobe Premiere for semi-automated tape logging.

Rather than the notepad, I envision the journalistic tool of the future to be a Blackberry or iPhone with real-time speech-to-text software running on it. And I want a “tag this” button on the phone, which I will push to mark the time whenever someone says something interesting. Immediately after the interview, I might review the transcripts and tweet any particularly interesting points from my mobile device. Back at the newsroom, I would cut and paste quotes from marked times, then upload the article and post the entire recording, including its transcript. Done, and anyone who wants to know more than my 200 word summary is free to roll the tape to do their own sleuthing.

Of course, this means that everything the reporter says will be public. This will be a change. It won’t always be appropriate, as with off-the-record chats or embargoed material. But most of all, I think it’s going to take getting used to the idea that our lives and our work are going to be a lot more public than they were.

On Tuesday I worked with a colleague to interview an Indonesian street performer. The recording on my iPhone also caught us in a brief, er, discussion about who was going to type up the transcription. I would be hesitant to put the recording online because of that, and because it reveals all the snarky comments, translation confusions, possible inaccuracies and in short, the entire messiness of the journalistic process. Traditional journalistic style is clean, well ordered, literally black and white. The real world isn’t like that, and showing our work is going to destroy some cherished myths.

But then, entire civilizations used to think that the King was descended from the gods. Some myths are worth losing. And we are losing them: LiveJournal, Facebook etc. are giving us new notions of privacy and with them, an outpouring of strikingly honest accounts of what it is to be human. Meanwhile, we’re going to start recording much more of our lives, and with much less effort. Imagine an audio recording of every moment of your life, with good search capability. Such a system would be a form of perfect memory. Life-logging is going to happen; the technology is already here, so I’d guess that teenagers are going to be growing up with it within a decade.

Recordings of candid moments will then become commonplace, and if we’re lucky, we’ll all become more forgiving as we gain a better understanding of human fallibility.

In this context, it won’t seem so scary for a journalist to post the unedited records of their encounters with reality. In fact, it’s going to become necessary. Everyone else will be doing it.

The Structure of Social Journalism

Shortest way I can describe how I think journalism must change: the internet is not just for distribution, but production too. I’m not saying that “citizen journalists” will be making all the news. I suspect a complex collaboration between many people, including something like a newsroom full of pro journalists. In this article I’m going to explore what that might look like, by asking what the component tasks are that make up “journalism”, and thinking about who can do those most efficiently. And I’m going to sketch out the design for a piece of social software to support this.

Here’s a list of things that professional journalists do:

  • decide what should be more broadly known
  • decide what should be more deeply investigated
  • collect information from sources both public and private
  • check that information for factual accuracy
  • construct narratives to make sense of that information
  • produce content to convey those narratives
  • publish and market that content

This list is by no means definitive or exhaustive. It’s just illustrative, a starting point for a thought experiment. Who could do each of these things best? And what tools to do they need to do it?

Having a network of people producing journalism around a newsroom is not a new idea. Jeff Jarvis has been discussing networked journalism since at least 2006, and naturally I think he’s on to something. In this essay I want concentrate on process and roles. If cheap networks make new types of collaboration possible, they also set the stage for new types of specialization. I think one of the problems of the traditional, mainstream media newsroom is that it it tries to handle the entire journalistic process internally, even the parts that it’s not actually very good at.

An example

On November 25, a video appeared on YouTube which appears to be the testimonial of a young woman recently fired from the credit card collections division of Bank of America. She had been allowing the bank’s most desperate customers to enroll in fixed-payment debt recovery schemes. Many of these customers are currently paying 30% interest as a result of recent rate hikes, so this was a great kindness. It was also against company policy.

The video is powerful. It’s an amazing first-person testimonial of the greed and heartlessness of large corporations.

So is this journalism?

Continue reading The Structure of Social Journalism

Five (Long) Videos about Journalism Transformed

Come, my fellow information geeks, and gather ’round the glow of monitors. The world is changing (it’s the internet) and the way we learn things is changing too. The blogosphere is blooming while journalists are being laid off. Is this good? Is this bad? I’ve spent far too much time trying to understand how everything is shifting.

And now you too can waste your time in learning! Here are five videos about journalism, blogging, tweeting, collecting and sharing information, and how stuff is generally changing. In no particular order:

1. “The Arab World on the Front Edge of Media”, by Moeed Ahmad, head of New Media for Al Jazeera

Moeed Ahmad talk

Figuring out which tweets from the Iranian protests are true. Tracking falling bombs in Gaza using SMS and open-source mapping mash-ups. Releasing war footage under Creative Commons licenses. Moeed has a seriously interesting job, and speaks with great eloquence about how his small new media team fits into a huge global news organization.

Continue reading Five (Long) Videos about Journalism Transformed