The village of Kangzhuang, in Henan Province, China, was built in 2006 next to the Tianrui cement factory (in background, above), to house villagers relocated after the government bought their land. A huge grey cloud of dust hangs permanently over the village. The villagers told me that the factory is shut down for a day or two whenever air quality authorities come to visit, then started up again as soon as they leave.

I visited Kangzhuang on another story. But eating dinner that night, covered in a film of dust, I suggested to a Chinese colleague that this issue of cheated environmental regulations deserved to be investigated further.

“That’s not news,” she said. “That happens all over China, every day.”

And she’s right: it’s not news. All chronic problems fade from attention, and old news won’t interest (most) readers, or sell papers. But we’re leaving the paper era, and this changes things. In the web era, documenting old, marginal, or incomplete stories is much more valuable — and much more affordable — than it used to be.

There has been much discussion of how web writing style must differ from paper writing style, but the difference in medium also changes what stories can and should be covered. Despite its physicality, paper is a deeply transient medium. Yesterday’s newspaper lines birdcages, but yesterday’s web stories will be showing up on Google five years from now. An editor selecting stories needs to be thinking about not only tomorrow’s page views but next year’s as well, and also, crucially, how the story will function in combination with stories from other outlets. There are close ties here to the concept of stock and flow in journalism, and the new-media notions of topic pages and context.

I ran into this paper-web cultural divide discussing a pitch for a story about the number of civilian casualties in the Iraq war. I had done a little bit of new reporting on the topic, but mostly I just wanted to write a thorough summary of the various estimates and a careful, clear analysis of how reliable each might be. An editor friend said that, since the studies I would be referencing were several years old, this wasn’t news and would be hard to sell to the outlet I was proposing; I said that I was planning to write an authoratative article on a topic of great international concern for a general audience, a piece which doesn’t really exist yet. I felt that this would be exactly the sort of public service that journalism is supposed to be, and further, we’d have a good shot at getting in the top five results on Google.

We clearly had somewhat different conceptions of what stories count as publishable journalism, which seemed to be derived from our focus on different media.

Not only is the web a permanent medium, it’s a distributed, accessible-from-anywhere medium (unless your government doesn’t want you to know about certain things, but there are ways around that.) A single report of environmental cheating at a cement factory is not going to change anything for the people who will get sick from cement dust, but hundreds of such reports all over China might. If editors, used to selling papers, think of stories as throwaway consumables rather than careful additions to a permanent store, they miss opportunities for collective action.

But how to fund such long-term, speculative projects? After all, every story costs money to produce. I think part of the answer lies in another medium-driven difference: the web is more amenable to journalism of different levels of quality and completeness. The New York Times aims to be “the paper of record,” which means it hopes always to tell the full story, and to never get a fact wrong. On the web, this is ridiculously inefficient. As a social medium, the web draws power from collaboration and conversation — say, between different papers in different places — and that process is severely hindered if only “finished” work makes it online.

Yes, there will always be a need for the authoritative voice and the carefully edited, sub-edited, copy-edited and fact-checked article. But what about all that other good stuff that journalists produce in the course of their work? What about the juicy trimmings that had to be cut from the main story, the tantalizing leads that the reporter hasn’t had time to follow up, and the small incidents that have meaning only in aggregate? The web demands that we put more online than we would publish on paper, and provides a place for information of all grades. In this new medium, amateur journalists (such as bloggers and thoughtful commenters) are often much more adept at creating value from information by-products than their professional peers. News organizations will have to find forms for publishing unpolished information, such as the beat blog.

The report of environmental malfeasance in Kangzhuang is not yet a story. It hasn’t been checked against other sources, we haven’t heard from the local government ministry about whether they did a proper environmental study prior to putting 2,300 people next to a factory, and anyway the journalists who visited the town (myself among them) are currently busy writing up an entirely different story. But this tidbit deserves to be aired in a public place, so that others can build on it in the future. This is a valuable public service that could be provided for low cost, a service that is possible only on the web. Will an industry trained in the paper era see this possibility?

Flame wars and jihadist rants and generally worthless behavior in the comments: that’s the problem I’m trying to solve here.

And I’m trying to do it while preserving anonymity. Internet conversation can get nasty when the participants are anonymous, which has led to proposals of tying all online identities to “real” identities. This is the wrong solution to the troll problem, because it destroys privacy in a serious way. I want to build discussion systems that allow anonymous comments, yet remain orderly, civil, and enlightening. I think this can be done with filtering systems based on reputation.

Reputation is a thing that sticks to an identity. Historically most people had only one identity, closely tied to their physical presence. But now, online, every one of us has multiple identities: think of how many user names and logins you have. There’s some consolidation going on, in the increasing acceptance of Google, Twitter, and Facebook logins across the web, and this is mostly a good thing.  But I don’t think we want to aim for a world where each person has only one online identity. Multiple identities are good and useful.

Multiple identities are closely related to anonymity. Anonymity doesn’t mean having no identity, it means not being able to tie one of my identities to the others. I want to be very careful about who gets to tie the different parts of me together. I’m going to give two arguments for this, which I’ll call the “does your mother know” and “totalitarian state” arguments. They’re both really important. I’d be really if sad if we lost anonymity in either case. And after I’ve convinced you that we need anonymity, I’ll talk about how we get people to behave even if they don’t leave a name.

Keeping the different facets of ourselves apart is the essence of privacy. We’ve always been different people in different contexts, but this was only possible because we could expect that word of what we did with our friends last night would not get back to our mother. This expectation depends upon the ability to separate our actions in different contexts;  your mom or your boss knows that someone in the community is going on a bender/having kinky sex/voting Republican, but she doesn’t know it’s you. The ability to have different identities in different contexts is intricately tied to privacy, and in my mind no different than setting a post to “friends only” or denying Amazon.com the details of your personal life. Although the boundaries around what is “personal” are surely changing, if you really think we’re heading toward a world where everybody knows everything about everyone, you’re mad. For one thing, secrets are immensely valuable to the business world.

And then there’s China. I live right next door to the most invasive regime in the world. The Chinese government, and certain others such as Korea, are trying very hard to tie online and corporeal identities together by instituting real name policies. This makes enforcement of legal and social norms easier. Which is great until you disagree. Every damn blog comment everywhere is traceable to you. Every Wikipedia edit. Everything. China is trying as hard as it can to make opposing speech literally impossible. This is not theoretical. As of last week, you can’t send dirty words through SMS.

When the digital panopticon is a real possibility, I think that the ability to speak without censure is vital to the balance of power in all sectors. Anonymity is important to a very wide range of interests, as the diversity of the Tor project shows us. Tor is a tool and a network for anonymity online, and it is sponsored by everyone from rights activist groups to the US Department of Defense to journalists and spies. Anonymity is very, very useful, and is deeply tied to the human right of privacy.

Right, but… how do we get sociopaths to play nice in the comments section if they can say anything they want without repercussions?

The general answer is that we encourage social behavior online in exactly the way we encourage it offline: social norms and peer pressure. We can build social tools into our online systems, just like we already do. A simple example is the “flag this” link on many commenting systems. Let’s teach people to click it when they mean “this is a useless post by troll.” Collaborative moderation systems — such as “rate this post” features of all kinds — work similarly.

Collaborative moderation is a really big, important topic, and I’ll write more about it later. There are voting systems of all kinds, and the details matter. Compare Slashdot versus Digg versus Reddit. But all of these systems rate comments, not users, and I think this makes them weaker than they could be at suppressing trolls and spam. Identities matter, because identities have reputations.

Reputation is an expectation about how an identity will behave. It is built up over time. Crucially, a throw-away “anonymous” identity doesn’t have it. That’s why systems based on reputation in various forms work to produce social behavior. There are “currency” systems like StackOverflow’s karma where one user can give another credit for answering a question. There are voting systems such as the Huffington Post’s “I’m a fan of (comment poster)” which are designed to identity trustworthy users. Even Twitter Lists are a form of reputation system, where one user can choose to continuously rebroadcast someone else’s tweets.

And in the context of online discussion, you use reputation to direct attention.

That’s what filtering is: directing attention. And this is how you deal with trolls without restricting freedom of speech: you build collaborative filters based on reputation. Reputation is powerful precisely because it predicts behavior. New or “anonymous” identities would have no reputation and thus command little attention (at least until they said a few interesting things) while repeat offenders would sink to the bottom. Trolls would still exist, but they simply wouldn’t be heard.

NB, none of this requires tying online identities to corporeal people. Rather than being frightened of anonymity and multiple identities, I think we need to embrace them. We need to trust that we can evolve the right mixes of software and norms so that collaboration overwhelms vandalism, just as Wikipedia did. This field is mostly unexplored. We need to learn how identity relates to trust and reputation and action. And we need to think of social software as architecture, a space that shapes and channels the behavior of the people in it.

Simply trying to make it impossible to do anything bad will destroy much that is great about the internet. And it lacks imagination.

Today I have been keeping Twitter window open, watching messages tagged #10yearsago scroll by. It’s striking. This is the sort of grass-roots expression of hopes and dreams that adventurous journalists used to travel the world for, and compile into coffee table books. Now we can all see it live for free.

aricaaa #10yearsago boys still had cooties. ah i miss those days!

davidwees Happy New Year! #10yearsago today I was in a dead-end job working in a warehouse. Now I love what I do and have a great family.

scottharrison: #10yearsago I was a sycophant and a drunk selling vodka to bankers in clubs. Grateful for God’s grace and sense of humor.

Sirenism #10yearsago I was eleven and one of my brothers friends tried to kiss me at midnight. I punched him in the nuts.

cosmicjester Holy Shit #10yearsago I met a girl at a friends birthday party, we both liked Red Dwarf and the Beatles. Then she became the girl.

shaunraney #10yearsago was the the saddest day of my life.

As striking as this is, I notice that almost all of the traffic is in English. The only other language reasonably well represented is Indonesian. Curious, though it is the 4th largest country by population, and social media are hugely popular here.

I’ve also really enjoyed watching the clock strike midnight in different time zones. Here in Jakarta, the NYE conversations of my friends in California — 13 hours behind — seem so last night. I’m nursing a hangover, they’re working on one.

It’s so easy to forget the world outside what you know. I hope that global media like Twitter will help us to remember everyone else. The technological means have arrived with a roar, but we’re still not really talking to one another. What is the next step?

Here are some problems I see with the implementation of the commenting system on the New York Times web site. Assuming that they want the discussions about their content to be taking place on their site. The way things stand now, I suspect that they’re actively sending many readers to Facebook instead of keeping them on nytimes.com.

• How come some articles allow comment and others don’t? Is the policy of which articles get comments explained anywhere? Arriving at Times content from a link, I’m confused about whether I can expect a good discussion or just broadcast.
• Even for articles that we are allowed to comment on, the comments are hidden. Sometimes there’s a pull-out quote (which is cool!) but more often we see only this:

  

• The number of comments on each article is not visible from the front page or the section pages. There’s no way for readers to see, at a glance, which discussions are hot.
• The “recommend” button on each comment is welcome, and serves as a useful way to filter comments. The “highlight” button which seems to appear on the more recommended comments is a little more obscure — does it just put the comment in the “highlight” list, or is there editor moderation involved? The “what’s this” tip doesn’t clear this up (click image below for larger)

  

• There is no comment view that is sorted for both relevance and freshness, which is the most useful way to track a discussion. Digg and others get this right by adding a time-weighting to a comment’s position in the list.
• There is no way to reply to someone else’s comment. This makes it impossible to have a real discussion on the site. Many other commenting systems organize comments into threads. By not supporting this, the Times is saying that we can talk to them, but not to each other.
• The comments on an article close after a certain point. Although I can see that this might be due to moderator workload issues, it’s also a way to drive away future traffic — as when that link goes viral a week later, or the discussion lasts for months.
• Speaking of which, the comments are moderated. This topic’s a little more complex: there are advantages and disadvantages to this. But I’d like to note that there are plenty of civil discussions happening on the internet in unmoderated places. The strategy of putting a “flag this” link on each comment that sends it for human review is relevant here.
• From the comments FAQ: “We appreciate it when readers and people quoted in articles or blog posts point out errors of fact or emphasis and will investigate all assertions. But these suggestions should be sent by e-mail.” Really? Why? Wouldn’t claims of possible errors of fact be the among the first things you’d want readers to see? Your comments are moderated anyway. And I’d like to point out that your own journalists and editors read the site too — you need those corrections up quickly for internal communication (see: the hilarious Washington Post vs. Public Enemy “911″ correction saga.)

In short, the comment system seems to be to have been designed by someone who has never responded to a message that says “there’s a great discussion about that going on at [link].”

Comments?