Oct 12 2009
I think I’ve found a useful distinction between the “search” and “news” problems. News organizations like to complain that search engines are taking their business, but that’s only because no one has yet built a passable news engine.
Search is when the user asks the computer for a particular type of information, and the computer finds it.
News is when the computer has to figure out, by itself, what information a user wants in each moment.
This definition has useful consequences. For example, it says that accurately modeling the user and their needs is going to be absolutely essential for news, because the news problem doesn’t have a query to go on. All a news selection algorithm can know is what the user has done in the past. For this reason, I don’t believe that online news systems can truly be useful until they take into account everything of ourselves that we’ve put online, including Facebook profiles and emails, and viewing histories.
And yes, I do want my news engine to keep track of cool YouTube uploads and recommend videos to me. This in addition to telling me that Iran has a secret uranium enrichment facility. In the online era, “news” probably just means recently published useful information, of which journalistic reporting is clearly a very small segment.
It’s worth remembering that keyword web search wasn’t all that useful until Google debuted in 1998 with an early version of the now-classic PageRank algorithm. I suspect that we have not yet seen the equivalent for news. In other words, the first killer news app has yet to be deployed. Because such an app will need to know a great deal about you, it will probably pull in data from Facebook and Gmail, at a minimum. But no one really knows yet how to turn a pile of emails into a filter that selects from the best of the web, blogosphere, Twitter, and mainstream media.
Classic journalism organizations are at a disadvantage in designing modern news apps, because broadcast media taught them bad habits. News organizations still think in terms of editors who select content for the audience. This one-size fits all attitude seems ridiculous in the internet era, a relic of the age when it would have been inconceivably expensive to print a different paper for each customer.
Of course, there are some serious potential problems with the logical end-goal of total customization. The loss of a socially shared narrative is one; the Daily Me effect where an individual is never challenged by anything outside of what they already believe is another. But shared narratives seem to emerge in social networks regardless of how we organize them — this is the core meaning of something “going viral.” And I believe the narcissism problem can be addressed through information maps. In fact, maps are so important that we should add another required feature to our hypothetical killer news app: it must in some way present a useful menu of the vast scope of available information. This is another function that existing search products have hardly begun to address.
Not that we have algorithms today that are as good as human editors as putting together a front page. But we will. Netflix’s recent million dollar award for a 10% improvement in their film recommendation system is a useful reminder of how seriously certain companies are taking the problem of predicting user preferences.
The explosion of blog, Twitter, and Wikipedia consumption demonstrates that classic news editors may not have been so good at giving us what we want, anyway.