I see a key open standard missing, crucial to the development of both user reading experience and publisher business models. Users want the “best” content from wherever it may have been published, presented beautifully and seamlessly within a single interface. Publishers want users to go to their site/app so that they can get paid, whether that’s through a subscription or by showing the user ads.

This tension is hugely visible in the content economy of today. It’s why Flipboard, Zite, and Google News are so loved by consumers yet so hated by publishers. It’s a manifestation of the “producers vs. aggregators” spat, a senseless culture war which reflects a legacy publishing industry structure that is ill-equipped to serve a digital public. This has spawned many lawsuits. These battles make no sense to the consumer, and indeed, the content supply chain is not the consumer’s problem. Nonetheless there is a real problem here, and lawyers alone cannot solve it.

What I start from is user experience. My user experience. Your user experience. I want whatever I want, all in one convenient cross-platform place. The product itself might be an expertly curated selection of articles, an algorithmic aggregator (Google News), a socially-filtered stream of content (Flipboard), or a system that tries to learn my content preferences over time (Zite.) The best method of content selection is far from settled, but it’s clear that it’s going to be very hard for a general-interest publishing brand to reliably attract my attention if all they can offer me is what they can create in-house. To adapt Bill Joy’s maxim, “most of the best content is on someone else’s site.”

The practice of pointing to content created by someone else within your product has come to be known, for better or for worse, as “aggregation”, though “curation” has also been used to describe the more manual version. (Personally I suspect distinction is meaningless, because algorithms embody editorial judgement, and there are strong hybrid approaches.) Because of the way the internet developed, many people have conflated aggregated content with free content. But this is not necessarily so. Aggregation has mostly been done by using links, and it’s not the aggregator who decides if the page on the other end of the link is free to view.

In an era of massive information abundance, filtering creates enormous value, and that’s what aggregation is. Aggregation in all its guises is really, really useful to all of us, and it’s here to stay. But linking as an aggregation method is starting to fall apart in important ways. It’s doesn’t provide a great user experience, and it doesn’t get producers paid. I strongly believe that we don’t want to discourage the linked, open nature of the internet, because widespread linking is an important societal good. Linking is both embodied knowledge and a web navigation system, and linking is incredibly valuable to journalism. Nonetheless, I see an alternative to linking that aligns the interests of publishers and consumers.

When Google News sends you to read an article, that article has a different user interface on each publisher’s site. When the Twitter or Flipboard apps show you an article they display only a stub, then require you to open a Safari window for the rest. This is a frustrating user experience, which Zite tried to remedy by using the readability engine to show you the clean article text right within the application. But of course this strips the ads from the original page, so the publisher doesn’t get paid, hence this cease and decist letter. For many kinds of content, somebody needs to get paid somewhere. (I’m not going to step today into the minefield of amateur-free vs. professional-paid content, except to say that both are valuable and will always be with us.) Payment means taking either some cash or some attention from the consumer. Lots of companies are working on payment systems to collect money from the consumer, and there have long been ad networks that distribute advertising to wherever it might be most valuable. What is missing is a syndication technology that moves content to where the user is, and money to the producer. The user gets an intergrated, smooth user experience that puts content from anywhere within one UI, and the publisher gets paid wherever their content goes.

This would be a B2B technology; payment would be between “aggregators” and “content creators,” though of course both roles are incredibly fluid and lots of companies do both at different times. To succeed, it needs to be a simple, open standard. Both complexity and legal encumbrances can be hurdles to wide adoption, and without wide adoption you can’t give consumers a wide choice of integrated sources. I’m imagining something like RSS, but with purchased authentication tokens. For the sake of simplicity and flexibility, both payment and rights enforcement need to be external to the standard. A publisher can use whatever e-business platform they like to sell authentication tokens at whatever pricing structure suits them, while merely expressing online rights — let alone enforcing them — is an incredibly complicated unsolved problem. Those problems will have to be worked on, but meanwhile, there’s no reason we can’t leverage our existing payment and rights infrastructures and solve just the technical problem of a simple, open, authenticated B2B content syndication system.

What I am trying to create is a fluid content marketplace which allows innovation in content packaging, filtering, and presentation. There is no guarantee that such a market will pay publishers anything like what they used to get for their content, and in fact I expect it won’t. But nothing can change the fact that there is way, way more content than there used to be, and much of it is both high-quality and legally free. If publishers want to extract the same level of revenue from you and I, they’re going to have to offer us something better than what we had before — such as, for example, an app that learns what I like to read and assembles that material into one clean interface. But it’s clear by now that no one content creator can ever satisfy the complete spectrum of consumer demand, so we need a mechanism to separate creation and packaging, while allowing revenue to flow from the companies that build the consumer-facing interfaces to the companies that supply the content. That means a paid syndication marketplace, which requires a paid syndication standard.

This idea has close links with the notion of content APIs, and what I am proposing amounts to an industry-standard paid-content API. Let’s make it possible for those who know what consumers want to give it to them, while getting producers paid.

Further reading:

• Zite incident shows why publishers need to enable automatic, controlled content distribution – Poynter
• Clay Shirky’s right that syndication’s getting disrupted — but not in the ways he thinks it is — NiemanLab
• Flipboard may be creating third distribution channel for mobile publishers – Poynter
• What is the open platform? – The Guardian
• Open APIs and News Organizations: A Study of Open Innovation in Online Journalism – International Symposium on Online Journalism 2011
• In 2011, news orgs will finally start to move past the borders of their own content – NiemanLab