Comments on: Why We Need Open Search, and How to Make Money Doing It http://jonathanstray.com/why-we-need-open-search Information, Culture, and Belief Tue, 24 Aug 2010 11:34:03 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: admin http://jonathanstray.com/why-we-need-open-search/comment-page-1#comment-599 admin Tue, 29 Sep 2009 02:24:10 +0000 http://jonathanstray.com/?p=1023#comment-599 80legs may doing exactly what I want! Thanks so much, Neil! Hmm... what do I want to build first? 80legs may doing exactly what I want! Thanks so much, Neil!

Hmm… what do I want to build first?

]]> By: Neil Kandalgaonkar http://jonathanstray.com/why-we-need-open-search/comment-page-1#comment-598 Neil Kandalgaonkar Mon, 28 Sep 2009 23:06:03 +0000 http://jonathanstray.com/?p=1023#comment-598 There's also 80Legs, a new startup which essentially rents crawling infrastructure at a relatively low fee. And another new startup, Spinn3r, tries to keep up with blog posts and Facebook activity in near-real-time. Finally, there's archive.org. I interviewed there -- didn't work out, but I learned a lot about them. Until now they've really seen their role as, well, an archive. They have a good amount of data, but they think about preserving web pages for the next 100 years, not opening it up to interactive experiments. Still it's a potential source of data... Of course none of these are Google. One of the reasons why Google is miles ahead of the competition is that they've worked out ways to make experiments on the dataset extremely cheap, at least internally. It's not impossible that they'll allow outsiders access to that for a fee, at some point. But that is still nothing like a true public archive. There’s also 80Legs, a new startup which essentially rents crawling infrastructure at a relatively low fee. And another new startup, Spinn3r, tries to keep up with blog posts and Facebook activity in near-real-time.

Finally, there’s archive.org. I interviewed there — didn’t work out, but I learned a lot about them. Until now they’ve really seen their role as, well, an archive. They have a good amount of data, but they think about preserving web pages for the next 100 years, not opening it up to interactive experiments. Still it’s a potential source of data…

Of course none of these are Google. One of the reasons why Google is miles ahead of the competition is that they’ve worked out ways to make experiments on the dataset extremely cheap, at least internally. It’s not impossible that they’ll allow outsiders access to that for a fee, at some point.

But that is still nothing like a true public archive.

]]>