Alle gegen Google

wer schon immer mal lust hatte eine suchmaschine zu bauen, aber keine lust hatte einen spider zu programmieren, der hat ab jetzt eine ausrede weniger. (FAQ)

Alexa claims it's the first time that developers, students and startups will be given inexpensive access to an industrial-scale web crawler -- the same technology used by industry giants like Yahoo (Yahoo Slurp) and Google (Googlebot).

'It sounds innocuous but it's big,' said Alexa CEO Bruce Gilliat. 'We're giving access to billions of pages and computing resources.... Users have never had this opportunity before. Big industry has ruled search, because it was the only player with access to the tools.'

Alexa spiders 4 billion to 5 billion pages a month and archives 1 terabyte of data a day. The new platform will allow developers to build their own search engines.

'If it is what they claim it is, it strikes me that this is nontrivial news,' said search industry pundit and author John Battelle. 'Anyone can crawl the web, but crawling and maintaining an index at scale is very difficult and very expensive. They are providing convenient access to something that was very dear.' (Wired News)