Thursday, June 10, 2010

Caffeine : New search index from Google

Google announced the completion of new web indexing system "Caffeine" this week.

This is what Google says about it:

Old index system had several layers, some of which were refreshed at a faster rate than others; the main layer would update every couple of weeks. To refresh a layer of the old index, the entire web would need to be analyzed, which meant there was a significant delay between when a page was found and when it is available as a search result.

With Caffeine, web is analyzed in small portions and search index's are updated on a continuous basis, globally. As new pages or new information is found, it is added to the indexes, which helps in finding fresher information than ever before. Apart from this, every second Caffeine processes hundreds of thousands of pages in parallel.