Google Caffeine: Indexing the Web Faster

Google announced today that new indexing system called Caffeine is live. The goal is to find fresh content faster, to index web pages faster and to update the index in real-time.

"Our old index had several layers, some of which were refreshed at a faster rate than others; the main layer would update every couple of weeks. To refresh a layer of the old index, we would analyze the entire web, which meant there was a significant delay between when we found a page and made it available to you. With Caffeine, we analyze the web in small portions and update our search index on a continuous basis, globally. As we find new pages, or new information on existing pages, we can add these straight to the index," explains Google.

Now that the index is updated in real-time, you'll find recent information faster, sometimes a few minutes after it's posted. Matt Cutts says that you'll sometimes find web pages that weren't indexed when you started typing your query. This is really incredible, even though it's still very difficult to rank recent web pages.

To find recent web pages, click on "past 24 hours" or "past week" in Google's right sidebar (the screenshot shows Google's old interface). You can even sort the results by date.