The speed of Google updates and indexing

(note: this is a guest post written by Gary Sims who runs Simo's Sagacity)

---

At the turn of the century Google's index wasn't very fresh, sometimes up to 3 or 4 months would pass before the index was updated. Then in the middle of the year 2000 Google started doing monthly updates which lead to what became known as the Google Dance. Fast forward to 2003 and Google started doing incremental updates to their index. That meant that the spider was sent out to update a small portion of the index on a daily basis. From then until now Google have been getting better and better at the incremental updates, in other words the index is updated faster and is fresher.

Their updates are go good now, under the right circumstances pages of popular websites can appear in the Google index in minutes and new sites can appear in hours.

Recently I started a new technology blog with some friends (Hi-Tech
Squad) and we had resigned ourselves to the fact that it would take weeks for Google to read our site and then only 1 page and it would take months before the site became integrated fully into the index. We we were surprised (and relieved) that our blog turned up in just 2 days. And now only a couple of weeks later we have over 80 pages in the index (including the comments to the blog posts). And Google are still keeping up the pace. The index is normally only 24-48 behind what we have on the blog and we are getting about 1/3 of our traffic the search engine.

How did we do it, first we got some links out their to our site, I posted on some of my other blogs about the new blog and also we heavily used Technorati and Digg. The result was that Google came, saw and conquered.

I think Google need to be congratulated on their freshness of their index and their on going effort to keep on top of the vast quantity of material which is created on the web every day,

Thanks Google.

Gary Sims
Simo's Sagacity