I've long wondered exactly how effective a sitemap is in getting a site fully indexed. Like the other Goops (Google hoops) I jump through, I do it without thinking. 90% of the sites that I build have are tracked through the Google webmaster tools and have been verified and sitemapped. I've never known exactly why I create a sitemap other than Google claims it makes a site more "Google friendly".
According to the
"Google uses your Sitemap to learn about the structure of your site and to increase our coverage of your webpages."
I haven't been able to quantify Google's claim, so I put it to a test. Here is how I set it up:
- I created three sites with unique content
- Each of the three sites had 6 pages
- The navigation structure was identical on each page - I used a "link bar" at the top and bottom of each page that linked to all pages in the website.
- The three sites had different topics. I originally wanted to create three identical sites with different URLs, but the chance that two of the sites would be seen as duplicate (and penalized) held me back. Instead, I went for three separate but equally innocuous topics about mundane tasks.
- Each site had 1 image per page and between 200 and 300 words
- The first site (Site A) was entered into the webmaster tools, verified, and had a complete sitemap submitted describing it's structure
- The second site (Site B) was entered into the webmaster tools and verified, but I refrained from submitting a sitemap
- The third site (Site C) wasn't even entered into the webmaster tools. Google had to find this third website naturally
The results of my research were surprising to say the least.
- The first site to be fully indexed by Google was Site B - the site listed in the webmaster tools and verified but not sitemapped. It took about 24 hours to be fully indexed.
- The second site to be fully indexed was the site without any listing in the webmaster tools (Site C). Google found it through a natural link and indexed it completely. Oddly, this was the last site to have it's first page indexed, but all were indexed at one time. Site C took almost 6 days to become fully indexed.
- The last site to become fully indexed was Site A - the website loaded into the webmaster tools, verified, and sitemapped. It was also the first site added to the tools and verified. It only took several hours (less than 8, but I can't be sure because I was sleeping) for the home page to get indexed, but more than 1 week for the entire site to be indexed.