Relevant results from search engines: Part 1

This is the first part in a serious of blog posts I'm going to be making regarding the relevancy of search results. I'll be comparing the results from the 3 major search engines (Google, Yahoo, and MSN). Here's how it will work:

• I will choose a keyword at random
• I'll do a search on each of the search engines using the exact same keyword
• Only the top 20 results (the first 2 pages) will be counted. This is because a majority of users don't go beyond page 2
• I will count each result returned that matches my search as a point. So a perfect score is 20, which should be nearly impossible.
• Results I will not count as relevant:
• Directories
• Wikipedia or Yahoo Answers links
• Manufacturers' links (if applicable)
• Links that are not relevant to my search, even if they match the keyword (for example, if I'm searching for a cat, as in a feline, then I wouldn't count results for a Caterpillar bulldozer)
• Any result that doesn't match what I had in mind for the keyword

The first keyword I decided on was "tool chest". Pretending to be a user in search of a tool chest for sale, I entered the keyword into all three search engines. I am only interested in sites that have tool chests for sale. Here are the results:

1. Google - 7 relevant results
2. Yahoo - 15 relevant results
3. MSN - 6 relevant results

Yahoo wins round 1 with a more than 2:1 margin of victory over both Google and MSN. While this may be an unscientific survey, Yahoo has been getting a lot better at returning relevant results.

We'll see in Part 2 (coming up next week) how the search engines fare when I use a more tricky keyword.

Stay tuned...