My first search engine relevancy test took some flak for not being scientific enough. This time around, I've set some strict guidelines that should make it a bit more fair to all the search engines. Yahoo won round 1, but I only tested one keyword and used only my interpretations as scoring guidelines. Here are the scoring guidelines for round 2:
- Only sites returned on the first page of search results will be used (as opposed to the top 2 pages in part 1).
- A result that directly links to content that fits my search objective will receive 3 points.
- A result that links to a page that, while it may not exactly answer my question, has the information on its site, and it can be found quickly will earn 2 points.
- Results that are Wikipedia entries, Yahoo Answers, About.com, etc. will receive 1 point. The reason for this is because these sites are so well known that if a user wants to search for the answer on them, they can go directly to those sites.
- Any result linking to a manufacturer or retailer/wholesaler that does not provide the information I'm looking for will not receive any points.
- Results returned to directories will receive -1 points.
- Completely irrelevant results will receive -2 points.
- Results from the same website will only be counted once.
The goal is to find instructions on how to fix a lawn mower engine. The keyword is long-tailed and fairly precise while the results might be pretty broad considering the number of different models and brands of lawn mowers out there. Note - I'm looking for answers on the internet with this keyword, and not for a book to purchase that helps me out.
- Google: 14 points
- Yahoo: 9 points
- MSN: 9 points
Pretending for a second that I'm a 14 year old XBox 360 fan boy, and I want to find a website with cheat codes for lots of games. The results should have a good variety of XBox 360 cheat codes for a good share of all the XBox 360 games on the market. If the initial page that pops up is selling product or giving reviews, I'm going to give it zero points. I want cheat codes for this keyword. If I can't find cheats 1 click away from my landing page, I'm moving on. I'll be awarding three points to sites that have actual cheat codes, or a list of cheat codes on the landing page.
- Google: 16 points
- Yahoo: 19 points
- MSN: 22 points
Keyword #3: "Buy a used laptop"
I need a laptop, but can't afford a new one (okay, I know that with the price of new laptops at ridiculously low prices, it would be stupid to purchase used - humor me). The results that link directly to a listing for a laptop for sale will get the full 3 points (including manufacturers). A result that doesn't directly sell a laptop on that page, but has used laptops for sale somewhere on the site gets 2 points. 1 point will be given to sites that may not sell laptops, but link to other sites that sell laptops. Sites giving advice, reviews, or information on purchasing used laptops will not be given any points.
- Google: 13 points
- Yahoo: 10 points
- MSN: 14 points
- Google: 43/90
- Yahoo: 38/90
- MSN: 45/90