It’s pretty clear that Experian Hitwise has a comfortable familiarity with Tom Smykowski’s “Jump to Conclusions” mat, at least in regards to comparing search result accuracy with Google and Bing. It’s either that or perhaps the “online competitive intelligence service” needs another lesson in correlation and causation.
In a recent report discussing Bing’s improved market share — something the tech journalism sector is going wild about, but if I’m not mistaken, 68 percent is far greater than 27.4 — Experian made the bold statement that, because Bing users click search engine results 81.54 percent of the time, compared to 65.58 percent for Google users, Bing’s results are more accurate than Google’s.
Of course, considering the spat between Google and Bing, and Bing’s use of at least a small percentage of Google’s search engine results, it adds more skepticism to Hitwise’s claim.
As many of you know, the principle idea in “correlation does not equal causation” is one thing does not necessarily cause another, or in the case of Google and Bing’s search result accuracy, one piece of data (more Bing users click the results) does not make bold conclusions true (Bing’s results are more accurate).
Taken at face value, it’s easy to conclude Bing’s results are 16 percent more accurate, but then, when you consider so many more people are using Google to conduct Internet searches, the margin for non-clicks for any number of reasons — incorrect spelling, refining the original search query — is a lot larger than Bing’s.
Furthermore, what is the average experience level of Bing users? If Bing users are comprised predominantly of “Internet noobs,” for lack of a better term, would they not be more inclined to click the first result they saw, even if it was relevant or not? If Google’s users are “more experienced,” would they not be more selective about which results they clicked?
And if that’s the case, doesn’t that mean the overall quality of search results — across the board, not just with Google — need to be improved?
As for Google’s results being less accurate than Bing’s, before making such a claim, much more research is needed: an extensive, side-to-side comparison of multiple queries, not just reliance on how frequently the results were clicked. Another obvious aspect to consider is the user’s experience level. An Internet veteran will not click everything they see; whereas an Internet novice using Bing because they liked their witty “search overload” commercials is not as selective.
In regards to Hitwise’s conclusion, as told via Internet chat lingo, the following statement comes to mind: More clicks != better quality of results.
You need more data to make such a profound conclusion.