Taming Google (or the price of free)
Search tools are not infallible, as anyone can testify who has taken content metrics sourced from Google, Technorati or their peers at face value, and been burnt. Here is some evidence, from a recent internal research project.
Chart 1 plots the number of results reported by Google Blog Search in a simple search string, banded by month using the advanced search interface.
Chart 2 plots the same results if you throw away the number of results reported by Google, and count actual results by hand.
There is a critical issue here for metatrend analysis – the old business cliche of garbage in, garbage out applies: if data are junk, no valuable insights will be created. Researchers working within social media need to understand the limitations of the tools that they use, and incorporate an effective validation process into their workflow.
A Metatrend method paper is in draft form, so more on the implications of accuracy and reliability in tool selection later.
