Questions can come to us in many forms and layers, as can the answers we seek. Research too can take many forms and layers. Most people are not researchers and do not understand how research works, whether or not it works or even why it works. Societal norms teach us to trust research even though many of us have no idea why we should.
Research mythology has its roots in this uncertainty mystique. Who gets to be the judge and the juror? How do we know the study was conducted correctly? How can we know for certain that the results are correct?
BIG DATA has capitalized on this uncertainty. At its core, BIG DATA distrusts research. It purports that if you do not have the actual, factual data in the aggregate then you cannot possibly come up with an accurate answer.
In truth BIG DATA has overstated its efficacy because it is impossible to have 100% of the actual/factual data on most topics. This level of actual big data completeness can sometimes be alarmingly low. In those cases where BIG DATA does not collect data by research sample balancing standards, the results can turn out to be misleading and not representative of the target populations.
Let’s talk about research
When the word ‘research’ is thrown about, most people conjure up political polls that can sometimes be all the way right and sometimes all the way wrong. We’ve all heard, have even silently mocked the claim of ‘plus or minus 4%’ margin of error. Let’s completely rethink this perception. If there is an outside 4% margin of error, that means that the accuracy rate would be 96%; in other words, the survey’s prediction would be accurate 96 out of 100 times! Granted, this may not be enough for politics or candidates split by a few hundred votes, but it is more than plenty for the rest of us.
Research has its flaws and limitations too but there is no escaping the fact that properly conducted research can attain incredibly high accuracy rates.
My first encounter with properly conducted secondary research accuracy involved a group of Toyota automotive dealers in Colorado. They challenged our research sources saying we could not possibly have any research specific to their dealer trading areas. They indicated they would change their tune if we could estimate the last quarter of sales for their dealerships without the benefit of their sales data. We expertly scaled national secondary surveys of new car buyers/intenders to the Colorado market trading areas. The estimate was within 2% of the quarter’s actual sales levels. This was not enough to convince them because they said what we produced was not possible, that we had to have somehow gotten hold of their quarterly dealership sales reports.
Over the course of three decades, I experienced the same secondary research accuracy phenomenon case after case. Whenever a particular secondary research finding was compared with the customer’s real- world data, they matched up every time.
Real world cases
Here are a handful of cases: Memorial Sloan Kettering claimed our area-by-area projections of female breast cancer survivors in Manhattan was so accurate they jokingly asked if we had hacked into their medical records; Another case involved a real-world comparison of a major Mississippi medical system’s estimates of disease states and utilizations by market; Another case involved accurately predicting what specific types of music would drive ticket sales for a local Sacramento concert venue; Another case involved the Texas University of Health in their teen pregnancy prevention initiatives. Projecting from a National study again, we reported Audience profiles within only two zip codes that were incredibly consistent with their door-to-door community audit.
Final thoughts
To be clear, there are very specific reasons for all these cases of real-world accuracy. In the first place, the studies were competently fielded by renowned research organizations such as Nielsen, GFK, Kantar and also several reputable primary research organizations. In the cases I cited, the research was conducted across tens of thousands of measures and meticulously documented to be representative of the U.S. population as a whole. The second reason relates to the accuracy of the Audience definitions that all the measures were based upon. The third reason involves certain methods of expertly projecting research from a larger geography into a smaller one. Can primary research also be used or integrated in this manner? Yes, with the correct methodologies it can.
As you press onward to get researched answers to your organization’s own pressing questions, remember the above three criteria. The most important of these three is having research that has been conducted correctly and whose respondents are representative of the surveyed populations. All findings derived from consumer research surveys can only be deemed accurate if they were fielded correctly in the first place.
The most successful solutions have been engineered in reverse. They started with the end purpose in mind, followed by the strategies, elements and steps required to achieve the desired outcome.
Send me a message or book a free 30-minute Intro meeting with me to get the conversation started!
Not sure what expertise you need? Our concierge is happy to discuss your needs and match you with the best suited consultant. On Cansulta, you can browse consultants by expertise, and connect with vetted professionals in Intro sessions FREE.
