There's no question marketing research can be expensive. Even if your company is doing it in-house, marketing research still requires time and staff. Even if the research is entirely secondary, and can be found in libraries or on the Internet, someone has to collect the information. There is considerable opportunity cost to conducting marketing research in-house, as the time an employee spends compiling, summarizing, analyzing, and presenting information cannot be dedicated to other projects. Yet, budget constraints often dictate tradeoffs that must be made for marketing research projects, and often these tradeoffs are made carelessly.
Two obvious ways companies scrimp on their research budgets are selecting a vendor solely on the basis of cost and deciding to perform the research in-house. Companies also make tradeoffs by using smaller sample sizes for surveys, choosing nonprobability over probability sampling methods, or opting for secondary over primary research, among others.
None of these tradeoffs is inherently bad. Indeed, when budgets are scarce, you may need to make several in order to balance the scope of your project against your budget constraints. A decision based on a little good marketing research is still more solid compared to totally unaided judgment. But the key word here is good. When cost becomes the overriding constraint for marketing research, companies run the risk of throwing the baby out with the bathwater.
Pitfalls of in-house research using inexpensive survey tools
Generally, when conducting a survey, you want to choose a sample that adequately represents the population in which you're interested. If your company is marketing a product or service to low-income Hispanics, conducting an inexpensive online survey is either going to result in several respondents who don't fit that demographic, or so few responses, as many low-income Hispanics are unlikely to have Internet access. Yet many companies make use of inexpensive online survey tools like SurveyMonkey for the very reason that it is inexpensive, and the results they obtain are useless because either the wrong people or too few of the right people respond.
Pitfalls of using nonprobability samples over probability samples
Companies have also tried to cut marketing research costs by substituting nonprobability for probability samples. Respondents in a nonprobability sample are chosen solely on the basis of judgment, unlike probability samples, which are chosen at random with each member of the population having an equal chance of selection. When someone at a shopping mall or on the street asks you to take a survey, you're being selected for a nonprobability sample.
There is nothing wrong with using nonprobability samples; surveys using such samples can be executed rather quickly if time is an issue. Furthermore, researching populations that are quite small and scattered (e.g., recruiting persons with a rare disease for a clinical trial) or where few published directories - the sample frame - about its members exists (e.g., medical coding professionals), can make probability sampling cost-prohibitive and unfeasible. However, generalizing results from a survey administered to a nonprobability sample to the true population can be difficult and highly error prone. As long as you understand this drawback and make allowances for it, you will be OK.
Pitfalls of using smaller sample sizes
Sample size is another popular way companies attempt to cut marketing research costs. Assume that a local pizzeria wants to explore the feasibility of offering delivery. The pizzeria needs to estimate the amount a household within its ZIP code spends on a typical pizza delivery order, and randomly selects 100 households from within the ZIP code for a survey. Further assume the pizzeria wants a five percent margin of error. The survey is executed and the pizzeria finds that the average household surveyed spends $15 on a pizza delivery. Figure in the plus or minus five percent, and the estimated average is between $14.25 and $15.75.
The problem emerges when the pizzeria tries to generalize this average delivery order to all households within that ZIP code. A sample size of 100 with a five percent margin of error is barely a 65% confidence interval. That means the pizzeria can be only 65% confident that the true average pizza delivery order in that ZIP code is between $14.25 and $15.75. If the pizzeria concludes that that order size is too small to justify offering delivery, but the true average turns out to be well above that range, the pizzeria has left money on the table. On the other hand, if the pizzeria concludes that the order size is large enough to justify offering delivery, but the true average falls below that range, it risks adding an unprofitable service.
The optimal sample size is the one that gives you the highest level of confidence and the most tolerable error margin required for you to make an objective marketing decision.
How to keep marketing research costs low while keeping quality high
When it comes to getting the most bang for your research buck, use the Pareto Principle as a starting point. The Pareto Principle, also known as the 80/20 rule, states that about 80% of your results will come from roughly 20% of your efforts. So, 80% of the information you need to make your decision will come from just 20% of the research questions you ask. Find out what those questions are and be sure they are asked. This process alone will help optimize the scope and cost of your research project.
Also, decide how much confidence you need and how much error you'll tolerate. Some decisions require more accuracy and/or confidence than others. But your sample size changes in proportion to these accuracy and confidence needs. The best way to start is by asking, "How much precision would I gain using a 95% vs. 90% confidence interval, and would that precision justify the extra cost to get it?" A 95% confidence interval with a five percent error margin requires a sample of 384 people, while a 90% confidence interval with the same margin of error requires a sample of just 271. Surveying those additional 113 people can add a couple thousand dollars to the cost of your project. Is the gain in precision worth that additional cost?
Also, does your project really need a probability sample? Not every research project does. If the purpose of your research is exploratory, you generally need neither a large sample nor a randomly generated sample. If you sell healthy meal solutions and want to understand issues busy moms face when trying preparing to prepare healthy meals for their children, you might simply run an ad in a local paper to recruit maybe 10 or 15 of those moms to either participate in a focus group or an in-depth interview. This qualitative information can be sufficient on its own, or can lay the groundwork for a future larger, quantitative (not to mention probability sample) study.
Thoroughly understanding your business problem will give you the best idea of the scope your research requires, the precision you need, and the money you should budget. If, after determining the necessary scope and precision, you find that the project is going to be prohibitively expensive, you can do one of two things. One alternative is to look at all the information you are seeking to collect. Then prioritize them in terms of their benefit to your marketing objectives. Those parts of the research project that will add the most value should be undertaken; the remainder can be delayed until funds or time are available. The other alternative is to do a cost-benefit analysis of the entire study. If you had only $20,000 budgeted for the study, but you find it will cost you $35,000, weigh that cost against the value of the insights. If, after the study, you are able to make decisions that increase sales - or reduce marketing costs - by more than $35,000, then it makes sense to make the case for more funding.
The expression, "you get what you pay for" rings especially true for marketing research.
Alex J. Caffarini
President - Analysights, LLC