In an ideal survey research world, it is preferable to work with a single probability-based sample as it provides the best representation of the target population. In the real world, however, cost and feasibility often prohibit the luxury of using purely probability-based samples. This is where different sampling methods come into play to reduce cost and improve feasibility, especially those that rely on online panels. All in all, online sampling isn’t ideal, since such samples are void of “organic” representation. If you can’t get generalizable results from your surveys, then what’s the point?
A blended (hybrid) sampling approach can offer an effective and practical alternative, through which multiple frames are used for sample selection—oftentimes a combination of probability-based and convenience samples from online (opt-in) panels. Further, we might start with a fully probability-based sample from a telephone or address frame, but then tap into online panels to supplement what we get from the main probability sample.
Taking a hybrid sampling approach sounds well and good, but just because you’ve gone hybrid doesn’t necessarily equate to unbiased survey results. Sampling from online panels is always a little tricky because if you don’t know what you’re doing, you can end up taking a seemingly inexpensive sample component, mix it with your precious probability-based sample and end up with a poor combination.
Sure, theoretically it’s preferable to have all or most of the samples be probability-based, but they are expensive. At the same time, you don’t want samples from opt-in panels dwarfing your precious probability-based sample. As a general rule of thumb, something on the order of no more than 50% of your sample should be coming from opt-in panels. Keep in mind that budget and other factors may dictate a higher or lower contribution.
The selection of samples from opt-in panels needs to be carried out sensibly. Equally important is the way you blend the probability and nonprobability-based sample components to produce a single database capable of producing reliable conclusions. It’s a little bit like chemistry when different materials are tossed into the mix to produce an alloy with higher-level properties; you have to be measured about it and get the ratios down just right using correct weighting and calibration adjustments.
As response rates continue to decline into single digit territory, even with fully probability-based samples, geodemographic weighting of survey data becomes essential. This is proven true since nonresponses are always different in nature. However, this issue will magnify with hybrid sampling when part of the sample may come from opt-in panels. Hence, in addition to basic weighting, additional calibration adjustments become necessary as well. This means going beyond geodemographics and applying corrections based on attitudinal and behavioral characteristics to ensure respondent representation for their population.
If you are looking to enhance your phone or address-based surveys and supplement them with samples from online panels, survey research scientists at MSG have decades of knowhow and hands-on experience to support your hybrid sampling methods. Our experts can assist you with sample selection, survey administration and questionnaire design, as well as state-of-the-art weighting and calibration procedures. Additionally, we can support you with reporting and analysis of data from complex surveys.
To learn more about our hybrid sampling products and services, click here, or contact one of our specialists.
For a deeper dive, watch Episode 06 of our Coffee Quip YouTube series, wherein the panelists discuss the intricacies and benefits of Hybrid Sampling!
