Tag: survey design

Featured

The Enduring Power of Mail Surveys in Modern Research

On By MSG BloggerIn Address Based Sample, Genesys

In the realm of quantitative research, where unraveling the intricacies of human thoughts and behaviors is paramount, mail surveys stand as stalwart companions. Embedded within fields like political and social sciences, social work, and education research, mail surveys offer an essential avenue to explore the “why” and “how” behind human actions. Contrary to the digital age’s sway, these surveys continue to wield unparalleled effectiveness and significance.

The Resilience of Mail Surveys

Amid the proliferation of research methodologies, mail surveys remain a steadfast choice, consistently outshining online, email, phone, and in-app methods. The statistics gleaned from April 2018 data by Pew Research and industry experts reaffirm this preference. Response rates demonstrate the following order of performance:

The Factors Fueling Mail Survey Excellence

Mail surveys’ superiority finds its roots in various factors, each contributing to their continued success:

  1. Trust: Well designed mail pieces with geographic specific salutations instill trust, shunning the skepticism often associated with online communications deemed as spam. The credibility of receiving a tangible incentive gift stands firm against the virtual maze of conditions often attached to online offers.
  2. Deliverability: Physical addresses offer reliability in comparison to email addresses prone to frequent changes without forwarding information.
  3. Noticeability: Amidst the clutter of emails and online platforms, physical mail emerges as a beacon of attention in a less congested environment.
  4. Convenience: Respondents can engage with the survey at their convenience, with the physical presence of the hard copy serving as a gentle reminder to participate.

Upholding Data Integrity

The bedrock of any research endeavor is the integrity of the collected data. Inaccuracies and respondent bias pose significant challenges. However, the revered status of mail survey methodology as a vanguard against these issues prevails, even in the era of digitization. Phone surveys are marred by ‘sample selection’ bias due to the dwindling landline use. Email and online surveys encounter ‘social desirability’ bias, as respondents tailor responses to fit a crafted image. Even in-person surveys sometimes fall victim to guarded responses. While method selection hinges on factors like time, cost, and respondent information, the quest for unbiased data reigns supreme.

Cost-Effective Efficacy

In the landscape of costs, mail surveys shine as beacons of cost-effectiveness. Medium-scale surveys (with 5,000 to 50,000 respondents) in 2018 incurred an approximate cost of $5,000. Comparable phone and in-person surveys incurred costs ranging from 50% to 150% more, respectively. Email and online surveys tout the lowest price tags, beginning at $20 to $500 monthly, albeit subject to additional costs for custom programming. Yet, factoring in data quality, survey mailing services reign as the prudent cost-effective choice.

When to Choose a Mail Survey

The decision to embark on a mail survey journey holds merit under several conditions:

  1. Data Quality: When impeccable data quality is non-negotiable.
  2. Accessible Population Data: When equipped with a comprehensive list of names and addresses or planning to acquire a sample.
  3. Audience Relevance: When the survey content resonates deeply with your target audience.
  4. Time Flexibility: When immediate results aren’t a pressing concern.

The Way Forward: Balance and Integration

While the digital realm promises a radiant future for research, challenges remain. Biases, data integrity, and cost-efficiency cast a shadow on the exclusive embrace of digital surveys. Embracing a multi-modal approach, synergizing both print and digital components, seems to hold the key to harnessing the best of both worlds.

Conclusion: Enveloped in Excellence

In a world undergoing rapid transformation, the enduring prowess of mail surveys stands tall. Their resilience in yielding quality data, overcoming biases, and delivering cost-effective solutions continues to resonate across the landscape of research. As technology and methodologies evolve, the measured and purposeful integration of mail surveys in research endeavors promises to illuminate the path forward with both wisdom and innovation.

For more information on MSG’s suite of sampling solutions, or mail surveys in general, contact one of our experts here.

Hybrid Sampling: Why a Blended Sampling Approach Is a Sensible Option

On By MSG BloggerIn Hybrid Sampling, PreVisionLeave a comment

In an ideal survey research world, it is preferable to work with a single probability-based sample as it provides the best representation of the target population. In the real world, however, cost and feasibility often prohibit the luxury of using purely probability-based samples. This is where different sampling methods come into play to reduce cost and improve feasibility, especially those that rely on online panels. All in all, online sampling isn’t ideal, since such samples are void of “organic” representation. If you can’t get generalizable results from your surveys, then what’s the point?

A blended (hybrid) sampling approach can offer an effective and practical alternative, through which multiple frames are used for sample selection—oftentimes a combination of probability-based and convenience samples from online (opt-in) panels. Further, we might start with a fully probability-based sample from a telephone or address frame, but then tap into online panels to supplement what we get from the main probability sample.

Taking a hybrid sampling approach sounds well and good, but just because you’ve gone hybrid doesn’t necessarily equate to unbiased survey results. Sampling from online panels is always a little tricky because if you don’t know what you’re doing, you can end up taking a seemingly inexpensive sample component, mix it with your precious probability-based sample and end up with a poor combination.

Sure, theoretically it’s preferable to have all or most of the samples be probability-based, but they are expensive. At the same time, you don’t want samples from opt-in panels dwarfing your precious probability-based sample. As a general rule of thumb, something on the order of no more than 50% of your sample should be coming from opt-in panels. Keep in mind that budget and other factors may dictate a higher or lower contribution.

The selection of samples from opt-in panels needs to be carried out sensibly. Equally important is the way you blend the probability and nonprobability-based sample components to produce a single database capable of producing reliable conclusions. It’s a little bit like chemistry when different materials are tossed into the mix to produce an alloy with higher-level properties; you have to be measured about it and get the ratios down just right using correct weighting and calibration adjustments.

As response rates continue to decline into single digit territory, even with fully probability-based samples, geodemographic weighting of survey data becomes essential. This is proven true since nonresponses are always different in nature. However, this issue will magnify with hybrid sampling when part of the sample may come from opt-in panels. Hence, in addition to basic weighting, additional calibration adjustments become necessary as well. This means going beyond geodemographics and applying corrections based on attitudinal and behavioral characteristics to ensure respondent representation for their population.

If you are looking to enhance your phone or address-based surveys and supplement them with samples from online panels, survey research scientists at MSG have decades of knowhow and hands-on experience to support your hybrid sampling methods. Our experts can assist you with sample selection, survey administration and questionnaire design, as well as state-of-the-art weighting and calibration procedures. Additionally, we can support you with reporting and analysis of data from complex surveys.

To learn more about our hybrid sampling products and services, click here, or contact one of our specialists.

For a deeper dive, watch Episode 06 of our Coffee Quip YouTube series, wherein the panelists discuss the intricacies and benefits of Hybrid Sampling!

Quality Starts with Survey Design: Tips for Better Surveys

On By MSG BloggerIn ARCS, Genesys, MSGLeave a comment

Marketing researchers are all facing two important challenges to data quality. First is the question of representativeness: with response rates plummeting, we need to make surveys shorter, more engaging, and easier for respondents to complete. Second is the issue of data accuracy: we must make sure that survey questions measure what we think they measure. Continue reading “Quality Starts with Survey Design: Tips for Better Surveys” →

Smart Survey Design: 3 Forgotten Pain Points to Avoid

On By MSG BloggerIn ARCS, Genesys, MSGLeave a comment

“Smart Survey Design” is a loose term (bordering on a catch-all) that you’ve probably heard pitched to you.  Maybe you have used it yourself when piecing together a study.

It’s not a hollow term, by any means. Smart design has advantages for both designers and respondents. Designing “smart” simply means maintaining data integrity, both in terms of capturing statistically relevant data as well as reducing the amount of bad data caused by poor survey takers (straight liners, short responders for OE’s, speeders, cheaters, etc.). Continue reading “Smart Survey Design: 3 Forgotten Pain Points to Avoid” →