Smart Survey Design: 3 Forgotten Pain Points to Avoid

“Smart Survey Design” is a loose term (bordering on a catch-all) that you’ve probably heard pitched to you.  Maybe you have used it yourself when piecing together a study.

It’s not a hollow term, by any means. Smart design has advantages for both designers and respondents. Designing “smart” simply means maintaining data integrity, both in terms of capturing statistically relevant data as well as reducing the amount of bad data caused by poor survey takers (straight liners, short responders for OE’s, speeders, cheaters, etc.).

That’s the basic idea, but there is one factor that often gets forgotten or ignored in a “Smart” design:  the respondent’s experience. You want your respondents to have a positive user experience, surveys with a human touch. They should feel good about taking the survey.

I’m not just talking about survey length or incentive, though those are certainly key tools in addressing the problem.  What I am referring to is the very way we talk to the respondent, the questions asked and how many times we ask that question.

It is easy for us as researchers to become so lost in our need for quality data that we forget the source of it—human beings. People are rational and emotional creatures. How do they feel about their participation?  It’s an important consideration, all too often ignored.

Identifying and avoiding potential pain points may not only help to reduce the number of scrubs and drop-outs, but also deliver better, more reliable data.

Have you ever been on a conference call where the speaker repeats the same point 5 times?  Did you like it?  Did you continue to pay attention or did you look at your phone or check your email?  Now imagine that same conference call. The speaker drones on with 4 more points that are roughly one hair’s width different from the original ones. Frustrating!

Plenty of studies out there get too repetitive in hopes of garnering nominal, ordinal, interval, and ratio data just to present the client with 4 different charts.  But you should ask yourself, how reliable are the opinions offered by a respondent that you have just bored and or annoyed?

Some repetition may be unavoidable, especially when you want to determine which of a group of stimuli is most attractive to your target, but you should not bludgeon the people who are meant to be helping you.

Pain point #2: Being too clever

“If you could be a tree, what tree would you be and why?”

This may be a good opener for your therapist to crawl around the workings and motivations of your mind, but some respondents may find such questions to be intrusive or something worse: “hogwash.”  They have signed up to take part in survey research, but they’re not lab rats!

We come back to the reliability question: how reliable is the data you are gathering if your respondent has been made uncomfortable and just wants to finish the ordeal and get out?

The prospect of getting “deeper data” out of your survey may be very alluring, but consider how appropriate those questions are for your audience.  Does a panelist really need to imagine their favorite restaurant as a spirit animal in order to tell you what their favorite sandwich is?

Pain Point #3: Being too “research-y”

While gathering data or even when trying to cut time off the length of interview in consideration for the respondents, questions might be presented impersonally or curtly. These rapid-fire “cold” questions, though absolutely focused, clear and concise, run the risk of boring a respondent into unintentional mental lethargy.

Questions can eliminate responders who have lost interest in your data set, but wouldn’t it be more beneficial to prevent the need for creating them in the first place?  You don’t have to write a narrative or tell a knock-knock joke to keep them engaged with the process.  Panelists are people. You should just remember to “speak” to them conversationally, instead of clinically prompting and probing for responses.

By being more aware of the respondent’s pain points and making a few tweaks to your surveys, you can improve completion rates, quality of open-ended responses and data integrity.  Better yet, it does all this without incurring any additional costs.

Remembering Dale Kulp

On what would have been Dale Kulp’s 66th birthday, we wanted to take a moment to remember a man who not only is responsible for the founding and creation of Marketing Systems Group but made innumerable contributions to the statistical sampling and survey research fields.  Dale’s career was already flush with accomplishment before he founded Marketing Systems Group in 1987.  He previously had worked for industry stalwarts Chilton, Bruskin and ICR (Now SSRS).  With MSG, he envisioned the development of a PC based in-house RDD sample generation system (GENESYS) that would become the cornerstone product of the company.

Aside from being the driving force behind the industry’s first in house sampling system, Dale was integral in the development of list-assisted RDD sampling methodology at a commercial level, which revolutionized the process for reaching probability-based samples of households. Through his many technical notes and various publications he remained vigilant about addressing the operational issues challenging the viability of this methodology, particularly those resulting from the unfolding changes in the US telephony.

Dale also started several Omnibus telephone surveys that not only continue to thrive 20 years after their launch; they have in at least one situation created their company.  Centris Marketing Science was created by Dale along with Paul Rappaport after realizing the value of the census block level data that the Omnibus survey collected.

Realizing that MSG should not be a one-trick pony, Dale continued to pursue other product lines that would benefit the survey research industry.  He assembled a team that included current MSG President Jerry Oberkofler and Vice President Reggie Blackman to develop the first automated screening process: GENESYS-ID.  Utilizing the technology and philosophy of GENESYS ID and applying it the survey research industry, PRO-T-S was born.  PRO-T-S was the first predictive dialer built exclusively for the research industry.  In 2004, Dale brought the ARCS Panel Management software under the MSG umbrella. ARCS is now one of the leading software packages in the sensory and pharmaceutical industries as well as a recruitment tool for large civic organizations.

Since Dale’s passing in late 2009, MSG has grown substantially but has remained attached to the vision, products and protocols that Dale Kulp laid out back in 1987.  Not only do the MSG folks wish Dale a Happy Birthday but we thank him for his vision, contribution and foresight.

TCPA Compliance

My name is Tim Antoniewicz and I am not a lawyer.  I am not legal counsel.  I once played Clarence Darrow in a junior High School production of Inherit the Wind but what I am about to say should in no way be considered legal advice.  However it may be considered helpful in sorting out the quandary that many researchers face when conducting research with a cellular sample frame.

In light of the new FCC regulations (6/18/15) that expand the TCPA, CASRO has provided the following guidelines:

The Federal Communications Commission approved new regulations that expand the Telephone Consumer Protection Act (TCPA). The new FCC rules broaden restrictions on autodialed calls to cell phones without differentiation for caller intent. 

ALL calls to cell phones made using an auto-dialer are PROHIBITED.

DO:

ALL calls to cell phones should be manually dialed.

Regularly update number databases to identify numbers ported to cell phones.

Don’t:

Use a predictive dialer OR an auto-dialer to call cell phone numbers.

Assume that human proximity to, or intervention in, the placement of an autodialed call to a cell phone provides exemption from the TCPA.

Staying educated and taking proactive measures are the key to compliance with the law. Here are some high-level guidelines to follow:

  • Mitigate the risk by verifying the types of phones numbers on your list
  • Get the consent of the current wireless subscribers to a number
  • Be aware of ethical considerations including respondent safety and privacy.

Marketing Systems Group can assist you in flagging likely wireless numbers, identifying ported landline-to-wireless numbers and real-time screening of active cell phone numbers.

MSG is committed to helping you navigate the ever changing regulation landscape from the TCPA and remain in compliance while accomplishing your research goals.

For more information about TCPA compliance, see http://www.tcpacompliance.us/

MSG in a Bottle

Welcome to our brand new blog for customers and industry observers. We’re calling it MSG in a Bottle, and I know you’re going to love following it.

One of the best things about my job as president of Marketing Systems Group is the opportunity to work with an inspiring, committed team of professionals. They constitute a brain trust of talent and experience. Their collective market research expertise and dedication to quality truly makes a difference across our entire product line — GENESYS® sampling system , PRO-T-S® dialer software and ARCS® all-in-one panel manager.

Since 1987, we’ve been delivering innovative solutions to the survey research community, and our staff continues to do amazing things, year over year.

I’m proud of these professionals. They are what make our products great. We learn from each other every day.

That’s all well and good for a company president, you might be saying, but what about me?

That’s why we’ve started this blog. We want you to benefit from the collective wisdom I see in action every day at MSG. I’ve asked this talented team of pros to share their insights and expertise with you.

At the MSG in a Bottle blog you will get our expert analysis of hot industry trends, stay informed with news on the survey research industry and the latest standards updates, learn how we’re positioning our products to meet customers’ needs, and get practical advice and tips on how best to use our products. Think of it as “news you can use.” Also, we hope to have some fun along the way, too. We want you to get to know us better. And we want to hear from you too. You’ll be able to join the discussion and share your feedback and suggestions via the comments section after each blog post.

I’m excited about this new channel for reaching our customers and the survey research community, and I’m confident that our media and marketing team will keep you up-to-date on the survey research industry topics that truly matter. We hope you’ll bookmark us and stop in frequently. New pieces will appear on a bi-monthly basis. Or better yet subscribe/follow us here, and never miss a new post when it arrives.

Thanks and happy reading,

Jerry Oberkofler