Polling: Advantages and Disadvantages to the Six Response Gathering Options

Currently, Cor Strategies and our partners at VictoryGeek use the following six methods to obtain responses when polling for our clients across the country:

  1. Automated Dialing
  2. Live Dialing
  3. Texting
  4. Online
  5. Mail
  6. Panels

Ideally, when executing a poll, we combine multiple methods to minimize methodology skew and maximize the poll’s confidence level.

Watch Collin Corbett and Cor’s Data Wizard Abe Levy’s overview of each response gathering method and the advantages and disadvantages of each:

 

Automated Dialing (aka IVR, Typically to Landlines)

This has existed since before our data team was even born. But today it’s hard to find a non-senior household that owns a landline, let alone would answer an unknown caller. Other than the rare instances where an entity has permission to automated dial cell phones (i.e. a unit of government), automated dialing has rapidly dropped off in usage, for good reason. That said, we continue to test this method by comparing automated results to responses gathered via other methods, and shockingly, the automated results still come very close to the other results once weighted. Automated gathering isn’t completely dead (yet), which is good news because this remains the cheapest option for gathering responses.

Advantages: It’s an inexpensive way to gather responses. Confidence level is surprisingly strong, though getting less reliable as the years go on and eventually this method will be obsolete.

Disadvantages: If you rely exclusively on automated gathering, you will likely end up with very skewed results. A majority of responses will come from seniors, and oversampling the 18-64 age demos to combat this skew is becoming increasingly difficult due to a dwindling number of landlines, especially since you can’t respondent match when calling a household.

 

Live Dialing (Typically to Cell Phones)

With cell phone usage at its peak, live dialing continues to be the most effective method for gathering responses. This method leads to high levels of accuracy and the maximum confidence level. It also has the advantage of being able to match respondents to your dataset. It is, however, the most expensive methodology due to the labor costs of completing the dialing. This is why many pollsters use live dialing for a portion of their overall response gathering plan, typically between 20%-40%. This increases the accuracy and confidence level of your poll while keeping costs down as much as possible.

Advantages: With the ability to dial cell phones, this is a very reliable method to reach younger and middle-aged respondents. With the ability to respondent match and ensure respondent comprehension, confidence level is the highest with this method. It’s also a lot easier to cluster samples based on various targets.

Disadvantages: It’s very expensive due to labor costs. This method is also slightly slower, depending on your budget for live dials.

 

Texting (Text-to-Online, Text-to-Automated, and Text Polling)

Texting for responses is a relatively new solution to gathering responses from cell phone users. There are multiple options, including conducting the entire poll over text, directing respondents to an online poll, and even obtaining an opt-in from a cell phone user via text to then poll them via an automated system. The challenge for this method has been participation—until recently, responses for this method were often capped due to the difficulty of getting people to engage. But as people are becoming more comfortable with this polling method, participation is increasing. This is excellent news, because texting is less expensive than live dialing.

Advantages: This is the least expensive way to gather highly accurate responses from cell phone users. Also, technologies are improving as this becomes more mainstream.

Disadvantages: Because this is often a multi-step process, extra precautions must be taken to safeguard the accuracy of your results. Participation is increasing but still often too low to conduct entire polls via this method.

 

Online

In an increasingly connected world, your target audience is always online. Even on their cell phones—many people use their phones for everything but texting or making calls these days. Online polling is an inexpensive way to reach your audience to generate the responses you need. This method allows your audience to respond on their time, which increases the participation rate. However, the results sometimes vary wildly—while this method can often achieve accurate results, it has a much lower confidence level because of too frequent instances of skewed results.

Advantages: This method is relatively inexpensive. It’s more convenient for participants which tends to result in solid participation rates. If done correctly, responses can be gathered fairly quickly.

Disadvantages: Accuracy for this method is the lowest of all response gathering methods. Respondent match is difficult and typically impossible to achieve.

 

Mail

Mail polling can be conducted on paper, with respondents mailing back their responses, or online, with respondents visiting a direct link to fill out the survey. Mail polling isn’t the most accurate of response gathering methods, but that may be acceptable if your main goal is to maximize participation, as mail surveys generate very high response rates. This method also allows you to provide much more information to your respondents, giving you a more informed response pool. The drawbacks are the reduced accuracy and confidence level of the poll, which can partially be overcome with appropriate weighting. Another challenge with this method is the cost, which can be as much or sometimes even more than live dialing.

Advantages: This method achieves the maximum response rate, so while it struggles to achieve fully scientific results, there are situations where that trade-off is worth it.

Disadvantages: The reduced accuracy isn’t always acceptable, so that must be considered. Also, mailing anything carries significant costs when you add up design, print, and postage. Additionally, this method is the slowest, so you will need to plan out your poll well in advance.

 

Panels

Panels have historically been great for market research, and while some polling outfits relied heavily on panels, their usage wasn’t widespread until recently due to the challenges and costs of creating panels. But technology has made panels increasingly more accessible, providing another response gathering method. Panels are a low-cost option for gathering responses from your target demographics that isn’t impacted by the limitations of cell phones and landlines. While respondent match isn’t possible, you are able to easily target specific demographics or even characteristics, which is extremely beneficial. This method currently has geographic limitations, working best for national or statewide polling, though over time the ability to utilize this method to poll smaller geographical areas may be possible.

Advantages: Unless developing your own panel, which has a significant up front cost, this method is relatively inexpensive. It’s also the best method for polling audiences that fit specific demographics and/or characteristics.

Disadvantages: Panels have limited participation so far, so until that grows, you will struggle to gather enough responses from smaller geographical areas. Also, this being a newer service being provided, it’s important you fully vet your provider.

Polling Methodology Graphic

Between these six methods, you can poll anything. Want to know what issues truly matter to your target audience? We can poll it. Want to know the most persuasive way to present your message? We can poll it. Want to know your state’s favorite pet? We can poll it.

Reach out to our data experts to determine which polling method is best for your situation. Let’s start gathering responses!

You might also enjoy: