Your surveys matter, which is why you craft those surveys so carefully, taking to craft well-thought-out questions, vetting any survey groups or panels, and selecting the right platform. You know the data gathered from a survey can produce results that change your business direction and dictate future success.
But despite all of that preparation and attention to detail, things can still go wrong.
One of the main challenges? Response bias. But what is response bias?
Survey Response Bias
You need to know exactly what response bias is, what causes it, and crucially, how to avoid response set bias in your surveys.
In this article, we cover it all.
What is Response Bias?
We should begin by defining response bias. This term refers to the various conditions and biases that can influence survey responses. The bias can be intentional or accidental, but with biased responses, survey data becomes less useful as it is inaccurate. This can become a particular issue with self-reporting participant surveys.
Bias response is central to any survey, because it dictates the quality of the data, and avoiding bias really is essential if you want meaningful survey responses. Leading bias is one of the more common types. An example would be if your question asks about customer satisfaction, and the options given are Very Satisfied, Satisfied and Dissatisfied. In this instance there is bias that can affect results.
To avoid bias here, you could balance the questions by including two of each of the positive and negative options.
Why Does Response Bias Matter?
Why Does Response Bias Matter?A survey is a powerful tool for businesses because it provides the ability to obtain data and opinions from real members of the target audience, which gives a more accurate assessment of market position and performance than any trial-and-error tests could ever produce.
When the whole idea of the survey is to get accurate data that can be used to make business decisions, having accurate data really is the thing that matters. Bad data can create poor insight and bad decisions from product output to overall business strategy and everything in between.
The consequences of this could be catastrophic for any organization.
Because important decisions will be made based on the survey outcomes, it is essential to get the survey right. The data that is generated is only useful when it is accurate and truthful. Indeed, making such critical decisions on inaccurate data or false information could lead to strategic errors being made that are potentially catastrophic. That is why avoiding response bias is so important to any organization.
What Types of Response Bias Are There?
One of the key things to avoiding response bias is to fully understand how it happens. There are several types of response bias that can affect your surveys, and the ability to recognize each one can help you avoid bias in your surveys as you create them, rather than spotting it later.
However, even with this understanding, it is always wise to have several people go through any survey design to check for possible causes of response bias before any survey is sent to respondents. This ensures that the resulting data is as accurate as possible.
We will cover the main types of response bias here, and we will provide examples of response bias to show just how easy it is to introduce bias within the survey.
Demand Characteristics
One of the more common types of response bias, demand bias, comes from the respondents being influenced simply by being part of the study. This happens as respondents actually change their behavior and opinions as a result of taking part in the study itself. This can happen for several reasons, but it is important to understand each to be able to deal with each form of this response bias problem.
- Participants who look to understand the purpose of the survey – For instance, if the survey is looking into a user experience of a website, a participant may see that the aim is to gather data to support making changes to layout or content. The participant may then answer in a way that supports those changes, rather than as they really think, resulting in untruthful or inaccurate responses.
- The setting of the survey or study – This is more applicable to surveys carried in person, where researchers conducting the survey can have an influence on the respondents, but it can apply to digital surveys too.
- Interaction between researcher and respondent – This can influence how the survey is approached. Note that if it’s a digital survey that researcher-to-respondent interaction is still possible, occurring in the email or message used to invite the respondent to participate.
- Wording bias can come into effect here as well. This type of bias influences the entire gamut of responses from individual or multiple participants. For instance, if the researcher knows the participant personally, even greeting them in a friendly manner can have a subconscious effect on the responses. This is as true in an email as it is in person, so by retaining a formal approach to all participants regardless of who they are you can ensure a uniform response from all participants.
- Prior knowledge of the survey – Whether the questions themselves, or the general aims of the survey, or how it is being put together, prior knowledge of some aspects of the survey deliver response bias. This is because the participants can become preoccupied with the survey itself, resulting in those participants second-guessing their own answers and providing inaccurate responses as a result.
Social Desirability Bias
This type of response bias results from participants answering sensitive questions with socially desirable, rather than truthful, answers.
The key here is how response bias questions are worded. To better illustrate this, here is an example:
“Do you think it’s OK to drink alcohol frequently?” is a question that society conditions us to say no to, even those who do drink heavily. A question like this cannot get a truthful response.
People know the appropriate answer and will give it regardless of their real view.
In fact, social desirability bias can work both ways. Depending on the question you can find both over-reporting and under-reporting of a particular viewpoint, and that can have a dramatic affect on the usability of the data eventually generated.
Extreme Responses
The name gives this one a way really, but it refers to the response bias that is characterized by respondents providing extreme answers to questions. This can manifest as extreme positive or negative responses, and both render the data ineffective.
This type of bias occurs most commonly in surveys that offer a scale for responses in order to rate individual components, whether that is numbers (such as 1 to 5, star ratings) or even a selection of statements (such as satisfied, mostly satisfied, somewhat dissatisfied, and dissatisfied).
What happens is that the answers tend to be either 1 or 5, 1 star or 5 stars, satisfied or dissatisfied. Few people choose the middle options. Through various response bias studies, it has been shown that there are cultural influences on this kind of behavior.
Culture is of course not the only cause of this behavior. Studies show that education level also matters. The lower the education level, the more likely the participant display extreme response traits.
Lastly, wording bias can also be a cause. Sensitive questions that offer blame to someone or something for a difficult situation will result in extreme responses too.
Think about the following question:
“Are family and peers partly responsible for a substance abuser’s behavior?”
There is no middle ground there for most people, and responses will be at the extreme end of whatever options are provided. While sometimes sensitive questions need to be asked, carefully structuring them to avoid the instant emotional response can avoid the extreme responses we are talking about.
Neutral Responding
This type of response bias is the exact opposite of extreme responding, as here the participant chooses the neutral answer every time. This is usually a result of the participant not being interested in the survey at all and is simply looking to answer questions as quickly as possible.
This inaccurate data is just as damaging and highlights just how important selection of participants can be for your research.
Acquiescence Bias
Acquiescence bias is a form of response bias where participants respond in agreement with all questions within the survey. In most cases, if your survey is well designed, that results in the participant agreeing with at least two contradictory statements. The answers provided this way are then no longer accurate or truthful.
The reality is that all of us have our unique view of the world, and it is highly unlikely anyone will agree with everything on a survey. To figure out how to combat this type of bias, let’s look first at how this kind of bias occurs.
There has been much research carried out on this kind of response. American educational psychologist Lee Cronbach theorized that it is a result of the participant looking at the question and actively searching for information from their own experiences to support it. Others have suggested it stems from a need to please researchers, but Cronbach’s approach is considered the most relevant.
As an example of this, a survey may contain two questions:
“Are you outgoing and social?”
“Which do you prefer: a quiet night in or a raging party?”
A respondent could answer yes to the first question, because we all like to think of ourselves as likable, social people. However, when faced with the second question, they may remember a nice night in they had enjoyed recently and answer with that. Those two answers may seem to contradict each other. Both, however, would be genuine, truthful answers from the participant’s perspective.
Dissent Bias
Finally, we have dissent bias. This is the exact opposite of acquiescence bias where the participant seeks to disagree with every statement or question the researcher makes.
With the same causes, dissent bias can be just as problematic for researchers as acquiescence bias, and it must be remedied in the same way.
How Do You Get Rid of Response Bias?
With the ability to recognize the types of response bias we encounter, and why they occur, we must also develop methods to avoid our own research being damaged by this issue. There are several things that every researcher can do to overcome response bias. Let’s me show you.
Understand Your Demographic
As we discussed previously, some demographics are more susceptible to certain types of bias. Paying attention to the ‘who’ you are asking is as important as ‘what’ you are asking.
You can put demographic understanding at the top of your research by asking yourself a few simple questions:
- What does my audience have in common?
- Why will they want to answer my survey?
- What is it about them I am interested in?
When you have the answer to those questions, you will have a better understanding of your audience and the type of questions you should be asking them.
Avoid Question Wording Bias
Careful question phrasing is critical to get an unbiased survey and responses. We talked before about inherent bias, but this also applies to questions that produce an emotive response and can cause acquiescence bias or extreme responding.
Follow these simple survey best practices to avoid word bias in your surveys:
- Look at the questions and ensure there is a balance
- Include as many positive responses as negatives
- Seek to avoid questions that will produce a powerful emotional response one way or the other
- Always look for phrasing that provides acceptable positive or negative answers
Diversify Questions
To keep participants focused, avoid using one type of question all the way through. Instead, mix binary response questions (yes or no) with those that offer a range of responses. This avoids the participant simply giving the same answer each time, and thus forces them to think about their responses.
This applies to more than just the style of question. Avoid asking questions about the same topic one after the other, as this can lead to reflex answers given without thought. Instead, mix up topics throughout the survey, so that the participant is not just automatically following what they answered last time and has to think about each question on its own.
Allow Participants to Say ‘No’
Participants often wish to give the best answer possible, but for some questions a participant may not have an answer. Feeling that they need to do something, they at times will give an inaccurate answer that you don’t want.
Having a ‘don’t know’ or ‘undecided’ option allows them to answer honestly without skewing your results. In fact, this minor addition to your survey could remove a lot more inaccurate answers than you think.
In addition, the ‘don’t know’ option can be a valuable data set of its own and can prove useful throughout subsequent analysis.
Effective Administration
As we have seen, several forms of response bias can be caused by participants being influenced by the researchers themselves, or the motives behind the survey. This makes every aspect of the administration of the survey crucial to its success.
The following are important steps every survey should follow:
- Researchers must remain neutral at all times – During all correspondence of any kind, maintain a professional, unbiased demeanor to ensure participants recognize the importance of the situation. Using the digital approach can remove some of these risks.
- Maintain survey integrity – Participants second guessing research motives or finding out motives before taking the survey both result in response bias. By maintaining the integrity of the survey and ensuring participants do not have additional information the risk is minimized. To ascertain if participants have any understanding of the survey motives, a short after-survey questionnaire can be used.
Avoid Emotionally Charged Terms
Your questions should be clear, precise, and easily understandable. That means simple, unbiased language that avoids using words that evoke an emotional (rather than through-based) response.
This includes words such as:
- Politician
- Terrorist
- Junkie
And so on. You want answers that are thought through, and these so-called ‘lightening’ words instead elicit an emotional response that is not as valuable for your research.
In addition, try and avoid using a lot of negative words, as these can have an effect on how the participant perceives the survey and skew their responses.
Finally, avoid word tricks. Be transparent with the audience and allow them to develop their answers rather than be guided to them.
All of these are relatively simple steps to deliver improved survey results. Removing response bias will help you acquire accurate, unbiased data that can be converted into informative, actionable insight and reflect the real views of your audience.
What Survey Tools Should Your Company Use?
Even knowing the various types of response bias your survey could be affected by, you still need to check and monitor the survey to check for problems and avoid inaccurate data.
Nextiva has two tools designed to do this for you, providing extemporary performance combined with ease of use for seamless integration into your workflow.
Nextiva Survey Analytics
Survey analytics provides business intelligence efficiency with a comprehensive feature set that tracks survey response data throughout your research. This provides simple, clear, visual presentation of the data you need.
Through this simple interface you can drill down to get a complete performance picture, analyzing results right down to the individual respondent if necessary. All data is easily accessible, saving time and frustration.
With a visual representation of aggregate responses to any question, you can quickly identify trends and anomalies as they are occurring.
Nextiva Surveys
A complete software solution for all your surveys, Nextiva Surveys provides the perfect platform for all your research. With a simple, fast design solution, your surveys will look great. And full customization ensures they always reflect your brand image.
With no coding required, you can get beautiful, rich surveys put together in just a few minutes, saving time and money without sacrificing quality or control.
There are templates for all types of questions, complete security, and various features including skip logic for the personalized experience that respondents love. Nextiva Surveys has all the tools you need to create response bias-free, effective surveys that produce exceptional results.
Platform free, you can deliver surveys in an email or via the web. The responsive design provides an exceptional user experience even on mobile devices.
When it comes to response bias-free surveys, Nextiva has your back. With our flexible, easy-to-use survey software and complete data analysis solution, creating beautiful, effective surveys that avoid response bias are just a few clicks away. Try them now and see how easy surveys can be.
Republished by permission. Original here.
Photo via Shutterstock
This article, “What is Response Bias and Why Does It Matter When Doing Business Surveys” was first published on Small Business Trends