What is Non-Response Bias?

S8-SA1-0231

Grade Level:

Class 7

AI/ML, Data Science, Research, Journalism, Law, any domain requiring critical thinking

Definition

What is it?

Non-response bias happens when people chosen for a survey or study do not answer, and those who don't answer are different in an important way from those who do. This difference can make the survey results misleading, as they don't truly represent the whole group.

Simple Example

Quick Example

Imagine a school survey asking students if they enjoy extra classes. If only students who love extra classes bother to fill out the survey, the results will show everyone loves them. But students who dislike extra classes might simply ignore the survey, leading to a biased view.

Worked Example

Step-by-Step

Let's say a local cafe wants to know if customers prefer tea or coffee.
---
Step 1: They hand out 100 survey forms to customers over a week.
---
Step 2: Only 40 customers fill out and return the forms. This is a 40% response rate.
---
Step 3: Among the 40 responses, 35 say they prefer coffee and 5 say they prefer tea.
---
Step 4: The cafe concludes that 87.5% (35/40) of its customers prefer coffee.
---
Step 5: However, many tea drinkers might have been in a hurry and didn't fill out the form, while coffee lovers were happy to share their preference. If the 60 non-responders were mostly tea drinkers, the cafe's conclusion is biased.
---
Answer: The cafe's survey suffers from non-response bias because the 40 responders might not represent the preferences of all 100 customers, especially if non-responders had different preferences.

Why It Matters

Understanding non-response bias is crucial for making good decisions based on data. In fields like AI/ML, data science, and journalism, it helps ensure that surveys and data collection truly reflect reality, preventing wrong conclusions. This skill is vital for future data scientists and researchers.

Common Mistakes

MISTAKE: Thinking that if you send out many surveys, non-response bias won't be a problem. | CORRECTION: The number of surveys sent doesn't matter as much as who responds. A low response rate, even from many surveys, can still cause bias if the non-responders are different.

MISTAKE: Assuming non-responders have the same opinions as responders. | CORRECTION: Always consider why someone might not respond. Often, those who don't respond have different experiences or opinions, which is the core of non-response bias.

MISTAKE: Confusing non-response bias with simply having too small a sample size. | CORRECTION: A small sample size is about not having enough people overall. Non-response bias is specifically about the people who *don't* answer being different from those who *do*, regardless of the initial sample size.

Practice Questions

Try It Yourself

QUESTION: A mobile company sends an SMS survey to 1000 users asking if they are happy with their data plan. Only 100 users reply, and 95 of them say they are very happy. What bias might be present here? | ANSWER: Non-response bias. Users who are unhappy might not bother replying, leading to an overly positive result.

QUESTION: A school wants to know how many students use public transport. They put a survey form in the staff room. Only students who frequently visit the staff room (e.g., school leaders, club members) fill it out. Why is this survey likely to be biased? | ANSWER: This survey has non-response bias because students who don't visit the staff room often (who might be the main users of public transport) are less likely to respond, making the results unrepresentative.

QUESTION: A popular online food delivery app asks its customers to rate their delivery experience immediately after an order is completed. Customers who had a very good or very bad experience are more likely to give a rating. Explain how this can lead to non-response bias in the average rating shown on the app. | ANSWER: This leads to non-response bias because customers with 'average' or 'okay' experiences might not feel strongly enough to leave a rating. If only extreme experiences (very good or very bad) are reported, the average rating shown might not truly reflect the typical customer experience, either making it seem better or worse than it is.

MCQ

Quick Quiz

Which of the following is the best example of non-response bias?

A survey sent to 1000 people, but only 50 respond.

A survey about mobile games where only gamers respond.

A survey about food preferences given only to vegetarians.

A survey where the questions are confusing and hard to understand.

The Correct Answer Is:

Option B is the best example because it highlights that the non-responders (non-gamers) are different from responders (gamers) in a way that affects the survey topic. Options A is about low response rate, C is selection bias, and D is survey design flaw.

Real World Connection

In the Real World

When you see ratings for products on e-commerce apps like Flipkart or Amazon, or movie ratings on platforms, non-response bias can be at play. Often, only people with very strong opinions (either very happy or very unhappy) bother to leave a review, while those with average experiences might not. This can skew the overall rating you see.

Key Vocabulary

Key Terms

BIAS: A tendency to lean in one direction, often unfairly or inaccurately | SURVEY: A method of gathering information from a group of people | RESPONDER: A person who answers a survey or participates in a study | MISLEADING: Giving the wrong idea or impression | REPRESENTATIVE: Showing or including a typical example of a group

What's Next

What to Learn Next

Next, explore 'Sampling Bias' to understand another way survey results can be skewed. It will help you see how the initial selection of people for a survey can also lead to unfair or inaccurate conclusions, building on your knowledge of non-response bias.