IPS - Day 34

Today's focus was on bias. I like to start off the discussion of bias with a look at a situation involving target shooting with a bow and arrow. I provide three type scenarios of missing the bulls-eye and ask students to consider what is error and what is bias.

The common perception of error is, many times, a statistical bias. And, statistical error is often not viewed as error, since this term is commonly associated with the idea of making a mistake.This helps students to understand that in statistics, an error is a natural occurrence of random fluctuation while bias is a systemic variation that pulls us off target from the population we are interested in studying.

I asked students to consider how bias may arise. They discussed this in their groups and we shared out as a class. The results included response bias from question wording, topic, interviewer and undercoverage.

I stepped through various bias, giving examples and firsthand experiences when applicable. The bias topics covered were:

• voluntary response bias
• convenience sampling
• undercoverage
• non-response bias
• response bias

Voluntary response bias occurs when a survey is offered and all responses made to the survey are counted. There is no attempt at structuring the sample. A good example of this are web site pop-up surveys. The pop-up is offered and anyone willing to respond is counted. Those who have a strong opinion, either positive or negative, tend to respond.

Convenience sampling is taking a sample through the path of least resistance. Whatever offers itself as the easiest way to gather data is taken. In business, customer surveys are often conducted by surveying the best customers because they are the ones that are most likely to respond.

Undercoverage occurs whenever a group within the population is either not sampled at all or enough. For a school survey, obtaining a sample of 80 students and then finding only 5 (less than 10%) of the sample is seniors would be an example of undercoverage of this population segment.

Non-response bias occurs when someone is asked to participate in the survey and refuses to respond. This is a wide-spread problem. When I first started working in marketing research, non-response rates were in the neighborhood of 20%. Today they are closer to 70%. This is a huge issue that is being extensively researched. The issue is that once someone doesn't respond there is no way to know why or what their characteristics are to adjust for the non-response.

Response bias results from any issues with the survey instrument itself or in the survey's administration. The length of the survey, the wording of the questions, the attire of the survey administrator, or the survey topic can all cause response bias. I participated in a survey one time in which the surveyor basically answered all the questions for me. This was response bias. 

We finished class by having students work through some questions about sampling designs and questions. They were to identify which sampling plan would have the least bias and which questions needed to be re-worded in order to reduce bias.

We'll go over these in class and then cover the idea of observational studies and experiments as two more ways to gather data.

Visit the class summary for a student's perspective and to view the lesson slides.