Ultimate and honest reporting of selection and confirmation bias

It’s been a long time since grading undergraduate lab reports. Prior to this term, I want to say it was Fall 2006, which was the last time I taught a UG course with a lab section where I did not have TAs teaching the labs.

It’s a pleasure to read them and help mold the next generation’s perspectives on methods utilized to generate knowledge. This particular opportunity has become a teaching moment on selection bias and confirmation bias combined.

Of a sample of 21 students, 6 students were selected to demonstrate the expected relationship between two variables measured in the lab……

Selecting subjects from among a larger sample to confirm a previously held belief - it’s never more clear than that. I am not sure if researchers stop doing this as they gain more experience; or if they simply get better at writing it up in a way that makes what has happened less clear.

Think of it this way; I just flipped two coins 21 times and selected 6 paired samples. There is a perfect association between these 6 paired flips; 4 of the paired flips were heads; and 2 of the paired flips were tails; therefore I can claim that there is a strong association whereby the result of one flip influences the second flip….



Leave a Comment