Back on April 5th I started posting about adjustment sets for making causal inferences as a step towards moving ahead. Adjustment sets are well described for inductive inference (which is the traditional approach to evidence based practice) and is the knowledge about universals generating arm of a knowledge based practice (the left side) of the KBP graphic:


After defining the adjustment set for induction and going over the general rules for identifying them from DAGs (graphical causal models - directed acyclic graphs) we then considered whether adjustment sets were possible (and/or important) for deduction and abduction (non-inductive adjustment sets). During the discussion on adjustment for deduction and abduction I was reminded of related limitations of RCTs from a KBP perspective and had to post on those limitations.

Based on all we have discussed thus far, adjustment is a good thing. An adjustment set provides information about what information is needed to reduce bias of a causal inference and causal inferences are critical to clinical reasoning. However, depending on causal structure, adjustment during study design (particularly sample selection or even recruitment) may lead to bias. Such a situation is called selection bias, and has also been referred to as Berkson’s paradox (Pearl). Let’s take the example from Pearl (2009 - Causality).

dagitty-modelLet’s take a graduate school of music that requires a high GPA ‘OR’ high musical talent for admission. Here “OR” is used as inclusive OR, not exclusive OR. Inclusive OR is true when either one or both of the conditions are true and only false when both are false. In every day language we commonly mean exclusive OR when using the term or (I will have the fish OR the chicken) which is true when either one of the conditions is true, but not when both are true). If we were interested in studying the relationship between GPA and musical talents on admission to the graduate school, by only selecting students admitted (by only recruiting from the school - people admitted), we have adjusted based on being a student which is dependent on being admitted.  Such an adjustment may force a negative association between GPA and musical talent even when such an association does not exist in the general population. This negative association is not a universal characteristic, it is bias due to the selection of this sample. To gain admission with average musical talent a student would have to have high GPA, and to gain admission with average GPA a student would have to have high musical talent. However, that negative association between GPA and musical talent is not necessarily a universal relation in the population.

Let’s find an example from our causal model from Krethong et al (2008):


For risk of selection bias we need to find a “collider” a variable which is thought to be caused by two other variables. From the Krethong example we can take “symptom status” which is thought to be due to “bio/physio status” and “social”.

Let’s say we want to do a study investigating the relationship between bio/physio status and social on symptom status in a group of patients in rehab following an exacerbation for heart failure.

dagitty-modelSince we start with patients in rehab the patients have a particular functional status - they cannot be home. If they could be home, chances are they would be home. So we are adjusting on functional status. Patient’s symptom status is either due to poor bio/physio status OR social factors and we may find an inverse association between these two variables that does not exist in HF patients that are not currently limited in function and requiring rehab.

Both of these examples are fabricated. I am in the process of trying to identify articles  that provide real published demonstrations that may be influenced by selection bias (though this is, thankfully, difficult as such bias should be caught in the peer review process).

The take home message is that selection bias needs to be considered when designing studies and attempting to draw inferences inductively from empirical observations to universals; and highlights more formerly how our practice experience (bound by place and time) as a mode of collecting empirical evidence is very limited for generating universals due to bias through selection - see the posts here and here.