Protecting against biasing actions in a knowledge based practice

In the last post we considered actively biasing behavior. As mentioned then there are many implications of such active behavior (biasing actions) to be considered. One such implication was pointed out in a comment on the last post by Nate Felton where behavior is changed resulting in an incorrect conclusion regarding the cause of the underlying mechanism and source of a patient’s primary signs and symptoms.

Since it’s foundation this blog has been, at it’s core, about the importance of causal reasoning for clinical practice. It proposes that cause, understood as a causal structure, represented by a causal model and communicated with directed acyclic graphs (due to their logical and probabilistic characteristics) are a bridge between the development of structured observations (research) and clinical practice (see the post “[Graphical Causal Models]”). As a bridge these models help make clinical research more clinically relevant for implementation in practice (see this post). Underlying this system is reliance on Bhaskar’s critical realist epistemology generally, his philosophy of science more directly. And critical to the system is the presupposition that, one, we can identify, understand and communicate causal structures of reality (involves moving from closed systems of experimental observations to real open systems that we encounter); and, two, that we can reason through these structures which involves identifying the relevant structure for a given situation in the real world (i.e. identifying when an existing causal structure is applicable in a given practice scenario,  identifying when (or why) a structure may not be completely applicable, and what the implications of the less then complete applicability are on the ability to use the structure when reasoning in practice.

Therefore, the research referred to in the previous post on actively biasing behavior must be considered. Actively biasing behavior can influence both of the above presuppositions. Such behavior can undercut both the development and use of causal models for practice. On the development side, actively biasing behavior can influence experiments (closed, structured observations), and moving from closed to open systems when we link together experimentally verified causes to build a causal model of knowledge for practice. On the practice side actively biasing behavior can greatly influence observations (in the selection and interpretation of tests and measures) and in the selection of correct causal model.

Addressing actively biasing behavior does give us the opportunity to address the two “sides” of knowledge based practice (development of causal models on one hand (research), and use of causal models on the other hand (practice)) and point out the practice benefits to an education in research skills for the clinician.

The same authors of the article referred to in the previous post have also studied the impact of an educational intervention on actively biasing behavior (here).

Implementation and Assessment of an Intervention to Debias Adolescents against Causal Illusions

Barberia I, Blanco F, Cubillas CP, Matute H (2013) Implementation and Assessment of an Intervention to Debias Adolescents against Causal Illusions. PLoS ONE 8(8): e71303. doi: 10.1371/journal.pone.0071303

The intervention introduced the participants to the concept of contingency described above. In the intervention, the participants learned that comparing the probability of an outcome in the presence and the absence of the potential cause is the normative manner of assessing the empirical evidence for a hypothesized causal link. In this sense, the emphasis was placed on the idea that the rate of cause-outcome co-occurrence, or P(Outcome|Cause), is necessary to infer a generative causal relationship between a potential cause and an outcome but is certainly not sufficient. The base rate of the outcome, or P(Outcome|~Cause), is also important to consider if one is to reach an appropriate conclusion. Understanding the necessity of considering this latter piece of information should encourage people to expose themselves to more instances in which the cause is absent. That is, to expose themselves to a lower P(Cause), which should, in turn, diminish the tendency to develop causal illusions.

In other words, the intervention taught its participants the basics of research methods. And, more importantly the intervention worked!

A DPT education should assure that it’s graduates not only understand how to reason about cause, but are fully aware of the pitfalls and understand the risks associated with reasoning about cause, and the potential for bias both in conduction of studies, the interpretation of study results and every day while practicing. This must include making connections between published tools for assessment of research (i.e. PEDro scale, AMSTAR) and the types of bias they are attempting to identify. The score on these scales does not indicate bias has occurred, but they indicate the potential for bias and the items in the scale each highlight a type of bias with a particular implication.

Here is one example of how a research method leads to bias that should be considered. Intention to treat analysis (ITT) attempts to eliminate issues associated with non random cross over of study subjects in a randomized controlled trial. Basically, the analysis is performed based on group allocation at the start of the study, not based on where subjects complete the study. So if a subject starts with a non surgical intervention but ends up crossing over and having surgery, they are analyzed as if they were in the non surgical group. This essentially biases the results toward the null hypothesis. It makes it more difficult to demonstrate a difference between the groups at the end of the trial period so that if a difference is demonstrated, in spite of the ITT analysis, we feel even more strongly that the intervention had an effect.

However, what if a null finding is what is found and is what is reported? For example, this recent study reports a null finding:

Delitto, A., Piva, S. R., Moore, C. G., Fritz, J. M., Wisniewski, S. R., Josbeno, D. a., … Welch, W. C. (2015). Surgery Versus Nonsurgical Treatment of Lumbar Spinal Stenosis. Annals of Internal Medicine, 162(7), 465. 

The study performed an ITT analysis, and a large proportion of the subject allocated to the physical therapy intervention switched to surgery within 10 weeks (31 out of 69 subjects). So what do we make of these findings? Here I suggest readers consider the difference between the figures provided (Figure 2 vs. the Appendix Figure). Figure 2 is based on ITT analysis (grouped based on initial allocation) and the Appendix Figure is based on treatment received. It turns out that the ultimate outcome (104 weeks) is similar with these two approaches, there just seems to be a slower progression to this outcome in the group receiving PT that did not switch to surgery. Overall the study was very well done and raises awareness and at least the question of whether surgery should be performed first, even for subjects that are qualified for surgery, and it opens up the possibility of identifying factors in the future that help better classify non surgical “responders” and/or surgical “responders.”

Leave a Comment