In the past post I tried to do a few things and I am not sure I did any of them very well. It started off on the correct path - wanting to teach about an adjustment set. But it then got bogged down into why I want to teach about an adjustment set, and did not do such a great job with that either.
To summarize (at risk of creating another post bogged down) - while the foundations of KBP are philosophical, there are highly practical consequences. Those practical consequences are directly linked to the use of causal models as a representation of knowledge for practice. Research (empirical observations and induction) provides information for the causal models and a critical rational process builds the causal models. A great way to represent a causal model is with a causal graph. A great approach for creating graphical causal models are DAGs (directed acyclic graphs), and a great system for creating DAGs is DAGitty.
DAGs (generally) and DAGitty (specifically) allow us to visualize the causal structure and build adjustment sets. An adjustment set is a “set of covariates such that adjustment, stratification, or selection (e.g. by restriction of matching) will minimize bias when estimating the causal effect of the exposure on the outcome (assuming that the causal assumptions encoded in the diagram hold).” (Drawing and Analyzing Causal DAGs with DAGitty, User Manual for Version 2.2, Johannes Textor)
An adjustment set is a set that includes the variables (sources of information) that should be considered when designing, conducting, analyzing or reviewing an empirical research study attempting to make a causal inference (or even attempting to demonstrate an association that the author is (either implicitly or explicitly) implying may underly a causal association). This is true whether it is a single study, or a systematic review of empirical studies. When Bradford Hill produced his “criteria for causation” for epidemiologists, an adjustment set is what he was referring to with the criteria: “consideration of alternate explanations.” It is what the study introduction should provide background about to justify the observational methods used in the study, and what the discussion should address the merits of the approaches used to adjust since this weighs greatly on the ability of a study to conclude whether a causal association has been observed.
An adjustment set is related to the causal structure (as depicted by the causal graph). Sometimes it is taken for granted that it is considered in a study. Assessment’s of study bias (such as the PEDro Scale) do not actually take into account the causal structure of a study, or whether the study design includes the variables that that are necessary based on the adjustment set. Part of the reason is that the need for an adjustment set is considered necessary for observational (not experimental) designs. But is there a limit to how much adjustment is provided by the randomization and control process of experimental designs? If not - would RCTs be more helpful if there was an explicitly stated causal structure that that design was based on and that the results contribute to? Also, if we do consider the causal structure and fully account for the necessary adjustment sets, don’t observational studies of people in their natural settings provide us with causal insight?