Confounding in RCTs

Confounding can occur in randomised controlled trials (RCTs)

How?

The randomisation process aims to equally distribute all variables in both (or more) study groups. This, however, is only successful if one considers a very large number of experiments (i.e., RCTs). In one single RCT, certain factors can be unequally distributed by chance, i.e., RCTs can still be "randomly confounded". For example, one group could by chance be older, smarter, or genetically different from the other group in some way.

The chance of RCTs being randomly confounded decreases with the number of RCTs and the sample size of an RCT. But how many is many? And how large is large? No one knows (god maybe). "Of course, this fact [increasing the sample size] is not an ironclad guarantee that confounding will not be severe, for our result could be one of the unlucky ones with severe confounding." Greenland 1990

Consequently, in RCTs it is still necessary to adjust the for possible confounders. Still, even if they do adjust the results for certain factors, the researchers might overlook some (or one) of these which may be relevant (i.e., which may change [confound] the results).

Many mistakes can be made when conducting a study (and some researchers intentionally make these mistakes, i.e., they tweak the evidence to their liking).

Sometimes it is suggested that, OK, the results of some RCTs are not reliable but the results of "well-conducted" RCTs are. The problem is that ... " ‘well-conducted’ rules out all of the things that almost always occur in practice" (Deaton & Cartwright 2018). What almost always occurs in practice is …
  • … that people drop out of a study, i.e., they stop participating in the middle of the study, and are then left out when results are analysed (also called attrition),
  • … that participants and/or researchers know who is in which group (intentionally non-blinded trials or unintentional unblinding in supposedly blinded trials),
  • … that other factors are unequally distributed between the different study groups after randomisation (post-randomisation confounding),
  • … that the selection process, i.e., the inclusion and/or exclusion of participants into one study group is different from the other group(s) (selection bias).
Sometimes (or most of the time [Kahan et al. 2015]) the randomisation process is not conducted perfectly.

In addition, there can be small problems (to say the least) in how things (the parameters of the study) are assessed (measured), which things are assessed, what statistical methods are used, how things are reported and which things are reported.

---------------------------

When it comes to non-randomised trials and observational studies (which are always non-randomized), including cohort studies, it has long been stated that: "I therefore conclude that probabilistic interpretations of conventional statistics are rarely justified, and that such interpretations may encourage misinterpretation of nonrandomized studies." (Greenland 1990). It other words, the presence of a relation of cause and effect is very uncertain and, for example, the "risk" reported in cohort studies should not misinterpreted by laypeople (or anyone) as actual risk, the way this word is commonly used.

---------------------------

Statistical adjustment can also introduce bias: "[...] Evidence from the model adjusted for compliance [of the participants to the study protocol, e.g., eating a certain food] is weaker than the model not adjusted for compliance however, since those who comply with interventions may be systematically different from those who do not. [...]." [Snetselaar et al. 2020]

---------------------------


"[…] one thing randomization does not do: It does not prevent the epidemiological bias known as confounding." Greenland 1990



References