Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-669899f699-rg895 Total loading time: 0 Render date: 2025-04-28T16:52:21.427Z Has data issue: false hasContentIssue false

5 - Bias in Randomized Controlled Trials

from Part II - Rethinking Research

Published online by Cambridge University Press:  13 March 2025

Karen B. Schmaling  and
Robert M. Kaplan
Karen B. Schmaling
Affiliation:
Washington State University
Robert M. Kaplan
Affiliation:
Stanford University
Get access

Summary

In hierarchies of research evidence, the randomized controlled trial (RCT) usually appears near the top of the pyramid. RCTs are usually considered to be free of bias; systematic reviews of the literature may exclude studies that are not RCTs. Although controlled trials are excellent methods for establishing causation, they do not assure freedom from systematic bias. This chapter explores biases that are common among RCTs. In particular, we report on the practice of systematically excluding study participants who do not meet specific criteria. It is not uncommon for 90 percent of potential volunteers to be turned away. Common grounds for exclusion include comorbidities, even though living with multiple chronic conditions is almost universal among older adults. The selection of the control group can also increase bias. Control groups might be selected specifically because they increase the likely difference between treated and control conditions. The implications of biases in our RCTs are discussed.

Keywords

generalizabilityhierarchy of evidenceinternal validityrandomized controlled trials (RCTs)representativenessselection bias
Type
Chapter
Information
Rethinking Clinical Research
Methodology and Ethics
 , pp. 97 - 116
Publisher: Cambridge University Press
Print publication year: 2025

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Vaughan, L, Espeland, MA, Snively, B, et al. The rationale, design, and baseline characteristics of the Women’s Health Initiative Memory Study of Younger Women (WHIMS-Y). Brain Res. 2013; 1514:3–11.CrossRefGoogle ScholarPubMed
Yetley, EA, MacFarlane, AJ, Greene-Finestone, LS, et al. Options for basing Dietary Reference Intakes (DRIs) on chronic disease endpoints: Report from a joint US-/Canadian-sponsored working group. Am J Clin Nutr. 2017; 105(1):249S–285S.CrossRefGoogle ScholarPubMed
Moberg, CA, Humphreys, K. Exclusion criteria in treatment research on alcohol, tobacco and illicit drug use disorders: A review and critical analysis. Drug Alcohol Rev. 2017; 36(3):378–388.CrossRefGoogle ScholarPubMed
Campbell, DT, Stanley, JC. Experimental and Quasi-Experimental Designs for Research. Ravenio Books; 2015.Google Scholar
Chambers, DA, Glasgow, RE, Stange, KC. The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013; 8(1):1–11.CrossRefGoogle ScholarPubMed
Shen, C, Ferro, EG, Xu, H, Kramer, DB, Patell, R, Kazi, DS. Underperformance of contemporary phase III oncology trials and strategies for improvement. J Natl Compr Canc Netw. 2021; 1(aop):1–7.Google Scholar
Fojo, T, Mailankody, S, Lo, A. Unintended consequences of expensive cancer therapeutics – the pursuit of marginal indications and a me-too mentality that stifles innovation and creativity: The John Conley Lecture. JAMA Otolaryngol Head Neck Surg. 2014; 140(12):1225–1236.CrossRefGoogle Scholar
Walia, A, Tuia, J, Prasad, V. Progression-free survival, disease-free survival and other composite end points in oncology: Improved reporting is needed. Nat Rev Clin Oncol. 2023; 20(12):885–895.CrossRefGoogle ScholarPubMed
Shen, C, Ferro, EG, Xu, H, Kramer, DB, Patell, R, Kazi, DS. Underperformance of contemporary phase III oncology trials and strategies for improvement. J Natl Compr Canc Netw. 2021; 19(9):1072–1078.CrossRefGoogle ScholarPubMed
Kaplan, RM, Irvin, VL. Likelihood of null effects of large NHLBI clinical trials has increased over time. PLoS One. 2015; 10(8):e0132382.CrossRefGoogle ScholarPubMed
Haug, N, Deischinger, C, Gyimesi, M, Kautzky-Willer, A, Thurner, S, Klimek, P. High-risk multimorbidity patterns on the road to cardiovascular mortality. BMC Med. 2020; 18:1–12.CrossRefGoogle ScholarPubMed
Green, LA, Fryer, Jr GE, Yawn, BP, Lanier, D, Dovey, SM. The ecology of medical care revisited. Mass Med Soc. 2001; 344(26):2021–2025.Google ScholarPubMed
Leinonen, A, Koponen, M, Hartikainen, S. Systematic review: Representativeness of participants in RCTs of acetylcholinesterase inhibitors. PLoS One. 2015; 10(5):e0124500.CrossRefGoogle ScholarPubMed
Anderson, TS, Ayanian, JZ, Souza, J, Landon, BE. Representativeness of participants eligible to be enrolled in clinical trials of aducanumab for Alzheimer disease compared with Medicare beneficiaries with Alzheimer disease and mild cognitive impairment. JAMA. 2021; 326(16):1627–1629.CrossRefGoogle ScholarPubMed
Cherubini, A, Oristrell, J, Pla, X, et al. The persistent exclusion of older patients from ongoing clinical trials regarding heart failure. Arch Intern Med. 2011; 171(6):550–556.CrossRefGoogle ScholarPubMed
Kaplan, RM. Diseases, Diagnoses, and Dollars. Springer; 2009.CrossRefGoogle Scholar
Bress, AP, Tanner, RM, Hess, R, Colantonio, LD, Shimbo, D, Muntner, P. Generalizability of SPRINT results to the US adult population. J Am Coll Cardiol. 2016; 67(5):463–472.CrossRefGoogle Scholar
Tan, YY, Papez, V, Chang, WH, Mueller, SH, Denaxas, S, Lai, AG. Comparing clinical trial population representativeness to real-world populations: An external validity analysis encompassing 43 895 trials and 5 685 738 individuals across 989 unique drugs and 286 conditions in England. Lancet Healthy Longev. 2022; 3(10):e674–e689.CrossRefGoogle Scholar
Pant, S, Lee, MS. Barriers to pancreatic cancer clinical trials enrollment. Oncology (Williston Park, NY). 2020; 34(10).Google Scholar
Kalbaugh, CA, Kalbaugh, JM, McManus, L, Fisher, JA. Healthy volunteers in US phase I clinical trials: Sociodemographic characteristics and participation over time. PLoS One. 2021; 16(9):e0256994.CrossRefGoogle ScholarPubMed
Abramson, JD. Sickening: How Big Pharma Broke American Health Care and How We Can Repair It. Mariner Books; 2022.Google Scholar
Kirsch, I. The emperor’s new drugs: Medication and placebo in the treatment of depression. Placebo. 2014:291–303.CrossRefGoogle ScholarPubMed
Kirsch, I. Antidepressants and the placebo effect. Zeitschrift für Psychologie. 2014; 222(3):128–134.CrossRefGoogle ScholarPubMed
Locher, C, Koechlin, H, Zion, SR, et al. Efficacy and safety of selective serotonin reuptake inhibitors, serotonin-norepinephrine reuptake inhibitors, and placebo for common psychiatric disorders among children and adolescents: A systematic review and meta-analysis. JAMA Psychiatry. 2017; 74(10):1011–1020.CrossRefGoogle ScholarPubMed
Enck, P, Klosterhalfen, S. Placebos and the placebo effect in drug trials. Handb Exp Pharmacol. 2019; 260:399–431.CrossRefGoogle ScholarPubMed
von Wernsdorff, M, Loef, M, Tuschen-Caffier, B, Schmidt, S. Effects of open-label placebos in clinical trials: A systematic review and meta-analysis. Sci Rep. 2021; 11(1):1–14.Google ScholarPubMed
Schwartz, LM, Woloshin, S, Lu, Z, et al. Randomized study of providing evidence context to mitigate physician misinterpretation arising from off-label drug promotion. Circ Cardiovasc Qual Outcomes. 2019; 12(11):e006073.CrossRefGoogle ScholarPubMed
Nicholls, SJ, Lincoff, AM, Garcia, M, et al. Effect of high-dose omega-3 fatty acids vs corn oil on major adverse cardiovascular events in patients at high cardiovascular risk: The STRENGTH randomized clinical trial. JAMA. 2020; 324(22):2268–2280.CrossRefGoogle ScholarPubMed
Webster, RK, Howick, J, Hoffmann, T, et al. Inadequate description of placebo and sham controls in a systematic review of recent trials. Eur J Clin Invest. 2019; 49(11):e13169.CrossRefGoogle Scholar
Demasi, M, Jefferson, T. What’s in the placebo? Trust the Evidence. 2023;(July 18, 2023).Google Scholar
Shader, RI. Placebos, active placebos, and clinical trials. Clin Ther. 2017; 39(3):451–454.CrossRefGoogle ScholarPubMed
Mohr, DC, Spring, B, Freedland, KE, et al. The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychother Psychosom. 2009; 78(5):275–284.CrossRefGoogle ScholarPubMed
Laursen, DR, Nejstgaard, CH, Bjørkedal, E, et al. Impact of active placebo controls on estimated drug effects in randomised trials: A systematic review of trials with both active placebo and standard placebo. Cochrane Database Syst Rev. 2023; 3(3):MR000055.Google ScholarPubMed
Toews, I, Anglemyer, A, Nyirenda, JLZ, et al. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: A meta‐epidemiological study. Cochrane Database Syst Rev. 2024; 1:MR000034.Google ScholarPubMed

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×