This last week, I was chatting with a medical specialist friend. We were discussing clinical trial data, COVID-19 and confirmation bias. The conversation drifted to real world data (RWD)/ evidence (RWE) and observational studies with historical controls. While I am a fan of real world data being used to form evidentiary hypothesis and help inform doctor decisions where clear evidence is lacking, I by no means think RWD is a substitute for Randomized Control Trials (RCT).
This discussion reminded me of a book I recently read, Malignant by Vinay Prasad MD. In chapter nine, Study Design 201, Dr. Prasad has a sub-section titled “Historically Controlled and Uncontrolled Trials.” It discusses the treatment CHOP (cyclophosphamide, doxorubicin, vincristine and prednisone) developed in the 1960s to ProMACE-cytaBOM developed at the National Cancer Institute in the 1980s. The short story is the NCI reported astounding results from a Historical Controlled Trial showing 86% non-Hodgkin Lymphoma patients had complete response, and the long term response appeared to be nearly 69%. These results were much better than CHOP.
Sounds great, but maybe too good to be true. In 1993, a RCT was performed to compare the two. The results were surprising. At three years, all groups had a 44% chance of being alive and the complete response rate for ProMACEcytaBOM was 30 percentage points less than previously reported (86% dropped to 56%). There are other examples of HCT being inaccurate, one of which saw a difference of 77% of HCTs compared to 20% RCTs finding a therapy beneficial. That’s quite a difference. As real world data and evidence might be considered a fancy name for historically controlled trials, the accuracy of such studies should be taken with a grain of salt.
With Precision Oncology coming into our lexicon, the discussion of whether to conduct a RCT is often not black and white. Chadi Nabhan MD recently hosted Vivek Subbiah MD, from MD Anderson Cancer Center, on his Healthcare Unfiltered podcast (A Precise depiction of Precision Medicine). I greatly enjoyed listening to this episode, though I think the promise and pitfalls are still evolving and seems to be somewhat tied to RWD/E.
Ultimately, this is an interesting space for biostatistics and health policy/ management.
PS: I highly recommend Dr. Prasad’s book.
