Yves here. This author no doubt chose easy-for-laypeople-to-understand examples to make his point about sloppy coronavirus studies. I’m sure our savvy readers can offer far more sophisticated advice on how to read medical studies, but it’s always useful to keep a few things in mind (and you’d be surprised to see how many studies fall short on these very basic tests):
How big was the sample population? I’ve been told by MDs who review medical research that they regard anything less than a sample of 100 (with a similar sized control) as not reliable. Smaller samples can be flukey. With coronavirus, there are presumably a lot of controls via the large population of the already sick; it would be nice to see a study take some trouble in creating its control out of available cases.
Is there bias in the sample? Age, ethnicity, gender, lack or presence of other conditions….
Are the authors careful in stating findings? Correlation is not causation!
Also watch what is being tested. For instance, the author discusses a preliminary hydroxychloroquine study. More studies are underway. But most appear to be evaluating the efficacy of hydroxychloroquine against coronavirus all by itself, when most of the experiments in the field I have read about are of either hydroxychloroquine + azithromycin or hydrochloroquinine + azithromycin + zinc sulfate. HIV is treated with a drug cocktail and that is where we may wind up with coronavirus. So a negative on hydroxychloroquine as the sole remedy does not establish it isn’t useful in combination with other drugs.
By Irving Steinberg, Dean for Faculty, USC School of Pharmacy; Associate Professor of Clinical Pharmacy & Pediatrics, School of Pharmacy & Keck School of Medicine of USC; Director, Division of Pediatric Pharmacotherapy, Dept of Pediatrics, LAC+USC Medical Center, University of Southern California. Originally published at The Conversation
It has been barely a few weeks since the coronavirus was declared a pandemic. The pace at which the SARS-CoV-2 virus has spread across the globe is jolting, but equally impressive is the speed at which scientists and clinicians have been fighting back.
I am a pharmacotherapy specialist and have consulted on infectious disease treatments for decades. I am both exhilarated and worried as I watch the unprecedented pace and implementation of medical research currently being done. Speed is, of course, important when a crisis such as COVID-19 is at hand. But speed – in research, the interpretation and the implementation of science – is a risky endeavor.
The faster science is published and implemented, the greater the chances it is unsound. Mix in the panic and stress of the current pandemic and it becomes harder to make sure the right information is communicated and adopted correctly. Finally, governing bodies such as the World Health Organization, politicians and the media act as sources of trustworthy messaging and policy making. Each step – research, interpretation, policy – has safeguards in place to make sure the right information is acquired, interpreted and implemented. But pace and panic are testing these safety measures like never before.
The process of taking an idea from theory through testing and eventually toward implementation has been refined in modern times to make sure medical studies and publications are truthful and accurate.
Once research is completed, investigators analyze their results and write a manuscript. They then submit it to a journal, where it is reviewed by experts in that field who assess whether the methods, analysis and conclusions are sound. If the paper is accepted, it is then further edited and published in a journal.
From there, groups like the WHO, medical societies and government agencies evaluate this and other evidence-based information to decide whether to establish new recommendations or change previous ones. It normally takes from several months to more than a year to go from submission to publication. But the rush to publish during this pandemic has shortened the time from submission to online publication to one to two weeks in numerous cases.
There has also been a huge increase in preprint publication – publishing studies online before they are adequately peer-reviewed – and these are a good example of the risk that comes with the rapid release of data.
On March 17, French investigators posted a prepublication clinical paper online touting the successful use of hydroxychloroquine in COVID-19 patients. Despite the media and government attention, the study was described by director of the National Institute of Allergy and Infectious Diseases Anthony Fauci as “anecdotal” due to the poor study design.
On April 3, the International Society of Antimicrobial Chemotherapy, the sponsoring organization of the very journal posting this prepublished article, agreed and stated “….the article does not meet the Society’s expected standard,” and “Although ISAC recognises it is important to help the scientific community by publishing new data fast, this cannot be at the cost of reducing scientific scrutiny and best practices.” The debate over the usefulness of hydroxychloroquine will likely continue until well-designed trials are completed.
The deliberate steps of scientific investigation, followed by editorial scrutiny, are guardrails. When these are disrupted there is a real risk that policy organizations may make consequential mistakes in spite of good intent.
When Pace Meets with Panic
Nothing better illustrates how trusted institutions can make misinformed recommendations than the recent fiasco over ibuprofen.
The most common early symptom of COVID-19 is fever, and ibuprofen is one of the most widely used drugs in the world to treat fever. In a letter published in The Lancet Respiratory Medicine, European researchers raised concerns that ibuprofen use could worsen COVID-19 symptoms. The idea is that since ibuprofen increases the quantity of ACE2 in human cells – the protein that the coronavirus uses to enter lung cells – the virus could infect lung cells more easily if a person was on ibuprofen. This was not a study nor did it present sufficient experimental evidence; it was simply a theoretical concern based on a mechanism.
Three days after the letter was published, the French health minister tweeted a message urging people to avoid ibuprofen for coronavirus associated fever based on four “cited” cases of people getting sicker after taking ibuprofen. These cases were never published in a journal. The French Health Ministry followed this with a broad ban on treating COVID-19 fever with nonsteroidal anti-inflammatory drugs like ibuprofen. The WHO tweeted an essentially similar warning. The media followed with more case anecdotes, dubiously relating worsening early symptoms with ibuprofen use and referring to the letter as a “study,” adding to the confusion and fear.
The Lancet letter also hypothesized that two other drugs commonly used to treat hypertension and diabetes – ACE-inhibitors (ACE-I) and angiotensin receptor blockers (ARBs) – could be problematic in people with COVID-19. However, the mechanism they put forward was incompletely described and neglected that a protein these drugs promote can be helpful in reducing inflammation and tissue damage in the lungs and heart.
This letter to The Lancet slipped past the safeguards in research and institutional and media interpretation, but one of science’s oldest pastimes – definitively calling out the errors of others – reestablished patience and perspective.
Clinicians and scientists pushed back swiftly, supporting the use of ibuprofen in COVID-19 patients. The support was outlined in a published literature review. In response, the WHO quickly reversed its position on ibuprofen.
There was a similar rapid response to the statements about ARBs. Within days, three prominent cardiology groups, including the American Heart Association, released a joint statement urging practitioners not to discontinue ACE-I and ARBs in their patients.
The risk-benefit ratio is always a clinical factor for the use of any drug in any patient. But the risk must be more than theory for the use of a drug to be discontinued or any major policy change to be implemented.
As the coronavirus rampages across the U.S., it is incredibly important to know whether commonly used drugs like ibuprofen or ARBs are risky, neutral or of therapeutic potential. There are ways to find out quickly. Researchers can look for correlations between the use of ibuprofen or ARBs and more severe infections or deaths, for example. And standard clinical trials can, should and are being done. There are several studies currently underway testing the effect and risk of ARBs for COVID-19 patients. But until the science is finished, it is foolish and potentially dangerous to flee from tested clinically important drugs.
Scientists and policymakers must take quick steps and avoid missteps. Proper scientific method and conduct of studies, carefully reviewed publications and cogent post-release interpretations are necessary safeguards that ensure the best and safest medicines are prescribed and provided. The pressure and desperation of the moment are forcing researchers and policymakers to be innovative and act quickly, but what is done should stay within the guiding concepts of medical research.