Lambert here: This post shows how to add narrative analysis to your toolkit of critical thinking skills.
By Roy Poses, MD, Clinical Associate Professor of Medicine at Brown University, and the President of FIRM – the Foundation for Integrity and Responsibility in Medicine. Originally posted at Health Care Renewal website
Tales of medical research misconduct seem increasingly prevalent in the media, and are getting increasing attention. They often may present a simple narrative, like this recent story in the New York Times, “An NYU Study Gone Wrong, and a Top Researcher Dismissed.”
The Narrative Arc
A Renowned Researcher
The researcher in question was one Dr Alexander Neumeister. Oddly enough, the article provided very little information about his background, but made it clear he was at New York University, and was a “top researcher.”
The Potentially Ground-Breaking Studies
The NY Times article noted that the studies were of “an experimental, mind-altering drug.” In particular,
In one of the shuttered studies, people with a diagnosis of post-traumatic stress caused by childhood abuse took a relatively untested drug intended to mimic the effects of marijuana, to see if it relieved symptoms.
The study was ground-breaking, in that
It’s a critical time for two important but still controversial areas of psychiatry: the search for a blood test or other biological sign of post-traumatic stress disorder, which has so far come up empty, and the use of recreational drugs like ecstasy and marijuana to treat it.
The drug was not identified, but the article noted that it was
a drug intended to produce some of marijuana’s effects, made by Pfizer
and was thus like the drug in “a French trial,” that caused six patients to be “hospitalized with severe neurologic problems.”
The study was apparently a small short-term randomized controlled trial
Some participants took the drug over a seven-day period; others took a placebo pill. The N.Y.U. team performed scans on each person to see whether brain activation patterns correlated with symptom relief.
The study called for recruiting 50 people with a PTSD diagnosis, according to study documents.
Research Misconduct Discovered
Dr. Charles Marmar, the chairman of the psychiatry department at N.Y.U., said that people working with Dr. Neumeister had reported concerns about the lab’s compliance with research standards.
This led to
The federal inspection, from July 16 to Aug. 5 last year, found that the research team had failed to assess at least three subjects 24 hours after they had taken the experimental drug, contrary to study protocol, according to the F.D.A. letter. In several instances, the agency found, Dr. Neumeister had falsified documents by signing a fellow investigator’s name on reports. ‘However, in fact, you or another study employee actually conducted these study procedures,’ not the colleague, the F.D.A. concluded.
The violations ‘jeopardize subject safety and welfare, and raise concerns about the validity and integrity of the data collected at your site,’ the F.D.A. said in a letter
Note that the article did not explain why the FDA was called upon to investigate this problem.
Aggrieved Research Subjects
The article focused on one Ms Diane Ruffcorn, who “writes a popular Facebook blog on trauma,”
‘I think their intent was good, and they were considerate to me,’ said one of those subjects, Diane Ruffcorn, 40, of Seattle, who said she was sexually abused as a child. ‘But what concerned me, I was given this drug, and all these tests, and then it was goodbye, I was on my own. There was no follow-up.’
After the trial, she was concerned because
Ms. Ruffcorn said she had several odd symptoms after the trial, including a hyper, wired sensation that occurred without the usual memories of abuse. For months, she tried to find out whether those reactions were tied to the experimental drug, but because the study was shut down and the data belonged to Pfizer, the N.Y.U. doctors could not tell her whether she had taken the drug or placebo.
Earlier this month, after much persistence, she found out that she’d taken the placebo. ‘It was a big relief,’ she said.
Note that the article did not explain by Pfizer owned the data, and would not reveal it.
The Researcher Punished
The researcher did not agree that things were so bad,
Georges Lederman, a lawyer for Dr. Neumeister, said there may have been protocol violations, ‘but N.Y.U. has taken the position that those violations were more egregious than we believe they actually were.’ The issues could have been easily remedied, he said, and noted that they did not cause the sponsor of the research, the pharmaceutical giant Pfizer, to shut it down.
Note that the article did not explain why Pfizer was empowered to shut such a study down.
In any case,
Dr. Neumeister and N.Y.U. continue to disagree over the seriousness of the research violations, both sides said. But the university has tossed out all of the data as unreliable, and tracked down the study participants to check on their health, Dr. Marmar said.
And apparently Dr Neumeister quit, or was fired, although the article only said NYU “parted ways with a top researcher.”
So, in summary, the story was that a prominent researcher was doing cutting edge research at a big university, but people onsite noted some problems, the government was called in to investigate, the investigation found problems, the research was stopped, and the researcher lost his job. However, while the article mentioned that Pfizer sponsored the study, Pfizer had control of the study’s data, and Pfizer had the power to shut the study down, the article did not comment on whether the involvement of Pfizer could have had any relationship to the narrative of alleged individual researcher misconduct.
Research Misconduct as a Problem with Bad Apples
Thus, in my humble opinion, this story followed the usual narrative arc of research misconduct stories: an individual scientist over-reaches, possibly in pursuit of fame and money, is discovered and punished, and things get back to normal. The implication is that research misconduct is a bad-apple problem, although fueled by a hyper competitive research environment. For example, last week the (UK) Times Higher Education Suppplement published an article entitled, “Is There a Problem with Research Integrity,” that opened,
For many academics today, research is not about pushing intellectual boundaries. It is not about investigating a fascinating issue so much as it is about churning out publications, demonstrating impact and generating revenue in order to meet the performance targets upon which institutional reputation and individual careers depend.
The temptation to cut corners is immense. Tricks include getting your name on a paper that you contributed little towards, or “salami-slicing” the same research across several publications. More seriously, some researchers falsify – misrepresent – their data, or even fabricate them entirely. Some universities tacitly encourage such behaviour and the boundary between academic integrity and malpractice is becoming blurred.
The current case seems to be on of attempted falsification, misrepresentation of research data.
Notice the use of the pronoun “you,” emphasizing that research misconduct is about individual misconduct. Similarly, tha THE article included commentaries by various individuals. One was by a “research integrity expert,” who began,
Having positive and preferably spectacular research findings is wonderful. It helps you to get a publication in a journal with a high impact factor, which will be cited often and may attract a lot of media attention. This is not only a pleasant ego boost but may also be instrumental in getting your next grant or strengthening your academic position. So, in an ever more competitive and metrics-driven scientific environment, it is tempting to make such results occur by any means necessary.
All this is true as far as it goes. But in my humble opinion, the usual research misconduct narrative is vastly oversimplified, as is the case reported by the New York Times.
We have been writing for years about massive problems with manipulation of clinical research to increase the likelihood that the results would satisfy vested interests, and suppression of research whose results remain unsatisfactory after such manipulation. The vested interests are most commonly pharmaceutical, biotechnology or device companies and those working with them. Such suppression and manipulation may make treatments that do not work look efficacious, and treatments that are dangerous look safe, and may lead to excess costs, and worse, harms to patients. This kind of research misconduct may be facilitated by individual researchers seeking fame and fortune, but is hardly an individual sport.
Focusing on individual research misconduct thus leaves the larger problem of vested interests dominating clinical research anechoic.
Looking carefully at the NYU/ Neumeister case as reported, and a little research on the web suggests that there may be more involved than just the conduct of one researcher. But that could only be confirmed, or refuted, by investigation beyond what this humble blogger can do.
A Pharmaceutical Company Sponsored, Likely Pharmaceutical Company Designed, Phase II Drug Study Gone Wrong
The NY Times article acknowledged, almost parenthetically, that the study on which the article was focused was sponsored by Pfizer, although it first did so in the context of Dr Neumeister’s lawyer arguing that the problems with the study were not that serious:
[he] noted that they did not cause the sponsor of the research, the pharmaceutical giant Pfizer, to shut it down.
Later, the article stated,
Pfizer said that N.Y.U. was responsible for conducting the trial,
the company had previously tested the same drug, known as an F.A.A.H. inhibitor, for osteoarthritic pain, without significant side effects. ‘The safety profile we observed does not preclude future development of our compound,’ a Pfizer spokesman said by email.
So this was not a case of a company funding a study merely to advance medical science. The implication was that the company was testing its own compound in hopes of seeking approval from the US Food and Drug Administration. That must be why it was the FDA that investigated the research misconduct, particularly to the extent that the conduct of the research violated a “protocol” to which the FDA was apprently privy.
More evidence that the study was under the control of Pfizer, rather than of Dr Neumeister, could be inferred from the problems Ms Ruffcorn had in determining whether she had taken the drug or placebo.
For months, she tried to find out whether those reactions were tied to the experimental drug, but because the study was shut down and the data belonged to Pfizer, the N.Y.U. doctors could not tell her whether she’d taken the drug or a placebo.
Note that the “data belonged to Pfizer,” not to NYU or Dr Neumeister.
In fact, in perhaps the only critical look given to this story, in a post on Neuroskeptic,
I believe the compound in question is PF-04457845.
I believe this because ClinicalTrials.gov lists a trial of PF-04457845 for PTSD, a trial which was recently terminated. NYU was one of the research sites. I also think that this trial is the fateful one, as it matches the NYT’s description of that study. Interestingly, ClinicalTrials.gov says that the trial was stopped ‘based on Pfizer portfolio prioritization and not due to safety and/or efficacy concern or change in benefit:risk assessment of PF-04457845′.
So given that the study was a small randomized controlled trials of patients, not of healthy volunteers, it appeared to be a Pfizer sponsored, Pfizer designed, Pfizer controlled Phase II study being done in the hope of eventually marketing PF-04457845.
As noted in an article about agreements between academia and industry on the conduct of randomized controlled trials(1),
Many randomized clinical trials (RCTs) are designed and sponsored by for-profit companies. Companies typically contract academic investigators to identify, recruit, and manage patients. Clinical research under these circumstances is a business transaction that bears the potential for conflicts of interest, including those regarding the publication of trial results
It also appears that Pfizer was spending a more than tiny sum on this work. A Politico article from 2014 revealed that Dr Neumeister at that time had a $1.7 million grant from Pfizer, presumably for this particular study. Thus this drug trial was likely providing NYU with more than negligible monetary support, most likely including salary support for Dr Neumeister.
Dr Neumeister apparently has had some previous involvement with pharmaceutical companies, and with Pfizer specifically. A search of the ProPublica Dollars for Docs 2009-13 database revealed that Dr Neumeister received consulting, travel funds and a more than $227,000 grant from Eli Lilly. Dr Neumeister apparently is currently on the advisory board for Fiorello Pharmaceuticals. In a 2015 article in the Journal of Clinical Psychiatry(2), Dr Neumeister acknowledged that he “has received consulting fees from Pfizer.”
So it seems that in this case a study which may not have been conducted according to research standards was likely a pharmaceutical sponsored, designed, and controlled Phase II trial done as part of an effort to seek approval for a new drug. Hence this case was not only about allegations of individual research misconduct, but about yet more problems with the implementation of commercially controlled human experiments designed to ultimately further marketing as well as science. Yet none of the public discussion so far of this case was about whether Pfizer had any responsibilities to assure the quality of the research in which it was so involved, much less whether interactions between the company, the university which was being funded by the company, and the researcher employed by the university but whose salary was probably partially underwritten by the company might have affected how the study was implemented.
There may be many problems with individual misconduct affecting clinical research. But failure to consider how this research is now mainly conducted within a commercial milieu seems to be missing the elephant in the room. If we cannot plainly discuss research misconduct as part of the larger picture of health care dysfunction, we will not be able to do much about it. True health care reform would help end the taboo on discussion about how powerful organizations and their wealthy and powerful leaders distort health care.
1. Kasenda B, von Elm E, You JJ, Blumie A et al. Agreements between Industry and Academia on Publication Rights: A Retrospective Study of Protocols and Publications of Randomized Clinical Trials. PLoS Med 13(6): e1002046. doi:10.1371/journal.pmed.1002046. Link here.
2. Mota N, Sumner JA, Lowe SR, Neumeister A et al. The rs1049353 Polymorphism in the CNR1 Gene Interacts With Childhood Abuse to Predict Posttraumatic Threat Symptoms. J Clin Psychiatr 2015; 76(12):e1622–e1623. Link here.
As you suggest, drug trials fall into a grey area. Big Pharma is looking to develop drugs which will make them money. Researchers want to further their careers and private incomes. Universities want to procure grants to fund their operations. All these focus on money. Just as in politics, money is the preferred agent, (does money have agency?) driving the interlocked processes. Such is the main perversion; the replacement of patient care by financial gain. I personally can’t think of a simple way to take money out of the equation, absent fully socialized medicine. Capitalist Medicine, a very subtle oxymoron. A social Darwinist might say that Socialized Medicine subverts Natures workings. Hah! Another appeal to the God of Death. Much fun and games can result from this dialogue.
A social Darwinist might say that Socialized Medicine subverts Natures workings
By the same logic, another social Darwinist might say that heads on pikes and falling in baskets are Nature’s workings.
Social Darwinist? Hmm. I like the term Darwin Socialist for the kind of heads on pikes kind of Darwinist. :-) We have to come to grips with the effect of corporate misconduct on our society. When the idea of a limited liability corporation was first utilized to limit financial liabilities for investors I don’t think people realized how effectively the idea could be exploited by the high functioning sociopaths among us to achieve their personal goals behind a corporate veil. A corporation is a conspiracy waiting to happen. Perhaps this is why those who use the word conspiracy are so stigmatized. In a corporate world, conspiracy is the default way of doing business.
This is a short, powerful clip that addresses the exact issue you have identified.
There’s a website for the full documentary, Future Dreaming, There are quite a few short clips on it, organized by topic. The ones I’ve watched so far are amazing. I wish the full documentary were on the web.
Patients who can’t afford medications can find themselves reliant on drug trials and taking risks just for treatment.
“For many academics today, research is not about pushing intellectual boundaries. It is not about investigating a fascinating issue so much as it is about churning out publications, demonstrating impact and generating revenue in order to meet the performance targets upon which institutional reputation and individual careers depend.” – that really sums it up. Well said!
Scientific research, like so much in a society, ultimately depends upon a culture of honor. People need to do the right thing even if it costs them or their friends. For people who are not saints (most of us), this means that you need an economy where not falsifying data (or taking bribes etc.) does not mean that you will get fired and you and your family will get permanently stuck in poverty.
As the United States increasingly merges with the overpopulated third world, it is not just low-skill labor that is harmed: all levels go down, because supply and demand applies to everyone (one is reminded that the policy that gave the United States people like Einstein and Fermi and Szilard was a very restrictive one, that pulled in the best from around the world while limiting the total numbers so that the overall job market was not flooded. Kind of like how companies like Google recruit talent…). So increasingly if you don’t come up with a spectacular result this year, you lose your job, and likely the university will let you go (because there are plenty more where you came from), and if you are lucky you will end up as a temp teaching courses at minimum wage. And forget what you see in the movies: spectacular results don’t occur on schedule, they are built up piece by piece over decades. So you can cut corners, or you can keep your job for a bit longer…
There is a reason that poor countries are so corrupt and riddled with nepotism: when people can’t earn a decent living (or perhaps, any living) by playing by the rules, they don’t play by the rules. I don’t blame the people, of course, I blame the politicians and economists that created those conditions.
In this country things are really not that bad yet, I think because people in senior leadership positions were raised in different conditions, and these people have senior tenured positions. So we are coasting on the past culture of honor. But as these people retire, things will fray fast.
And it’s not just science. How often have you had to bribe a government employee to get something routine done? Possibly never in your life. How soft we have had it. How much we take that for granted. How much we shall miss it when it all goes away…
We are seeing this as well in clinical medicine. Gresham`s Law prevails and indications for surgery and treatment are really whether there is enough evidence to cover one`s butt for malpractice (a very low hurdle). Outcomes don`t matter which was the basis for indications years ago. When one goes to the third world to see a doctor they use the first method….is their money, will they get in trouble if they do it……..and this method of care has now been brought over to the US.
This has been going on for years. Marcia Angell regularly reports on such abuses in the NY Review of Books. Among other things, apparently such drug-company-sponsored research entitles the drug companies to suppress results that are unfavorable.
Others (like Marianne Mazzucato) report that 75% of pharmaceutical innovation comes from government-sponsored research.
In contrast to the actual innovation coming from government research, Angell reports 55% of gross profits from private big Pharma companies are devoted to marketing, while only 15% go to R&D–typically targeted at extending the patent life of existing drugs. Think “time-release Viagra”…
Yet, thanks to the efforts of lawmakers like Bob Dole, even the results of land grant college research can be privatized!
Don’t take any drugs that are new. Take ones that have been on the market and have had time for problems to show up. This article was not about corruption in psychotropic research, but the
corruption there is equally upsetting. Robert Whitaker’s “Anatomy of an Epidemic” describes the
shoddy and corrupt research that goes as science.
1. My initial impression is that Pfizer has a drug and they are searching for a “disease” . As PTSD is a growning market, why better than to come up with a solution drug.
2. A recent article that the interpretation of scans may be wrong. What if the interpretation error rate in 70% not 5%? http://www.pnas.org/content/early/2016/06/27/1602413113
A drug to mimic the effects of marijuana. Heh.
This disgusting nexus if pharma research – absence of control if the research results is very well describes by ben goldacre in bad pharma.
The concept of “publish or perish” has been around for a very long time. The first time I heard this expression was in the 60’s (1960’s). Although the pressure may be much higher now.
What I think is most disturbing in this piece is the ownership of the data and given Ms. Ruffcorn’s difficulty in finding out what she was give in the test is even more disturbing. Why did it take so long? Was it because Pfizer’s lawyers were arguing over telling her, seems unlikely. Was it because Pfizer’s lawyers were arguing over whether or not to lie to her, the likelihood of being caught and the consequences of being caught, seems more likely.
If they told her she had taken the drug being tested they would have possibly opened themselves to a lawsuit or at least unfavorable information about the drug. Would Pfizer lie to avoid that? Do wild bears . . . LOL.
“the data belonged to Pfizer”
This kind of thinking is antithetical to the medical profession: Physician do no harm.
Corporations have trade secret protections for intellectual property. While they are entitled to a private right of action when breaches of their data hurt their bottom line, they should not be entitled to obfuscation of data that demonstrates harm to research subjects. All research subjects should be informed if they have received a drug or placebo after a study is completed.
The publish or perish consequences for “top researchers” who are PhRMA whores puts us at great risk. Peer reviewed journals should be publishing data of drug failures. Otherwise we should expect cherry picked results… and lots of black box warnings in years to come.
Vested interestes dominate research, policy, politics and economics generally and are clever about covering up for their negligence or failures. A view to the big picture is necessary to see through those cover-ups which this article does. Another facet of this story is how marijuana research itself has generally been stifled, outlawed, bogged down in red tape, obscured, and ignored unless the research echos the prevailing view of officialdom that marijuana is bad. The ruse is almost up in terms of a consensus that marijuana should be a Schedule 1 Substance, being the most dangerous. Its efficacy at relieving a wide range of ailments has been documented, often being more effective than prescription drugs and without as many side effects. Post Traumatic Stress Disorder is one serious malady that many sufferers claim can be alleviated with marijuana.
What if there were a conspiracy – might a drug company like Pfiser hope that marijuana’s illegal status be prolonged so they could find a substance that mimics marijuana’s effects and patent it? The market would be much better if prohibition were in effect. Or being as how all the compounds contained in the marijuana plant work synergistically and trying to patent so many synthetic compounds might involve too much research, might a biotechnology company like to modify the genetic material of a common weed to thereby patent it?
The answer to your questions in the last paragraph is YES! And Pfizer is not the only Big Pharma working on such ideas. These companies would dearly love to be able to mimic the effects of marijuana with a patentable drug. Unfortunately for them, fortunately for us, research is already demonstrating that the “entourage effect” of the multiple components of marijuana cannot be replicated by a single-molecule drug. For example, different ratios of CBD (cannabidiol) to THC in medicinal marijuana can produce more or less effective results in treating different maladies and even for different patients with the same malady.
There’s just as much fraud and ‘fudging’ in non pharma controlled studies. It’s apalling. I have friends who work in prestigious places and see it regularly. It’s hard to make progress when half of the truths we believe are lies.
Okay, everyone needs to calm down and take a deep, deep breath.
Have you done so? Try again if you are still too agitated to read information that is not horrific about this system.
1. Pfizer COULD NOT investigate the study being conducted. That would violate FDA rules. The researcher HAS TO BE independent of the company. It is the FDA’s job to investigate the researcher.
2. Pfizer should have notified the woman in question whether or not she had taken the active or placebo far sooner. The woman signed away most of her rights to sue, so Pfizer really did not have a financial incentive to withhold the information.
3. Although a lot of pharmaceutical research is sponsored by the government, that research is in the very, very early stages. It costs a fortune to get a drug approved by the FDA. The vast, VAST majority of that money is spent by the company bringing the drug to market. They then get maybe ten years to try to recoup that expense before the drug goes off patent and generics take over their market. That is a very difficult hurdle to cross.
4. There are bad people doing bad research. There are good people doing good research. There are likely good people doing bad research. There are usually not very many bad people doing good research. That is why the FDA has protocols that have to be followed. That is why the FDA checks. That is why we need an FDA that is not under ANY pressure from politicians to approve things for the politicians’ friends.