Seeding Trials
By Bruce W. McNulty, MD
This article is part of our MedRIB project and was published September 17, 2008.
Medical Research Integrity Blog (MedRIB)
Physicians in training are taught that when ordering tests and prescribing drugs, they should do so based on sound medical evidence, which, by and large, comes from medical journals. But what would be the effect if the evidence in these journals were presented in such a way as to mislead readers? If done cleverly, one might believe that an inferior medication or test was the best medical option. Who would benefit from this? Above all, those with a financial stake in the sale of such tests or drugs. And yet, the implications are serious. Not only could this lead to the provision of substandard care, it also puts at risk the credibility of American research. Is this a problem? This is the question that will be explored in MedRIB.
Vioxx is a medication in the NSAID group, similar to Motrin/Ibuprofen, but one that had the apparent advantage of not causing the gastric/stomach side effects that this group of medications caused. Given the many patients with chronic pain and arthritis that need these medications on a daily basis, this medication had the potential for a huge market in the U.S. and abroad. However the medication was withdrawn from the market by its manufacturer Merck Pharmaceuticals, after an increase in cardiovascular death was noted in clinical trials. The potential misrepresentation of this evidence has been widely discussed. There has been extensive reporting on matters pertaining to Vioxx, but a helpful overview of the events appears in Wikipedia at http://en.wikipedia.org/wiki/Vioxx. (For further information, readers may be interested in the book just being released in mid-September 2008 called, Poison Pills: The Untold Story of the Vioxx Drug Scandal, by Tom Nesi. Although we have not yet had a chance to read this book, it appears to be a thorough review of the Vioxx case.)
Rather than review the Vioxx case generally, however, I would like to discuss a paper published in August 2008 in the Annals of Internal Medicine entitled “The Advantage Seeding Trial: A Review of Internal Documents,” by Ross et al. The link can be found at http://www.annals.org/cgi/content/full/149/4/251 while an accompanying editorial can be found at http://www.annals.org/cgi/content/full/149/4/279
Although long suspected, but difficult to prove, the concept of a ‘seeding trial’ is this: just prior to or at the time of a new drug’s launch into the market, the pharmaceutical
companies who developed the drug conduct a ‘study’ that enrolls selected doctors to prescribe a drug to patients under the guise of a research study, when, in fact, the purpose is to fulfill marketing objectives.
Clearly, if this is being done, it would be difficult to prove with certainty. The issues with the ADVANTAGE (Assessment of Differences between Vioxx and Naproxen To Ascertain Gastrointestinal Tolerability and Effectiveness), trial discussed in this paper were only discovered because the authors were expert witnesses for plaintiffs involved in the litigation against Merck with regards to Vioxx. This gave these physicians access to internal documents at Merck that made the true purpose of the ADVANTAGE trial apparent to them.
The article describes how, with review of internal company documents, in January 1999, before the launch of Vioxx (but after FDA approval), the marketing division conceived the ADVANTAGE trial. Six hundred investigators were enrolled and 2785 patients were randomly assigned to Vioxx vs. 2772 to generic naprosyn. A Merck marketing slide described the goal of the trial was for investigators to “gain experience with Vioxx prior to and during the critical launch phase”; clearly not to assess the GI side effects of the medication, which was the stated purpose of the trial.
The author’s review of the documents describe 3 key themes related to the design of the trial: it emerged from the marketing department with a marketing objective; the data was collected analyzed and disseminated by the marketing division; and the company did not reveal the true purpose of the trial to the physician or patient participants or their review boards.
The authors were able to find a memo nominating the trial for a marketing division award! They quote directly from the nominating memo:
First, the trial was targeted to a select group of critical customers. The clinical trial program for VIOXX focused primarily on specialists. While they would be critical to the early uptake and advocacy for VIOXX, the large majority of prescriptions in the arthritis and analgesia market (~60%) come from primary care physicians. The ADVANTAGE trial utilized this important group of prescribers as investigators…the design of the trial focused on demonstrating the value of VIOXX to this important audience.
The paper describes the first author on the trial, Dr Jeffrey Lisse, a physician not employed by Merck, telling the NY Times that he did not have a role in data collection or analysis, but that the company came to him after the study was complete and initial paper was written, saying ‘ we want your help on this paper’.
In their discussion Ross et al go on to describe how at least 3 elements of seed trials that are harmful:
• Full consent of patients in not possible without disclosing the purpose of the study
• Good research practice is at risk when marketing divisions design and conduct studies. To fulfill the marketing objectives, doctors are targeted based on certain demographics, not on scientific principles
• The study may have little scientific merit. In fact, seeding trials in which the medication has yet to receive FDA approval may cause patient harm for marketing purposes only
Given how difficult it may be to identify this practice, increasing awareness is an initial step. Ross et al. appropriately state that greater transparency into the clinical trial process — including clinical trial registration, and requirements for study protocols to be included with institutional review board submissions — may help better identify this practice. Clearly patients and doctors must be made aware of the true purpose of a study if they are to give informed consent.
The Case of Ketek
By Bruce W. McNulty, MD
This article is part of our MedRIB project and was published July 31, 2008.
Let’s go back about a year to an article in the April 19, 2007 edition of the New England Journal of Medicine, which describes the FDA approval of the antibiotic Ketek (telithromycin) and the subsequent problems associated with it. The full text article can be found at :http://content.nejm.org/cgi/content/full/356/16/1601, and reply from the FDA can be found at : http://content.nejm.org/cgi/content/full/356/16/1675.
The article is entitled The FDA and the Case of Ketek, by David Ross, M.D. Ph.D., who is a former FDA physician and was involved in the review of this medication. The paper presents a time line of the FDA’s review of this medication for eventual approval for use in bacterial sinusitis, exacerbation of chronic bronchitis, and community acquired pneumonia. According to Ross, in the first round of reviews, safety concerns were raised regarding drug interactions, effects on vision, and issues with liver toxicity. The FDA requested that the manufacturer, Sanofi-Aventis, undertake a study to investigate the safety of the medication in typical patients. That investigation, known as, “study 3014” was undertaken to look at adverse effects of this medication and compare them to a well known popular antibiotic, Augmentin. More than 1800 physicians, many of them new to clinical research, were recruited and paid up to $400 for every subject they enrolled in the study. In 5 months, 24,000 subjects were enrolled. According to Ross, soon after the study began a routine investigation of the physician who enrolled the most patients (over 400), revealed fraud and patient fabrication. Ultimately, a large number of research sites were referred for investigation, but the study was still allowed to be used in the FDA approval process. If you follow the time line in the paper, it can clearly be seen that there were serious problems with the study. Yet, despite all the issues raised, the drug was approved. In 2006, cases of Ketek-associated liver damage and failure began to surface, and the medication now has a ‘black box’ warning placed by the FDA. This has led to congressional investigations of the process of drug approval by the FDA. Type in Ketek in any search engine and you will find links to more law firms than health sites. This also got a fair amount of notice in the lay press, in articles such at this: http://www.post-gazette.com/pg/06164/697890-114.stm
This case also shone a light on the practice of using ‘non-inferiority’ trials. To understand this, we should first note that there are conditions that often self-resolve with no treatment at all such as sinusitis, ear infections, and ‘bronchitis’. With these conditions, the appropriate first step in testing should be to compare the trial drug against a placebo. That way, only if the drug shows some significant benefit against a placebo, should it be considered for approval. With a “non-inferiority” trial, the new medication is tested against another drug on the market and if it is shown to be ‘non inferior’ to the older medication, it may be given approval – as was done for Ketec. Thus, this medication, which upon its wide scale release was found to have a variety of toxicity issues, was approved for rather benign conditions such as sinusitis and bronchitis. These conditions are often self limiting, and had this drug been tested against a placebo, it may have been shown not to be needed at all! The question must also be asked that if this new medication is really not better than an older (and invariable cheaper and better tested medicine on the market for years) medicine, why approve it at all?
So what we have illustrated by this case is a medication approved by the FDA for largely benign self limiting conditions, which appears to not have been properly tested, putting patients at risk. Furthermore, the credibility of the FDA in protecting consumer well-being in these matters was damaged.
Concerns with Contract Research Organizations
By Bruce W. McNulty, MD
This article is part of our MedRIB project and was published July 09, 2008.
Two articles on Contract Research Organizations (CROs) have appeared in the New England Journal of Medicine in the last 8 months that deserve a few comments.
In the Oct 4th, 2007 issue, Miriam Shuchman, MD wrote an article entitled, “Commercializing Clinical Trials – Risks and Benefits of the CRO Boom.” The article describes something that many are likely unaware of, which is that a lot of research, primarily from pharmaceutical companies, has moved out of the companies themselves, as well as universities, into what has become known as Contract Research Organizations or CROs. The article provides great detail on the structure and functions of these organizations. It can be found at http://content.nejm.org/cgi/content/short/357/14/1365, although a subscription is required to read the entire text.
The article describes how in about five years CROs have grown from a $7 billion to an almost $18 billion industry and are known for the speed at which they can deliver results. It notes that in 2004, the 10 largest CRO’s enrolled more than 640,000 patients in clinical trials looking at the effects of medications and therapies.
According to Shuchman, the problem arises in part because “drug regulations were written in the 1970s, about a decade before the contract research industry emerged, and thus don’t really address the problem of CRO accountability.” Dr. Shuchman does not explain in what ways industry regulations do not apply equally to big pharma and CROs, however, what the article clearly indicates is that in outsourcing research generally and clinical trials in particular, the CROs are able to skirt regulatory controls such as those intended to ensure subject safety, methodological rigor, and the sharing of experimental outcomes. This idea was echoed by Michelle Mello from the Harvard School of Public Health, who stated that CROs “fly under the radar.”
According to Shuchman, because of the lack of clarity as to how these regulations apply to CROs, there have already been problems associated with FDA approval of certain medications, including Ketec, which we will discuss in the next post. Shuchman’s paper also discusses other concerns as well, such as the inexperience of the research staff, lower pay scale, and pressure on the CRO’s to deliver positive research to continue to get contracts. An alternative model called Academic Research Organizations in a university setting uses faculty members to oversee research. These tend to be less efficient that CRO’s however, and also face considerable pressure to show results that support the sponsoring company’s study designs and protocols or risk losing the contract to a CRO.
Unrelated to the actual quality of the research itself, but important to be aware of as part of the discussion, is the treatment of study subjects themselves. If you look at the May 29th, 2008 NEJM at http://content.nejm.org/cgi/content/full/358/22/2316?query=TOC
you will find a fairly frightening description of where test subjects may be found to participate in clinical research. It raises the issue, “Is it ethically problematic to pay poor people to test the safety of new drugs?” The practice of paying people to participate in trials is not new, but by moving this research out of the labs of the drug companies or universities, into the less regulated world of CROs, is raising concerns about the ethical dimensions of this work. According to the article, only 16% of academic health centers provide injured subjects with free care, as for the CROs, the number is not reported. Amazing! There is an implied warning here: be careful before you sign that waiver.
The article goes on to point out that many of these issues are rising from the transformation of clinical research “into a business” rather than something pursued as academic research. Unfortunately, the FDA seems ill-equipped to provide the oversight needed in this area of research. Neither article implies that CRO’s don’t and haven’t done strong work. However, the concern that remains is the lack of oversight when dealing with human test subjects and medical research needs to be addressed.
A Growing Awareness of a Problem: Focusing Our Microscopes on the Elephant in the Lab
By Bruce W. McNulty, MD
This article is part of our MedRIB project and was published April 24, 2008.
It would appear that public awareness of the issues we are discussing concerning the credibility of medical research data is increasing. It is not hard to find frequent references to this in the popular press. Two recent examples, one in The Chicago Tribune and another in The New York Times, provide examples of the problem of research integrity and how some doctors are taking measures to safeguard their reputations.
Let’s look first at the troubling story in the April 6, 2008 edition of the Chicago Tribune, which can be found at the following link:
http://www.chicagotribune.com/news/chi-birth-control1apr06,0,7409040.story
The article entitled “Drugmaker hid contraceptive data,” by Gardiner Harris and Alex Berenson, reports that according to internal company documents, for years, Johnson and Johnson “obscured evidence that its popular Ortho Evra birth control patch delivered much more estrogen than standard birth control pills, potentially increasing the risk of blood clots and strokes.” According to the article, this has resulted in more than 3000 lawsuits against the company in which it has been claimed that “use of the Ortho Evra patch caused heart attacks, strokes and, in some cases, death.” Documents made public due to the lawsuit show that even before FDA approval, the company’s own research revealed that the patch delivered far more estrogen than that which was reported to the FDA. The FDA then received multiple reports of problems with the patch from 2002-2006, but did not warn the public until 2005. What is not clear from the article is what data the FDA relied on to approve the medication and from where it originated. The company now hopes to avoid being successfully sued, based on the fact that the medication was FDA approved, despite the inaccuracies of its estrogen dosing. The legal argument is called pre-emption and will be ruled by the Supreme Court in the fall. Stay tuned.
Indicative of just how damaged the credibility of industry funded research has become, take a look at this article from The New York Times, found at the following link: http://www.nytimes.com/2008/04/15/health/15conf.html?pagewanted=1&_r=3&ref=health&adxnnlx=1208279239-xqTqRAQmrU5NBWie7VgYhQ
Entitled, “Citing Ethics, some Doctors are Rejecting Industry Pay,” the article notes that due to increasing public awareness of conflict of interest issues, some prominent physicians have made the decision to stop taking payments from food, drug and medical device companies. If they do work for these companies, they do it pro bono. The article describes the striking story of three physicians, among them, Dr Peter Libby, chief of cardiovascular medicine at Brigham and Woman’s hospital in Boston. As a highly respected scientist, his views were often solicited and and like many well known researchers, for many years he served as a speaker and consultant paid by the pharmaceutical industry. Recently, after spending four years working for free with PBS to create the TV series “The Mysterious Human Heart,” his work was accused of bias by ‘bloggers’ and others. Although there is nothing to suggest that his work was compromised by the monetary relations he had with the pharmaceutical companies, he came to the decision that he could no longer continue “under suspicion,” and for this reason, he decided to sever his ties to industry as a paid speaker and consultant. Two other prominent doctors who made similar decisions to avoid ‘the asterisk’ next to their names associated with physicians who choose to accept these payments are profiled in this article. As always, these are complicated issues. Dr. Libby indicated that he would continue to speak at forums sponsored by the pharmaceutical industry and consult for companies, but he would no longer accept payments. “It is not worth it to be under suspicion,” he said. So we see here the dilemma: some good scientists have been sensitized to the conflict of interest risks and they are taking a stance to safeguard their reputation and personal integrity. The question remains, how can we safeguard the reputation and integrity of the industry after it has been seriously damaged? Certainly, much work is needed.
(#003 2008-04-24)
Research Integrity and Evidenced Based Medicine
By Bruce W. McNulty, MD
This article is part of our MedRIB project and was published March 13, 2008.
As we discussed in our previous blog entry, central to our understanding of medical research integrity is the notion of Evidenced Based Medicine (EBM). Physicians in practice like to think that they treat their patients based on firm scientific knowledge that follows from sound clinical studies such as those that are typically performed in a double-blind type fashion to avoid the possibility of chance and bias. Past reports have shown, however, that physicians often base their practices on other things: their anecdotal experience, advice from colleagues, local practice patterns, and advice from industry sales representatives. Following these studies, a few years back, a movement started called ‘Evidence Based Medicine’. Its concept is clear and straight forward: treat patients based on the best, most highly rated medical evidence.
The goal of those involved in EBM is to create easily disseminated information for physicians that will help them consistently manage the day to day medical issues that they encounter in their practice. For instance, The American Heart Association (AHA) may bring experts together to review current literature and then provide evidence-based guidelines on the treatment of victims of acute myocardial infarctions, i.e., heart attacks. The AHA experts may then go as far as creating goals for doctors and institutions to meet in terms of percentages of patients that get certain recommended treatments or medical interventions. The extent to which these goals are met can then be used for assessment purposes to generate a physician or hospital “report card” by insurance companies or the federal government’s Centers for Medicare and Medicaid Services (CMS). Regional data may become public record, and reported on the front page of local papers with top performing hospitals being able to tout their record of excellence. CMS has also begun a reimbursement system called pay for performance in which the payment of hospitals and physicians can be either positively or negatively affected based on meeting EBM goals. More can be found at the CMS website at: http://www.cms.hhs.gov/MedicaidSCHIPQualPrac/04_P4P.asp. Thus the stakes are high for both health care providers, and the patients they serve.
What happens, however, if the studies on which evidence based guidelines are created are biased? Clearly, the financial implications are significant. Let us restate our question: What happens if a scientist’s or company’s financial interests exert undue influence on the research that is being conducted, presented, and published? There is growing evidence that such vested interest often has a profoundly negative influence on the conduct of medical research. This blog will look back at some examples of what has occurred in this regard and, where appropriate, we will try to propose some solutions to this very serious problem.
As a start, let’s consider a paper recently published in the New England Journal of Medicine (January 17, 2008, Volume 358:252-260, No. 3): “Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy” by Erick H. Turner, M.D., Annette M. Matthews, M.D., Eftihia Linardatos, B.S., Robert A. Tell, L.C.S.W., and Robert Rosenthal, Ph.D. A link to the summary of this article can be found at: http://content.nejm.org/cgi/content/short/358/3/252?query=TOC
This article looks at studies that were registered with the FDA on the efficacy of antidepressants. The authors analyzed the registered studies and were able to divide the studies into those that showed a positive or beneficial effect of antidepressants vs. those whose results were either negative or questionable at best. They then looked at which studies were then actually published in the medical literature for doctors to see, and found that the vast majority of positive studies were published, and the majority of negative ones were not. They also found that many of the questionable studies, in their opinion, were often ‘spun’ to be more beneficial than the data supported in their conclusions. Turner et al. found that if one contrasted published studies on the effectiveness of antidepressants to all the registered FDA studies, 94% of the published literature provided evidence in support of the benefit of antidepressants, whereas if unpublished studies were included in the analysis, only 51% of clinical trials showed positive results from the use of antidepressants.
This presents a clear illustration of the issue of publication bias. For a variety of reasons, articles showing a benefit of treatment may be more likely to be published than those showing no benefit. It may be that a journal would prefer to publish a study that has a positive outcome and could affect patient management. It may be that those doing the work are selectively submitting positive studies to journals because the work is being sponsored by the manufacturer of the medication or treatment being tested. What the medical practitioner is left with in this case however is published data showing the vast majority of studies supporting the use of antidepressants. Following the creed of Evidenced Based Medicine, a physician would seem to be on solid ground by basing treatment on the published findings. CMS and medical societies could point to sound published evidence to create treatment goals for the diagnosis of depression. Yet clearly the entire story has not been told by the published literature.
It would seem that there is a solution to publication bias: that any study that wants to be published must register prior to the initiation of the trial. And even if not published, all data from such trials must be readily available to medical practitioners as well as the experts creating guidelines for us to follow. Evidence based medicine must be based on all the evidence available, not just that selected by those that may have a vested interest in the results.
MedRIB: The Medical Research Integrity Blog
By Bruce W. McNulty, MD, Robert E. McNulty, Ph.D.
This article is part of our MedRIB project and was published February 12, 2008.
Today’s medical practitioners are facing a serious problem: the information they depend on to treat patients is sometimes wrong or based on misinformation. Worse still, the problem is not simply a result of the general limitations of human knowledge or the unavoidable variations in practitioners’ knowledge. The problem is more serious because it follows from a systemic breakdown within the current research establishment that generates the information essential to the professional practice of medicine.
The Goal of this Blog
We are not denying that medicine is progressing, but its progress is being undermined because the research on which doctors and other healthcare professionals rely is too often skewed, biased, or misleading in order to support commercial interests. The most prominent example of this pertains to the pharmaceutical industry. Let us be clear, we recognize that the partnership between the medical community and the pharmaceutical industry is vital and symbiotic. But when advances in medical research are subverted for the sake of commercial interests, the consequences can be catastrophic. In the short term, patients may die. In the long term, the effects are deeply corrosive to the medical profession as a whole — the credibility of the research establishment itself is put in question, and with that, the dividing line between science-based medicine and commercially motivated quackery will become blurred. Patients will be fully justified in wondering whether the prescription they are being given represents the treatment most conducive to their health or most conducive to the bottom line.
In this Medical Research Integrity Blog (MedRIB), our goal is to initiate a discussion that focuses on issues pertaining to the integrity of research in those fields that support the medical profession, including pharmaceuticals, medical technology, and services supporting the medical field, etc. — and among these, the greatest attention will be given to pharmaceutical research, since it is here where the problems appear most serious. Besides focusing on problems that we believe deserve greater public attention, we will also look for solutions.
What Do We Mean by “Medical Research Integrity?”
We understand medical research integrity to be found in the intersection of medical ethics, research ethics, and business ethics. When the ethics in these three areas are sound, so too will be the research on which those in the medical field depend. We could go to great lengths to describe the characteristics of ethically sound medical research, but for now at least, this strikes us as unneeded. The essence of what guides us is this:
- Medical decisions should be made to serve the best interests of patients, by contributing to their health and well-being.
- Medical research should be conducted with strict observance to the best scientific practices, adhering rigorously to principles of objectivity and impartiality;
- Business interests should never be allowed to exert any distorting influence on the presentation of medical research that would in any way undermine the guiding principles of sound medical and research ethics.
Medical research integrity is what we take for granted when we read research reports. However, the integrity of medical research is damaged whenever medical and research ethics are compromised for the sake of commercial or related interests. If violations to medical research integrity were rare, there would be no need for this blog. This is not the case. The words in this blog, we hope, will be a prelude to reform and a renewed commitment to research integrity.
Who We Are: Applied Ethics, Inc. and You
For the foreseeable future, in this blog we hope to offer a periodic examination and conversation on the integrity of research pertinent to the medical field. The primary author of MedRIB is Bruce W. McNulty, MD. Bruce received his MD degree from Georgetown Medical School in 1986. After his medical schooling, he attended a residency in Internal Medicine in Boston. He has practiced full time emergency medicine since finishing his residency, first in the Boston area, and for the past 16 years in Chicago. He is board certified in Internal Medicine and Emergency Medicine. He has spent a significant portion of his career in administrative medicine, acting as Chairman of Emergency Departments in Chicago (and currently holds that position at a busy ED in a Chicago hospital). Bruce is also the Vice President of Applied Ethics, Inc.
MedRIB’s editor is Applied Ethics, Inc.’s founder, president, and chair, Robert E. McNulty, Ph. D. After having one career in international business, he returned to graduate school and received a Ph. D. in philosophy and education from Columbia University, Teachers College. The focus of his graduate research was ethics, and after stints at teaching at Columbia, the State University of New York at New Paltz, and at Bentley College, he joined the Center for Business Ethics at Bentley College where he is the director of programs.
Besides Bruce and Bob, MedRIB will enlist the views of other well qualified guest writers. Finally, we welcome your comments, suggestions, and any kind of feedback you would like to give us. Please address these to . For reasons primarily to do with manpower, at least initially, comments will not be posted directly on the blog. However, we will read your comments carefully and, where appropriate, we will respond either by email or through the blog.
In that regard, please do let us know what you think. The need for reform is undeniable – we hope MedRIB can help.