When Clinical Trials are Meant for Marketing, not Science

The development of the randomized controlled clinical trial (RCT) was one of the major scientific advances in clinical medicine.  RCTs provide a major part of the evidence underlying evidence based medicine.  RCTs provide a major source of data used by the US Food and Drug Administration, and similar agencies in other countries, to decide whether to approve drugs or devices to manage particular clinical problems.  Unfortunately, with the rise of the RCT came a rise in attempts to suppress and manipulate clinical trials by those with vested interests, often in selling the products and services the trials can evaluate.

Many of the examples we have discussed were of attempts at manipulation or suppression of clinical trials originally done to provide evidence for product approval, or even ostensibly to advance clinical science.  Yet a relatively new article covering evidence revealed from litigation about the drug gabapentin (Neurontin) originally made by Parke-Davis, first a part of Warner Lambert, now part of Pfizer suggests that many clinical trials may not be done to advance science, or even just to provide data to regulators, but only to market products.

The article is Vedula SS, Goldman PS, Rona IJ, Greene TM, and Dickersin K. Implementation of a publication strategy in the context of reporting biases: a case study based on new documents from the Neurontin litigation.  Trials 2012; 13: 136.  Link here.

Methods Summary

The article described a case study based on documents revealed in 2008 litigation.  The purpose of the study was:

to describe the implementation of a publication strategy for off-label marketing of gabapentin, within the context of reporting biases and spin of Pfizer and Parke-Davis’s clinical trial findings, for four off-label uses: migraine prophylaxis, treatment of bipolar disorders, neuropathic pain, and nociceptive pain. 

The study examined documents revealed through the 2008 litigation.  It concentrated on documents relating to the marketing of Neurontin for four off-label indications, that is, for four potential reasons to use the drug which had not yet been approved by the US FDA.  It focused on documents discussing strategies for marketing for these indications, and then documents about particular company sponsored RCTs done for these indications.

Strategies to Sell Drugs

The documents examined by the investigators showed that Parke-Davis prepared "marketing assessments" for four possible indications for Neurontin.  The assessments discussed two possible strategies.  The "indication strategy" would be to conduct trials for the purposes of providing data to the FDA in the hope that the FDA would then approve the new indication for the drug.  The "marketing strategy" would be, in the words of an internal company memo quoted in the study, to conduct clinical trials and

to disseminate the information as widely as possible through the world’s medical literature

In other words, the publication strategy involved using RCTs first for marketing, rather than scientific or regulatory purposes.

How the Trials were Controlled by Marketers 

The publication strategy involved not simply doing trials and publicizing their results, but controlling the messages conveyed by the trials to make sure they primarily supported marketing.  This was done by having the company's marketing department control the content of the trial reports.

 A Neurontin Publications Subcommittee (NTN PSC) was formed within Pfizer and Parke-Davis to implement a publication plan. Minutes from meetings between the NTN PSC and Medical Action Communications (MAC), a medical writing company, indicate that a list of key messages, guiding the content of published reports related to the trials of gabapentin for off-label indications, was developed based on a branding guide

Thus it appeared to be that marketers, not physicians or scientists were in control of supposedly scholarly research publications appearing in medical journals.

The marketers controlled content by controlling those who wrote it.

 A standard operating procedure related to publication of affiliate-driven manuscripts was identified in internal company documents dated October 16, 2002, and it sheds further light on the publication planning process (see Figure 3). (The term affiliate in this context refers to Pfizer’s foreign affiliates, that is, corporations related to Pfizer by either shareholdings or other means of control, including subsidiary, parent, or sibling corporations). According to the internal company documents, 'affiliate-driven manuscripts' were written for Pfizer and Parke-Davis by MAC and sent to the authors for approval. Each article was coordinated by a manuscript team, consisting of representatives from the medical and marketing divisions of the company. The documents also indicate that all affiliate-driven manuscripts should be forwarded to the NTN PSC for review. One of the objectives of manuscripts being reviewed by the NTN PSC was to ‘ensure that they are in-line with current product messages and areas of interest’

Manipulating the Dissemination (and Analysis) of Research Results

Note that the control of publication by marketers describe above involved ghost-writing of articles by medical education and communications companies (MECCs).  Later the article explained that these ghost writers were "not appropriately acknowledged" in the resulting published articles.

The Vedula et al study article provided examples of "spin" in the writing of again supposedly scholarly research publications used in fact to support marketing, and controlled if not composed by marketers.

 We identified spin in publications related to 8/12 trials included in our analysis.... We classified the following as spin: emphasis in the published report on outcomes that were not specified in the study protocol (Study 879–201) ...; conclusions that did not match study findings described in the internal company research report (Study 945–220) ...; extensive rationale to explain away statistically non-significant (unfavorable to the sponsor) findings (Study 945–209; 945–291; No study number - Gorson) ...; conclusion of treatment effectiveness from an uncontrolled study (Study 945–250) ...; emphasis on statistically significant secondary outcomes despite negative findings for the primary outcome (Study 945–271) ...; and an explicit description of an attempt to spin study findings (as described in internal company emails) (Study 945–306).

Note that while most of these examples of spin involved manipulation of the dissemination of study results, that is, doubtful, biased or fallacious arguments based on apparently unbiased data, they also involved manipulation of the analyses (in italics). 

The article also described manipulation of dissemination involving the timing of publication, including delaying publication of an article despite the wishes of the study's investigators because the results were not statistically significant, and hence not favorable for marketing. 

Summary

Documents revealed by litigation about Neurontin in 2004 provided insights about how pharmaceutical and presumably other kinds of health care corporations may conduct systematic, deceptive stealth marketing campaigns to promote their products and services (look here).  We noted that initial media coverage of documents revealed in the 2008 litigation also suggested the existence of a systematic "publication strategy" to control dissemination of results of particular trials, while suppressing trials whose results could not easily be spun to provide support of marketing objectives (look here).

Now the new paper by Verdula et al fill out our knowledge of this case.  The paper suggests that randomized controlled clinical trials may be done not to advance science, or even convince regulators, but primarily to market their sponsors' products.  Thus some significant proportion of the clinical research literature, the literature that physicians and other health professionals have relied upon to make evidence-based decisions for their patients, may exist mainly for marketing purposes.  Even the most rigorous methods used by clinical epidemiologists to review research are meant to discover problems that arose from human error or the inevitable trade-offs made when research is done in the real world, but not deliberately introduced biases and defects meant to promote vested interests.  Thus it is not clear that even the most "evidence-based" medical decisions are based on real scientific evidence rather than the spinning of marketers.

Thus health care professionals, policy makers, researchers, and the interested public need to be even more skeptical about arguments made to promote innovative treatments and other clinical interventions.  However, it is not clear that even rigorous skepticism can defend the integrity of evidence based medicine from marketing disguised as clinical research.

Going forward, we must consider erecting an impregnable barrier between clinical research and those whose primary interest is to make money by selling health care goods and services.  If we do not do that, we will forever need to worry that we really have no idea what "works in medicine," and whether any particular test, treatment, or program provides benefits that outweigh its harms. 

The Editorial that Wasn't: Evidence for Systematic Research Manipulation Undetectable by Critical Review

Woe to those of us who have been advocates for evidence-based medicine.  A short description of the evidence-based medicine is medicine whose practice is informed by critical, rigorous review of the best available evidence from clinical research as revealed by systematic search of the published research literature, as well as by the clinician's understanding of biology and the medical and biopsychosocial context, and by the patient's own values.

Evidence-based medicine is based on some key assumptions.  One is that a systematic review will reveal all the results of research studies that are relevant to the issue at hand.  A second is that while the research studies may be flawed and imperfect, they are reported honestly.

Unfortunately, as we have repeatedly discussed, there is more and more evidence that a systematic review will not reveal all relevant results, because research studies may be suppressed, perhaps often, when their results are unfavorable to vested interests.  (Look here for further discussion.)  There is also considerable anecdotal evidence that the design, implementation, and analysis of research studies may be manipulated to make the results more likely to favor vested interests.  (Look here for further discussion.)

An article by Vedula et al just published in the renowned New England Journal of Medicine suggests that such manipulation might be systemic, and that the reporting of manipulated studies may not clearly show what manipulation was done. (1)

To summarize, the authors got access through legal proceedings to internal study protocols and research reports from clinical trials sponsored by Parke-Davis (later merged into Pfizer Inc) of gabapentin (Neurontin) for a variety of clinical problems other than seizures.  Gabapentin was originally marketed as an anti-seizure drug, but Pfizer later "admitted guilt for off-label marketing."  (We discussed the stealth marketing campaign for Neurontin here.)  Vedula et al compared the primary outcome variables specified in the original research protocols, internal research reports, and any publications of the trials' results.  They identified 21 trials, 13 of which were published.  For 12 of the 13, the authors had access to the internal protocol, report, or both.  The main results were:

For 8 of the 12 published trials, there was a disagreement between the definition of the primary outcome in the protocol and that in the published report.... Sources of disagreement included the introduction of an entirely new primary outcome in the published report (in the case of 6 trials); failure to distinguish between primary and secondary outcomes in the published report, even though the protocol did distinguish between them (2 trials); relegation of a primary outcome in the protocol to a secondary outcome in the published report (2 trials); and failure to include in the published report one or more primary outcomes specified in the protocol (5 trials).

Furthermore, it appeared that failure of published articles to clearly and fully report results in terms of the original, pre-specified primary outcome variables occurred when these comparisons were not favorable to gabapentin. As the authors summarized:

Thus, trials with findings that were not statistically significant (P≥0.05) for the protocol-defined primary outcome, according to the internal documents, either were not published in full or were published with a changed primary outcome.

As shown in Figure 3, all the changes that took place between what was specified in the protocol, what was known before publication (as presented in the internal company research reports), and what was reported to the public led to a more favorable presentation in the medical literature of gabapentin's efficacy for unapproved indications.

They concluded:

We are concerned that the reporting practices observed in our analysis do not meet the ethical standards for clinical research or maintain the integrity of scientific knowledge. Fair and honest treatment of patients enrolled in clinical trials of any kind requires full, open, and unbiased reporting. Journal publication, a formalized platform for scientific discourse and dissemination of knowledge, should not be used as a marketing tool for off-label drug use.

Reporting biases such as those we describe here increase the likelihood that interventions will appear to be effective when they are not. Such biases can lead to the omission of negative findings in systematic reviews of intervention effectiveness and in evidence-based guidelines. For example, the 2005 Cochrane systematic review regarding the effectiveness of gabapentin for acute and chronic pain concluded that it is effective on the basis of published findings and should now be updated with the inclusion of unpublished information made available through litigation.

I believe that the article by Vedula et al is particularly important because it shows what appears to be systematic manipulation of the analysis and reporting of multiple clinical trials of the same drug (but for different indications) that had the effect of making the drug appear efficacious when it likely was not. Furthermore, the manipulation was concealed. The published research articles did not completely describe what the intended primary efficacy outcome variables were, did not provide results in terms of these variables, and instead provided results only in terms of variables that were chosen post-hoc as new primary outcome variables.

To address that latter point, I independently reviewed three of the research publications cited by Vedula et al.(2,3,4) All three were noted to have reported significant results favoring gabapentin in terms of a primary efficacy outcome variable that was not identified as such in the corresponding studies' original research protocols or reports (see the supplementary data provided with the New England Journal article.) Per my review, none of the three published articles offered any hint that what they reported as primary outcome variables were not the variables originally chosen in that capacity, nor did they identify what those original primary outcome variables were, or how comparisons made using them turned out.

The implications of the article by Vedula et al are very important. Hence, I was surprised that the article appeared without an accompanying editorial to discuss these implications, and that its publication did not generate much media interest.

So that gives me an opportunity to comment further.

One could start with the implications for evidence-based medicine. As noted above, a short description of the evidence-based medicine is medicine whose practice is informed by critical, rigorous review of the best available evidence from clinical research as revealed by systematic search of the published research literature, as well as by the clinician's understanding of biology and the medical and biopsychosocial context, and by the patient's own values. Evidence-based medicine depends on critical, rigorous review, but the review process is generally done under the assumptions that research publications honestly describe what was done and what its results were. The review process was never designed to detect dishonest reporting or find information that was deliberately concealed. Manipulation of research (design, analysis, and implementation), concealment of that manipulation, and outright suppression of research threaten the foundations of evidence-based medicine. Yet the article by Vedula et al is part of a growing body of evidence that such manipulation, concealment and suppression are widespread, and done to serve vested interests, often commercial.

That is a huge problem for proponents of evidence-based medicine, but also for physicians who want their practices to be based on science, for patients who want their care to be based on science, and for all those in society who see the advancement of medical science as a way to improve peoples' lives.

As senior author Professor Kay Dickersin noted in an interview with Bloomberg "The trouble is, as a scientist, the publication has always been held up to me as the truth. It's the scientific record. What this study indicated is we can't believe that record."

Furthermore, to be a bit more concrete, most physicians, patients, and policy makers depend on what appears to be honest clinical research to make decisions about individual care and health policy. Deliberate and deceitful manipulation of clinical research to favor sponsors' products has likely lead to excessive use of and payments for drugs and devices that are less effective than advertised, if not useless or dangerous. Thus, it is likely that such manipulation is partially responsible for ever increasing health costs and poor health outcomes.

Finally, we need to start thinking about how we can detect and compensate for manipulation of clinical research in the past, and deter such manipulation in the future. One possible deterrent would be, as was noted by Vedula et al, detailed clinical trials registries that contain complete information about trial protocols.  For this to be effective, there need to be mechanisms to assure compliance, and penalties for non-compliance.  Moreover, since clinical research is now global, the registries must have global scope, and enforced assurance of compliance must also be global.

Registries might decrease future manipulation and suppression of research.   No one has suggested, as far as I know, a systematic way to detect and correct for previous manipulation. It would require a major, global investigative effort to uncover manipulation, and it would be a major scientific and policy endeavor to reveal most suppressed research and correct most manipulations.

However, before anything is done, patients, physicians and policy makers must acknowledge and understand the problem.  Yet it seems that even discussing these may be topics that are very uncomfortable for some of us. The longer we shrink from addressing them, however, the worse will be the results for patients, physicians, science and society.

References


1. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored tirals of gabapentin for off-label use. N Engl J Med 2009; 361: 1963-1971. Link here.


2. Mathew NT, Rapoport A, Saper J et al. Efficacy of gabapentin in migraine prophylaxis. Headache 2001; 41: 119-128. Link here.


3. Vieta E, Goikolea JM, Martinea-Aran A et al. A double-blind, randomized, placebo-controlled, prophylaxis study of adjunctive gabapentin for bipolar disorder. J Clin Psychiatr 2006; 67: 473-477.  Link here.


4. Caraceni A, Zecca E, Bonezzi C et al. Gabapentin for neuropathic cancer pain: a randomized controlled trial from the gabapentin cancer pain study group. J Clin Oncol 2004; 22: 2909-2917. Link here.