Restoring Invisible and Abandoned Trials: A Creative Approach to a Public Good; Now a Creative Approach to Implementation is Needed
Margaret A. Winker, MD, Senior Research Editor, PLOS Medicine, and Virginia Barbour, MD, Chief Editor, PLOS Medicine, on behalf of the PLOS Medicine Editors.
The power of the concept that making results of all clinical trials accessible to researchers, clinicians, and the public to promote transparency and make the best evidence available for informed clinical decision making is not only self-evident, but is illustrated by the number of organizations that have signed on to the AllTrials initiative (of PLOS, among others) as well as its translation into 30 languages. However, it is one thing to support trials and transparency and it is quite another to actually get the job—of analyzing, authoring, peer reviewing, and publishing the trials–done. Getting the job done for unpublished, distorted, or abandoned trials is even harder and is the goal of the initiative Restoring Invisible and Abandoned Trials (RIAT), published today in the BMJ with an accompanying editorial signed by, in addition to BMJ’s Clinical Epidemiology Editor and Editor-in-Chief, PLOS Medicine Chief Editor Ginny Barbour and myself on behalf of the PLOS Medicine editors.
As a first step, RIAT authors Peter Doshi and colleagues offer to potential “restorative” authors around 165,000 pages of previously confidential company research, many of which pertain to drugs currently prescribed for patients. Restorative authors are encouraged to declare their interest, encourage the owners of the data time to submit them, then adopt the trials, write them up for publication, and submit them to participating journals. However, this apparently simple process to achieve a public good opens a barrel full of worms when putting it into practice. Therefore, while PLOS Medicine enthusiastically supports the RIAT initiative, we also raise some of the issues this initiative poses. We consider this new approach to publishing studies an experiment, and we wish to get input from stakeholders–researchers, authors, industry, and the public—regarding what they think of the proposal, the issues, and the proposed solutions. We will gather feedback, begin the experiment, be as transparent as possible, and publish on our experience.
Doshi and colleagues have outlined the steps a RIAT author needs to take in submitting a RIAT article (their Box 2):
Proposal for restoring invisible and abandoned trials (RIAT)*
1. Obtain clinical study reports and any other study data
2. Collect documentation of trial abandonment
For unpublished trials—No primary publication detected by systematic search of the literature and/or confirmation from original trial sponsor or current responsible party that no publication exists
For misreported trials—Evidence of misreporting (ideally, published letters or other articles in the scientific literature or documentation of communication with the original trial publication author(s) detailing the misreporting) and a failure to correct the scientific record.
3. Issue a “call to action” by publicly registering your possession of data sufficient for publication
At least initially, this should be by an electronic response to this article and should include, as a minimum, trial identifiers, number of participants, date completed, publication status, pages in your holding, and level of access to trial data. This declaration offers original sponsors and trialists an opportunity to publish or formally correct their studies within the next 365 days. Send a copy of the rapid response by email to trial sponsors (and authors, for published trials), requesting confirmation of receipt.
4. Collect documentation of the need for restoration
Save time stamped copies of all rapid responses to this article (or other relevant websites) to document the time elapsed and consequent need for third party restoration.
5. Presubmission inquiry to RIAT friendly journal
Present editors with documentation from steps 1-4 and seek confirmation of editors’ interest.
6. Prepare and submit manuscript according to RIAT procedures
• Include explanation (with references) in the Introduction of why this trial is being restored
• Provide auditable record of decisions (use RIATAR template), documenting which parts of the clinical study report (page number and paragraph) were used
• Report analyses specified in protocol
• Denote any analyses that were not prespecified
• Make all underlying data available electronically
Just as any other study, a RIAT study submitted to PLOS Medicine will undergo review by the editorial group, an Academic Editor, and external peer reviewers if it meets our criteria for scope and methodology. Ultimately, published studies must meet our criteria for methodology, ethics and reporting and addresses the major biological, environmental, social, or political determinants of health. However, RIAT raises unique issues, some of which are outlined in their Box 3:
Issues for further discussion and development*
• Should the original trial investigators have the right to join a group of RIAT authors?
• Should RIAT authors contact original investigators for help in interpreting trial documents?
• Should old trials that are going to be subject to RIAT publication be registered in trial registries such as ClinicalTrials.gov (assuming this becomes possible)? Should registration preclude other restorative author teams from using the same data?
• Should RIAT publication related clinical study reports and audit records be stored on the publishing journal’s server or is a publicly available database required?
• Where should the declaration of intent to publish a RIAT trial be published?
• How should RIAT publications be cited? Should restoration be clearly identified in the publication title or the authorship byline? If yes, how?
• What (if any) additional rules or safeguards should RIAT authors apply to ensure patient confidentiality of individual patient data and clinical report forms that are already in the public domain?
• For misreported articles, what level of narrative detail regarding the misreporting should the RIAT publication contain?
• Should RIAT authors seek approval from an appropriate research ethics committee before publishing a trial with unclear prior ethical basis?
While some trials may have not been reported because the evidence did not support the manufacturer’s hopes of efficacy, in other cases the trial may have been flawed, or worse, unethical. The trial may not have been registered in the timeframe required of the journals that require prospective trial registration (which includes PLOS Medicine). Rejecting the trial is the usual approach to these issues, but then how will these trials be made available? It is certainly possible that some RIAT studies will never find a home, and the trials unpublishable in a peer-reviewed journal may need to find another way to see the light of day (in, for example, clinicaltrials.gov). On the other hand, submitting trials to science’s standard of best evidence, the scrutiny of peer review and editorial review, revision, and editing, seems ideal and something we want to encourage. Therefore, the approaches below differ in some cases from the journal’s usual policies, which will not change. Published studies will be designated as RIAT to ensure that this distinction is clear and the explanation regarding RIAT-specific steps will be included in the study Methods, since they are integral to the study report.
Establishing restorative authorship: How will restorative authors determine whether the investigators intend to publish? It is not the intent of the initiative that investigators lose the opportunity to author their own research. Therefore, in addition to establishing that the investigators or funders did not respond to the RIAT proposal article via a BMJ Rapid Response, restorative authors will need to provide evidence that they have contacted the investigators and the investigators have either indicated that they do not intend to publish or that the investigators did not respond over a 60 day period. This time period helps to ensure that restorative authors will not scoop impending publication and avoids a greater delay that would defeat the purpose of the initiative.
Authorship criteria: The RIAT authors state, “RIAT authorship, whether of industry or non-industry trials, will not usurp proper credit for a trial.” Just as any other author, restorative authors must meet the ICMJE authorship criteria : “1) substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; 2) drafting the article or revising it critically for important intellectual content; and 3) final approval of the version to be published. Authors should meet conditions 1, 2, and 3.” The first criterion will need to be met by “analysis and interpretation of data” since the other steps have already been completed. Therefore, restorative authors should meet authorship requirements using established criteria.
Trial registration: The trials on the list provided by Doshi and colleagues were with rare exception completed before 2005, and therefore would be eligible for late trial registration under the current trial registration policy. The trials are unlikely to have been registered, and one goal of the AllTrials initiative is for all trials to be registered. However, restorative authors do not meet the criteria for the “responsible party” able to register a trial as designated by NIH. Therefore, when restorative authors contact investigators regarding their intent to publish the trial, we suggest that they ask the investigators to register the trial. Since the original investigators have little incentive to do so, we also ask clinicaltrials.gov to consider allowing restorative authors to register clinical trials, perhaps with a RIAT designation, to help fulfill the AllTrials goal.
Ethical oversight: RIAT studies submitted by the original investigators require the same ethical oversight of institutional review board (IRB) approval and patient informed consent as all other studies. However, restorative authors may not have access to all the documentation that the original investigators did. We believe it is essential for studies to have professional ethical oversight. One possibility would be for restorative authors to work with their IRB to have the study protocol reviewed for evidence of ethical lapses. If this review is performed, the IRB review should be included with the submission. If IRBs are not willing or able to do so, we may involve an ethicist in the peer review process to help identify ethical concerns. We are interested in feedback from stakeholders, particularly ethicists and IRBs, as to the feasibility of these proposals.
Hopefully someday the suppression of clinical trial results will not be possible and RIAT will no longer be necessary. In the meantime, these and many other specifics will need to be worked out. We expect this to be a work in progress. However, if we do not begin, we will not achieve any progress toward the ultimate goal of transparency of clinical trials and access to all available data for making the best clinical decisions. We look forward to your comments and feedback on our experiment.
[…] sure were your conclusions? Two medical journals will start publishing heretofore confidential research documents that will allow people to evaluate previous research […]
This initiative is a very worthy idea, but I can see one massive problem: who is going to pay for it?
The costs of writing all these papers — which, let’s face it, aren’t going to write themselves — is not even mentioned in either your blogpost or the BMJ article to which you refer.
Do you have any idea how this initiative might be funded?
[…] far, three medical journals, the BMJ, PLoS One and PLoS Medicine, have signed on to the idea and agreed to consider RIAT manuscripts, provided that authors first […]