Skip to content

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

PLOS BLOGS Speaking of Medicine and Health

EMA’s Release of Regulatory Data: Possible Fall out for Journals and Research Synthesis

On 2 October the European Medicines Agency’s (EMA) published the final version of its policy on prospective release of clinical study reports (CSRs) of trials submitted by sponsors in support of Marketing Authorisation Applications (MAAs).

I have summarized its content and made preliminary comments here

The policy

Image credit: gosheshe, Flickr
Image credit: gosheshe, Flickr

In brief, from 1 January 2015 the proactive policy will join the current reactive policy and allow prospective access to incomplete CSRs at two levels: a “viewer” level of access and a “researcher” level. The former will allow on-screen viewing only with a simple registration procedure, while the latter will require proof of identity but allow download and OCR searches to be carried out on CSRs. Prospective seems to mean two things: after a regulatory decision has been taken, pharma will format and write CSRs for direct web posting, and once you have obtained access credentials, you will not have to ask every time you want a CSR and wait some months (and in some cases be turned down).

Its meaning

The practical importance of the policy hinges on the dawning realization of the difference in accuracy, completeness and trustworthiness of published versions of trials compared to their CSR counterparts. Trials may not be published, some may not even be registered (especially the older ones) posing decades-old problems for readers, users and, from my point of view, evidence synthetisers. Reporting bias (cherry picking) is the broad term which encompasses the scores of different types of bias present in publications. The range from the well-known publication bias to the less known swamping bias (when important information on a trial is missed amidst a sea of other information). CSRs in theory cut through all this by presenting exhaustive structured narratives and results of trials. This is essentially part of what regulators see.

The policy has some as yet unclear legal aspects and sets out equally unclear rules for redaction of CSRs (based on the preservation of commercial interests and protection of participant privacy). It also does not explain why some parts of the CSRs will not be released.

The proactive EMA policy will provide an easily accessible window to look at CSRs. When you read the statements “placebo controlled” or “matching placebo” or “double blind” in a journal article you can now go and check if that is really the case. And if, for example, the certificate of analysis1 is missing from the CSR that you are accessing, you can ask EMA why that is so, introducing an unheard of degree of accountability.

An ever growing body of scientific evidence shows that journals do not give the whole story (although some go to extraordinary lengths).  This point reflects simple arithmetic: the ratio of CSR pages to publication pages for the same trial can be as much as 8000 to 1.  To maintain their credibility journals will have to either provide access to CSRs, or stop publishing trials altogether and offer commentaries on trials only. One further option, that of asking for the complete CSR to be made available for each trial on submission, is also at present unrealistic, as current peer reviewers are unskilled in making sense and reviewing a CSR, especially if time constrained. My experience also tells me that every trial should be looked at in the context of the whole trial programme, for example to avoid a piecemeal approach to the assessment of potential harms.

There may be other scenarios such as slowing the rate of publication of trials to allow thorough review of the linked CSR, the growth of a skilled cohort of well-paid peer reviewers or the publication of structured summaries linked to CSRs on the EMA webpage or a combination of these.

Evidence synthesis

Accessibility of CSRs, although not a panacea for reporting bias, should also improve the reliability of research synthesis. As more information on the planning, intended and real conduct of the trial will be available, a good review will be the one that searches regulatory and pharma websites, reconstructs and presents the whole trial programme of interest and includes and reviews CSRs. A good review will also contact the trials sponsors to obtain confirmation that there are no invisible trials and trawls through registers, black and grey literature to check that this is so.

But all this will take a lot of time and require skills and tools to be developed. Businesses which produce reviews in months will require years to do the same, or fall behind.

It is likely that the whole infrastructure to produce synthesis studies will have to change and become much more professional, flexible and intensive, or fall behind. For the record, I don’t believe in automatic methods of data extraction to facilitate dealing with CSRs, but perhaps I am too old-school and will be proved wrong.

So, increased accountability should work for everyone and should keep CSR writers on their toes, open regulators to being quizzed, revolutionise publishing and produce better synthesis assessments.

1 A certificate of analysis is an independent certification that the active and control pills, liquids and so on used in the trial have the same colour, taste, aspect and content apart from the active principle (in the case of a comparison with placebo), by batch number.

Tom Jefferson, reviewer, Cochrane Acute Respiratory Infections Group, 00187, Roma, Italy.

Competing interests: TJ was a co-recipient of a UK National Institute for Health Research grant (HTA – 10/80/01 Update and amalgamation of two Cochrane Reviews: neuraminidase inhibitors for preventing and treating influenza in healthy adults and children—http://www.nets.nihr.ac.uk/projects/hta/108001).

In addition:

TJ receives royalties from his books published by Blackwells and Il Pensiero Scientifico Editore, Rome. TJ is occasionally interviewed by market research companies for anonymous interviews about Phase 1 or 2 pharmaceutical products. In 2011-2013, TJ acted as an expert witness in a litigation case related to oseltamivir phosphate; Tamiflu [Roche] and in a labour case on influenza vaccines in health care workers in Canada. In 1997-99 TJ acted as consultant for Roche, in 2001-2 for GSK, and in 2003 for Sanofi-Synthelabo for pleconaril (an anti-rhinoviral which did not get approval from FDA). TJ was a consultant for IMS Health in 2013, and in 2014 was retained as a scientific adviser to a legal team acting on the drug Tamiflu (oseltamivir, Roche).

 

Back to top