Aid Displacement: The story continues
by Jocalyn Clark, Senior Magazine Editor, and Virginia Barbour, Chief Editor, PLOS Medicine
Today in PLOS Medicine we publish two commissioned Perspectives from leading groups working in the area of foreign aid effectiveness. These follow the retraction of an essay in PLOS Medicine by one group (Rajaie Batniji and Eran Bendavid) that criticized the work of another group published in The Lancet (Christopher Murray and colleagues). The two Perspectives are intended to contribute to the ongoing and vital debate about whether and to what extent foreign aid may lead to less domestic health spending by recipient governments.
We wanted to share with our readers the process of how the PLOS Medicine essay came to be retracted, which is unprecedented at this journal, but which we believe was an important and necessary step for integrity and transparency. We recognize that retractions may trouble authors and readers, but setting the scientific record straight was more important to us than any reputational threat we or the authors might experience. We took the issue seriously and carefully considered how to manage the post-publication concerns.
Concerns about the essay were first raised by David Roodman in a blog post and in several comments on the paper. Other readers contacted us as well. These comments led us to request clarifications from Drs. Batniji and Bendavid, who engaged in discussions without hesitation and were proactive in making the corrections as needed. When it became clear there were errors in how the data were both analysed and reported in the essay, we consulted the Committee on Publication Ethics (COPE) guidelines for retracting articles, which is the current best practice for correcting the science literature when serious errors have been identified.
The question for us was whether a formal Correction to the article was appropriate, or a full Retraction. After discussion with the authors, and consultation with the statistician who provided reviews on the original paper and its revision, the editors at PLOS Medicine decided to ask the authors to retract their paper due to its inaccuracies. They did. We accepted the authors’ explanations for the errors in statistical model choice and reporting, which was the basis for the retraction and is detailed in the Retraction Notice; there is no suggestion of misconduct. In the Retraction Notice the authors state:
“On May 8, 2012, we published an essay that contained a reanalysis of data used by Lu et al. (2) to examine whether development assistance for health leads to displacement of public health spending by recipient governments. In the essay we indicated that we used a fixed effects linear regression, while in Table 4 we reported the results of an analysis using a linear regression without controlling for fixed effects. Upon discovery of this mistake, which resulted from a miscommunication between the authors, we immediately informed PLoS Medicine and moved to correct the record. Our essay concluded that the findings in the analysis by Lu et al. were strongly influenced by outliers and questionable data. However, when our analysis is done using a linear regression with fixed effects, the results show a similar effect size and significance level to that indicated in the original article by Lu et al. As a result of this error, we now believe the following statements in our essay lack justification:
1. “the association between DAH and displacement of government health expenditures is not robust after exclusion of a small subset of data.”
2. “The trends [in the analysis by Lu et al.] are driven by outliers…”
3. “While there does appear to be an association, it is too tenuous, too dependent on problematic model selection, and inconsistent.”
In the Retraction Notice the PLOS Medicine editors apologized to readers for any inconvenience caused by the publication of this article in the journal. Retraction Watch noted the retraction.
We don’t believe the story about aid displacement ends here.
The problems with the original essay notwithstanding, what is clear is that the question of whether (and how) foreign aid displaces domestic health expenditures remains important and unanswered. Recent commentary has again critiqued the original Lancet article, to which the authors have responded, and there continue to be concerns raised about access to data and code on which the article is based. We intend to continue to be part of that conversation and thus commissioned the two Perspectives we publish today:
- Batniji R, Bendavid E (2013) Considerations in Assessing the Evidence and Implications of Aid Displacement from the Health Sector. PLoS Med 10(1): e1001364.
- Murray CJL, Dieleman JL, Lu C, Hanlon M (2013) More Data and Appropriate Statistical Methods Needed to Fully Measure the Displacement Effects of Development Assistance for Health. PLoS Med 10(1): e1001365.
We also do want to continue the discussion about better ways of correcting the literature and intend to write more on that in 2013.
As ever, we invite readers to comment on any of the issues raised on this blog or on the articles.