Outdated Evidence at the WHO: Case of Rwanda

As the world’s coordinating authority on international public health issues with the mission of, among other priorities, “articulating evidence-based policy options and monitoring and assessing health trends,” one would expect the World Health Organization to take seriously the responsibility of assuring that the evidence derived from these assessments is accurate and up-to-date. As the case of Rwanda illustrates, however, policy options articulated by the WHO can often be based on outdated and misleading evidence.

Tuesday, December 13, 2011
Dr. Agnes Binagwaho

As the world’s coordinating authority on international public health issues with the mission of, among other priorities, "articulating evidence-based policy options and monitoring and assessing health trends,” one would expect the World Health Organization to take seriously the responsibility of assuring that the evidence derived from these assessments is accurate and up-to-date. As the case of Rwanda illustrates, however, policy options articulated by the WHO can often be based on outdated and misleading evidence.
 
Country-level health data for Rwanda is often years behind, despite the fact that the Ministry of Health regularly makes publicly available the latest annual data from our Health Management Information System (HMIS) and our Demographic and Health Surveys (DHS). In fact, the country office of WHO even has member seats on the national steering committee for planning, implementing, and analyzing the DHS. Yet after all of the painstaking effort undertaken by Rwanda and our partners (including UNICEF, the CDC, ICF Macro, and the WHO) to ensure accurate and representative information, the WHO delays for years before updating figures in reports and databases.
 
Validated preliminary results from Rwanda’s 2010 DHS have been publicly available on the Ministry of Health website, the ICF Macro website, and accessible through a simple Google search since early July 2011 (see http://www.measuredhs.com/publications/publication-PR7-Preliminary-Reports.cfm), yet the WHO continues to disseminate country-level statistics for key health indicators from 2005 and 2007 for Rwanda. For example, the WHO World Health Statistics 2011 report cites Rwanda’s under-five child mortality rate as 111/1,000 live births while the 2010 country DHS figure is 76/1,000. In fact, even the 2007 DHS figure is below the current estimate, at 102.6/1,000 – by exactly what metrics is the WHO adjusting their data?
 
If Rwanda’s own fully-validated indicators from the DHS which WHO advised are not sufficient, one would still expect those in Geneva to take note of data and analysis from their M&E partners in the United States and Europe. But the internationally respected Institute for Health Metrics and Evaluation of Seattle published a comprehensive report on child mortality in The Lancet (one of the world’s most respected peer-reviewed medical journals) three months ago with an under-five child mortality figure of 75.4/1,000 (see http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)61337-8/abstract), and the WHO still does not move to revise its figures.
 
There are many other examples of outdated and inaccurate WHO data on Rwanda. The recent World AIDS Day 2011 report (a wonderful report in all other aspects) lists Rwanda’s coverage rate for antiretroviral therapy among HIV-positive children in need as 45%, while the Ministry of Health’s publicly available HMIS figure for 2011 is 82%. WHO estimates for Rwanda’s coverage of BCG (against pediatric tuberculosis) and measles vaccinations in 2010 are 75% and 82%, respectively, while the 2010 DHS cites figures of 99.1% and 95.0%.
 
I suppose we should not be so surprised by these inaccuracies, as a 2011 WHO report identified the whole of Rwanda as a high-risk area for yellow fever despite the fact that we have not seen a confirmed case since the mid-1950s (and the WHO’s own database shows 0 cases over the past decade).

While delays in providing accurate and up-to-date information to inform international public health policy may not be new, they are unacceptable and do have consequences. The WHO should be made fully accountable to ensure that reliable new information is incorporated within a reasonable time frame into its reports and databases so that the phrase "evidence-based” does not become an empty statement.

The Author is the Rwandan Minister of Health