Automatgenererad bild.

1 April 2015

EviEM sceptical about World Bank’s criticism

A misunderstanding about the requirements for reviewing research to provide sound decision support — this, according to Mistra EviEM, underlies the World Bank’s assertion in a recent report that systematic reviews can be unreliable.

The Mistra Council for Evidence-Based Environmental Management (Mistra EviEM) carries out systematic reviews of research in several areas to provide decision support for politicians and government agencies, based on the best possible scientific documentation. A review of research on the impact of reindeer grazing on mountain vegetation was published recently.

Earlier in the spring, however, the World Bank asserted in a report that systematic evaluations are not very reliable. The report was based on a study of six evaluations of attempts to improve schoolchildren’s learning in low- and middle-income countries. According to the report’s authors, these evaluations drew widely differing conclusions although they dealt with the same theme, and this was a sign that the method was not reliable.

Criticism missed the mark

But when experts from Mistra EviEM investigated this troubling conclusion in depth, it was the criticism instead that proved to be unreliable. The report was examined in cooperation with the Africa Evidence Network, which also works to compile scientifically based decision support. It turned out that in many cases the reviews were not good enough and the report authors had misunderstood what makes a review ‘systematic’.

To be capable of providing a reliable basis for decisions, the whole evaluation process must be carried out systematically, according to the strict guidelines followed by Mistra EviEM and other organisations. Their conclusion was thus that systematic reviews may indeed be regarded as reliable summaries of existing knowledge.

Mistra Webbredaktör