SLUTREPLIK DN DEBATT 10/11. Bo Rothstein: My article about the lack of attention to research ethics in the method that has been awarded this year’s prize in economics to the memory of Alfred Nobel's has generated some rebuttals. Here are my final remarks.
En utskrift från Dagens Nyheter, 2019-12-07 03:51
Artikelns ursprungsadress: https://www.dn.se/debatt/repliker/remarkable-that-the-prize-committee-cant-see-the-ethical-problems/
Peter Fredriksson, Torsten Persson and Jakob Svensson, who are members of the prize committee, claim that there are no serious ethical problems in this research. Then one has to wonder why Angus Deaton, the Nestor in economic development research and who received this award in 2015, in a recently submitted report delivered an almost devastating critique of the ethics of this research.
He writes, ”Some of the RCTs done by western economists on extremely poor people in India, and that were vetted by American institutional review boards, appear unethical, sometimes even bordering on illegality, and likely could not have been done on American subjects”. Fredriksson et al. further claims that the Swedish legislation on research ethics is not applicable to this economic research because it ”primarily concerns medical research and research in clinical psychology”. This is not correct, no exception for social science research is made neither in the law nor in its preliminaries.
Nor have I, as they argue, claimed that the award-winning researchers has been lacking permission for their experiments. What I have pointed out is that two of them highlighted the benefits of trying to obtain (and often succeed in obtaining) exemptions from the review boards examining ethical issues in research from informing the subjects about the purpose of the research and that participation is voluntary. Not getting medicine when you have a parasitic disease or having to use contaminated drinking water must reasonably be within what is considered by the law to be something that affects these children ”physically and mentally”.
Having to change school to get the medicine that the researchers know cures the disease or to be forced into probably long-distance transport to get safe drinking water is a strange idea of what should be considered voluntary participation. In the above-mentioned report, Angus Deaton asks the question ”How is informed consent handled when people do not even know they are part of an experiment?” Furthermore, it is very strange that they do not see it as a problem that more than half of the research articles using this method and published in the leading journals lack information on how these serious ethical problems have been dealt with.
Furthermore, the three members of the prize committee claim that the distinguished philosopher Peter Singer has given a ”green light” to this method. But Singer points out exactly the same as I do, namely that it is central that the people you experiment with get to know what the study is about, that participation is voluntary and that they can leave at any time. It is precisely this information requirement that the award winners claim should be avoided by exploiting the opportunity to obtain exemptions from the ethical review boards. In light of this, it seems very strange that the members of the prize committee argue there are no ethical problems with the method to which they have awarded this prestigious prize.
Against my criticism that this research is unable to address the probably most serious problem of poverty, namely corruption, my critics refer to a research article that they claim shows the opposite. However, that article actually comes to the exact same conclusion as I do, namely that although these experiments would find factors that help to reduce corruption, this will usually not work because the policy that worked in the experiment then has to be implemented by corrupt authorities. As summarized in the article, the measures ”were undermined by the local administration that exploited a loophole in the design of the program… .18 months after the program was launched, it no longer had a positive effect”. Thus, we are dealing with one of the many null results produced by this so called ”gold standard” method.
Economists Niklas Bengtsson, Martina Björkman-Nyqvist, Andreas Madestam and Miri Stryja argue that my concern about the so-called ”Hawthorne effect” (that participants in experiments change their behavior precisely because they know the purpose of the research and not because of the measures one wants to try) is unjustified. One of the most notable studies using this experimental method was performed in 2004 by the above mentioned economists Martina Björkman-Nyqvist and Jakob Svensson. The experiment included 50 health clinics in rural Uganda. In half of them, the villagers were given opportunities to communicate with the health care staff and to (in informal ways) hold them accountable. When compared with the health clinics where no such arrangements were established, the researchers found a huge positive impact on child mortality and other measures of population health. This report has had a very large international impact and has resulted in hundreds of millions dollars of aid funds being invested in these types of arrangement.
Recently, however, three American researchers have presented a report in which they replicate the Swedish researchers' experiments in Uganda (ten years later) but at no less than 376 clinics. The result is that the measures that Björkman-Nyqvist and Svensson came up with as effective now produce a null result in terms of improving child mortality and other measures of population health. My interpretation of the explanations given to this dramatic difference is that the two Swedish researchers' positive results were very likely due to the experiment being affected by precisely the Hawthorne effect. Thus, the medical staff improved their efforts because they knew they were in an experiment that aimed at improving health care and not at all because of the measures that the researchers thought they were testing. Thus, contrary to what the four development economists claim, this null result shows that my critics have every reason to worry that these Hawthorne effects may contaminate the results of this kind of research.
When it comes to improving public health for poor people, it is worth considering what the hundreds of millions of dollars that the report says have been invested in the measures suggested by the two Swedish economists, and that seem now to have been proven ineffective, instead could have been used for.