Experts meet to discuss reaching a consensus on what the evidence tells us
This is the first of two blogs written about the “Sanitation and health evidence consensus meeting”, convened by WHO in Seattle on May 24 and 25 of 2018. It was written by Jan Willem Rosenboom and Radu Ban, who are both Sr. Program Officers on the Water, Sanitation and Hygiene (WSH) team at the Bill & Melinda Gates Foundation.
Introduction: What is this about?
It is hard to imagine that making improvements in sanitation wouldn’t play a role in improving health. After all, we know that shit spreads disease and the F diagram shows us that sanitation is an important tool in blocking the transmission of pathogens from one person to the next, thus lowering exposure. And sure enough: we have strong evidence about the effectiveness of sanitation interventions and improving health and human capital outcomes from rigorous historical studies, from high- as well as low- and middle-income countries.
At the same time, looking at the specific impact of programmatic sanitation interventions, it can be hard to figure out what the evidence is really telling us. On the one hand, a systematic review of the whole body of evidence on sanitation and health (carried out by Freeman et al. in 2017) suggests that sanitation protects against diarrhoea, active trachoma, some soil-transmitted helminth (STH) infections and schistosomiasis. It also improves height-for-age scores of children (i.e. it decreases stunting, which is an important measure of human capacity). On the other hand, several recent sanitation intervention studies have found limited or no impact on different health outcomes. The table below (copied with permission from a presentation by Tom Clasen), provides a summary of key findings from the most recent sanitation studies:
- Fewer observed flies and feces; no change in fecal contamination of water
- Fewer observed soiled hands and less fecal contamination of water
- Except in the study arm considering just water quality improvements
This seeming lack of agreement is confusing, and partly in response to questions from practitioners, on May 24 and 25 of this year WHO convened a meeting of experts to review the existing evidence and reach a consensus about what it is telling us. The group of experts consisted of researchers across multiple disciplines who had written extensively on the topic of sanitation and health. We thought it was necessary to reach consensus among researchers before engaging, in a unified voice, the practitioner community.
At the same time, to make sure that the concerns of practitioners would be considered in the meeting, we published a “request for input” online (through the SuSanA network as well as the Sanitation Updates blog) and we will summarize the responses here. But first…
What did we expect?
When asking practitioners what questions they would like to have answers to, we really expected four (or maybe five) main responses:
- What is the minimum level of sanitation coverage and use a community needs to achieve before we can expect to see an impact on health?
- Is there a “minimum standard” of sanitation that needs to be achieved to get health impacts, and if so, what is it? (in other words, where should we be on the sanitation ladder?)
- What do we know about the role of children’s feces, and what do we do about it? (and the same question for animal feces)
- How important are supplemental investments in water supply and hygiene when making sanitation improvements?
(The fifth question we sort of expected was one asking how good the studies were that the evidence comes from).
While all these questions were put to us, there were many, many more in addition. There was a strong emphasis on research methods and much less reflection on direct practitioner concerns (i.e. there were more questions about how to obtain valid evidence, than about how to use the evidence that exists most productively). This really took us by surprise (and we suspect that a number of those responding to our call for input were themselves researchers, rather than practitioners).
What did we get?
We received a total of about 60 responses from 22 individuals and organizations, unevenly distributed among Europe (9), North America (5), Asia (4), Africa (3) and Latin America (1). Some of the responses came with elaborate context, others just as a list of clearly worded questions.
These fell into a number of more or less discrete categories; there were questions dealing with research methods, study results and their (external) validity, study quality, measures and indicators to use, programmatic approaches, the sanitation chain and other WaSH interventions, and there was one question asking how to influence donors and NGOs to act on the evidence. The full spreadsheet with questions can be downloaded from the SuSanA website here.
It’s hard to summarize the questions here, as they are so varied in their outlook, and range from very straightforward (“Can these study results be generalized?”, or “How important is herd immunity, effective pit emptying, and waste treatment?”) to very elaborately contextualised. In addition, a number of the responses were not really questions, but suggestions to take specific study results into account, or observations about the recent studies. So to get a taste of the depth and the breadth of the questions, it’s best to head over to Susana.org and open the spreadsheet. To keep the consensus meeting focused, we selected ten questions for consideration, and these are listed in the next section.
1. How was the evidence we have so far produced?
There are two types of evidence:
- Historical studies and cross-sectional evidence, which show a strong link between sanitation and child diarrhea and growth
- Randomized controlled trials (RCTs), which are limited to a 2-year period at most and show mixed results, with some studies finding a link while others do not (possibly due to threshold effects or poor implementation).
Have sustainability and non-household sanitation been taken into account?
2. Which outcome measures are most useful, or needed, for evaluating the performance of sanitation programs, to capture their impact both within and beyond health? Have toilet use and cleanliness, the (un)reliability of diarrhea as an indicator and local exposure routes been taken into account?
3. What can be drawn from the recent studies regarding the hypothesis that exposure to environmental pathogens contributes importantly to stunting?
4. To what extent should programme evaluation go hand-in-hand with impact evaluations? Lack of impact could be due to poor implementation or limited uptake.
5. What is the role of observational, non-experimental studies versus RCTs? Should we pay more attention to complex system modeling for sanitation investment decisions?
6. How can research results be interpreted for other settings? What is the external validity, and how can we increase our confidence in applying insights generally?
7. How important is herd immunity, habitual and exclusive use, and maintenance (cleaning, effective pit emptying, and waste treatment)? Look at SaniPath studies: eye watering how much shit people are exposed to in their environment. Investments in sanitation alone may not be enough to impact health.
8. Are we able to say anything about the efficacy of reaching different rungs of the sanitation ladder for achieving health outcomes?
9. Given the reasonable theory-based view that the health impacts of sanitation interventions in high-density locations are likely to be dependent on intervention coverage levels: what evidence is available to support links between coverage levels and health effects? Can we achieve an expert consensus despite lack of strong empirical proof?
10. How much impact should anyone realistically expect ANY sanitation intervention to have when (a) young children, including infants, spend a substantial amount of time playing outside of the domestic house and yard, and (b) sanitation interventions do not install barriers between animal vectors and young children? Which minimum sanitation threshold needs to be reached in crowded, low-income settings to achieve a health impact?
Following the completion of the meeting, a few of the participants have been working on a consensus statement. Once that statement has been finalized, it will be published and at that point, we will release a second blog that will reflect on the consensus, in light of the questions that were asked.