Effectiveness, Scale, and Sustainability in WASH: A Review

1. Introduction

It seems unnecessary to restate the primacy of water, sanitation and hygiene (WASH) to human development and well-being. Yet, despite over half a century of the development project, over 10% of the world’s population do not have access to clean drinking water; a figure which doubles when applied only to rural areas. Over one third of the world’s seven billion inhabitants – and more than half of those that live in rural areas - do not have access to adequate sanitation facilities, more than 20 years after the end of the International Decade for Drinking Water Supply and Sanitation (JMP, 2012). Bilateral, multilateral and NGO money has been channelled into this sector throughout this period and this most serious of development concerns persists.

This paper’s first objective is to assess the impact of existing WASH programmes and, in order to do so, Section 2 outlines a clear methodology. This section is divided into two parts, the identification of evaluation criteria and the identification of the methods for data selection. The paper draws, firstly, on systematic reviews and synthesises the data within them to inform the analysis. However, the paper then identifies a clear selection bias in systematic reviews which results in a distortion in the conclusions made on the impact of WASH programmes. This paper, therefore, subsequently aims to incorporate some of this missing data in the form of individual programme level data and case study material.

In the subsequent analysis, the paper assesses interventions in the WASH sector according to three criteria; effectiveness, sustainability and scale. By using this lens, the paper is able to categorise aid approaches and demonstrate the implications of the approach adopted by WASH programmes for the impact that the programme has. Section 3, then, is a review of existing reviews of WASH programmes incorporating some additional non-experimental academic and programme level data.

Section 4 analyses not just the results of programmes but how and why those results were achieved. In order to do this, it is necessary to determine different approaches to interventions in the WASH sector. These are categorised as direct delivery, knowledge and skills transfer, and systemic change.

Section 5 then uses the categories outlined in the previous section to examine how the choice of approach is likely to impact on results. This section is important in managing expectations for development programmes and their funders.

Section 6 provides some indicative and experimental case study data on where elements of a systemic change approach to development have been adopted in the WASH sector. Data from Kenya, Malawi, Bolivia and Mali are used to show where systemic approaches have the potential to deliver change and the limitations of current attempts to approach WASH interventions in a more systemic way.

Section 7 addresses the unique and influential political economy of WASH, both as part of a justification of the adoption of a systemic approach to WASH interventions and as a caveat to its success. Micro and macro level political, economic and moral concerns are addressed.

Finally, Section 8 draws together the previous sections by summarising arguments on how WASH programmes should be analysed, both in their criteria for success and how this success is achieved.

2. Methodology

2.1 Evaluation Criteria

Before an analysis of the performance of WASH programmes can be conducted, it is necessary to identify the criteria for their evaluation. This paper uses the academically influential Carter et al. (1999) as its base for defining the criteria as they remain relevant almost 15 years after the paper was written. Carter et al.’s stated requirements were impact and sustainability, which are adapted for use here.

Firstly, effectiveness. The paper looks at whether the interventions have been effective in improving access to clean water, sanitation and hygiene services. Implicit within this is an analysis of the metrics for improvement and for whom access has been improved. Using Carter et al.’s classification of problems in WASH, i.e. water supply, excreta disposal and wastewater disposal, programmes will be judged by the impact they had on alleviating these problems.

Secondly, sustainability. Programmes in the WASH sector will be assessed for the longevity of their impacts; whether the effects of the programme continue beyond the period of donor input. ‘Sustainability’ is used widely, pluralistically, and often incoherently in development. It is beyond the scope of this paper to engage fully with the various definitional challenges of ‘sustainability’ but, as this paper is concerned with performance in the post-project period, sustainability should be taken to mean a permanent change in the adaptive capacity of a given WASH system to cater for the needs of its target beneficiaries in this regard.

Finally, scale. The paper examines the extent of changes in the scale of WASH[1] problems. That is whether, through the activities designed to address WASH problems, their extent has decreased. Where the first consideration examines quality of change, this consideration examines quantity. Implicit within this is, to a limited extent, an analysis of the returns to investment. Programmes with greater benefit to investment ratios can provide more positive lessons from which future programme designs can draw.

2.2 Methods and Selection Criteria

Interventions in the water sector number into the hundreds and consistent, robust data on those that effectively reduce WASH problems, but more acutely on those that are ineffective in doing so, is rare. Given this limitation to what is an ambitious research agenda, and the absence of an unlimited research budget to carry out the primary research necessary, this paper aims to build on Carter et al.’s (ibid) paper together with Waddington et al.’s (2009) systematic review. The paper consolidates existing reviews of completed wash programmes and evaluates the programmes against the criteria of effectiveness (defined by the programme’s own goals), sustainability (defined by a combination of a theoretical assessment of how gains might be perpetuated and ex-post results where available) and scale (defined as how much change, by their own measure, the programme has affected and how this has altered in the post programme period).

This paper itself is not a systematic review. Systematic reviews have become to the assessment of a particular policy in development what randomised controlled trials (RCTs) have become to the analysis of individual activities or interventions; a self-proclaimed ‘gold standard’ in comparison to which all other evidence, no matter the volume, is deemed secondary. Systematic reviews are very useful in certain circumstances where there is a high degree of comparability in the interventions being assessed, the contexts in which they are being assessed, and in the interventions themselves (Taylor, 2013). Unfortunately, even their strongest proponents at 3ie[2] have begun to question their utility, and indeed misinterpretation and misuse, for assessing change in complex development programmes (Mallett et al., 2012; Waddington et al., 2012). They rely on only the most robust data based on a set of stringent criteria. However, the majority of donor programmes cannot provide such data and, more pertinently, do not provide it on programmes that were viewed as unsuccessful. This can result in a selection bias against large numbers of programmes.

Furthermore, the barriers to entry to academic journals, from which the majority of data for systematic reviews is drawn, are high, not just from the peer review process but also in capturing the zeitgeist in the editorial process. Where findings are not novel and simply reinforce established theory, the probability of publication in the social sciences is low. Moreover, there are only a limited number of academics working on WASH in developing countries and each tends to have a regional specialism or focus. There are large areas of the world from which there is little or no formal data collection on WASH programmes. A significant number of small and experimental programmes, therefore, are not represented in the academic literature.

For the purposes of this paper, one of the most detrimental omissions from current literature on WASH programmes is sustainability. The number of studies that have collected primary data on how effective interventions were five years or more after their initial implementation is small. As this study aims to address the relationship between the impact a programme has and the scale and sustainability of that impact, the pool of literature from which to draw that included data on all three aspects was limited.

As such, this paper is set up partially as a ‘systematic review of systematic reviews’ but also includes more informal data from programme documents as case studies to ensure wider coverage[3].

In addition to the ‘review of reviews’, several further studies are included where the methodologies for data collection are far from robust. The volume of evidence is low in this area and the intention of this paper is, in light of the results of existing systematic reviews, to link theory more closely with what little empirical data does exist to argue for an alternative strategy. Methodologies for data collection can be found in the source material in each case.

3. Review of Reviews

There are very few reviews that examine the criteria outlined here. The majority of papers use the Cochrane Method for systematic reviews. This excludes a great deal of qualitative and non-experimental quantitative data which could greatly influence the conclusions of the studies. Studies on attitudes towards food hygiene, for example, could reveal a great deal about how people make the connection between hygiene, health outcomes and, ultimately, poverty as their ability to earn and medical expenditures are affected. It would be difficult to conduct empirical observational analysis of hygiene behaviours in people’s homes in many parts of the world for cultural reasons. Mortality from diarrhoeal diseases is measureable to a reasonable degree of accuracy, but the majority of incidents of diarrhoea will not be accurately reported as no medical treatment will be received and the stigmatisation of the illness means it is unlikely to be reported in a survey (Weiss, 2001). An ethnographic study of one slum area in Nairobi suggests that in the majority of poor households, children experience episodes of diarrhoea on a monthly basis (Thieme, 2013).

3.1 Effectiveness

The nature of the interventions is incredibly diverse and, as such, any comparison of effectiveness is difficult. Some programmes attempt to reduce the cost of getting clean water to households while others try to reduce the distance people have to travel to obtain it. Therefore, in trying to assess the effectiveness of interventions in the WASH sector, programmes can only be assessed in terms of what they set out to achieve.

If one accepts that Cochrane review methodologies will yield different results from those that include qualitative data, such reviews still provide valuable information for a comparison such as this. Effectiveness is the area that the majority of studies included under such a methodology have focused.

In terms of the morbidity rates from diarrhoea in childhood, in aggregate, interventions to improve water quality are seen as more effective than those that improve water supply (Waddington et al., 2009). Intuitively, access to water in one form or another is near universal; humans can survive only a matter of days without water. Therefore increasing the supply of or access to poor quality water may even increase the chances of infection.

In terms of the most effective strategy overall in reducing morbidity from diarrhoeal disease in childhood, some studies see hygiene and sanitation as more important while others have found that hygiene education and water quality interventions have yielded greater effectiveness (Waddington et al., ibid). These analyses of potential solutions to WASH problems reinforce the fact that their impact is situationally specific; a fundamental misunderstanding of the RCTs used to evaluate interventions in systematic reviews. One RCT of a hygiene and sanitation promotion and provision programme in Kenya amongst girls found that rates of infection actually increased significantly amongst girls as the construction of new facilities led to a decrease in their usage of them due to unfamiliarity (Greene et al., 2012).

Within water quality, far greater improvements are seen in the target variable if improvements are made at the point of use rather than the source, due to risk of recontamination. Technical studies also examine the effects of hand-washing and find it to be significant. However, one meta-analysis finds a reduction of half while two other meta-analyses find a reduction closer to one third, again emphasising the importance of context in the effectiveness of any given intervention.

Technical data plays a significant part in the design of programmes in this field, far more so than in programmes which focus solely on economic development, for example. Biologically, humans behave in an almost uniform fashion across their geographical distribution. However, behaviourally, populations differ significantly based on economic, social and cultural factors. As stated by Waddington et al.:

While the programme theory is clear and logical, interventions are embedded in social systems which have a strong bearing on their uptake and impact in the real world (2009; 15).

Three of the papers reviewed by Waddington et al. (ibid.) factor time considerations into their evaluation of WASH interventions and find substantial savings resulting from the interventions assessed (Wang et al., 1989; Blum et al., 1990; Pattanayak et al., 2007). However, no attempt has been made to systematically evaluate the impact of WASH interventions on people’s lives in any integrated way. This review does not attempt to address this gap in the literature by developing a model for how to assess the quality of life impact. It is neither possible nor ethical to estimate how much further one would walk to avoid an illness for their child. This review simply attempts to show how the context specific and pluralistic impacts of WASH interventions should be taken into account when making programming decisions.

Beyond technological and knowledge and skills transfer initiatives, there have also been institutional measures implemented to address WASH problems. One of the key and most contentious debates in this area is the privatisation of the water sector. While there is no opportunity to explore this culturally and politically inflected issue at length here, it is sufficient to state that results have been mixed. Although experiences differ, where developing countries have increased private sector involvement in water supply, water quality and levels of service have improved. However, the impact of this on the poorest members of society is less clear. For poor people in rural areas, access has been reduced as there is little appetite or incentive for the private sector to provide services to marginalised communities, while these impacts vary by region (Budds and McGranahan, 2003; McDonald and Ruiters, 2005; Prasad, 2006; Barrera-Osorio et al., 2009; Olarreaga et al., 2013).

One market development programme in Kenya conducted a survey of water points for consumer’s willingness to pay for clean water working through local water boards to ensure maintenance of supply networks. The programme identified a small number of areas with clear viability in this regard but found many others where there was a ‘viability void’ (Hitchins et al., 2004) with potential consumers being unable or unwilling to pay (Tiwari, 2013).

Systematic reviews are comparative and have demonstrated which of a range of interventions is more effective in impacting on their target variable. For example, improvements in water quality are more likely to reduce rates of childhood morbidity from diarrhoeal disease than improvements in water supply but there is disagreement on whether improvements in hygiene and sanitation are more effective than those in water quality. Predictably, point of use improvements are more effective than improvements at source and hygiene behaviour is also an important variable. The question remains as to the transferability of these lessons and the selection bias against ineffective programmes.

Some of the data taken from outside of systematic reviews has demonstrated a great number of ineffective interventions. A Water and Sanitation Programme (WSP) assessment of a number of different sanitation interventions in six areas of Cambodia found very little evidence of effectiveness. All involved heavily subsidised or government funded construction of different types of toilet, some of which were connected to a sewerage system. Projects ranged from three to five years in length and samples for the study averaged 1000 individuals with access to the 47,000 toilets provided through the various schemes. Household contribution to the sanitation project ranged from 2-29%. In one “community-led total sanitation” (CLTS) project – a participatory method of incorporating communities in an appraisal of their own sanitation needs – it was found that only 15% of community members were using the latrines provided as part of the scheme. Across all the projects, an average of 37% of people with access to toilets were still practicing open defecation, 59% of children were witnessed defecating on their own premises, and only 15% of toilets had running water in or near the toilet (WSP, 2012).

The WSP examples demonstrate the need to understand the contexts within which WASH programmes take place and to match demand with supply. People could be provided with the most effective sanitation hardware available but they are unlikely to alter their habits unless they are aware of the incentives to do so. Furthermore, these evaluations were made at the end of the programme period as a snapshot of effectiveness and do not factor in depreciation of the hardware or attrition over time.

Technical questions around WASH problems have definitive answers and so are easily addressed. Does providing a village with cleaner water reduce the rates of diarrhoeal disease? Yes. There are also ways of comparing interventions to ascertain which has the greatest effect on a target variable. With such data to hand it would represent poor policy design or programme management if the programme could not achieve its stated goals. The important questions relate to how can these approaches be delivered so as to achieve sustainability and scale and, reciprocally, can approaches that have the potential to achieve sustainability and scale be effective? As Waddington et al. concede:

“Ensuring sustainability over time and diffusion across populations is of fundamental importance if the benefits of water, sanitation and hygiene interventions are to be maintained when intervention activities come to an end” (41).

These are the questions which the remainder of this paper aims to address.

3.2 Sustainability

Sustainability is difficult to measure. Of the data that is available, there is very little which examines the effect of a programme on changes in WASH problems after a programme has ended. A further complication is the absence of institutional memory or historical public record of the exact nature of interventions carried out by programmes. This is the area in which data becomes most scarce and, where available, the efficacy of interventions begins to look far less favourable.

Of the studies included in Waddington et al.’s (ibid.) meta-analysis on childhood diarrhoea, the criteria for inclusion had to be expanded and still only five studies examined the effect of interventions more than one year after the external input had ended. Despite evaluations of sustainability taking place in the short-term after the end of an intervention, in some cases impact was still seen to decrease over the course of the programme.

In the absence of data on programme costs in relation to ex-post programme impacts (as such evaluation is very rarely funded), researchers have devised other measures of sustainability, such as cost effectiveness. This involves a range of calculations concerning the cost of interventions and measures of the benefits experienced as a result. One such measure is Disability Adjusted Life Years (DALYs). What this means in WASH is that if you can prevent a single child’s death it is worth 30 DALYs and given that feco-oral diseases account for 85% of preventable DALYs (with mortality from diarrhoeal disease mainly affecting children), the impact can be significant (Rosen and Vincent, 1999). Thus, if cost effectiveness is measured by the ratio of input cost to increase in DALYs, low cost interventions such as hygiene promotion which can result in even a small reduction in the number of child deaths, have a far greater impact by this metric than any intervention which may reduce the number of work days lost to illness amongst adults, increases in perceptions of quality of life etc. (Cairncross and Valdmanis, 2006). However, the prevention of a child’s death should only be measured as having a full impact on DALYs if there is no chance of this change being reversed over time. As a stylised example, if a programme installs water purification technology in a village’s water supply eliminating childhood mortality from diarrhoeal disease amongst the village’s 100 children then it is common to assume that this has resulted in an additional 3000 DALYs. However, if the technology fails one year after the intervention then the rate of mortality will resume at the previous, if not a higher rate, owing to increased consumption based on the assumption that the water is safe.

Where WASH interventions are delivered directly by a development organisation to a beneficiary, there are a great number of less-formal studies demonstrating a reduction in the extent or effectiveness of the change brought about by a programme over time. In Sierra Leone, five years after the end of a project to install a large number of water points, 88% were not delivering consistent year-round water supply, 17% of which were broken entirely (Ministry of Energy and Water Resources, 2012). In Swaziland, 41.4% of water points mapped by the government in their most recent mapping exercise were not fully functional (Department of Water Affairs, 2012). Further instances of significant proportions of water and sanitation infrastructure and hygiene campaigns that have been paid for, in whole or in part, by donor funding being inadequate or dysfunctional abound in Ghana, Democratic Republic of Congo, Tanzania, Kenya, Iraq, Pakistan, and Nepal among a list spanning the majority of developing nations (Asian Development Bank, 2009; Evans et al., 2009; Behrens-Shah, 2011; Kanagasivam and Danielson, 2011; Adank, 2012; European Court of Auditors, 2012; Listening to Dar, 2012).

Engineers Without Borders (Canada), began to recognise the unsustainable nature of a number of their hardware related interventions in developing countries and, as a consequence, in 2008, took the unusual step of publishing details of their interventions which had not been effective in their now annual Failure Report. While these reports have not gone as far as to address the reasons for failure, they do represent a rare recognition of where innovations have been unsuccessful. Unfortunately, this has not been embraced throughout aid agency and NGO programmes in WASH, partially due to the incentives of employees, contractors and NGO’s being misaligned with making failures public. There is, therefore, a systematic bias in the data selection for academic publications and subsequent systematic reviews. Who pays for the initial research to be done and who owns the rights to its publication? Which organisations participate in qualitative data collection? Even when academics conduct research entirely independently, the visibility of projects which donors and implementing organisations do not publicise is likely to be limited.

Hygiene promotion is arguably one of the more sustainable forms of directly delivered intervention as, if effective, it results in a behaviour change amongst the treatment group and a limited degree of information transmission can be expected. Wilson and Chandler (1993) conducted a study in Lombok, Indonesia showing that two years after the end of an intervention which freely distributed and promoted the use of soap, 79% of the treatment group continued to buy and use the soap. While a positive result, the attrition rate of 21% in just two years means the incentives to continue with the change in behaviour were not clear to the beneficiaries. Further, there is no consideration made of how this information dissemination may be replicated between treatment and non-treatment groups.

It is likely that hygiene promotion activities need to be repeated from time to time—say, every five years (Cairncross and Valdmanis, 2006; 785)

Two years prior to this statement, one of the same contributing authors (Cairncross and Shordt, 2004) examined the sustainability of hygiene promotion interventions in eight countries for up to nine years after the end of the intervention and found that behavioural change was, broadly, sustained amongst the treatment group, although some cultural variations were found.

Caincross and Valdmanis (2006) also state as a reason for cost-effectiveness being underestimated in WASH, “[I]f a sustainable low-cost sanitation industry can be developed, it will have an interest in promoting its own product” (790), although there is no recognition of the potential for a development intervention to bring such a change about.

While water quality interventions were found to be more effective than water supply interventions, their sustainability does not yield such positive results. LIjima et al. (2001) found a 70% reduction in the use of pasteurisation four years after a programme designed to promote the technique had ended. A range of other studies find that where mechanisms for the improvement of water quality are freely distributed, there is a significant reduction in usage following the end of the programme (Clasen et al., 2006; Sobsey et al., 2008; Kremer et al., 2009). No assessment is made of the reason for these attrition rates and, while it is likely to be both geographically and intervention specific, it could be attributable to the physical maintenance of the equipment, the communication of the incentives for continued use, or the requirement for an increase in user inputs i.e. paying for the fuel for pasteurisation etc.

In summary, the sustainability of interventions in the WASH sector, based on the limited data available, is low. This is likely to be reduced further if unpublished and programme level data were included, given the incentives to disguise failure. The data that is available relates only to the treatment groups and gives no consideration to the wider effects. If existing approaches to WASH interventions cannot deliver sustainable results among the target group, what are the possibilities for replication among non-treatment groups? Combining global annual ODA for the next decade could not provide and maintain adequate WASH infrastructure to all of the world’s 2.5bn poor people by aiming to deliver these services directly. There is, therefore, a need to assess the scale of WASH programmes.

3.3 Scale

Scale and sustainability are inherently linked and so their separation here is artificial to a degree. There are two dimensions that must be considered in demonstrating the scale aspects of a programme’s overall impact. Firstly and most simply, there is an estimation of the number of people that have received the intended ‘effect’ (as identified by the effectiveness criteria) of the programme over time. This includes the period after donor involvement has ended for the entire time that the programme’s impacts continue to be seen. The second dimension is a caveat to the first; that of attribution. Where a programme’s impacts continue beyond the period of donor input, to what degree can they claim that the change that continues to happen is because of the programme’s initial activities? Is the creator of the internet responsible for all subsequent ecommerce? Of course, there is a degree to which he is but he is not solely responsible and the degree to which an initial innovation causes subsequent change is likely to decrease over time.

The above considerations make scale a difficult variable to estimate, and this difficulty varies for different types of programmes. The majority of studies do not attempt to make any assessment of the number of beneficiaries from a particular intervention. This is because the effects they aim to have are directly on the people to whom they provide the goods or services.

In an approach that aims to deliver goods directly to its target group, the scale of a programme is simply the number of goods that can be bought within the programme budget. There is no intention of replication. For knowledge and skills transfer programmes – programmes which aim to directly deliver education or training – a simple count of the number of people that participated may be conducted. Where hardware, such as a toilet, is installed in a village, the number of people that will use that hardware will be counted. This, in fact, frequently underestimates the scale of a programme as the toilets may, until they require maintenance, continue to be used in the post programme period.

No attempt is made here, therefore, to measure the scale of common approaches and activities in WASH programmes as it is a relatively simple function of the budget that has been allocated to them. However, it is important to highlight that this should be a consideration of WASH programmes given the severity of the problems and the decreasing per capita resources available to address them.

4. An Analysis of Approaches

The above review has shown that existing reviews have concentrated on the effectiveness of WASH programmes in achieving their stated goals at the expense of considering the sustainability and scale of the change they affect. Further, there is a systematic bias against the selection of failed programmes owing to the high barriers for inclusion and the incentives for reporting on ineffective programmes. Despite this bias, there are a great number of WASH programmes that are shown to be ineffective and there are significant variations in impact in different locations. This questions the search for a silver bullet and focuses attention more closely on why these programmes are failing to achieve their goals, particularly in the areas of scale and sustainability.

In order to move towards a more comprehensive analysis of impact of WASH programmes, there is a need to move beyond a technical analysis of what programmes do and the relative merits of activities, towards an analysis of the overall approach they take to determining what these activities should be and the way in which they implement them.

The following stylised typology aims to identify the key features of these approaches. The intervention activity or activities are determined by an analysis of what the problem is.

Table 1: Typology of Approaches to WASH Problems

4.1 Direct Delivery

Programmes following this approach aim to give things to people. These things are primarily in the form of goods such as soap or toilets. Programmes are often relatively short-term and aim to address the symptoms of WASH problems. Determining which goods and services should be provided is often informed by the type of analysis, contained in the first section of this paper, on effectiveness. As such, direct delivery programmes are likely to provide hardware to improve the water quality at point of use, such as purifiers, or provide community toilets as these were seen as more effective solutions relative to alternatives in addressing WASH problems in the short term.

4.2 Knowledge and Skills Transfer

These are programmes that aim to change behaviour based on the direct input of training or education to a target or ‘treatment group’. The difference here is that the change brought about by the programme is intended to last beyond the programme’s lifespan as, once behaviour has been positively changed and the benefits are realised by the individual, they are unlikely to discontinue that behaviour. However, there are two primary problems which limit sustainability and impact.

Firstly, there is little incentive for the individual to disseminate information beyond their immediate family and the incentive to put the information or training into practice is often unclear. Preventative behaviours are more difficult to institutionalise than curative behaviours and so telling people to wash their hands so as to prevent disease where it is not culturally ingrained is likely to be difficult owing to a lack of a local and obvious correlation with disease. More generally, this approach fails to take adequate account of the incentives of actors within a system, as explored in Section 7.

Secondly, there is no capacity for the adaptation of the learning to new external developments. Knowledge and skills transfer initiatives commonly consist of a one-off transferral of information. For example, a programme might teach people to utilise a low-cost solution for the conservation of water resources which results in a 70% increase in household water resources. Subsequently, technologies are developed which result in a far larger increase in available water resources but there is no mechanism in place for the transmission of this information to the beneficiaries in the absence of external support. Furthermore, under such a programme, this technology would necessarily have to be developed externally and, potentially, in a contextually inappropriate way for the beneficiaries. No endogenous capacity will be built within the system for the development of locally appropriate solutions.

4.3 Systemic Change

This is the most complex of the approaches outlined here and relies heavily on continued and ongoing analysis of the drivers of and incentives for change in any given context (see DfID and SDC, 2008 for a full guide to the approach). Fundamentally, this approach analyses not just a problem but the many smaller problems – or underlying causes - within a broader system which contributed to its creation. In addressing a single problem such as diarrhoeal disease, a systemic approach is likely to identify a series of situationally specific additional problems which contribute to the creation and perpetuation of the original problem. These may include, for example, property rights issues in urban areas which deter people from investing in infrastructure to provide clean water or sanitation facilities, institutional problems such as a lack of competition within the water sector owing to poor regulations or corruption, or the absence of information on the commercial potential of hygiene products meaning producers do not engage in promotion to poor consumers or the development of products for low income markets. Indeed, one of the activities engaged in by a systemic change programme might be aimed at increasing the capacity of players within a system through knowledge or skills transfer. However, this would be done in such a way as to ensure that the generation of capacity is made endogenous to the system and also that the skills and knowledge are adaptable to alterations in external factors.

Operationally, too, systemic change requires a different approach to addressing these problems in placing sustainability at its core. The direct involvement of development programmes is kept to a minimum. In each case, where the programme does become directly involved in addressing a given constraint within a system, this will be done in partnership with a local organisation or institution and there will be a clearly defined exist strategy where the incentives for the transferral of ownership of a given change over time are clear.

The expected results of such a programme are far longer-term and it is anticipated that impact will grow over time through the imitation and adaptation of a given innovation. Crucially, and in contrast to knowledge and skills transfer programmes, systemic change programmes aim to affect a change in a system whereby there is capacity to adapt to exogenous stimuli. Where there is incentive for a local partner to adapt to these changes, whether through additional sales or electoral votes, they are more likely to do so.

5. Implications of Approaches for Evaluation

There are different risk profiles associated with different approaches to assessing WASH problems which affect the likelihood of programmes satisfying their goals in this regard. If, for example, a direct delivery programme aims to build ten public toilets while a systemic change programme intends to work with a toilet manufacturer to promote the benefits of good sanitation to a community and, as a consequence, convince the community to pay for ten public toilets to be built, then the latter approach is far riskier than the former, making it less likely to achieve its goals. However, the relative benefits of these two approaches are able to be assessed more completely once the sustainability and scale of these interventions is considered.

Knowledge and skills transfer programmes, such as the education programmes examined in earlier sections of this paper, have a lower initial impact than direct delivery programmes as there is no guarantee of uptake. This also makes them slightly more risky than direct delivery programmes.

Systemic change programmes will not deliver immediate impact. They rely on relationships being built and extensive analysis of systems and interrelated systems. If a programme can successfully affect a change in the adaptive capacity of a system then, through demonstration and replication, these benefits will continue to reach more and more people ad infinitum. They are however, inherently more risky, as the execution of a strategy is not within the control of a given programme.

The scale of change is a direct product of the sustainability of the change, with the caveat that attribution to the programme must be considered. Scale of change is more difficult to calculate for indirect interventions than for direct interventions but the total number of people that benefit from the programme, if successful, is potentially far greater.

Figures 1 and 2 are graphical representations of the theoretical benefits of the three approaches and the degree to which this change can be attributed to a programme respectively.

While, in theory, systemic change can continue to reach more and more people and continue to improve outcomes for poor people over time, the degree to which this is attributable to the activities of the programme is likely to decrease significantly and form an asymptotic curve along the time axis. Initial changes in the target variables are likely to be solely attributable to the programme but, as time progresses in the post-programme period, the influence of the initial intervention decreases in significance. The asymptotic element indicates that, without this intervention, change may not have happened.

6. Systemic Experiments in WASH

Of the WASH interventions analysed in Section 3 of this paper, the vast majority represent either ‘direct delivery’ or ‘knowledge and skills transfer’ approaches to WASH problems. However, data is emerging from a number of experimental programmes and private sector initiatives that are adopting elements of a systemic change approach to a greater or lesser degree. A systemic approach involves a thorough examination of the roles of actors in the core of a system (producers, consumers etc.), supporting functions and their associated inter-linked systems (input suppliers, logistics etc.) and the rules governing transactions (formal and informal institutions, regulations, etc.) in determining outcomes within that system. A development programme, private sector initiative, or government adopting this approach would analyse why a given problem exists by looking at the incentives of different players within a system to maintain or alter current structures.

Three examples are explored here, one led by a private company, one led by government and a third which is a development programme led by an NGO. These examples of systemic treatments of WASH problems will now form case studies to demonstrate the potential for a different approach in the sector. Several caveats should be noted here. Firstly, no programme has explicitly followed a systemic change approach in the WASH sector, instead adopting elements of the approach without fully embracing its principles. Secondly, methods for data collection (referred to by programmes as monitoring and evaluation) are unlikely to be as rigorous as those in peer-reviewed journal articles. Furthermore such data is far more difficult to collect in programmes which adopt a systemic change approach (Taylor, 2013). For the reasons discussed above, this programme data is valuable as it allows access to results that would not otherwise be available. However, conclusions should be treated as indicative and reasons for further investigation into the effect that the approach taken in WASH programmes has on their impact, incorporating effectiveness, sustainability and scale of change into any analysis. They should not be treated as definitive conclusions.

6.1 Finding Markets at in Poor Urban Areas: CCS in Kenya

One programme example took the form of a hybrid of a Corporate Social Responsibility (CSR) strategy and a Base of the Pyramid (BoP) commercial strategy adopted by SC Johnson (SCJ), a US-based transnational consumer goods company. As documented by a 15-month ethnographic study 2009-2010, the company began to engage in some experimental projects in the sanitation sector in Kenya. SCJ began to examine ways to access the markets of poor urban areas and had to modify their business model in order to do so. While the original intention was both commercial and social, if successful, the firm hoped to create a profitable, sustainable market which also resulted in economic and health benefits for the local community. Working through local youth groups in slum areas of Nairobi, SCJ established the Community Cleaning Services (CCS) franchise. CCS would provide uniforms and would purchase cleaning products wholesale from SCJ. The youth groups that effectively formed branches of CCS would then offer households the use of well-maintained, clean communal toilet facilities in exchange for a small fee, $0.70 per month. By 2009, CCS were working with over 20 entrepreneurs, operating over 100 toilets and had operations in the majority of Nairobi’s low income communities.

In 2010, SCJ withdrew support for CCS and the company began a partnership with PLAN International. Plan International is an NGO that, albeit on a ‘no subsidy basis’ implying a move towards commercial viability, provided pro bono support in the form of technical assistance and marketing to CCS. The original model was not profitable for three main reasons. Firstly, the cost of training the individual franchisees was too high and any rise in the price of inputs (SCJ products), which were the main source of revenue, would have deterred franchisees from continuing their business (although there appears to have been little research on optimal pricing). The second problem compounded the first; no entrepreneur could expand their business sufficiently to pay for higher price inputs – including labour as there was a high rate of attrition of staff within CCS as the business of cleaning toilets was not a desirable employment opportunity for those with a number of other low-income generating activities available - because of the ‘territorial economic zoning’ which meant they could not expand their customer base. In the informal economy within these informal settlements, the economic boundaries of a group’s activities were defined by the social norms and, often physically, enforced boundaries of their territory. Finally, there were also issues around the need for further sanitation marketing which was seen as the responsibility of the company (SCJ) rather than the franchisee at the local level.

SCJ analysed the market and, based on their own incentives and those of others within the system, entered the sanitation market by subsidising some of the functions of the system in the short-term with a clear exit strategy. SCJ’s original approach evidently sought systemic change as its ultimate goal. Interventions in the market were experimental, multi-scalar, and there was a clearly defined strategy to end external support. They sought to work through local partners and targeted national level regulatory constraints and local level institutional constraints to develop the system. However, the approach did not fully analyse systemic constraints. Local level institutional constraints (territorial economic zoning) and research did not extend to an analysis of the incentives for business expansion and competition. PLAN International’s intervention adopted a knowledge and skills transfer approach through the direct provision of sanitation marketing. The system now requires a continual outside input in order to operate and PLAN do not have a clearly defined exit strategy from the sanitation marketing system.

Ultimately, CCS failed as a profitable sustainable business venture, but perhaps succeeded in reducing diarrhoeal disease. The key systemic constraint in this case is the absence of effective institutions allowing successful entrepreneurs to expand their business, gain economies of scale and make continued import of appropriate inputs cost-neutral or positive for a supplier such as SCJ or an alternative provider. Where markets are thin – SCJ was the only firm offering this wholesale and branding service – the major players within that market need to be innovative in order to optimise their business model as competition within the market will not provide this information. While this is not primarily a WASH programme, it demonstrates how incentives can align, given the right regulatory environment, to provide sustainable health outcomes through a primarily economic programme (Thieme, 2013).

6.2 Steps to Sustainability: WfP in Bolivia and Malawi

Water for People (WfP) is an international NGO which implements WASH programmes in developing countries. In 2008, they explicitly switched to a more market-oriented approach; moving away from directly building infrastructure to the integration of local sanitation enterprises. There are numerous examples from the countries that WfP works in of how this switch to a consideration of systemic goals has led to more sustainable outcomes. This has been realised in the form of greater public and private sector investment in WASH facilities, services and maintenance at a local level.

Two countries will now be used as comparisons for how the approach has fared differently in practice; Bolivia and Malawi. In Malawi, WfP directly trained a number of people (called Small Sanitation Entrepreneurs – SSEs) at the village level in the building of low-cost latrines that, if utilised correctly, would reduce rates of diarrhoeal disease, together with developing basic business skills. SSEs were only allowed to sell at the village level with the rationale that this would provide an easily accessible market for each and allow each to succeed. SSEs were also provided with seed capital to purchase materials. On the supply side, WfP saw the main problem as the ability to pay and so designed flexible payment plans and an option for partial payment through composting (‘humanure’).

The approach used here was a mixture of direct delivery, knowledge and skills transfer and systemic change as the seed capital and training were directly delivered but the flexible payment options offered the potential for the model to persist with the target beneficiaries without outside involvement. It is a clear demonstration, though, that a systemic change approach must be fully embraced in order to be sustainable and scalable. The detrimental impact of the directly delivered and knowledge and skills transfer elements of the approach, together with those of previous development programmes, was clear.

The SSE programme was largely unsuccessful for a number of reasons. Firstly, the initial market analysis was insufficient. This meant that the distortionary effect of previous WASH interventions on the market was not initially envisaged. Potential consumers saw the latrines as inferior to those that had been provided for free through other development programmes. Secondly, the local social and cultural context was not fully understood. Potential consumers were aware of the subsidies given to SSEs and saw this as unfair, despite not fully understanding the economics of the situation. Further, there was a lack of understanding of the interlinked market for humanure, on which some household’s ability to pay was predicated. Owing to a lack of compelling evidence and an effective communications strategy, farmers were of the opinion that this would not be an effective fertiliser and so were only willing to pay a fraction of what they would pay for a commercial fertiliser which would increase their yield by an equivalent amount. Finally, and perhaps most crucially, WfP created a market distortion through the programme itself in limiting SSEs to their local areas. There was no competition between providers and innovation would not yield reward. There was no opportunity for successful SSEs to expand. In developed economies hundreds of businesses fail every day, it is the essence of competition and innovation. There were other flaws in the programme design such as the lack of exit strategy in the training markets and the lack of research demonstrating incentives but the success of one of the 70 masons supported by the programme demonstrated how these problems can be overcome by skilled entrepreneurs (Sparkman, 2012; Water for People, 2012).

Globally, WfP focus on both water and sanitation and a different approach to incorporating systemic change goals can be seen from in each country context. In the water sector in Bolivia, WfP have targeted the universal provision of clean water, while at the same time, attempting to achieve sustainable, scalable change in the water sector. In water, unlike in sanitation and hygiene, technological solutions are unlikely to be able to play as significant a role; there is a need to transport a good from one point (whether that is underground or from a surface level source or processing facility) to another (where people live) on a daily basis.

WfP recognised the key constraint on water services being provided to poor people, many of whom had both the means and desire to pay for these services, as being the absence of a coordination function. Analysis demonstrated that municipal governments were the most appropriate partner to provide this coordination function and, indeed, had the desire to do so but could not owing to a lack of funding through central government. Central government had the explicit aim of providing water services to the poor but had been unable to do so effectively. WfP funded municipal governments in order to demonstrate the utility of this approach to central government and then gradually reduced their contribution to this process over time as the value of the approach became clear. WfP were simultaneously working with other partners to ensure that the demand, logistics and supply functions were performed adequately, aligned with the incentives of all actors involved to do so, all of which are in line with a systemic approach to the problem.

However, in Bolivia, as in every country for which data is available, WfP have been unable to withdraw from the market completely. WfP see their role in Bolivia as supporting local governments to fulfil their mandate of providing water and sanitation services to their constituents. They seek to do this by funding the provision of goods and services to those from whom there is no realistic prospect of cost recovery whilst also facilitating a market development approach to cater to those from whom there is.

The systemic approach was not fully adopted in Bolivia as WfP saw, from the outset, a continuing role for outside support in order to provide services where it was not economically viable or politically necessary for public or private companies to do so. Deeper analysis of the problem might have yielded alternative avenues for exploration such as advocacy functions, positive pricing strategies and political dimensions (as explored in Section 7) to the system which may have allowed for an exit from the market.

WfP’s Sanitation as a Business programme rightly states that market-based development should be seen as a means for catalysing a process that will lead to universal coverage of a given service to the whole population – a situation in which a plurality of public and private actors are likely to be involved (Sparkman, 2012).

6.3 Government-led Commercialisation: Water User’s Associations in Mali

A more publically oriented solution to the problem of water provision can be seen in the case of Mali. Throughout the 1990s Water User’s Associations (WUAs) in rural Mali, were providing poor quality services at unaffordable prices, which resulted in high rates of diarrhoeal diseases and frequent disruptions to water supply. The government’s analysis of the sector identified the lack of capacity of WUAs as the key constraint to improving the coverage and quality of water provision. Consequently, the government established a water advisory unit (CCAEP) which began to offer a range of financial and technical consultancy services to WUAs with a view to improving efficiency within the organisations. These fees were not charged at a commercial rate but were cost neutral to the government ministry in which CCAEP was established. As a result of the CCAEP’s interventions, costs to consumers fell by over 60% and there is improved monitoring and transparency of accounting, improving trust between WUAs, consumers, and institutional actors (Collignon and Vézina, 2001).

Again, strong analysis of the system identified not the symptoms of a problem, in this case an under supply of clean water, but traced it back through its sequential and mutually reinforcing causes of inefficient management, leading to poor quality services, leading to high prices, all of which ultimately stemmed from a lack of capacity within WUAs. The failure to overcome this constraint was due to an absence of both demand for and supply of essential services to improve this capacity. The government managed to fill this gap on a cost-neutral and sustainable basis due to its perpetuation being in the interests of all of the actors within the system – the consumers who got better quality and cheaper services, the WUAs who made more money and more satisfied customers (of which, by the very nature of WUAs, they were some), and the government who moved towards their goal of universal access with its associated moral, economic and political incentives.

The small number of examples here demonstrate some of the ways in which approaches to WASH problems can be more systemic and can begin to consider sustainability and scale in designing their interventions. However, these examples also demonstrate the difficulties in pursuing such a strategy. Systemic change requires careful and on-going analysis in order to fulfil its goals. As identified in the analysis of approaches section, systemic change is, inherently, a riskier undertaking and these early applications provide evidence of some of the potential pitfalls of employing such an approach. However, the Mali example in particular shows how sustainable, large-scale change is possible in the WASH sector. There is a clear contrast between this example, where macro politics were a driver of systemic change, and the Kenyan case, where micro-politics were a contributory factor in preventing it, highlighting the importance of political economy in the analysis of systemic change.

7. The Political Economy of a Mixed Good[4]

The outcomes realised in each of the above cases were significantly, positively or negatively, influenced by the political economy of their given context. For systemic change to be effective in any sector requires a political economy analysis examining the incentives of public, private, and third sector actors. In different geographical contexts a plurality of configurations are likely to be arrived at reflecting the individual political economy of that context. These factors which influence the configuration of the system are particularly evident in WASH as a sector characterised by mixed goods which exist in a politically charged environment.

WASH, like other mixed good such as education and health, has been influenced by a wider re-appraisal of the role of the state that has taken place over the last two decades. Dogmatic and idealised positions have been questioned in development owing to pragmatic assessments of the quality of and access to sevices for poor people (Chang and Rowthorn, 1995; Besley and Ghatak, 2001; Rufín et al., 2003; Mehrotra and Delamonica, 2005).

In that context, different negotiated settlements will be reached in different situations as elements of WASH are considered quasi-public, social or private goods in different contexts (Rogers et al., 1998). In the US, commonly seen as the bastion of liberalised economies, the current funding structure includes 80% public ownership by local governments with consumers charged on the basis of cost recovery. Improvements in infrastructure are often financed by grants, low-interest loans from federal government or public-private partnerships (American Rivers, 2013). In the UK, the system has been privatised to a far greater degree through a franchise model, displaying a plurality in systems of provision in developed economies.

These negotiated settlements are based on different political, economic and moral drivers of change in different contexts. There may be an economic rationale for improving WASH service provision as, for example, the provision of clean water means a lower burden on a public health system or an increase in the available workforce and greater tax contributions. In many developing countries, however, public health provision is limited and tax collection is ineffective and, here, political considerations may be more influential.

In the case of WfP, the political will to address WASH problems is demonstrated by the reduction in subsidy levels from WfP over time for water service provision and maintenance. However, the ongoing subsidy demonstrates that the economic incentives are not sufficient, or at least sufficiently evident, for the government to seek a more sustainable solution. WfP’s continued subsidy has created a distortion in the market by an external player which makes a more sustainable solution less likely to emerge. If public funding cannot be stretched to the extra 3% necessary to remove the function of WfP from the market, then perhaps a levy on private water companies or business consumers may be necessary to fill this funding gap. WfP are also exploring the feasibility of using micro-finance and small bank loans to fund maintenance and improvements (in effect facilitating change in the finance system) on the basis that productivity gains resulting from better health outcomes in poor communities would offset this cost.

Where public policy and practice is more responsive to the concerns of the electorate, there may be political incentives for governments to provide WASH services. For those without access to adequate WASH services and where it is not economically rational for the private sector to address these problems, there may be political motivation for a government to tackle the issues, because users themselves constitute an important political voice. Often, however, the most economically marginalised in society, with the most severe WASH problems and the least ability to address them, are also the most politically voiceless.

A further political consideration in the WASH sector is the international political environment. Water is considered a human right by many (Gleick, 1998; Scanlon et al., 2004). National governments are under pressure to ensure that clean water is provided to their citizens in order to receive the continued support of the international community, particularly through donors. The UN has been a key driver of pressure in this area with resolution 64/292 – the Human Right to Water and Sanitation. These are key considerations as governments try to attract investment and allocate spending accordingly. There is pressure to address these problems quickly which can affect the capacity to address them sustainably and the approach taken to doing so.

Of course, government involvement in WASH need not be through direct provision or subsidy. There are a range of other functions required in the WASH sector – related to regulation, licensing, arbitration, quality and coordination – where government are best place to engage in the context of private sector involvement (Perry et al., 1997; Komives, 2005). This formal institutional context can be used to overcome some of the problems created by the informal institutional context and vice versa. In the case of CCS, providing greater security and improved regulatory frameworks in informal settlements would allow for the expansion of successful entrepreneurs beyond their ‘economic zone’ and potentially even into the formal economy.

Moral considerations also play a role in shaping WASH outcomes. There are issues where politicians maintain a stance which is not necessarily in their political or economic interests, including in the provision of aid itself, but where they see a moral imperative to do so (Henson et al., 2010). Where neither political will nor moral obligation for change exist amongst politicians, a vital part of the political economy is the role of advocacy, which can shape the information environment impacting on the incentives of different actors – and potentially reconfigure the power dynamics within the WASH system. While, in practice, this function is often played by NGOs and development programmes, the sustainability and scale of the change is reliant on making this function intrinsic to the system. If poor people are not sufficiently empowered as to make their voices heard then it is the responsibility of development programmes that seek to serve their interests to make them heard. There will often be local partner organisations that do have political influence, or at least can attract support for the cause, for whom this would represent a key concern. A development programme might work with them to build capacity to better represent the interests of poor and marginalised people.

Such moral considerations can also be used positively in a competitive market where support for universal provision of WASH services has some currency. It may be possible to work with one company to make this a unique element of their offer; a small increase in fees to provide affordable water and sanitation to poor communities.

Political economy concerns, therefore, are an inherent part of the WASH sector in a way in which is more pronounced and important than is the case in many other sectors. In determining an approach to addressing WASH problems, political, economic and moral considerations, together with the interaction of the three, and the incentives of actors within a local or national political system must be included as variables in any analysis. This provides a further reason why a systemic approach is more appropriate for the WASH sector; considering interventions in isolation from the political economy in which they exist is to ignore a key determinant of the outcomes they will result in.

8. Conclusions and Future Challenges

The first conclusion to be drawn from the analysis contained here is that current approaches have been largely unsuccessful, particularly in respect of the sustainability and scale of their impact. In order to understand why this has happened, analysis needs to go beyond the specifics of which activities have been more successful than others in relative terms to look at the approaches that have been taken to decide what these activities should be. Such an analysis leads to understanding approaches to development as a trichotomy; direct delivery, knowledge and skills transfer, and systemic.

The direct delivery and knowledge and skills transfer approaches, which predominate development approaches to WASH problems, are ineffective, temporary and limited to the relatively small number of people at whom they are targeted. Analysis of these approaches has been incomplete, focussing only on short-term goals. There is, therefore a need for a different approach to programme design, implementation and evaluation, making scale and sustainability of change central to each.

Systemic change offers great potential as shown in the examples from Kenya, Mali, and Bolivia, which, while they haven’t been entirely successful, show that a more integrated approach in the WASH sector can lead to longer-term benefits. These are appreciably innovative and experimental incursions into a politically charged sector dominated by granting access to target groups in the short term. As such, these examples should be treated as promising innovations rather than policy prescriptions.

One of the major reasons why current approaches to WASH problems have failed, including the limitations to the success of more systemic approaches, has been a failure to fully understand the political economy of the environments in which the programmes are operating. This is highly influential in determining the incentives of actors within the WASH system and their willingness to change and, therefore, must be fully understood in any attempt to embrace a more systemic approach.

The positive empirical signs displayed here are reinforced by theoretical compatibilities with a systemic approach in the WASH sector. WASH problems are complex and require solutions which include a range of public and private sector actors, a thorough analysis of the formal and informal institutions governing a system, effective transmission of information, and the involvement of third sector actors to perform the role of advocacy. These are all issues which are analysed in and central to systemic change programmes and so there is a theoretical cohesiveness in addressing WASH problems in this way.

References

ADANK, M. 2012. It's About Sustainable Water Services, Not Just About Pumps and Pipes. Assessing and Monitoring Water Services in Ghana. Presentation to World Water Week, Stockholm: IRC.

AMERICAN RIVERS 2013. Drinking Water Infrastructure: Who Pays and for What (and how)? In: RIVERS, A. (ed.).

ASIAN DEVELOPMENT BANK 2009. Impact of Rural Water Supply and Sanitation in Punjab, Pakistan. In: DEVELOPMENT EFFECTIVENESS COMMITTEE (ed.).

BARRERA-OSORIO, F., OLIVERA, M. & OSPINO, C. 2009. Does Society Win or Lose as a Result of Privatization? The Case of Water Sector Privatization in Colombia. Economica, 76, 649-674.

BEHRENS-SHAH, P. 2011. Sustainability of Water Supply Systems in Kenya: Ex Post Assessments of 100 Water Supply Systems Constructed/Rehabilitated by Welthungerhilfe 2006-2009. In: WELTHUNGERHILFE (ed.). Berlin.

BESLEY, T. & GHATAK, M. 2001. Government versus private ownership of public goods. The Quarterly Journal of Economics, 116, 1343-1372.

BILLIG, P., BENDAHMANE, D. & SWINDALE, A. 1999. Water and sanitation indicators measurement guide, Food and Nutrition Technical Assistance Project, Academy for Educational Development.

BLUM, D., EMEH, R. N., HUTTLY, S. R., DOSUNMU-OGUNBI, O., OKEKE, N., AJALA, M., OKORO, J. I., AKUJOBI, C., KIRKWOOD, B. R. & FEACHEM, R. G. 1990. The Imo state (Nigeria) drinking water supply and sanitation project, 1. Description of the project, evaluation methods, and impact on intervening variables. Transactions of the Royal Society of Tropical Medicine and Hygiene, 84, 309-315.

BUDDS, J. & MCGRANAHAN, G. 2003. Are the debates on water privatization missing the point? Experiences from Africa, Asia and Latin America. Environment and Urbanization, 15, 87-114.

CAIRNCROSS, S. & SHORDT, K. 2004. It does last! Some findings from a multi-country study of hygiene sustainability. Waterlines, 22, 4-7.

CAIRNCROSS, S. & VALDMANIS, V. 2006. Water supply, sanitation and hygiene promotion. Disease Control Priorities in Developing Countries ed. Jamison D, Breman J, Measham A, Alleyne G, Claeson M, Evans D, Jha P, Mills A and Musgrove P. New York: Oxford University Press.

CARTER, R. C., TYRREL, S. F. & HOWSAM, P. 1999. The impact and sustainability of community water supply and sanitation programmes in developing countries. Water and Environment Journal, 13, 292-296.

CHANG, H.-J. & ROWTHORN, R. 1995. The role of the state in economic change, Oxford University Press.

CLASEN, T. F., BROWN, J. & COLLIN, S. M. 2006. Preventing diarrhoea with household ceramic water filters: assessment of a pilot project in Bolivia. International journal of environmental health research, 16, 231-239.

COLLIGNON, B. & VÉZINA, M. Independent water and sanitation providers in African cities full report of a ten-country study. Taller Soluciones Exitosas al Saneamiento en Zonas Desatendidas, 2001. SEDAPAL, 68.

DEPARTMENT OF WATER AFFAIRS 2012. Water and Sanitation Point Mapping Project Report 2012. In: GOVERNMENT OF THE KINGDOM OF SWAZILAND MINISTRY OF NATURAL RESOURCES & ENERGY (ed.).

DFID & SDC 2008. The Operational Guide for the Making Markets Work for the Poor (M4P) Approach. In: SDC & DFID (ed.).

DUVENDACK, M., HOMBRADOS, J. G., PALMER-JONES, R. & WADDINGTON, H. 2012. Assessing ‘what works’ in international development: meta-analysis for sophisticated dummies. Journal of development effectiveness, 4, 456-471.

EUROPEAN COURT OF AUDITORS 2012. Special Report 13: European Union Development Assistance for Drinking Water Supply and Basic Sanitation in sub-Saharan African Countries. In: UNION, E. (ed.). Luxembourg.

EVANS, B., COLIN, C., JONES, H. & ROBINSON, A. 2009. Sustainability and equity aspects of total sanitation programmes-A study of recent WaterAid-supported programmes in three countries: global synthesis report.

GLEICK, P. H. 1998. The human right to water. Water Policy, 1, 487-503.

GREENE, L. E., FREEMAN, M. C., AKOKO, D., SABOORI, S., MOE, C. & RHEINGANS, R. 2012. Impact of a School-Based Hygiene Promotion and Sanitation Intervention on Pupil Hand Contamination in Western Kenya: A Cluster Randomized Trial. The American journal of tropical medicine and hygiene, 87, 385.

HENSON, S., LINDSTROM, J. & HADDAD, L. 2010. Public Perceptions of International Development and Support for Aid in the UK: Results of a Quantitative Enquiry. IDS Working Paper, 353.

HIGGINS, J. P. & GREEN, S. 2008. Cochrane handbook for systematic reviews of interventions, Wiley Online Library.

HITCHINS, R., ELLIOTT, D. & GIBSON, A. 2004. Making business service markets work for the rural poor–a review of experience. Small Enterprise Development, 16, 10-23.

JMP 2012. Joint Monitoring Programme. New York: WHO/UNICEF.

KANAGASIVAM, S. & DANIELSON, B. 2011. Evaluation of NCA WASH program in Iraq 2005-2011. In: NORWEGIAN CHURCH AID (ed.).

KOMIVES, K. 2005. Water, electricity, and the poor: who benefits from utility subsidies?, World Bank-free PDF.

KREMER, M., LEINO, J., MIGUEL, E. & ZWANE, A. P. 2009. Spring Cleaning: Rural Water Impacts, Valuation and Property Rights Institutions. National Bureau of Economic Research.

LAGARDE, M., HAINES, A. & PALMER, N. 2007. Conditional cash transfers for improving uptake of health interventions in low-and middle-income countries. JAMA: the journal of the American Medical Association, 298, 1900-1910.

LIJIMA, Y., KARAMA, M., OUNDO, J. O. & HONDA, T. 2001. Prevention of bacterial diarrhea by pasteurization of drinking water in Kenya. Microbiology and immunology, 45, 413-416.

LISTENING TO DAR 2012. Report 15: Access to Water. Twaweza.

MALARCHER, S., MEIRIK, O., LEBETKIN, E., SHAH, I., SPIELER, J. & STANBACK, J. 2011. Provision of DMPA by community health workers: what the evidence shows. Contraception, 83, 495-503.

MALLETT, R., HAGEN-ZANKER, J., SLATER, R. & DUVENDACK, M. 2012. The benefits and challenges of using systematic reviews in international development research. Journal of development effectiveness, 4, 445-455.

MCDONALD, D. A. & RUITERS, G. 2005. The age of commodity: Water privatization in Southern Africa, Earthscan.

MEHROTRA, S. & DELAMONICA, E. 2005. The Private Sector and Privatization in Social Services Is the Washington Consensus ‘Dead’? Global Social Policy, 5, 141-174.

MINISTRY OF ENERGY AND WATER RESOURCES 2012. Sierra Leone Waterpoint Report. In: MINISTRY OF ENERGY AND WATER RESOURCES (ed.).

OLARREAGA, M., PAMPILLON, N. G. & BORRAZ, F. 2013. Water Nationalization and Service Quality.

PATTANAYAK, S. K., DICKINSON, K. L., YANG, J.-C., PATIL, S. R., PRAHARAJ, P. & POULOS, C. 2007. Promoting latrine use: Midline findings from a randomized evaluation of a community mobilization campaign in Bhadrak, Orissa. development.

PERRY, C. J., ROCK, M., SECKLER, D., ROCK, M. T. & SECKLER, D. W. 1997. Water as an economic good: A solution or a problem?, IWMI.

PRASAD, N. 2006. Privatisation results: private sector participation in water services after 15 years. Development Policy Review, 24, 669-692.

ROGERS, P., BHATIA, R. & HUBER, A. 1998. Water as a social and economic good: How to put the principle into practice, Global Water Partnership/Swedish International Development Cooperation Agency Stockholm, Sweden.

ROSEN, S. & VINCENT, J. R. 1999. Household water resources and rural productivity in Sub-Saharan Africa: A review of the evidence, Harvard Institute for International Development, Harvard University Cambridge, MA.

RUFÍN, C., RANGAN, U. S. & KUMAR, R. 2003. The Changing Role of the State in the Electricity Industry in Brazil, China, and India. American Journal of Economics and Sociology, 62, 649-675.

SCANLON, J., CASSAR, A. & NEMES, N. 2004. Water as a human right?, IUCN.

SOBSEY, M. D., STAUBER, C. E., CASANOVA, L. M., BROWN, J. M. & ELLIOTT, M. A. 2008. Point of use household drinking water filtration: a practical, effective solution for providing sustained access to safe drinking water in the developing world. Environmental science & technology, 42, 4261-4267.

SPARKMAN, D. 2012. From Beneficiaries to Businesses to teh Big Picture: Monitoring for Sustainability in Market Based Approaches to Sanitation. In: PEOPLE, W. F. (ed.).

TAYLOR, B. 2013. Evidence-based Policy and Systemic Change: Conflicting Trends? Springfield Centre Working Paper Series, 1.

THIEME, T. 2013. Trash and Toilets: ‘Hustling’ and the Informal Economy in Mathare, Kenya. In: UNPUBLISHED PHD THESIS (ed.).

TIWARI, C. 2013. Towards Strengthening the Gains of the Water Sector Reforms in the New Dispensation. In: SNV (ed.) Annual Water Sector Conference. Nairobi.

WADDINGTON, H., SNILSTVEIT, B., WHITE, H. & FEWTRELL, L. 2009. Water, sanitation and hygiene interventions to combat childhood diarrhoea in developing countries, International Initiative for Impact Evaluation Delhi.

WADDINGTON, H., WHITE, H., SNILSTVEIT, B., HOMBRADOS, J. G., VOJTKOVA, M., DAVIES, P., BHAVSAR, A., EYERS, J., KOEHLMOOS, T. P. & PETTICREW, M. 2012. How to do a good systematic review of effects in international development: a tool kit. Journal of development effectiveness, 4, 359-387.

WANG, Z., SHEPARD, D., ZHU, Y., CASH, R. A., ZHAO, R., ZHU, Z. & SHEN, F. 1989. Reduction of enteric infectious disease in rural China by providing deep-well tap water. Bulletin of the World Health Organization, 67, 171.

WATER FOR PEOPLE 2012. Rural Sanitation Marketing. In: WATER FOR PEOPLE (ed.) Proof Point.

WEISS, M. G. 2001. Cultural epidemiology: an introduction and overview. Anthropology & Medicine, 8, 5-29.

WILSON, J. M. & CHANDLER, G. N. 1993. Sustained improvements in hygiene behaviour amongst village women in Lombok, Indonesia. Transactions of the Royal Society of Tropical Medicine and Hygiene, 87, 615-616.

WSP 2012. Economic Assessment of Sanitation Interventions in Cambodia. In: PROGRAMME, W. A. S. (ed.).

[1] For simplicity ‘changes in WASH’ will be used as a catch-all to denote any change in Carter et al.’s (ibid; 13) “components of water and sanitation problems in developing countries”. A similar attempt to encompass the diversity of WASH interventions was made by Billig et al. (1999) as “improved hygiene behaviours and access to sanitation solutions and adequate water” but it was felt that for this research, quality dimensions also needed to be captured.

[2] A centre for impact evaluation set up by multiple donors in 2008 which conducts a large number of systematic reviews.

[3] Systematic review methodology: broadly follows Cochrane Handbook for Systematic Reviews of Interventions (Lagarde et al., 2007; Higgins and Green, 2008; Malarcher et al., 2011; Duvendack et al., 2012).

Using Scirus – a comprehensive aggregated database of academic articles and grey literature – the search terms ["water, sanitation and hygiene" AND "sustainability" ANDNOT "environmental sustainability"] in the entire document and ["systematic review" OR "meta-analysis"] in the title of the article were used. The phrase “water, sanitation and hygiene” is sufficiently orthodox within the field to make the sound assumption that any review would have had to include it, if not adopt it for the purposes of the review. As one of the principle purposes of this paper is to examine the sustainability of WASH interventions, it was important to include some reference to it in the selection of material. While there are synonyms for “sustainability” this term too has sufficient orthodoxy (although multiple meanings) that it would be highly unlikely that any paper that passed comment on long-term impact would not have included it. Indeed, the understanding of the term sustainability was sufficiently broad that the qualification of [ANDNOT “environmental sustainability”] was required so that it was the sustainability of the developmental outcomes that was under consideration and not simply the impact on the environment, which was to be one consideration amongst many. The search for the terms “systematic review” and “meta-analysis” used in all comprehensive analyses of this type, was limited to the titles of the papers to avoid articles being included which simply cited such reviews.

While there are provisions under the Cochrane system for the inclusion of non-randomised studies, there is a systematic bias against a number of types of intervention for which randomisation is inappropriate or under-resourced. The Cochrane system, too, has provisions for qualitative research which is to be embedded within or associated with the review but it is clear that such evidence should be used in the design and review of questions and methodologies rather than in the drawing of conclusions.

[4] A mixed good shares characteristics of a private good in that it is both rivalrous and excludable, but it also provides significant non-rivalrous, non-excludable positive externalities which are not revealed by the market.

---

* this article was first published on the Springfield Centre website. Please cite as Taylor, B., 2013, Effectiveness, Scale and Sustainability in WASH Programmes – A Review, Springfield Working Paper Series (2), The Springfield Centre, Durham.