Unconditional: The limits of evidence

If there is a progression in international development thinking towards a more favourable assessment of a Citizen’s Income, it involves more than one trajectory – and they are not parallel. Support for social protection, broadly, and cash transfers in particular, is increasingly a consensus position. The balance of views on the importance of transfers being unconditional and being universal however is far less clear. This article leaves aside the question of universality and addresses evidence on the benefits of conditionality – concluding that current efforts to demonstrate the superiority of unconditional transfers may be destined for disappointment, and suggesting additional approaches that may be of use.

As a starting point, we take Ian Orton’s summary of the findings of his study of social protection for the International Labour Organisation (ILO). [note]Citizen’s Income Newsletter, issue 2 for 2011[/note] He drew out two main conclusions. First, he found supporting evidence for a Citizen’s Income (CI) approach from social transfers for older people and for children. Second, however, Orton found ‘a mixed message’ from studies of transfers to the active population:

The key impediment to using the [ILO evidence matrix] to support CI revolves around whether it is the conditional nature of many of the STs that is pivotal in producing the positive results they have delivered. Does conditionality make the difference? If conditionality is not the overriding factor, then perhaps we can conclude that the unconditional and universal nature of a CI could deliver results similar to those documented in the ILO matrix. There is not space here to discuss this debate in full, but suffice to say that the precise role played by conditionality in delivering positive outcomes is not clear.

Finally, Orton notes that current government and World Bank preferences appear to be shifting towards conditionality; and that ‘this poses some concerns for those proposing universal and unconditional cash transfers. The political prospects of a CI would be better if the trend were against conditionality.’

Before moving to consider the evidence base on cash transfers and conditionality – which has been significantly strengthened in the last few years – it is worth noting the strength of the consensus on social protection of which the ILO position is part.

This is seen most clearly in the context of the live discussions over the post-2015 framework which will supersede the Millennium Development Goals. Broad support within the UN system is seen in the thematic think piece jointly authored by the Economic Commission for Africa, the ILO, the UN Conference on Trade and Development, the UN Department of Economic and Social Affairs and UNICEF (2012). The title says it all: ‘Social protection: A development priority in the post-2015 UN development agenda.’

Support extends across the spectrum of development NGOs also. In the drafting of Save the Children’s outline post-2015 proposal (2013), to which I contributed, for example, social protection was one of the most keenly discussed areas. The first of ten goals proposed includes a target to ‘Establish a global social protection floor’, which itself draws on ILO work, and proposed indicators to capture both the percentage of GDP spent on social protection and the percentage of the population covered. In general, the detail of proposals is geared towards cash transfers.

Evidence on cash transfers

As Orton (2011) indicates, however, this consensus does not extend to transfer payments being of an unconditional nature. In common with many other areas of development, transfers have seen an explosion of randomised control trial (RCT) testing in recent years. Three review papers produced for the UK’s Department for International Development (DFID) allow us to consider the balance of this evidence.

Hagen-Zanker et al. (2011) review evidence on the impact of 21 unconditional cash transfer (UCT) schemes, 18 conditional ones and 7 employment generation schemes (EGSs), focusing on effects on income (‘money-metric’) poverty. A second paper by DFID’s own staff (Arnold et al, 2011) considers a much broader range of impacts, and underlying studies. Finally, Kabeer et al. (2012) provide a more thorough review of conditional cash transfers (CCTs) only, with a focus on the economic impacts.

Cash transfers diagram

Hagen-Zanker et al (2011, HZ hereafter) is the only study that specifically aims to compare conditional and unconditional transfers. The authors’ figure 1 gives a sense of the sweep of evidence considered, and how 35,000 documents eventually reduced to 37 studies in the review. Unfortunately the main finding in regard to un/conditionality concerns the quality of evidence – specifically, that (p.vii):

Most of the high quality studies examined conditional cash transfer programming, and/or programmes from the Latin America region… In addition, greater methodological consistency in terms of analytical approaches adopted, and indicator selection and definition is needed to enable more meaningful cross-programme performance analysis, so that robust comparisons of performance both within intervention types (UCTs, CCTs, EGSs) and between types can be drawn, in order to inform future programming decisions.

On balance, HZ find stronger evidence for positive income and expenditure results from cash transfers than employment guarantee schemes, and for conditional compared to unconditional transfers – but the authors are clear that this reflects a lack of evidence on UCT effects, rather than evidence of a lack of effect of UCTs. Overall, the picture that emerges is of a failure to design either interventions or evaluations in such a way as to facilitate rigorous statistical comparison. The actual value of the thousands of studies examined, in terms of robust evidence on the value of conditionality, is ultimately close to zero.

Arnold et al. (2011) provide a literature survey rather than a systematic review, which allows them to focus more on specific findings rather than seeking the breadth of evidence to draw overarching conclusions. Their discussion of the relevant evidence leans against conditionality, highlighting the lack of evidence to support stronger results – even in terms of the narrow target indicators for conditionality. The major World Bank study of Fizsbein and Schady (2009) is cited in support of this finding; Fizsbein and Schady are notable for emphasising conditionality not in support of target outcomes but as offering potential to strengthen political acceptability of transfers – an important point to which we return below.

Arnold et al. (2011) also cite a World Bank randomised control trial carried out in Malawi (Baird et al., 2010) which was constructed to give a specific comparison and found that – compared to UCTs – conditions yielded no additional benefit in terms of increased school attendance and reduced dropout rates.

Finally, Kabeer et al. (2012) provide what looks the most robust review so far, but restrict themselves to conditional cash transfers only, recommending a separate study for UCTs. In light of HZ it seems unlikely sufficient evidence in fact exists; so a question for those interested in a Citizen’s Income is whether or not to prioritise investment in randomised control trials to this end.

The limits of evidence on conditionality

In the preceding section we have taken more or less for granted three implicit positions of much of the literature, namely: (i) that conditionality is the default position, so evidence is required to overturn the presumption in its favour; (ii) that aggregate effects on target populations are appropriate as the main measures; and (iii) that RCTs are the main, or even only, type of appropriate evidence. Each of these can be, and has been, challenged.

Ravi Kanbur (2013) takes aim at the first and second in his deliberately provocative paper, ‘Social protection: Consensus and challenges’ (p.16): ‘The somewhat unconditional support for CCTs is disconcerting. There seems to be very little questioning of them in the policy discourse. One is reminded of a similar situation with microfinance fifteen years ago. A more sober assessment of the microfinance phenomenon and its impact on the poor is now coming into view.’

Kanbur goes on to question the implications of the argument that CCTs may be more politically acceptable. He makes the case that conditioning tends to target ‘normal goods’ (those for which demand increases with household wealth), such as education and health. Almost by construction then, those households that meet the conditions for consuming these goods are likely to be the better-off in the target population – so that introducing conditionality to improve the middle class acceptability of transfers may have the effect of making them regressive, thereby exacerbating, for political reasons, the very inequality which the transfers may originally have been intended to challenge.

Kanbur goes on to argue that the onus should be on the supporters of conditionality to demonstrate its worth, rather than on its opponents to demonstrate the reverse. In the current climate, however, this seems an unlikely shift – which leaves a Citizen’s Income proponent, once again, with the question of whether to pursue new, high-quality RCT evidence, with the aim of changing the balance of presumption.

The efforts of Citizen’s Income proponents – notable among them Guy Standing – to do just this are commendable. Evidence from the trial in Namibia is indicative only, but compelling; while the major trial underway in India promises to generate exactly the kind of systematic RCT evidence that the debate currently relies upon. Evidence on the distributive impact of conditionality could be of particular value.

There are, however, two reasons to be cautious about the likely return to further efforts in this direction.

One is that the wealth of recent work means that additional pieces of evidence, even of high quality, will not necessarily provide the silver bullet to change policymaker positions. Rather, they are likely to add to that base of knowledge, undoubtedly improving but at the same time being diluted by their position as one among many.

The second is perhaps a more fundamental reason to be concerned about the current dynamic in relation to evidence on conditionality. The limits of RCTs are increasingly well documented (see e.g. my colleagues Lant Pritchett’s views here: www.oxfamblogs.org/fp2p/?p=12562), in particular in regard to their non-applicability in wider circumstances. Some proponents believe that RCTs can themselves begin to answer these questions (eg Bold et al., 2013, document RCT findings on the scaling up of education interventions in Kenya).

It is the political questions that RCT approaches to conditionality seem least likely to be able to capture, however. Some aspects of dis/empowerment could potentially be assessed in RCT conditions; as perhaps could questions of wider public support, given sufficient scale and time; but to attempt to use RCTs to understand the implications of conditionality, and related questions around direct and indirect tax funding, in terms of long-term state-citizen relationships seems likely to prove a fool’s errand.

Implications

Where does this leave a development-focused Citizen’s Income proponent? A good question, with no easy answers.

Some pointers may come from the literature on tax and development, which perhaps offers an appropriate parallel. I once wrote an angry piece about the damage done by the ‘tax consensus’ (Cobham, 2007) – by which, to put it briefly, major donors including the International Monetary Fund had pushed a succession of ‘reforms’ with the result that most developing countries had backed off from direct taxation (and trade taxes), and pursued sales taxes including VAT instead.

These reforms were justified largely in the name of efficiency, and perhaps with the underlying thought that this was the direction of travel in rich countries and therefore represented appropriate ‘modernisation’. Setting aside evidence on the administrative demands of running VAT systems, what was most striking about the dominance of the tax consensus was that it effectively put aside concerns about inequality and politics. To caricature only slightly, tax could be regressive because developing countries were better at doing redistribution through expenditure (hence for many years Latin America had the same type of market income inequality as the UK and USA, but, due to the absence of progressive taxation, much higher final consumption inequality).

At the same time, there was little consideration for the political impacts. Meanwhile, the evidence has grown that direct taxation is fundamental to the state-citizen relationship, and to the development over time of accountable governance (eg Ross, 2004; Mahon, 2005).

The pendulum has begun – though only begun – to swing towards a more holistic understanding of the role of taxation in development, and away from a simplistic and narrow focus on certain technical aspects. Arguably, this is the bigger shift that is needed on the expenditure side. Pursuit of additional RCT evidence on conditionality may ultimately reinforce a consensus about the appropriate basis on which to assess the impact of transfers. Wider analysis of the relationships between social transfers, citizenship and governance may provide a better path towards reassessment of the potential of unconditional Citizen’s Income measures to contribute towards broad-based human development progress.

 

References

Arnold, C. with T. Conway and M. Greenslade, 2011, ‘Cash transfers’, DFID Evidence Paper.

Baird, S., C. McIntosh and B. Ozler, 2010, ‘Cash or condition? Evidence from a randomized cash transfer program’, World Bank Policy Research Working Paper 5259.

Bold, T., M. Kimenyi, G. Mwabu, A. Ng’ang’a and J. Sandefur, ‘Scaling-up what works: Experimental evidence on external validity in Kenyan education,’ Center for Global Development mimeo.

Cobham, A., 2007, ‘The tax consensus has failed!’, Oxford Council on Good Governance Economy Recommendation 8.

ECA, ILO, UNCTAD, UNDESA & UNICEF, 2012, ‘Social protection: A development priority in the post-2015 UN development agenda’, UN System Task Team on the Post-2015 UN Development Agenda – Thematic Think Piece.

Fiszbein, A. and N. Schady with F. Ferreira, M. Grosh, N. Kelleher, P. Olinto and E. Skoufias, 2009, Conditional Cash Transfers: Reducing present and future poverty, Washington, DC: World Bank.

Hagen-Zanker, J., A. McCord and R. Holmes with F. Booker and E. Molinari, 2011, ‘Systematic review of the impact of employment guarantee schemes and cash transfers on the poor’, ODI Systematic Review.

Kabeer, N., C. Piza and L. Taylor, 2012, ‘What are the economic impacts of conditional cash transfer programmes? A systematic review of the evidence’, Technical report, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

Mahon, J., 2005, ‘Liberal states and ?scal contracts: Aspects of the political economy of public ?nance’, paper presented at the annual meeting of the American Political Science Association.

Orton, I., 2011, ‘The International Labour Organisation’s analysis of social transfers worldwide augurs well for a Citizen’s Income in the context of middle and low-income countries’, Citizens Income Newsletter, issue 2 for 2011.

Ross, M., 2004, ‘Does taxation lead to representation?’, British Journal of Political Science, 34, pp.229-249.

Save the Children International, 2013, ‘Ending poverty in our generation: Save the Children’s vision for a post-2015 framework’.

 

Alex Cobham works for the Centre for Global Development and is a trustee of the Citizen’s Income Trust

Footnotes