More evidence, less poverty

The title above is the mission statement of the organization I work for here in Tamale, Innovations for Poverty Action or, as it is more commonly called, IPA (people in the NGO world love acronyms!)

In broad terms, what we do at IPA is partner with other development NGOs or country governments (e.g. the government of Ghana) to conduct impact evaluations that test the effectiveness of their programs on key outcomes they care about. We work in a wide range of sectors – education, financial inclusion, social protection etc. – so those outcomes could be anything from school enrollment rates to agricultural productivity.

The main methodology we use to do this work is RCTs, which stands for a randomized control trial. These are commonly used in medicine and are known as clinical trials, but the idea is essentially the same in the international development sector – we take a sample and randomly assign half to be in the treatment group and the other half to be in the comparison group. The treatment group receives some intervention (whatever the NGO or government program provides) while the comparison group receives nothing. Having the comparison group is critical because it tells us what would have happened in the absence of the program i.e. the counterfactual.

We collect data on both groups, and at the end of the study we evaluate the outcomes of the treatment group relative to the comparison group in order to determine whether the intervention made those who were treated better off, since the randomization ensures that the two groups should have been more or less identical prior to the study. The diagram below should help illustrate this concept.

If interested, you can read more about RCTs and why we at IPA use them to conduct evaluations here.

My official role with IPA is that of a senior research associate. It is not a particularly meaningful title, but basically what I do is manage one of the ongoing impact evaluations we are conducting here in northern Ghana. It is a study called Comparing Livelihood Approaches for the Ultra-Poor in Ghana (you can read the full IPA study description here).

Back in 2016, the lead researchers on the project randomly selected 7,330 households across 4 regions in northern Ghana to participate in the study. They were selected based on a poverty scorecard test, since the goal was to target the poorest households within each community where we would be working.

I won’t go into the weeds on the various treatment arms, but you can think of there being two main interventions: 1) a mental health intervention, which took place in the form of cognitive behavioral therapy sessions administered at the community level and 2) an economic intervention, which took place in the form of a livelihood program that our partner, an NGO called Heifer International, had been implementing previously and was willing to be evaluated on. The livelihood program has a lot of components: a productive asset transfer (households choose whether they want to receive goats, sheep, pigs, poultry, etc.), coaching visits to help them manage the asset, consumption support, access to savings, and health insurance. The idea behind this “package” style intervention is that there is no silver bullet for graduating someone out of poverty; rather, it takes many different things working together to help someone sustainably improve his or her livelihood.

Thus, we ended up with a study design in which there were 4 main treatment groups: 1) households that received only the mental health intervention 2) households that received only the economic intervention 3) households that received both interventions and 4) households that received nothing i.e. the comparison group. We are now three years into the study and conducting the final round of data collection on all these households. Once we do, we will aim to test the impacts of the various interventions. Of particular interest will be comparing those households who received both interventions to the households that only received one or the other, to examine whether there are added gains to treating both the psychological and economic dimensions of poverty.

When I began the role in August, my job was wholly focused on getting ready for the survey we were preparing to conduct on our sample of 7,000+ households. That meant programming the survey instrument, setting up a data management system, recruiting/hiring/training surveyors, procuring field logistics (tablets for data collection, backpacks, ID badges, etc.) and the like. It was an intense first few months, and was especially nerve-wracking as the time approached to actually launch the survey.

Now, we are fully in the swing of things – about 10 weeks into the survey period with things going more or less smoothly (there is always some shit hitting the fan, but at this point it’s become expected so I don’t stress too much). Usually once a week or so, I accompany one of our field managers to the actual communities we’re working in so that I can help monitor how the survey teams are doing . It is one of my favorite aspects of the role because I get to witness firsthand the very personal, very genuine interactions that our team has with the actual study participants.

When I sit in on interviews, I am always amazed at how the respondents – regardless of their economic situation – so generously give us their time and attention, so willingly engage with us even though they know they don’t receive any money for doing so. When we enter a compound, we are immediately greeted (“Despa” for good morning or “Anterri” for good afternoon), followed by a flurry of movements as someone rushes to find us stools or chairs to sit on, as we are their guests. We usually seat ourselves in a shaded hut, with chickens or goats casually strolling through every now and then to pluck at stray maize kernels on the ground.

For me especially, it is eye-opening to see how the data is collected at its very source. I sit there knowing that the words exchanged between respondent and surveyor will eventually translate into a mere observation within a larger dataset, the nuance lost and the sentiment removed. That data will be aggregated, analyzed, and summarized – used as evidence of how the program works and communicated to policymakers or other researchers. But in the moment, the people speaking with us are what matter most. They give my work meaning and they humble me immensely.