measurement issues

Martin Ravallion, who is a senior researcher on anti-poverty policies at the World Bank, has (along with others) identified three major issues which can stymie anti-poverty program evaluation. Selection bias, which I discussed in my last post, is probably of greatest concern in the microfinance context, but there are also spillover effects and measurement problems. Spillover effects occur when the program being evaluated has a positive impact on not just its recipients but also, to some degree, on adjacent non-recipients. (Here's a trivial but hopefully intuitive example: there is a program targeted toward someone who owns a bar. The program results in the bar's business improving. The owner of the all-night pizza joint next door, who did not have access to the program, also sees an improvement in business as the result of all those drunk people next door. Now imagine that the bar owner is in your treatment group and the pizza-joint owner in your control group, and that you can't necessarily see the connection between the two, and try to figure out what, if any, the impact of your program is.)

And then there are measurement problems, which were brought up in the comments of my last post. One of the central issues here is that it is extremely common for the "businesses" run by microfinance clients to be very small and very, very informal. Not only is it unlikely that they keep records, it is unlikely that if you asked them you could get a consistently accurate answer to the question "what are your average weekly profits?" or even "what were your profits last week?" The finances of the business are often just too intermeshed with the finances of the household, and borrowers have no reason to keep track of profits because they don't necessarily pay taxes on business earning specifically (or perhaps on any earnings at all).

It is the case that some MFIs do trainings and counseling sessions to encourage microentrepreneurs to determine this type of information, which is logical insofar as you want to know something about the business prospects of the business you are lending to. In India I asked about weekly profits, and I also got information from IMED on profits, but the two almost never matched up very well for an individual borrower. Which leads me to another point, which is that the MFI's records may not even be that great, even if the organization is in general pretty well-run.

Some microfinance clients do have slightly larger business and keep better records, and there has been research done in which economists were able to gather the data necessary to estimate an actual profit function in order to look at how microfinance affects the microenterprise in a more formal way. But these borrowers tend, of course, to be a bit less poor, which is great for them, but doesn't yield information that is necessarily applicable to very poor people (this is actually a concern with all microfinance: many people argue that the loans just don't go to the "poorest of the poor" and that therefore microfinance isn't exactly the poverty cure-all that some people make it out to be).

Of course all of this says nothing about being able to study the effects of microfinance on something other than the borrower's business, such as nutrition or household wealth (both of which are covered in my survey). You can ask someone how long they've had a TV or running water, of course, but it's much harder to ask them about their eating habits or smaller-item spending patterns going back into the past.