A central problem with measuring the imact of microfinance (or, to some extent, any anti-poverty program) is that you have to find a way to isolate the effect of the program itself so that you don't ascribe good (or bad) outcomes to the presence of a loan when they are in fact the result of something else. The classic example from microfinance is selection bias. Most microfinance borrowers run very small, informal businesses ("microenterprises" in the development lingo), but of course some people are better entrepreneurs than others, and as a result some people will be more successful than others. It's pretty easy to imagine, furthermore, that someone with good entrepreneurial skills might be systematically more likely to seek out a microfinance loan (because they're more ambitious, resourceful, willing to take risks, etc). Therefore microfinance might appear to make businesses more successful, when in fact all that you're seeing is that successful people are more likely to have microfinance. It could also be the case, on the other hand, that people who have experienced random bad outcomes ("negative shocks" in the lingo) might be more likely to seek out microfinance, so microfinance could appear to have no impact or negative impacts simply because many of the people who got loans were worse off to begin with.
The bottom line is that you can't compare borrowers to non-borrowers and see who is doing better and ascribe what you find to microfinance, because there is no reason to think that microfinance clients have the same characteristics (and therefore the same average outcomes) as non-clients. There are several different ways to get around this (my thesis, for example, looked at outcomes for individual borrowers before and after credit, rather than comparing borrowers to non-borrowers), but probably the best way is to do a randomized field experiment. What this would mean in a perfect world is that you have randomly assigned treatment and a control groups, with both groups consisting of microentrepreneurs who want and are eligible for microfinance. The treatment group get loans, the control group doesn't. The problem, obviously, is that you have to identify a bunch of new borrowers, but then randomly deny half of them credit for a period of time. Although that kind of control group methodology is common in, say, clinical drug trials, you can see how it might be logistically difficult to manage with microfinance loans (especially when an area has more than one MFI, so that control-group members could potentially seek loans elsewhere) and how MFIs might be wary on ethical grounds.
What my advisor is attempting, in Ghana and in India (with myself and my fellow IDECer JP as his research assistants, since he's not actually going to either of those places this summer), is a more feasible iteration of a randomized field experiment. It uses the concept of an instrumental variable, which is basically a way of measuring a causal relationship indirectly so as to subvert some of the bias problems that may be inherent in that relationship. In the microfinance context, we want an instrumental variable that is strongly correlated with receiving a microfinance loan, but has absolutely nothing to do with whatever positive outcomes we expect microfinance to cause.
In this case, what I will be doing is creating random treatment and control groups of microfinance-eligible microentrepreneurs who aren't currently MFI clients. The people in the treatment group will be encouraged to take a microfinance loan (essentially we'll heavily market microfinance to them), and the people in the control group won't be. People in either group are then free to get a microfinance loan if they choose to. The idea is that, hopefully, lots more of the people in the treatment group (the ones who had microfinance marketed to them) will decide to get loans, so that there will be a very strong correlation between being in the treatment group and having a microfinance loan. At the same time, having been told how great microfinance is will not, in itself, have any conceivable impact on whether or not your business does well or your family eats more meat, etc.
Thus if we want to know if microfinance increases a person's likelihood of buying a productive business asset (of course, we already know it does, because I said so in my thesis, and I'm always right) we can estimate the relationship between the encouragement "instrument" and the likelihood of buying a business asset. Since we know that the encouragement itself has no impact on buying a business asset, any estimated relationship between the two can be attributed to microfinance. The upshot in all this is that our instrument (the encouragement) is completely random--people in the treatment group should have, on average, characteristics identical to those of the people in the control group--so bias problems are completely avoided.
That's the theory anyway. This particular methodology hasn't been used for a microfinance field experiment before, so we simply don't know how well it's going to work. If take-up of microfinance isn't significantly higher in the treatment group, we're kind of screwed. But if it does work, the results should be both interesting and econometrically sound, which is great.
6 comments:
Out of curiousity, is it possible to compare borrowers to themselves? I know data isn't perfect, but on a grand enough scale wouldn't it be significant to compare year-over-year enterprise growth prior to a loan and after a loan? That would eliminate selection bias to a degree.
Dude! Adam Smith reads your blog?
;)
ALL the great, dead economists read my blog. even milton friedman, although i'm pretty sure he just reads it to make fun of me.
anyway, adam, it is possible, and in some ways ideal, to compare borrowers to themselves. that's essentially what i was doing (or part of what i was doing) in my thesis.
there are two problems, however, one practical and one theoretical. in practical terms, you need information on borrower outcomes spanning a relevant period of time, probably at least a couple of years. this means that you have keep track of the people that you surveyed and go back and survey them again at least one more time, which can be logistically tricky (although it's certainly not impossible, and that's the intention with what i'm doing--the people i survey this summer will be re-surveyed next summer by 1st year IDEC students). if you're only doing a one-shot survey, you can create panel data by using the restrospective methodology that i used in my thesis (asking borrowers about past events and using the info to create past years of data), but this limits what you are able to study and can have accuracy problems.
the theoretical issue with comparing borrowers to themselves is that by relying on a differencing approach (meaning that you're looking not at, say, income levels, but differences in incomes from one period to the next), you can't measure the impact of anything that is time-invariant. you can't look at differences in the impact of microfinance between men and women or between different religious groups or between borrowers with different types of businesses, because gender, religion, and business type don't generally change over time.
so what i'm doing is gathering the first round of data in what will become panel data (meaning it is data across borrowers and over time). the data will be able to be used to both compare borrowers to each other and to themselves over time, with bias being controlled for in either case, which is fairly ideal.
I certainly see what you're saying regarding the inherent problems. But do most businesses keep records? Can't you tell how they are doing merely by a little forensic accounting? Would that create another, more difficult selection problem by only seeing businesses who a)kept records, and b)were willing to let you examine said records?
tom- the short answer to your questions are "no", "maybe", and "yes", but the points you bring up are worth discussing in a bit more detail, so please refer to my next post.
Aren't there some mighty assumptions here? In particular, that most of those you market to will take the loans? If not, the self selection problem would seem to be just as severe as if you had not bothered to establish your control group.
Post a Comment