Jeffrey D. Michler, Dewan Abdullah Al Rafi, Jonathan Giezendanner, Anna Josephson, Valerien O. Pede, Elizabeth Tellman
{"title":"Impact Evaluations in Data Poor Settings: The Case of Stress-Tolerant Rice Varieties in Bangladesh","authors":"Jeffrey D. Michler, Dewan Abdullah Al Rafi, Jonathan Giezendanner, Anna Josephson, Valerien O. Pede, Elizabeth Tellman","doi":"arxiv-2409.02201","DOIUrl":null,"url":null,"abstract":"Impact evaluations of new technologies are critical to assessing and\nimproving investment in national and international development goals. Yet many\nof these technologies are introduced and promoted at times and in places that\nlack the necessary data to conduct a strongly identified impact evaluation. We\npresent a new method that combines remotely sensed Earth observation (EO) data,\nrecent advances in machine learning, and socioeconomic survey data so as to\nallow researchers to conduct impact evaluations of a certain class of\ntechnologies when traditional economic data is missing. To demonstrate our\napproach, we study stress tolerant rice varieties (STRVs) that were introduced\nin Bangladesh more than a decade ago. Using 20 years of EO data on rice\nproduction and flooding, we fail to replicate existing RCT and field trial\nevidence of STRV effectiveness. We validate this failure to replicate with\nadministrative and household panel data as well as conduct Monte Carlo\nsimulations to test the sensitivity to mismeasurement of past evidence on the\neffectiveness of STRVs. Our findings speak to conducting large scale, long-term\nimpact evaluations to verify external validity of small scale experimental data\nwhile also laying out a path for researchers to conduct similar evaluations in\nother data poor settings.","PeriodicalId":501273,"journal":{"name":"arXiv - ECON - General Economics","volume":"75 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - ECON - General Economics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.02201","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
Impact evaluations of new technologies are critical to assessing and
improving investment in national and international development goals. Yet many
of these technologies are introduced and promoted at times and in places that
lack the necessary data to conduct a strongly identified impact evaluation. We
present a new method that combines remotely sensed Earth observation (EO) data,
recent advances in machine learning, and socioeconomic survey data so as to
allow researchers to conduct impact evaluations of a certain class of
technologies when traditional economic data is missing. To demonstrate our
approach, we study stress tolerant rice varieties (STRVs) that were introduced
in Bangladesh more than a decade ago. Using 20 years of EO data on rice
production and flooding, we fail to replicate existing RCT and field trial
evidence of STRV effectiveness. We validate this failure to replicate with
administrative and household panel data as well as conduct Monte Carlo
simulations to test the sensitivity to mismeasurement of past evidence on the
effectiveness of STRVs. Our findings speak to conducting large scale, long-term
impact evaluations to verify external validity of small scale experimental data
while also laying out a path for researchers to conduct similar evaluations in
other data poor settings.