Search results

Privacy Enhancing Technologies (PETs) are at the forefront of efforts to protect privacy and enable responsible data use. But what are they and how are they used? Dan Cassara, former Project Manager for the Digital Credit Observatory (DCO), provides a case study of how PETs can be used to enable better financial lending practices across countries in the third part of our series on 'How PETs Make Data Work for All.'Read parts one and two. Learn more about PETs in our Data Privacy Lab.Credit: Maksym Yemelyanov via Adobe StockAccording to GSMA's 2023 State of the Industry Report, it took 17 years for the first 800 million mobile money accounts to register globally, but just five years for the next 800 million accounts. The expansion of other digitized financial services has also accelerated, with the proportion of mobile money providers offering digital credit nearly doubling in the past two years. Though the surge of digital financial services across low- and middle-income countries (LMICs) is well documented, the implications for data protection and compliance have received less attention until recently. This blog post builds off of parts one and two in this series exploring how Privacy Enhancing Technologies (PETs) can enable more responsible data sharing for the public good.Many mobile money providers and fintechs operate in multiple markets. For example, Safaricom, which catalyzed the rise of mobile money by launching M-Pesa in 2007, recently began operating in Ethiopia, its tenth market. Because many fintechs use machine learning algorithms to predict loan repayment and create credit scores for consumers, new markets provide not just growth, but also new data to improve their models. However, new markets can also create regulatory and compliance challenges.Data Protection ChallengesRegulatory efforts to safeguard data have often included data protection laws: 71% of all nations worldwide have data protection laws, including roughly half of the countries the UN refers to as Least Developed Countries. These laws generally govern what data can be collected, how it is processed, and how it can be shared. In particular, they often limit cross-border data sharing. Kenya, the birthplace of mobile money and a hub for fintech firms in Africa, passed a Data Protection Act in 2019 that states personal data cannot be "transferred outside Kenya, unless there is proof of adequate data protection safeguards or consent from the data subject."A fintech aiming to aggregate data from customers in multiple countries to predict consumer behavior may not be able to do so because of these kinds of data sharing restrictions. As a result, it might face higher default rates and pass those costs on to consumers in the form of increased interest rates. Instead, the fintech could turn to homomorphic encryption (HE), which refers to methods for analyzing data while it is still encrypted. (Analyzing data has traditionally required that the data be decrypted, making it vulnerable to hacks and misuse.) In this way, it may comply with data sharing regulations, as underlying data is not available across borders, and still learn from the data.Using Homomorphic Encryption (HE) to Protect PrivacyTo illustrate the idea underpinning HE, consider a scenario before electronic communications where Alice wants to send a secret message to her colleague, Bob, but is worried it could be intercepted. So Alice puts the message in a box, locks it, and sends it to Bob. The message is secure, but there's a new problem once Bob receives it: he doesn't have the key. Bob puts his own lock on the box and sends it back to Alice. After she removes her lock and sends the box back to Bob, he can remove his lock and finally access the message. This approach added extra steps, but it ensured that the contents of the box were never revealed to anyone except Bob.In practice, the secret message is often a desired query or computation, like the total number of transactions that customers initiated last quarter. Suppose there are just two customers who made 4 and 7 transactions, respectively; with access to a list of customers and their transaction histories, it's a simple task to calculate that 11 transactions were made. But if the bank uses HE to protect customers' data, the analysis must incorporate a set of procedures to ensure the encryption, computation, and decryption processes return the correct answer. There are many such systems, but most rely on a set of public and private keys that are similar to the locks Alice and Bob used to lock and unlock the box — or encrypt the data and decrypt the results — but with a few important differences. First, the dataholder (the bank) has both keys. Second, some predefined operations can be performed on the data in the box without ever needing to see inside.For simplicity, let's also assume that a given encryption system and set of public keys transform the values 4 and 7 into encrypted values of 31 and 212 (the math can get complex, interested readers should consult the endnotes). When the query is attempted, instead of 4 + 7 = 11, the system, operating on encrypted data, gets 31 * 212 = 6,572 (for technical reasons, 31 and 212 are multiplied, not added; see the endnotes for more info). The encrypted value of 6,572 clearly does not equal 11, but this is expected in this encrypted space. To return to unencrypted numbers, the bank's system uses a decryption function that relies on private keys. With some hand-waving (see notes), the computer returns the correct answer of 11 to the bank, who never saw the underlying transaction histories.The Future of HEsThe processing power needed to privately analyze data using HE limited real-world applications until recent advances in computing power, but financial services applications are now growing. HE was used in combination with Multiparty Computing to enable data from non-Belgian subsidiaries of a Belgian bank to be used in its machine-learning model, even though EU privacy regulation prohibits the cross-border transfer of this data. Researchers constructed a privacy-preserving machine learning model that did not require any of the underlying data from the non-Belgian subsidiaries to be shared, and all private data remained encrypted throughout the computational process.Industry's growing reliance on big data and the increasing prevalence of data misuse has led many countries to pass data protection laws. These laws often empower people to have greater control of their data, and provide necessary safeguards. The cost is that data sharing — and the learning and innovation it can facilitate — becomes more burdensome, if not impossible. Privacy enhancing technologies like homomorphic encryption can play an important role in facilitating responsible data usage to expand access to financial services and to drive down costs, without sacrificing the data protections that people deserve.For more detail on how HE works, review this example.Homomorphic Encryption for Cross-Border Computation was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

This year, CEGA was recognized by the American Economic Association (AEA) for our efforts to promote diversity and inclusion in the field of economics and within our center. In this post, Maya Ranganath, Associate Director of Global Networks and Inclusion, reflects on CEGA's inclusion strategy, pointing to unequal power dynamics in global development research and discussing how CEGA ensures that our work — externally and internally — is consistently aligned with our core values.CEGA Fellows, from left to right: Jaah Mkupete, Fola Aina, Hellen Namawejje, Caroline Sitienei Koech, Arnold Musungu, Ojiri Enahoro Innocent, and Muthoni Ng'ang'a, alongside Maya Ranganath at PacDev 2024 | Credit: CEGAAt CEGA, we regularly reflect on what it means to produce evidence for decision-makers in low- and middle-income countries (LMICs) from our position at an elite academic institution in the United States. In recent years, we have increasingly focused on how scientific narratives are "owned" by privileged groups and have updated our diversity, equity, inclusion, and justice (DEIJ) strategy to address this dynamic. While inclusion has always been core to CEGA, we have consistently updated our strategies to match evolutions in the development research ecosystem.CEGA's theory of change centers our DEIJ goals: our third pillar is to "make the evidence ecosystem more inclusive." We pursue three interlocking objectives in support of this:To center the voices of underrepresented groups — including researchers, decision-makers, and study participants in LMICs — in social science research and policy debates.To ensure that rigorous evidence exists to help decision-makers in LMICs address injustices in society.To create a diverse and welcoming community at CEGA, respect and reflect the communities in which we live and work, and encourage the broader evidence-informed policy ecosystem to seek the same.We pursue these objectives by investing in LMIC scholars and institutions, supporting inclusive and equitable research, promoting ethics in research, diversifying our networks and team, and creating an environment where our staff and researchers feel a sense of belonging. Below we highlight some of the most important aspects of our DEIJ strategy.Investing in LMIC Scholars and InstitutionsSince 2008, CEGA's Global Networks initiative has bolstered the leadership of scholars from Africa, South Asia, and other low-income regions by equipping them with the tools, skills, networks, and funding they need to thrive. We organize short-course workshops, host semester-long fellowships, and provide extensive follow-on support. To date, CEGA has hosted 85 scholars at UC Berkeley, Northwestern University, and virtually. The fellowships provide the opportunity to audit relevant courses, present work in seminars, apply to competitive funding calls, receive mentorship, and network with faculty members and students. These scholars have received nearly $2 million in grant funding to promote their independent research activities and mainstream of development research globally. Program alumni started the Network of Impact Evaluation Researchers in Africa (NIERA) in 2018, an independent organization of former fellows that works to advance decision-focused evaluations in sub-Saharan Africa.Importantly, the capacity-strengthening support we provide to LMIC scholars is designed to meet the demand for training and mentorship in the short- and medium-term. Longer term, we are eager to support the transfer of ownership of these types of programs to LMIC institutions, to leverage their growing capacity and resources to train and mentor the next generation of African scholars.As a learning organization, CEGA is committed to pursuing open inquiry and debate about the barriers to — and opportunities for — inclusion across the evidence-informed policy ecosystem. Through our Collaboration for Inclusive Development Research (CIDR), we are working with NIERA to develop an evidence-based theory of change for inclusion that delineates which stakeholders are best placed to add value at various stages of the research-to-policy pipeline.Supporting Inclusive and Equitable ResearchMany research approaches in development economics reinforce Western notions of "epistemic superiority" over indigenous ways of knowing. CEGA has long encouraged an inclusive approach and a plurality of methods and disciplines that, together, make our research more accessible and more relevant to a broader range of stakeholders. While we continue to prioritize the use of impact evaluations where feasible and appropriate, we also support studies that leverage quasi-experimental methods, qualitative methods, data science, and mixed-methods approaches.As a re-granting institution, we recognize our power to fund research equitably and inclusively and support research that challenges unequal power structures. Our research granting policy formalizes how and when research teams should consider DEIJ criteria when awarding new grants. Typically, when evaluating research proposals, we include criteria that assess the diversity of the research team and of the larger portfolio. We have also developed a "co-authorship statement" with our grant awards outlining how researchers should be credited for their contributions, encouraging greater equity within research teams.CEGA prioritizes research activities that address persistent inequities and injustices in society. This means moving beyond sub-group analysis and conducting research with transformational potential for traditionally disadvantaged groups. For example, CEGA recently established a "Gender & Agency" theme to support gender transformative research aimed at dismantling harmful power structures and promoting gender equity. Additionally, our domestic-facing portfolio, the Opportunity Lab, hosts an initiative on "Racial Equity in the Labor Market," which examines the role of state and federal wage and employment policy in reducing racial disparities and promoting greater equity of economic opportunity in the United States.CEGA's Berkeley Initiative for Transparency in the Social Sciences (BITSS) works to democratize the way social science is produced, scrutinized, and shared by realigning incentives for knowledge generation and use. For example, the BITSS catalyst program invests in scholars from LMICs to encourage transparent and reproducible research. Tools and standards that foster open access to data, code, and published research make it easier for LMIC researchers to access and advance new research. Additionally, the Social Science Prediction Platform (SSPP) provides a tool for sourcing input from community members, amplifying local knowledge, and helping to inform better research.Diversifying our Network and TeamCEGA continues to pursue a more diverse staff and network of affiliates. We have audited and refined our hiring processes, writing more inclusive job descriptions, disseminating them to diverse institutions, and developing a more equitable interview process. While the work of diversifying our organization will never be "finished," our efforts have yielded some progress: currently, 27% of our staff come from underrepresented backgrounds (as defined by UC Berkeley), up from 9% only three years ago. To track our progress, we conduct annual demographic surveys with staff, measure staff perceptions of the Center's equity and inclusivity, and enable anonymous feedback mechanisms. Meanwhile, we recently revamped our compensation framework to ensure that staff of different backgrounds are compensated equitably and transparently. We also audited and adapted our affiliate nomination process to ensure it is equitable and conducive to a diverse network. Importantly, we maintain a variety of staff forums to discuss our positionality in development research and reflect on current events in our communities.CEGA's DEIJ work is managed by a dedicated working group that I lead as the Associate Director of Global Networks and Inclusion, and guided by an internal DEIJ strategy and set of key performance indicators. We are energized by the AEA's recognition of our efforts over the years, while acknowledging that we have a long way to go to achieve our DEIJ goals. In this spirit, we welcome your comments and feedback on our approach. We further commit to sharing our learnings, including our challenges and failures, openly and transparently. Please stay tuned for more updates on DEIJ in the near future.CEGA's Award-Winning Approach to Promoting Diversity and Inclusion was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

Celebrating the DAISI Regional Scholars Program — Two-Years StrongThe Regional Scholars Program has taken center stage at the Digital Agricultural Innovations and Services Initiative (DAISI) nearly two years after its launch. The DAISI Regional Scholars Program is integral to the DAISI initiative, which is co-managed by the Center for Effective Global Action (CEGA) at the University of California, Berkeley, and the Abdul Latif Jameel Poverty Action Lab (J-PAL) at the Massachusetts Institute of Technology and promotes research opportunities for scholars based in sub-Saharan Africa and South Asia.In this blog, Abel Mkulama, Program Associate at CEGA, highlights the DAISI Regional Scholars Program's successes since its launch in 2021.Credit: Adobe StockBuilding Bridges in ResearchGenerating evidence that can positively impact policy decisions requires a deep understanding of the local context and relationships with policymakers. Though researchers from low- and middle-income countries (LMICs) often possess both, according to a UNESCO report [1], they only produce two percent of the world's research output. This is mainly due to limited access to research funding and research tools. However, impact evaluation capacity in LMICs is now increasing [2], thanks in part to programs such as the CEGA's Global Networks Program and the J-PAL Scholars Program that provide direct research training, mentorship, and funding opportunities for researchers based in LMICs who are using innovative approaches for reducing poverty.In 2021, CEGA and J-PAL launched the Digital Agricultural Innovations and Services Initiative (DAISI), which funds research to rigorously evaluate programs that increase the availability, quality, and reach of bundled digital agricultural solutions and services for small-scale agricultural producers in South Asia and sub-Saharan Africa. As part of DAISI's activities, the DAISI Regional Scholars Program advances funding opportunities and fosters mentorship for researchers interested in impact evaluation methodologies who hold a PhD and are based at academic institutions in South Asia and sub-Saharan Africa.Mentoring Scholars On the RiseDAISI's success is closely tied to our partnerships, strong network of researchers, and collaborative culture. The DAISI Regional Scholars Program affirms that local researchers are uniquely positioned to leverage contextual knowledge of the regional policy landscape and stakeholders to identify and evaluate relevant topics that can translate into meaningful policy action for achieving their nation's development goals.The program includes mentorship activities, fostering collaboration by pairing a Scholar with a CEGA or J-PAL-affiliated researcher who is based in a leading university and has experience conducting randomized impact evaluations. The mentorship provides space to share knowledge, engage in meaningful discussions, and explore research ideas. Scholars also receive constructive feedback while gaining hands-on experience in impact evaluation methods. Additionally, DAISI intends to convene Scholars for targeted interaction, fostering connections with both CEGA and J-PAL staff. To date, we have organized one virtual convening and expect to host more gatherings in the future.Embedded in the underlying framework of this program is an approach that invests in and nurtures talent through global academic collaboration. We believe that mentorship serves as a crucial lever for the next generation of researchers involved in the evidence-informed policy movement."DAISI isn't merely about submitting and closing projects; it's a collaborative, community-built effort where collective wisdom drives progress." — Fréjus ThotoProject SpotlightsOne DAISI Scholar, Fréjus Thoto, explored the possible effects of a digital, bundled service on agricultural and farmer outcomes like access to inputs and credit in the Benin Republic. Reflecting on the impact of the DAISI mentorship, Thoto said, "As someone who is new to impact evaluations, the mentorship aspect of the DAISI Regional Scholars Program has been beneficial. My mentor provided insightful guidance on impact evaluations and shared valuable knowledge for evaluating digital applications in the agricultural sector. DAISI isn't merely about submitting and closing projects; it's a collaborative, community-built effort where collective wisdom drives progress."Another Regional Scholars project, Digital Climate Change Adaptation Services for Sustainable Agricultural Productivity in Nigeria, led by Toyin Olowogbon and Segun Fakayode, explored a digital advisory service bundled with weather-based index insurance for small-scale maize producers in Nigeria. Sharing their DAISI Regional Scholar experience, the team said,"The mentorship program is the most valuable aspect of the DAISI Regional Scholars program; it helped us develop and refine our research concepts and methods. This mentorship approach is the fastest way for scholars to advance their research and career development goals."Impact Evaluation ResourcesCEGA, J-PAL, and our partners offer various opportunities to drive inclusion in the social science and global development ecosystems. Click the links below to learn more about application and eligibility requirements.CEGA's Fellowship Program invests directly in cultivating the talents of junior African researchers by providing residential (at UC Berkeley and Northwestern) and non-residential impact evaluation fellowships.J-PAL Scholars Programs provide funding, mentorship, and training opportunities to researchers from low- and middle-income countries. We currently have Scholars Programs in sub-Saharan Africa, South Asia, and the Middle East & North Africa.Every month, the CEGA Global Networks Opportunities Newsletter publishes funding, events, and career opportunities for scholars from Africa and beyond. Sign up for CEGA's Opportunities digest here (choose "Global Networks).To check on the status of future DAISI Requests for Proposals, please visit our website.ReferencesUNESCO. UNESCO Science Report: Towards 2030. Paris: UNESCO, 2015.Julia Kaufman, Justin Hurley, Janeen Madan Keller, and Erin Collinson. Taking Stock of Organizations with Impact Evaluation Capacity Headquartered in sub-Saharan Africa: A New Database and Landscaping Analysis. Center for Global Development Policy Paper 305, 2023.Celebrating the DAISI Regional Scholars Program — Two-Years Strong was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

Addressing Evidence Gaps to Facilitate Strategic Decision Making for Displaced PopulationsHumanitarian crises are fueling an increase in the number of forcibly displaced people around the world, yet a lack of quality evidence surrounding interventions and programs makes it difficult to decide how to allocate limited funding. In this blog, Forced Displacement Research Associate Mansi Kalra outlines CEGA's new effort to close the evidence gap, produce new insights to guide policymakers, and improve the lives of displaced people.Azraq Refugee Camp in Jordan, built to house Syrian refugees | WikimediaAn Unmet Need for EvidenceToday, low- and middle-income countries are disproportionately affected by displacement, with about three-fourths of the world's poorest people living in fragile contexts. Humanitarian crises in Afghanistan, Palestine, Syria, and Ukraine have generated an unprecedented increase in the number of forcibly displaced people around the world. Most decision-makers lack the resources to support these nearly 2 billion people. Faced with such urgent needs, it's critical that leaders can access the right evidence to make effective decisions.Unfortunately, very little causal evidence exists on what works best in conflict and crisis settings. To fill these critical knowledge gaps and help improve lives, CEGA will expand the evidence base by scaling ongoing research and expanding a coordinated portfolio of research through its existing Forced Displacement Initiative. At last December's Global Refugee Forum, CEGA outlined its four-step approach:To build research capacity, CEGA will launch a fellowship program that supports scholars affected by displacement or who study displacement in their home countries.CEGA will collect new rounds of data in two longitudinal panel studies that report on economic inclusion, social protection, and other policy-relevant outcomes.CEGA will also invest in partnerships that facilitate collaboration between research, policy, and practitioner communities concerned with forced displacement.CEGA will develop and promote a set of ethical guidelines for researchers who work with displaced populations.Centering Displaced VoicesCEGA is committed to supporting people directly impacted by forced displacement by providing opportunities for leadership, learning, and talent development to researchers with lived experiences of displacement. Our proposed fellowship program will equip scholars with the skills, networks, and resources they need to develop their careers and lend their voices to research and policy debates while increasing the evidence base that leaders can use to drive innovative, sustainable, and locally-led solutions to displacement.Expanding Panel Datasets to Create Lasting Public GoodsOne of CEGA's unique contributions is the creation of longitudinal panels. These studies track displaced populations over time and measure a wide array of demographic, social, and financial data. The first of these studies is the Syrian Refugee Life Study, a representative panel of Syrian refugees in Jordan that generates evidence for key policy questions. Currently, data and materials from the in-person 2020 partial survey round are available on the Harvard Dataverse, and additional rounds of data will be made publicly available in the coming months. This open source public good is a highly beneficial tool made available to the research community at large. CEGA also led the creation of the Kenya Analytical Program on Forced Displacement, a panel of 6,000 refugee households and 3,500 Kenyan. These projects are conducted in partnership with the United Nations High Commissioner for Refugees and other decision makers, thereby ensuring that policymakers are included in this work. Through the creation of these datasets, CEGA is expanding the availability of robust data and closing the evidence gap.Creating Collaborative Relationships for Informed Decision MakingAs funding for humanitarian efforts becomes increasingly constrained, it is critical that the limited funds are directed to the most cost effective and impactful interventions. To advance this effort, major donors can invest in the creation of basic evidence and expand critical experimentation. One such example is the Conrad N. Hilton Foundation Refugees Initiative which recently collaborated with CEGA to develop a rapid review of youth livelihoods interventions in displacement settings.CEGA's review of the evidence finds:A dearth of quality evidence available to guide decisions about the effectiveness of interventions among refugee and displaced populations;Limited impact evaluations of interventions for displaced populations that integrate cost analysis into their design;Multiple layers of context that make it difficult to generalize and adapt policies and programs "off the shelf" from one setting to another.These findings raise a key question: What is the strategy for an evidence-based approach to investing in interventions that work when the relevant evidence is so limited and where the complex context constrains the options? Expanding future investments in research will be critical to answering this question, and CEGA is actively working to build these necessary partnerships.Grounding Our Work in EthicsDisplacement settings are particularly prone to ethical challenges and clear guidelines are critical to building trust with respondents and ensuring that researchers do no harm. CEGA is developing a set of practices to incentivize and institutionalize an ethical approach in our displacement research, and is working with partners to ensure these practices are applied effectively. The Center's aim is to develop a framework for our own research and disseminate these findings to the larger research community.The Realities of Forced DisplacementToday, more than 100 million people are forcibly displaced. The need for a robust evidence base to inform critical responses has never been higher, and this work never more pressing. To learn more about CEGA's Forced Displacement Initiative and ongoing research, please refer to this link.UNHCR Global Trends Forced Displacement Flow: depicting refugees, asylum-seekers, and others in need of international protection displaced from 1975–2022.Evidence Generation: A Critical Tool for Understanding Displacement was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

A New Public Tool for Planning Cost-Effectiveness ResearchRigorous research on cost is needed to generate reliable and transparent estimates of cost-effectiveness — but a high quality cost study has to be planned prospectively.CEGA Program Scientist Liz Brown launches a new Costing Pre-analysis Planning template to foster collaboration between cost and impact evaluation research. This work was developed by CEGA's Cost Transparency Initiative and we welcome your feedback.A young woman sells tomatoes in a local market using her mobile phone to process the transaction. | Credit: Confidence Nzewi via Dreamstime.comJust one in five impact evaluations include any type of cost evidence (Brown & Tanner, 2019), which means that decision makers frequently lack key information to compare per beneficiary cost-per-impact across studies. CEGA's Cost Transparency Initiative has developed a new tool for researchers to help generate a rigorous estimate of the cost-effectiveness of an intervention, improve confidence in their findings, and provide actionable costing insights for policymakers. The publicly-accessible Costing Pre-analysis Planning template aligns impact estimates of an evaluation with final cost estimates, generating a rigorous and transparent estimate of cost per impact per beneficiary at the close of the study.CEGA developed and field-tested our cost pre-analysis planning tool on five cash-benchmarking impact evaluations completed in 2023. Throughout this engagement, we encountered wariness about the reliability and reproducibility of cost estimates and uncertainty about the underlying need for cost "research" among academic research economists. CEGA now routinely completes Costing Pre-analysis Planning templates in advance of conducting cost research, with costing-pre analysis plans in place on three active impact evaluations, Lishe Bora, Titukulane, and Takunda.The Costing Pre-analysis Planning template is designed using LaTeX so that it can either efficiently integrate with the pre-analysis plan (PAP) of an impact evaluation or serve as a standalone draft of a costing report that is fit for publication. The tool includes a structured outline and prompts to motivate the cost analysis, define the primary and secondary cost research questions, and identify the cost-relevant features of the intervention design and delivery.Cost pre-analysis plans document cost study designs and transparently share data, methods, and planned empirical analyses. As with PAPs for impact evaluations, costing PAPs minimize the risk of "cherry picking" specifications, such as narrowly-defining cost inclusion criteria that do not reflect the full cost of delivery or that only produce interesting cost-efficiency metrics. The tool improves collaboration around the use of sensitive and highly detailed expenditure data and the chances for better partnerships with decision makers on their cost research questions."Cost Is The Easiest Part" — A Common MisconceptionIt may seem straightforward to calculate the cost-effectiveness of an anti-poverty program: simply divide the total budget of the evaluated program by the number of households served, and compare it to the benefits per household. But there are several drawbacks to this approach:Expenditures in accountancy do not capture all relevant costs. There can be large deviations between budgets and actual expenditures — both in terms of the total cost and how resources are allocated across activities and time. Generally, actuals are the gold standard of cost data. However, the accountancy of the primary implementing partner(s) is a key source of data for this exercise. It has all the hallmarks of administrative data with some well-known advantages and limitations. We find that careful review of the chart of accounts, interviews, and supplementary data are needed to catch mis-classified expenses, to disaggregate "catch-all" categories of cost, and identify incompleteness.The development intervention and the development program may not be identical. Careful identification of treatment activities is needed to align the costing with the intervention that is the subject of the impact evaluation and the specific beneficiary population(s) of the evaluation. For example, our costing of a complex, large scale, bundled program of activities during a five-year intervention targets a subset of activities within the more 'complete' program. This well-established program has its own theory of change and targeting criteria. Our work isolates the costs of the intervention activities apart from the broader program.Different research questions require different cost data. It is not always evident which costs should be included in a cost evaluation. The criteria for inclusion or exclusion will depend largely on the research question at hand. For example, disentangling the cost of bundled intervention activities requires the identification of separate inputs for each component of the intervention. In this case, using the entire program budget as a proxy for cost would bias cost per beneficiary estimates by incorporating costs that are irrelevant to the research questions of concern to decision makers.Cost should match the estimand. To be valid, the estimate of cost per beneficiary should match the estimand that is the focus of the impact evaluation. Attention to the cost implications of sample attrition and non-compliance is needed to improve the accuracy of cost per beneficiary estimates.Implementation fidelity and costing accuracy. The real-world challenges of implementing a randomized evaluation complicates its costing. For example, a common assumption is that control groups incur no cost. However, this assumption fails when there is contamination between treatment and control groups. It is important to proactively plan the costing and track the intervention's implementation fidelity so that we can quickly identify anydata challenges that affect the accuracy of the costing.Variation in cost per beneficiary. A simple estimate of budget per beneficiary ignores a lot of important variation in cost per beneficiary. Beneficiaries who are more remote or more disadvantaged may incur a higher cost to reach. It might also prevent understanding how specific activities contribute to cost. A pre-analysis plan may help to plan out how to identify these differences.A completed costing pre-analysis plan template explains what the costing involves, how it fits with the impact evaluation research, and what data is needed to make it operational. The structure of this tool facilitates coordination not only within evaluation research teams but also with implementing partners and donors.Leaders need cost evidence to support the scale-up of effective interventions. For researchers to provide these data and insights, the field needs to develop and disseminate more open and transparent high quality tools for costing. We hope this new tool accelerates these efforts. Please join us by getting involved in the Costing Community of Practice.A New Public Tool For Planning Cost-Effectiveness Research was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

Muhammad Zia Mehmood is a PhD candidate in the Business and Public Policy Program at the Haas School of Business, UC Berkeley. In his job market paper, Zia studies the demand for and potential of business trainings provided over text messages to impact outcomes for micro-entrepreneurs in Kenya. This study was supported by CEGA's Development Economics Challenge initiative. This blog post was originally published on the Econ That Really Matters blog.A small business welder at work. Credit: Adobe StockIntroductionSmall businesses form the economic backbone of low-income countries and strengthening these enterprises is fundamental to alleviating poverty. Poor management practices is a major factor constraining firm productivity in these contexts, and over $1 billion is spent annually to address this constraint by providing business trainings to entrepreneurs. However, most of these are conventional, in-person, classroom-style trainings, which are expensive and hard to scale, and can exclude those who are unable to participate in person. Due to their low costs, scalability, and reach, phone-based trainings are gaining popularity as a potential solution, but there is limited evidence on whether remotely provided trainings are effective for micro-entrepreneurs in low-income settings.In my job market paper, I study the demand for and potential of text message-based business trainings using a field experiment in Kenya, in which access to an SMS-based training was randomized across 4,700 micro-entrepreneurs. I estimate short- and longer-run impacts using phone-based surveys conducted three months (Midline: 307 observations) and twelve months (Endline: 2,780 observations) after the intervention. I also elicit ex ante predictions for 12-month treatment effects from researchers through the Social Science Predictions Platform (SSPP), to assess whether the main findings depart from existing priors. Finally, I measure demand for the trainings through Take-It-Or-Leave-It (TIOLI) offers and the Becker-DeGroot-Marschak (BDM) willingness-to-pay elicitation method for a subset of the sample.Context and InterventionAccording to a 2016 nationwide survey of small businesses in Kenya, 90 percent of micro-entrepreneurs had never received any type of business training. This was reflected in their business practices: More than three-fourths did not advertise any of their products, over two-thirds didn't keep any business records, and less than a tenth accounted for prices set by their competitors when choosing their own prices.I partnered with a local firm specializing in digital content development and dissemination to implement an SMS-based training aimed at addressing these management gaps and others. Available in English and Swahili, the training modules covered best practices, including marketing, advertising, pricing, record-keeping, and stock management. The content was structured around stories about the decisions of hypothetical micro-entrepreneurs in different scenarios. Users accessed the trainings through self-paced engagement with an interactive chat-bot, which sent bite-sized chunks spanning about 150 text messages. The entire training could be completed in five to seven hours, and all content was retained indefinitely on users' phones. Weekly text reminders were sent to those who stopped engaging, and these reminders stopped if engagement was resumed or after two consecutive months of inactivity.The primary sample for the study was sourced from a list of micro-entrepreneurs compiled by my implementation partner and a local microfinance institution. Half of the study sample consists of female micro-entrepreneurs, and roughly 45 percent is based in rural areas. The average individual was about 35 years old, and had completed almost twelve years of education (high school level).Figure 1: Screenshots of user engagement with the chatbot. Note: This figure shows screenshots of interactions with the SMS-based chatbot as it pushes out content to users. In this context, most micro-entrepreneurs set prices just based on their buying costs, without accounting for prices of their competitors, so the content pushes them to change their pricing strategy. Credit: Muhammad Zia MehmoodResultsThree months after the intervention, I find that the SMS training increased knowledge and adoption of best practices by 0.20 and 0.33 standard deviations, respectively. I also find large positive, but statistically insignificant, effects on business performance in the overall sample, and significant positive effects for younger (below-median) micro-entrepreneurs on sales (109 percent increase), profits (38 percent increase), and business survival (11.6 percentage points increase). These positive effects for younger entrepreneurs are driven by higher engagement with the content, and larger effects on time spent on business, and loan amounts applied for and received.However, these positive results dissipate in the longer run; twelve months after the intervention, I see no effects on knowledge and adoption of best practices, as well as business sales, profits and survival. Additionally, the positive effects on business outcomes observed for younger entrepreneurs at three months disappear after twelve months. The lack of long-term impact was likely driven by micro-entrepreneurs abandoning all interactions with the content within the first few months of the intervention. The survival curve in Figure 2 shows how all cumulative aggregate engagement with the platform ended by May 2022 — five months into the intervention.Figure 2: Survival curve of interactions with chatbot. Note: This figure illustrates how interactions with the chatbot were distributed throughout the study period. The plot shows reverse cumulative engagement over time; for example, it shows that 80% of all the interactions with the chat-bot throughout the course of the study, had ended by 4/1/2022. The shaded areas represent the time-spans during which the Midline and Endline surveys were conducted. Credit: Muhammad Zia MehmoodFigure 3: Predictions vs observed treatment effects. Note: This figure shows how predicted treatment effects for the Endline compare with observed Midline and Endline effects. Error bars represent 90% confidence intervals. Credit: Muhammad Zia MehmoodFigure 3 illustrates how these results compare with predictions for 12-month treatment effects elicited ex ante through the SSPP. I find that SSPP researchers overestimated the engagement levels, both in terms of the proportion of the treatment group that would start engaging with the content (50 percent vs 30 percent) and how much training content the average user would complete after twelve months (40 percent vs 7 percent). Furthermore, predictions for the 12-month treatment effects on knowledge and adoption of best practices are somewhat similar to observed effects at three months, but significantly overestimated in light of observed 12-month treatment effects. Effects on business performance offer a similar story: SSPP predictions for the 12-month treatment effects on sales and profits are similar in magnitude to effects observed at three months (albeit statistically insignificant), but they grossly overestimate the effects at twelve months.Additionally, notwithstanding the low engagement and lack of longer-run effects, I find positive demand for SMS-based trainings among micro-entrepreneurs; both methods of elicitation — the TIOLI offers and the BDM exercise — reveal that individuals are willing to pay a small amount for an additional SMS-based training, suggesting that they value access to the content.Policy ImplicationsThese results indicate that SMS-based trainings are unlikely to improve outcomes for micro-entrepreneurs in the long run, despite their growing popularity in low-income and less accessible settings. These findings also highlight the lack of engagement with trainings as a major challenge that limits the potential of remotely-provided information-based support.Further, the forecasting exercise reveals that social science researchers overestimate the potential of SMS-based trainings to improve outcomes for micro-entrepreneurs, and the findings from this study are thus contrary to priors. Updating these priors is important because policymakers and practitioners often rely on social science experts to make decisions about how to invest in remote, information-based support programs.Lastly, the results on willingness to pay suggest that engagement with the content might not reflect the actual demand for SMS-based trainings, pointing towards possible behavioral drivers constraining engagement. To capitalize on the full potential of digital content delivery in low-income settings, further research is needed to shed light on how to encourage engagement with remotely provided content.Short Messages Fall Short for Micro-Entrepreneurs: Experimental Evidence from Kenya was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

Carson Christiano, CEGA Executive Director, outlines CEGA's top priorities in 2024, designed to expand the way we define and achieve "impact" in the evidence-informed policy ecosystem.Credit: Ronald Cuyan via UnsplashFor fifteen years, CEGA has supplied decision-makers in low- and middle-income countries (LMICs) with rich evidence, insights, and tools they can use to identify cost-effective solutions for reducing poverty and improving lives. As we've matured, we have become wiser to the ways in which our efforts may — and may not — be driving meaningful and lasting policy change. In recent years, we have turned the microscope on ourselves, looking closely at our successes and failures and contemplating ways to boost the return on investment in CEGA's work. We are especially proud of the investments we've made to make evidence more cost-effective, more transparent and reproducible, and more inclusive.This year, we're re-committing to driving systematic change in the global development community and expanding our imagination of what our impact can be. Our priorities include:1. Launching new research initiatives in priority areas.As the world continues to contend with overlapping crises, there are several thematic areas where more evidence is urgently needed. CEGA faculty and staff are working to build research agendas and partnerships to support for new work in the areas of Gender & Agency, Conflict & Security, and Forced Displacement. Meanwhile, we are working to ensure that all of our thematic areas address the persistent threat of climate change, which in turn has exacerbated both conflict and forced displacement, especially for marginalized groups — and low-income women and children in particular.2. Building open science infrastructure.A key pillar of CEGA's work is to make evidence better. As such, we're constantly striving to improve the quality and credibility of the data, tools, and analytical methods used to make consequential policy decisions and drive large-scale social impact. This year, we're expanding investments in our Cost Transparency Initiative (CTI), which is developing tools and standards for rigorous intervention costing. We're promoting adoption of the Social Science Prediction Platform (SSPP), which enables timely predictions of social science research, and the Social Science Reproduction Platform (SSRP), which crowdsources and catalogs attempts to assess and improve the computational reproducibility of social science research. We're also excited to contribute a highly collaborative, novel effort to build a comprehensive, open-access, and searchable library of results from social science RCTs in low- and middle-income countries. By consistently documenting study design, intervention features and context, effect sizes, and measures of certainty and credibility, the envisioned Impact Data and Evidence Aggregation Library (IDEAL) will dramatically accelerate the translation of evidence into action by allowing users to quickly and painlessly access relevant information for a given set of studies. Once established, IDEAL will facilitate everything from qualitative systematic review to quantitative meta-analyses, making evidence-based decision-making easier for all across the development research ecosystem.3. Promoting the use of novel data science tools and approaches.CEGA's embrace of multidisciplinary and mixed methods has allowed us to generate new types of insights for decision makers, thus diversifying and expanding the number of tools in their toolkits. For example, the use of novel data sources (like satellite imagery and cell phone metadata) and data science approaches (including applications of AI and machine learning) by CEGA researchers allows them to paint a more complete or accurate picture of what's happening in a given geography or sector than they would relying on traditional data alone. This is particularly important in conflict or climate change-affected countries where household survey or government census data may be woefully out of date or insufficient for high-stakes decision-making. This year, CEGA is scoping new activities and partnerships that elevate the use of AI-based tools by researchers and policymakers for targeting, deploying, and rigorously testing a wide variety of global development solutions.4. Putting ethics and inclusion front and center.We're mindful of our position as a Global North institution working on challenges facing people in the Global South, and are deeply committed to driving both ethics and inclusion in this ecosystem. Our Global Networks program, which has brought over 70 scholars from East and West Africa to the US for semester-long fellowships in impact evaluation since 2012, continues to thrive and expand. This year, we announced a new collaboration with the Partnership for Economic Policy (PEP) to create more training and mentorship opportunities for promising African scholars. At the same time, we're helping to set new standards for ethics in development research, for example by studying the practices and preferences surrounding the returning of research results to communities. And our Collaboration for Inclusive Development Research (CIDR) is examining — using both qualitative and quantitative methods — how the inclusion of African scholars can influence evidence-informed policymaking, and the obstacles that remain in doing so.As CEGA matures and the world around us continues to shift, we're striving to update how we define and articulate "impact" — not only in terms of our investments in research and evidence, but also our investments in methods, training, and research dissemination. In other words, we're beginning to measure success not just by the specific programs or policies that have been informed by CEGA evidence — although that is important of course — but also by the ways in which the entire global development ecosystem has shifted towards the effective use of evidence. This year, we're prioritizing efforts to better track and learn from our past experience and proactively integrating these lessons into our work.At CEGA, we're motivated by the opportunities that lie ahead and stretching our imaginations about the kind of impact we can have. We can't do it alone — we're proud of our collaborations with public, private, and non-profit partners, especially with those in the Global South, and look forward to seeing what we can do together this year (and beyond!) to make global development decision-making more cost-effective, innovative, and inclusive.Innovating for Impact was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

Incorporating Remote Sensing Data Into Randomized EvaluationsThis post was written by Kelsey Jack, CEGA affiliated professor and Associate Professor of Environmental and Development Economics at UC Santa Barbara, and Jack Ellington, Policy Associate at J-PAL Global. This post was originally posted on the J-PAL website.Credit: Pranavan Shoots via Shutterstock.comA growing number of economists are incorporating remotely sensed (RS) data — satellite data in particular — into their studies. For randomized evaluations, remote data collection offers alluring possibilities: lower data collection costs, a longer time series of data both before and after an intervention, geographic spillovers, and more. However, the initial allure may obscure some practical challenges.In a new set of guidelines, CEGA affiliated professor Kelsey Jack (University of California, Santa Barbara) and Kendra Walker (University of California, Santa Barbara) — with contributions from CEGA affiliated faculty Tamma Carleton and Robert Heilmayr, along with Jenny Aker, Seema Jayachandran, Namrata Kala, Rohini Pande, Ben Moscona, Sebastien Costedoat, Carlos Muñoz Brenes, and Johanne Pelletier — outline some of the opportunities and challenges associated with using remote sensing data in randomized evaluations. The guidelines provide resources and recommendations to help social scientists, practitioners, and their collaborators effectively leverage RS data in their evaluations.Remote sensing refers to collecting data from a distance. Examples of sensors used to collect RS data include on-site monitors, manned or unmanned aircraft systems, and satellites. Predictive models and machine learning methods are often used to interpret raw RS data, such as classifying a satellite image of a piece of land as forested or not. This interpretation stage is typically necessary to allow the researcher to use the data for their analysis.The guidelines are organized around three main reasons that researchers conducting randomized evaluations might wish to include RS data: (1) increase statistical power, (2) measure different or more objective outcomes and (3) extend analysis to more time periods or locations. For example, RS data may be especially useful when evaluating environmental or agricultural interventions, such as forest cover, crop yields, land use, wildfire smoke and pollution concentrations, since environmental outcomes are often difficult to measure through surveys alone.While the use of RS data in impact or program evaluation is not new, using RS data in randomized evaluations presents both new challenges and opportunities. Most notably, because randomized evaluations typically involve a substantial amount of researcher discretion over design decisions and primary data collection, researchers can tailor their sample, collect primary data, and interpret RS data to make the most of this new and exciting data source.The guidelines are structured around the three primary motivations for incorporating RS data into a randomized evaluation and use case studies from Jack's own experiences using RS data to evaluate the impact of rainwater harvesting techniques in Niger and of payments for ecosystem services on crop burning in India. In this blog, we will briefly highlight selected challenges associated with each of these motivations and examples of how to avoid common pitfalls.Using remote sensing data to increase statistical powerThe larger a study's sample size, the more likely that the researcher will be able to detect the effect of an intervention if it exists. However, researchers often face logistical or financial constraints that make it difficult to collect primary data for a large number of participants. RS data can help predict outcomes for study participants not included in a survey or other primary data collection, making it possible to include more observations in the study, thereby increasing statistical power.While RS data may be used to increase sample size, it also introduces a new source of measurement error since outcomes are typically predicted. If the error is sufficiently large, statistical power may not improve much relative to analysis using the smaller set of primary data. For example, if field observations of crop types are used to train a prediction model that achieves only 60 percent accuracy with the RS data, researchers may be better off just running regressions with the field observations. Non-classical measurement error, particularly if it is correlated with treatment, may introduce new forms of bias. For example, if the crop type observations can only be obtained in the treatment group, and treatment affects crop choices, then the model may be systematically more accurate in predicting outcomes in the treatment group than in the control.Increasing the number or quality of outcomes measuredThere are cases where RS data may be more objective, accurate, or inexpensive than primary data collected through surveys. However, some primary data will usually be necessary to calibrate or train the RS model. Raw RS data can be difficult to make sense of without primary data to compare it to. Therefore, designing appropriate primary data collection is important, and may differ from what would be collected in the RCT if no RS data were involved.One key consideration is linking the relevant unit of intervention in the RCT to the RS data. For example, if agricultural outcomes are of interest, then the researcher needs to know the spatial location of both treatment and control fields. Measurement error will be considerably higher if only geographic points — rather than field perimeters — are collected. Notably, spatial locations must be collected for both treatment and control fields.RS data may be particularly useful when outcomes are difficult to measure through standard survey-based techniques. For example, illegal activities, such as (in some settings) deforestation or crop residue burning, may be susceptible to substantial reporting error in surveys, but could be more accurately measured with RS data — providing adequate primary data can be collected, of course. Where primary data for training a model cannot be obtained from ground-based methods such as surveys or spot checks (for example, in a conflict zone), a small sample of very high-resolution satellite imagery may provide an alternative approach to constructing a dataset for training or calibrating the RS model.Extending measurement to locations or time periods outside of the main study sampleResearchers may also want to use RS data to examine the impacts of an intervention outside of the original time period or sample of the evaluation, but as with any statistical analysis, care must be taken when conducting out-of-sample analysis.First, out-of-sample extrapolation requires an assumption that the relationship between primary (training) data and RS data is the same between the original sample and the extended sample. For example, a land use model trained on data from a set of villages in the original evaluation may or may not perform well for a larger sample of villages that may have been affected by spillovers.Similarly, the same land use model trained at a single point in time may be poorly suited to predicting the evolution of land use into the future as a result of the treatment. There will almost always be some differences in background characteristics — weather patterns, economic conditions, landscape, etc. — between the main study sample or time period and the extended sample which may cause a model from the original sample to interpret new RS data inaccurately (referred to as "model drift").Collecting new primary data for the extended sample and recalibrating the RS model can help with both accuracy and interpretation. If researchers can identify potential opportunities to use RS data to measure spillovers or long-run effects early on, they can design the initial evaluation to make measuring these outcomes easier down the line.TakeawaysIncorporating remote sensing data into randomized evaluations has tremendous potential to measure outcomes that would otherwise be difficult or expensive to study with traditional surveys and may be especially useful for evaluating environmental interventions that require physical measurements like land cover. However, RS data are not a panacea and researchers need to take these considerations into account from the time they start designing their evaluations to determine whether and how to use RS data.For more thorough guidance, additional practical considerations, and examples; check out the guidelines here.Incorporating remote sensing data into randomized evaluations was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

Research shows that women are less likely to point out and penalize mistakes in science, and publish fewer comments and failed replications in scientific journals. What does this mean for the social sciences?BITSS Program Manager Grace Han interviews David Klinowski (Visiting Assistant Professor at the University of Pittsburgh Katz Graduate School of Business) about his forthcoming publication, "Voicing Disagreement in Science: Missing Women."Credit: Suad Kamardeen via UnsplashWhat does it mean for the state of social science when half of all published studies fail to replicate? A recent Nature article reported that, according to some studies, 35–70 percent of published results don't support replication with new data. "Often, researchers cannot even reproduce results when using the same data and code as the original paper, because key information is missing," the authors wrote.Despite glaring issues with the credibility of published research, replicating results remains disincentivized, and stark inequalities in their production and publication exacerbate this challenge. At the 2023 BITSS Annual Meeting, David Klinowski, visiting assistant professor at the University of Pittsburgh Katz Graduate School of Business, discussed determinants of gender gaps in education and labor markets. His forthcoming publication, Voicing Disagreement in Science: Missing Women, shows that women are less likely to point out and penalize mistakes in science and publish fewer comments and failed replications in scientific journals.We talked to David about why women are missing from academic criticisms and what this means for the social sciences. Responses have been edited for length and clarity.Grace Han: Your research indicates that women are missing from critiques and failed replications. Why do you think that is?David Klinowski: Many factors may influence a researcher's decision to criticize a published paper, some of which may have a gender component. For example, a researcher may be more or less willing to express criticism depending on her level of seniority or whether she has coauthors to pursue the criticism with. If women tend to be more junior than men, or have fewer coauthors, then these differences could contribute to the gender gap in critiques and failed replications.Opportunity costs may also play a role. Our profession values critiques and replications much less than it values novelty. Researchers may naturally choose to focus on doing original work rather than attempting to replicate or criticize others' work.Women may also suffer backlash to a larger degree than men for criticizing work. Research has found that women are often penalized for displaying male-stereotypical behaviors, such as assertiveness. If writing a critique or a failed replication is perceived as confrontational, then women may be reluctant to write these papers for fear of backlash. It is also possible that women criticize as much as men do but their criticisms are more often rejected during peer review.What are the impacts of women's comparative absence from such criticisms?DK: It is hard to say. Publishing a critique in a top journal could advance the critic's career. She could gain recognition from her peers, and citations to her work. And the process of replicating someone's work could be a great learning experience. But if negative consequences such as backlash are larger for women, then it may be entirely rational for women not to participate.We should also worry about what impact the lack of female-authored critiques and failed replications has on science itself. Is the process of self-correction in science slower as a result? Are there views and insights that never emerge as a result?What does the relative absence of women from critical publications say about the scientific literature and publishing process at large?DK: One way to interpret the results is that they illustrate how difficult and fraught it is to express post-publication criticism. It is not always easy to find out whether a replication exists, or to know what a critique may imply about the original paper. Original authors may be terrified of scrutiny, and may not be eager to share their code, data, or other details of their work.What can be done to increase women's participation in academic criticisms? Is this wise given the potential risks and professional blowback?DK: I think a promising approach is to make it easier and more rewarding for researchers to participate. To even start to scrutinize or replicate a paper, researchers must be able to access its data, code, and methods. BITSS does a great job at educating scholars on why and how to do transparent and reproducible research.One way to prevent authors from feeling singled out when their work is criticized or replicated may be to conduct replications via a "replication audit," an organized effort by multiple researchers to replicate a large number of studies. The Replication Games [convened by I4R,] is one such effort. Researchers may find it more attractive to participate in replication audits than to replicate a single study on their own.Afterwards, replicators become coauthors of a meta-paper that combines the different replications conducted, but their names are not associated with the particular paper they replicated. To increase participation of women and other researchers in academic criticisms, we need more of this type of careful design of the incentives and institutions.In March, alongside the American Economic Association's Committee on the Status of Women in the Economics Profession and Economists for Equity at UC Berkeley, CEGA will co-host the Berkeley Replication Games, which will focus on the participation of underrepresented researchers.To find out more and register, check out our event page.The Gender Gap in Academic Criticism was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

Dan Cassara (CEGA Project Manager, Financial Inclusion) and Ariana Zapanta (IPA Research Manager, Financial Inclusion) introduce a new synthesis on the relationship between the digitization of credit and development. Explore the full report or a condensed summary.The first digital credit loan was made in Kenya in 2012 [1]. By 2019, these loans had spread across the globe, with billions of dollars of credit disbursed to consumers in low- and middle-income countries (LMICs) [2]. Though the proliferation of quick and easy loans made demand for these products clear, their impact on consumers was uncertain.While proponents hailed digital credit as a transformative development with the potential to drive financial inclusion and reduce poverty, reports of fraud, exploitation, and over-indebtedness led to concerns that many digital credit users suffered financial harm. Until now, a shortage of rigorous evidence has made it difficult to answer questions about the true impacts of the first wave of digital loans. Even as a sufficient body of evidence has formed to evaluate early digital credit products, rapid evolution in the market and the digitization of new credit products over the past decade has further complicated the challenge of fully understanding the impacts of digital credit today. Insights from early evaluations should be applied to critically assess newer forms of digital credit and focus future support on more productive and impactful products.A New SynthesisSupported by the Bill & Melinda Gates Foundation, Mobile Instant Credit: Impacts, Challenges, and Lessons for Consumer Protection is a collaborative report from the Center for Effective Global Action (CEGA) and Innovations for Poverty Action (IPA) that contributes to a shared vocabulary, reference base, and conceptual framework to advance discussion on the relationship between the digitization of credit and development. The report draws on evidence mainly from CEGA's Digital Credit Observatory (DCO) and IPA's Consumer Protection Research Initiative (CPRI). Both initiatives have supported extensive research on digital credit and approaches to better protect consumers and support their financial health.This new report should temper both the fears of critics and optimism of proponents. The current evidence on digital credit suggests the average consumer has not benefited economically, but did see modest improvements in subjective well-being. Causal studies did not find that recipients were harmed by digital loans, but descriptive evidence showed a rise in problems such as debt stress, price shrouding, overcharging, and predatory collection practices. When considered alongside the proliferation of digital credit, these challenges necessitate new consumer protections.A small-business owner can be seen serving patrons in her restaurant in Nkhoma, Malawi. | Credit: Bill & Melinda Gates Foundation/Barbara KinneyKey EvidenceThe report focuses on studies investigating the welfare impacts of digital credit, the forms of misconduct associated with digital credit, and the effectiveness of consumer protection tools. It emphasizes the narrower topic of Mobile Instant Credit (MIC) — small digital loans that are primarily marketed and used for consumption — and airtime loans, which together comprised the first wave of credit digitization. As digital credit continues to evolve with new products like Buy-Now Pay-Later, overdraft facilities, and Pay-As-You-Go schemes, this report synthesizes research on MIC to generate insights that can inform digital credit-related policy more broadly.The report's main findings include:The current evidence on MIC and airtime loans reveals modest improvements in subjective well-being for the average consumer. Most loans are small — the average loan was less than US$3 in half of the studies on the welfare effects of digital loans — which may limit how large the impacts can be. Although the majority of evidence indicates MIC has not affected financial outcomes, including resilience, consumption, and financial health, studies that examined subjective well-being found a positive impact. One research team speculated — with the caveat that more evidence is needed — that the ability to access resources in times of need can relax constraints that damage mental health, consistent with prior studies [3, 4].The absence of negative impacts contrasts with robust descriptive evidence on the diverse consumer harms associated with rapidly digitizing credit, including rising debt stress, price shrouding and overcharging, predatory collections, and fraud. The rapid digitization of finance in many regions of the world has revealed new and growing consumer protection risks. While digital credit does not appear, on average, to harm consumers' financial health, ability to save, or overall spending, descriptive evidence from digital finance and mobile money, which underlies MIC and digital credit, reveals misconduct and other risks.Upgrading consumer protection regimes is imperative given the increasing digitization of finance and credit. Fortunately, there are promising approaches to improve market monitoring and empower consumers. Despite existing challenges, a handful of causal studies and pilot interventions confirm the effectiveness and feasibility of interventions to address consumer protection concerns. New data sources and collection methods, along with advanced data analytics, suggest that a next-generation approach to market monitoring would enable regulators to move towards preventative interventions. Further, several interventions directly targeting consumer and provider behavior have empowered consumers in early evaluations, and may be effective if scaled up.Policy InsightsDigital credit is growing and evolving rapidly, with providers increasingly using digital lending to offer more than just small consumer loans and targeting a wider array of sectors, from agricultural finance to e-commerce. The digital credit model has the potential to enable growth as reduced costs, faster underwriting, and better information on prospective borrowers may lead to more suitable and sustainable products, particularly in more productive parts of the economy. Evidence on Mobile Instant Credit and airtime loans is particularly informative for emergent forms of consumer-oriented digital credit, such as Buy-Now Pay-Later and overdraft products. However, the effectiveness of many forms of digital credit, particularly those focused on productive activities, remains unexplored, and regulatory frameworks are still being developed in many countries, leaving consumers at risk. More research is needed to understand who benefits from digital credit, and why.There are also clear dangers to maintaining the status quo in unregulated or underregulated markets. Although causal studies have not found clear negative impacts of MIC and airtime loans, decision-makers should consider a broader set of evidence. Robust descriptive studies highlight diverse consumer protection challenges associated with digitizing credit, and there is evidence that the most vulnerable people are more susceptible to fraud and exploitation as they enter the formal financial system via digital credit. Though risks have grown, initial research on new market monitoring techniques and interventions to directly support consumers have shown promise, and could pave the road to more proactive regulatory approaches.Despite what remains unknown, there are reasons for optimism. Digitization has catalyzed rapid growth in financial inclusion, and addressing known policy issues now can be instructive for informing management and regulation of digital credit products more broadly, including the potential movement of digital credit into more productive parts of the economy. Mobile Instant Credit: Impacts, Challenges, and Lessons for Consumer Protection gathers existing evidence on the relationship between the digitization of credit and development, while clarifying the need for further research. Whether the next evolution of digital credit improves upon the first to provide less risky and more transformative solutions will depend on the evolution of commercial product design and public policy to monitor markets and protect consumers.References:[1] Totolo, E. (2018) Kenya's Digital Credit Revolution 5 Years On. FSD Kenya, Blog. https://www.fsdkenya.org/blogs-publications/blog/kenyas-digital-credit-revolution-5-years-on/[2] Cornelli, G., J. Frost, L. Gambacorta, P.R. Rau, R. Wardrop, T. Ziegler, (2023) Fintech and big tech credit: drivers of the growth of digital lending. Journal of Finance and Banking, 148, 106742. https://doi.org/10.1016/j.jbankfin.2022.106742[3] Haushofer, J., E. Fehr, (2014) On the Psychology of Poverty. Science 344, no. 6186: 862–867. https://www.science.org/doi/10.1126/science.1232491[4] Banerjee, A., M. Faye, A. Krueger, P. Niehaus, T. Suri, (2020) Effects of a Universal Basic Income during the pandemic. UC San Diego Technical Report.The Promise and Harms of Digital Credit: What Does the Evidence Say? was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Digital Agricultural Innovations and Services Initiative (DAISI) is jointly managed by CEGA and J-PAL, and is generously supported by the Bill & Melinda Gates Foundation. This post was authored by Sarah Coughlin (J-PAL) and was originally posted on the J-PAL website.West African Farmer Manually Watering His Cabbage Field During An Arid Rainy Season | Credit: Riccardo Niels MayerWorsening droughts, more frequent and severe floods, erratic temperatures, and unpredictable seasons: these are just some of the disastrous consequences of climate change that wreak havoc on agriculture. Small-scale farmers in low- and middle-income countries (LMICs) typically have limited social safety nets to protect them against the growing threat of weather-based shocks. An urgent policy priority, both to increase farmers' own productivity and profits and to contribute to stabilizing food security throughout these regions, is to provide accessible forms of risk protection. J-PAL's updated Policy Insight reviews 23 randomized evaluations on common interventions intended to boost farmers' risk protection and help them make optimal farming decisions.Agricultural decisions in the face of weather-based riskFarmers are faced with many decisions at the beginning of the agricultural season, including which crops or seed varieties to plant, how much to spend on inputs like fertilizer, and how much land to plant. Higher upfront investments in productive agricultural technologies like fertilizer, seeds, and pesticide can increase yields and profits at the end of the season. However, choosing to make these investments may seem unwise if there's a chance they will not pay off due to an unexpected extreme weather episode. This uncertainty leads to a cycle of underinvestment and low productivity that can trap small-scale farmers in poverty.One common tool to protect against risk is agricultural insurance, which typically pays farmers back for their recorded crop losses due to an adverse weather event. Weather index insurance is an innovation developed to reach small-scale farmers in LMICs by basing payouts on an easily observable variable, like rainfall at a local weather station, which is compared against historical norms and used as a sliding-scale index of farmers' anticipated crop loss.Insurance can protect farmers against shocks — but most are not willing to buy itOverall, studies showed that farmers purchase index insurance at such low rates that insurance is often not profitable for lenders, inhibiting the growth of self-sustaining markets. However, overcoming learning, trust, or credit constraints can help boost take-up in some cases.When insured farmers receive a payout or see their peers benefit from insurance, they are more willing to buy it the following season — but these effects dissipate if farmers pay for insurance and do not later receive a payout. Social learning and financial literacy trainings can boost insurance take-up, but their effects similarly do not always persist over time. Tailoring insurance to farmers' unique seasonal incomes is also key, since farmers have limited cash on hand at the beginning of the season when they have to pay for insurance. While pairing insurance and credit products has not been shown to increase take-up in most cases, offering farmers flexibility to repay insurance after harvest time is a promising alternative, particularly for the least wealthy farmers.When insurance is both appropriately tailored to farmers' local contexts and is heavily subsidized, farmers do purchase it. In these cases, insurance is effective in helping farmers cope after a shock, and this protection also incentivizes farmers to shift their production decisions to higher risk and higher reward choices, like weather-sensitive cash crops. With this added benefit of boosting investments in both good and bad weather years, when insurance is designed in accordance with farmers' local contexts, it may be an effective form of social protection targeted at small-scale farmers.Insurance is just one tool to protect against risk: The role of risk-mitigating technologiesNew agricultural technologies like drought- and flood-tolerant seeds and climate-smart practices such as rainwater harvesting can help make production itself more resilient to risk, rather than allowing damages to occur and compensating for them with insurance payouts. Pairing insurance and climate-smart technologies may also provide additional benefits than offering them on their own. While improved seed varieties help lower crop losses during moderate weather events, farmers are more willing to make risky production decisions with the addition of insurance.Looking forward: Crowding in research on innovative technologiesWhile risk-mitigating technologies are promising complements to insurance to provide comprehensive protection that both lowers farmers' risk exposure and helps them bounce back after a shock, more research is needed on how to bundle these strategies effectively. The Digital Agricultural Innovations and Services Initiative (DAISI), jointly managed by J-PAL and the Center for Effective Global Action and generously supported by the Bill and Melinda Gates Foundation, was launched in 2021 to shed light on digitally-enabled, bundled agricultural services for small-scale farmers. A core component of DAISI's research agenda is to disentangle which elements of a bundled service are key to its success — and how to design bundled services to effectively alleviate small-scale farmers' productivity constraints.Small-scale farmers are at the forefront of global food security, and their livelihoods are also uniquely vulnerable to the effects of climate change as the intensity of weather shocks accelerates globally. As world leaders emerge from COP28, policymakers should continue to prioritize identifying and scaling evidence-based solutions tailored to farmers' local contexts to boost global food security and protect farmers' livelihoods.Using Insurance and Climate-Smart Technologies to Help Farmers Cope with Weather-based Risk was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

Privacy Enhancing Technologies (PETs) are at the forefront of efforts to protect privacy and enable responsible data use. But what are they and how are they used? Dan Cassara, Project Manager for the Digital Credit Observatory (DCO), provides a case study of how PETs can be used in public health responses in the second part of our series on 'How PETs Make Data Work for All.' Read part one.Young African delivery man with package and smartphone wearing surgical face mask for coronavirus outbreak | Credit: Sabrina via Adobe StockPrivacy Enhancing Technologies (PETs) aim to preserve the insights of a dataset and the privacy of the individuals in the data. The first case study in the How PETs Make Data Work for All series showed how Multiparty Computation could enable banks to work together to enhance regulatory efforts without compromising sensitive customer data. In other cases, researchers may already have a central dataset, so the challenge isn't in pooling the data but in sharing the insights that can be gleaned from it. PETs can help with that, too.Mobility data from cell phones are collected in centralized datasets and can play an important role in improving humanitarian relief efforts and public health responses. This data, which tracks location over time, can help model the spread of epidemics, track displaced populations after natural disasters, detect government violence against citizens, or provide insight into population movements.While collecting and analyzing mobility data can lead to large social benefits when done ethically, it also carries risks for individuals, their privacy, and their autonomy. For example, the New York Times reported how cell phone location data could be used to identify political protesters from past demonstrations and learn about their current daily activities. The places people visit — trips to a place of worship, a bar, or a doctor's office — reveal a person's life and carry different social meanings. In the wrong hands, mobility data could be used to target religious minorities, track an ex-partner, or blackmail political officials.Privacy enhancing technologies like differential privacy can play a pivotal part in the ethical use of data for social good. Consider a hypothetical pandemic scenario in which decision makers want to gauge compliance with stay-at-home orders: can differential privacy address location and mobility privacy concerns in this context? And conversely, how does the context of a pandemic response inform the design of systems that rely on location and mobility data?Call Detail Records (CDR) — metadata like the date, time, duration, and location of phone calls — can be used to compute statistics on mobility. Of particular interest for our example is population flow between distinct regions in a given time period, an important factor in determining the spread of disease. Analyzing sensitive data demands that we first think through privacy risks to the data used and the statistics generated. The WHO provides 17 guidelines for ethical public health surveillance; the five below are related to the collection and sharing of public health data:[3]: Surveillance data should be collected only for a legitimate public health purpose.[4]: Countries have an obligation to ensure that the data collected are of sufficient quality, including being timely, reliable and valid, to achieve public health goals.[13]: Results of surveillance must be effectively communicated to relevant target audiences.[15]: During a public health emergency, it is imperative that all parties involved in surveillance share data in a timely fashion.[17]: Personally identifiable surveillance data should not be shared with agencies that are likely to use them to take action against individuals or for uses unrelated to public health.Best practices such as sharing data only when necessary and de-identifying data as early as possible can help protect individuals. These steps, combined with the public health need in our hypothetical, satisfy WHO guidelines 3, 13, and 15, but guideline 17 may not be met. A growing body of research shows that true anonymization with these approaches is nearly impossible, meaning these mobility statistics could leak the presence or absence of an individual. Enter differential privacy.Privatizing Statistics: Differential PrivacyWe can privatize statistics by adding random deviations or "noise" to them, ensuring no one can learn too much about any individual in a dataset. How much is known about an individual depends on the circumstances and the statistics we wish to publish. While adding noise to a dataset increases the level of privacy, it also decreases the accuracy of the data — this is known as the accuracy-privacy tradeoff.Imagine we'd like to know more about compliance with stay-at-home orders related to COVID-19. We could produce a histogram showing the count of people who traveled certain distances, adding noise to make the counts differentially private. This provides insights on general compliance with travel restrictions, as well as the risks associated with long distance travelers who could contract or distribute the disease in many places. To better understand the importance of choosing the right statistics for the job, we can also contrast our histogram with other statistics. Though average distance traveled may seem like a more intuitive choice here, because the maximum distance an individual can travel is quite large, the average may be sensitive to their presence or absence and a lot of noise is needed to mask their data's contribution. Adding more noise reduces the accuracy of our estimates and highlights the accuracy-privacy tradeoff. Although it may seem more invasive to produce an entire histogram and share more data points, because an individual cannot change counts nearly as much as they could an average, much less noise is needed to privatize the histogram. Thus, the histogram enables us to gain more privacy for each bit of noise we add, and to produce meaningful insights for policymakers without earning "too much" about any individual in the data, even if the statistics were leaked.Big data and new technologies have created many opportunities to improve the distribution of aid and the delivery of public health interventions, but they come with many risks, including threats to individual privacy. Modern day mobility data can play a useful role in policymaking and evaluation, but presents ethical dilemmas. Thankfully, PETs and a well designed analysis allow for the production of useful insights to public health officials in an ethical manner that limits infringements on privacy.Mobility Statistics for Public Health Response was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

Maya Ranganath (Associate Director, Global Networks and Inclusion) speaks to Ronald Mulebeke (2019 CEGA Fellow) about his reflections on a return to Berkeley four years after his fellowship. Mulebeke's fellowship was supported by the East Africa Social Science Translation Collaborative (EASST).Ronald Mulebeke outside the CEGA office in Berkeley, CA | Maya RanganathCEGA invests deeply in our fellows each semester, but the relationship doesn't end there. Fellows go on to join the Network of Impact Evaluation Researchers in Africa, a network of African scholars seeking to advance impact evaluations of development programs, conduct workshops at their home institutions, and receive funding to pursue the brilliant ideas they develop while at Berkeley. While we reunite yearly at the annual Africa Evidence Summit, staff also keep in touch throughout the year with fellows to follow their professional and personal developments, from new papers to new babies.It's a happy occasion when fellows come back to Berkeley for a visit. Recently, 2019 CEGA Fellow Ronald Mulebeke returned to campus to work with his faculty mentor, CEGA affiliated professor Stefano Bertozzi, on his CEGA-funded study "Implementing Supportive Supervision and Behavior Change Communication at Private Health Facilities in Uganda." I sat down with Ronald to talk about what originally attracted him to the fellowship, what brought him back to Berkeley, and his advice for future fellows.Maya: Ronald, it's great to see you back in Berkeley! It's special when we have a fellow return after a few years. To start from the beginning, can you tell me about what originally brought you to the fellowship, and what your experience was like?Ronald: When I joined the fellowship in 2019, my interest was to improve my skills in impact evaluation and cost-effectiveness analysis. The fellowship was an interesting time of learning. I loved the chance to network with people, organizations, and groups involved in diverse research activities. The seminars were so eye-opening; I came from a health background and quickly realized that impact evaluation concepts applied to many different sectors. I loved the opportunity for interdisciplinary collaboration.It's great to hear that you expanded your thinking during the fellowship. Did your current projects start at that time? Could you tell me a little more about them?My work is about helping private health facilities provide better malaria treatment by improving their ability to "Test, Treat, and Track" malaria (a policy established by the World Health Organization). I submitted a concept note on this for the fellowship application and developed that into a full proposal with my mentors while at Berkeley. In 2020, I received a research grant from CEGA to conduct a feasibility study of our randomization process. We wanted to see if supportive supervision would work effectively in private health facilities owned by different individuals with varying capacities. We also wanted to test the best way to randomize. After the pilot study, we are planning to apply for a second grant. We implemented the project — with some disruptions due to the COVID-19 pandemic — — but, overall, it has progressed well.So what brings you back to Berkeley now?My current visit is primarily to finalize the initial results of the study and strategize with Stef [Stefano Bertozzi] for the next phase of the project. We needed the time in person to work together. This time around, the focus has been on collaborative work and building a strong research team. I have connected with other faculty members at Berkeley for their expert input on the project's future applications. This visit has been more study-focused and didn't involve coursework.Can you talk about the major changes in your life and career after the fellowship? How did the fellowship impact your work?After the fellowship, I maintained contact with CEGA staff and faculty and continued working with them on my main project. I also collaborated with CEGA researchers on a project to increase mask use in Uganda during the COVID-19 pandemic, and on the new initiative "Impact Evaluation for Evidence in Decision Making (IEED)."IEED is a joint program between CEGA and Makerere University in Uganda to improve capacity for impact evaluation within Makerere and connect with policymakers to start demand-driven studies. I have been able to ingrain impact evaluation in three institutions, working with the Office of the Prime Minister and two other government agencies. Because of the expertise I gained through the fellowship, I was also appointed a board member of the Mildmay Research Center, Uganda.Additionally, I have been involved in mentoring and training activities as a member of the Network of Impact Evaluation Researchers in Africa (NIERA).Did you learn anything specific during the fellowship that you are passing on to junior researchers?The fellowship emphasized the importance of networking, learning from mentors, and working in teams, which I've been able to apply in my career. I learned the importance of networking with policymakers and other stakeholders right from the beginning and involving them in a project's design, implementation, and dissemination stages. I have maintained a good working relationship with my mentor Stef, and I believe learning from experienced researchers and establishing collaborations is crucial for growth. When I work with junior researchers, I really emphasize how important this is.I have also mentored junior researchers and helped one successfully apply to the fellowship: I'm excited that Nneka Osadolor from the University of Benin in Nigeria is a resident fellow this fall.What advice would you give someone just starting their fellowship?I would give four key pieces of advice. First, be clear about your goals. Next, be open to new ideas (and to receiving constructive criticism!). Third, make sure you network proactively: connect with mentors, other fellows, and professionals, seek advice, have conversations, and build meaningful relationships. Last, keep in mind that it's important to balance technical skills and conceptual understanding.Looking ahead, what are your career goals?First and foremost, I aim to complete my PhD in Medical Sciences at the University of Antwerp this year. Beyond that, I'm passionate about doing more implementation science research. I want to delve deeper into how policies can be effectively implemented and scaled up through the application of research. I hope to contribute to discussions and make a meaningful impact in this area.That sounds like an important area of focus. Do you have any specific plans after completing your PhD?It's still a little early to say. I envision pursuing opportunities in academia or research institutions where I can continue my work in implementation research and impact evaluations. Ultimately, I want to contribute to improving the effectiveness of policies and their scalability. I'm grateful that the fellowship gave me a great foundation for this.A fellow returns to CEGA: Reflections, learning, and growth four years later was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

On September 28, CEGA hosted its annual Evidence to Action (E2A) conference, this year titled "Realigning Tech for Social Impact." It brought together researchers, policymakers, practitioners, funders, community members, and students to take stock of the benefits and harms technological interventions have had over the last two decades, with an emphasis on the experience of communities in low- and middle-income countries. Sean Luna McAdams, program manager for the Data Science for Development portfolio, shares key takeaways from the event.Carson Christiano (left), Executive Director of CEGA, and Mohamed Abdel-Kader (right), Chief Innovation Officer and Executive Director for the Innovation, Technology, and Research (ITR) Hub at USAID | Matt KrupoffNew technology benefits from a simple and powerful narrative: it can help solve complex problems, often by doing more with less. The printing press accelerated our ability to share information over long distances. Social media has made it nearly costless to communicate with anyone, anywhere in the world. But new technologies can have unintended effects. The cotton gin turbocharged an economic model that relied on enslaved African labor and ill-gotten land, with terrible consequences. More than one account at E2A emphasized this nuance of the effects of technological interventions.Over the course of the day, participants surfaced a set of complementary best practices when designing, testing, and integrating tech into existing policy, governance, and social welfare systems. The first speaks to the limits of expertise: a technically excellent product is necessary but not sufficient for social impact. Additionally, the ultimate goal of scaling a solution is often underspecified. Scaled by whom, for which communities, and at what scale (national, regional, global)? Making these answers explicit can improve estimates of the trade-offs to scaling and ensure the technological solution is the right fit for its intended problem.Beyond Technical ExcellenceAt E2A, the best examples of technology improving lives and achieving large-scale adoption in low- and middle-income countries relied on interdisciplinary teams, incorporated community input, and planned for maintenance and capacity building to ensure sustainability. Interventions developed with these elements can cultivate community trust, clarify their value proposition for potential users, and dynamically adapt and improve their product. Dan Fletcher, Professor of Bioengineering and Faculty Director at the Blum Center at UC Berkeley, reflected that the initial prototype of Cellscope failed during field testing despite its technical excellence because it was developed only by engineers. Subsequent iterations involved experts from the social sciences, improving the product and leading to greater success.Cellscope is a mobile phone-based microscopy tool that helps health workers identify a variety of pathogens to more quickly diagnose and treat those affected and test the potability of water. | Cellscope teamHowever challenging, collaboration across disciplines is essential for successful interventions. Mohamed Abdel-Kader, Chief Innovation Officer and Executive Director for the Innovation, Technology, and Research (ITR) Hub at USAID, reminded us of the value of proactively including people "whose lived experience can provide different perspectives." For USAID, this means ensuring that 25 percent of its funding is dispersed to local organizations. Daanish Masood, Advisor on AI Alignment at the UN's Department of Political and Peacebuilding Affairs, echoed this call by highlighting the importance of "epistemological pluralism" to more effectively engage community members who often perceive and value their world using different frameworks. By bringing together multidisciplinary teams, innovators can develop technologies that will better meet the social reality in which they will be deployed.Not only does diversity make technology more effective, it also helps to minimize potential harm. For example, Josh Blumenstock shared how his team gathered input from rural communities in Togo to inform the development of a social transfer targeting software his team co-created with the Ministry of Digital Economy and Transformation. Zoe Kahn, a PhD student on Blumenstock's team in Togo, conducted ethnographic research to better understand how members of rural communities would interact, understand, and appraise the digital social transfer system they were building.Finally, technically excellent solutions must build the social consensus and requisite buy-in across different contexts to tailor and maintain these tools iteratively. This requires planning for the long-term, building in plans for deprecation, and creating institutions — whether private companies, volunteer-based organizations, or capacity-building initiatives — to maintain and update these innovations to make them sustainable over the long-run.What Does Success Look Like?We should not assume that we have the same vision of a successful technological innovation in the social impact space. We often tout "scaling" as the final life stage of tech; for many, it serves as the ultimate goal and proxy of success. Yet, Brigitte Hoyer Gosselink, Director of Product Impact at Google.org, distinguished two paths to integrating technological interventions: a tailored, individualized solution in close partnership with a decision maker and centralized, replicable systems that can be deployed across time and space reliably.Our Technology Adoption for the Public Good panel delved into the public sector's challenges and opportunities in helping to scale proven solutions. | Matt KrupoffShould a technological intervention in global development be global in scope? We need a better sense of the trade-offs in scaling up from local to global to answer this question empirically. Some systems may be better suited to function at national or regional scales. Collaborators should plan to measure performance reliably across scale to tailor solutions accordingly.As Jane Munga, Fellow at the Carnegie Endowment for International Peace, reminded us: even if we develop good tech solutions for citizens who have a mobile phone and access to the internet now, there is a sizable portion of the world's most underserved who would be excluded due to lack of access. We need to expand the size of the pie, improve the quality of its ingredients, and increase the diversity of the culinary team that bakes it. Digital technology has a role to play in achieving more equitable social and economic development world-wide. How transformative that development will be depends on us.Technology Is Not a Panacea, but It Can Help Solve Specific Problems was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.

Key Takeaways from the Africa Evidence Summit CIDR PanelAt the 2023 Africa Evidence Summit, panelists Jeanine Condo (Chief Executive Officer, The Centre for Impact, Innovation and Capacity building for Health Information Systems and Nutrition), Rose Oronje (Director of Public Policy and Knowledge Translation, and Head of Kenya Office, AFIDEP), Aurelia Munene (Founder of Eider Africa), and Constantine Manda (Assistant Professor of Political Science, UC Irvine) joined moderator Daniel Posner (Professor of International Development, UCLA) to explore the varied incentives for African scholars to publish. Amy Shipow (Project Manager, CEGA Global Networks) and Maya Ranganath (Associate Director, CEGA Global Networks and Inclusion) synthesize the discussion to shed light on the publication gap and generate insights on how barriers can be alleviated to allow African scholars to participate fully in scholarly publication.Jeanine Condo speaks on the panel at the 2023 Africa Evidence Summit | Luft Ventures"The research infrastructure on the [African] continent remains very low. Few publications are a reflection of the limited investments governments are putting into research […] There are biases in journals. As a PhD student studying in the US, I was told that I should have my Western professor as a co-author to get published." — Rose Oronje, Director of Public Policy and Knowledge Translation, and Head of Kenya Office, AFIDEP.The Collaboration for Inclusive Development Research (CIDR), co-led by CEGA and the Network for Impact Evaluation Researchers in Africa (NIERA), aims to shift norms in global development research towards a more inclusive ecosystem. At the 2023 Africa Evidence Summit in Nairobi, we organized a panel to discuss how differing incentives and resources contribute to publishing disparities between researchers from high income countries (HIC) and those from low- and -middle-income countries (LMICs). The panel also probed the role of journals, universities, and researchers in HICs in exacerbating (or mitigating) this problem.We present the major themes of the discussion below, which shed light on the barriers and opportunities that African scholars face along the education to evidence-use pipeline.Co-authorship with scholars from HICs can serve as a helpful career stepping stone; however, it is only a small step in changing the research ecosystem to produce more African scholarship.Dr. Condo shared that working with and publishing together with CEGA affiliated professor Paul Gertler was an important lever for her career. Yet, African scholars often face internalized pressure (due to external biases) towards co-authoring with scholars from HICs. When Dr. Oronje asked the nearly 500 Summit attendees, "How many African scholars here are first authors when you submit a publication?," hardly anyone raised their hand, leading her to share that not even she submits academic publications as a first author.Panelists shared the structural obstacles African researchers face to conduct their own rigorous social science research.Since most African universities are "teaching" institutions and prioritize this over research, scholars are left with limited time to pursue their own research agendas. Moreover, faculty are rarely trained to mentor or share resources with their students — challenges that are further exacerbated by the significant time constraints they face. Access to scholars with experience publishing at top-tier universities is also privileged for HIC students, who can more easily build subject-matter expertise simply by attending office hours and speaking with other students and faculty. Without this similar exposure to the most relevant literature and resources, African students instead spend more self-directed time finding, reading, and engaging with a variety of potentially less relevant work."Last year, I was challenging the Vice Chancellor of the University of Rwanda — I am breastfeeding but I am expected to publish three papers. How can I have more space and time to be an equal competitor?" — Jeanine CondoFemale scholars face additional barriers to publishing, even at the highest levels of academia.Dr. Condo recently published a paper on gender inequities in publishing, which outlined that in papers from sub-Saharan Africa (SSA), men comprised 61% of first authors, 65% of last authors, and 66% of single authors. A recent survey of over 200 alumni of SSA STEM PhD programs confirmed that women obtain less university and external funding for graduate studies than their male counterparts. Female panelists shared that they had to push back against senior leadership with regard to career expectations in the face of reproductive and domestic responsibilities; they are expected to achieve the same results as their male counterparts but are constrained by additional, invisibilized labor.Given these constraints to African-led authorship, what are the potential solutions to incentivize and support publishing?"Money is the problem and the solution," Dr. Manda shared. Existing funds should be leveraged to support and nurture African researchers so that journals will be eager to publish their work. Dr. Condo recently wrote a grant to research the effects of providing monetary incentives for researchers to publish, instead of accepting small consultancies, where their contributions were less likely to be recognized by name. Preliminary results suggest that this is an effective incentive for SSA researchers. Similarly, panelists cited the dedicated research and pilot funding that CEGA provides to African scholars helps them pursue their own research interests.Investing in the quality of African journals is essential, and can be accomplished through different mechanisms.Panelists urged researchers to publish important work in African journals, which can signal the quality of the journal, can make findings more accessible, and uphold the ethical responsibility to share results with the communities of study. Dr. Oronje also emphasized the need for African scholars to sit on the editorial boards of such journals.Concurrently, HIC journals are making strides to improve geographic equity; for example, PLOS recently announced a policy that authors conducting research outside of their country of origin will be asked to complete a questionnaire that details the ethical, cultural, and scientific considerations taken to uphold inclusivity in their research, including if local authors are included among the authorship list. Making journals open access was another solution proposed by the panel. Initiatives like the Northwestern Research Feedback Project provides LMIC scholars, who have a desk-rejected paper from the Journal of Development Economics (JDE), the opportunity to be matched with a HIC scholar for feedback. While the author is not reconsidered for JDE, it is hoped that the paper can be improved for other submissions.Both Africa-based and HIC-based non-governmental organizations can complement African universities by offering dedicated mentorship to African scholars.Through her work as the founder of Eider Africa, panelist Aurelia Munene is striving to decrease the sense of isolation researchers experience, especially among recent graduates looking to publish. Her organization offers mentorship on essential research and writing skills as well as supports enhancing curricula at African universities. Additionally, panelists mentioned the importance of identifying resources (such as Afrobarometer, the Harvard Dataverse, or datasets of published papers) and training students to use them.While the panel began with discussion about differing incentives to publish, the conversation evolved to mirror CIDR's fundamental goals: how can we best support African researchers so that journals will actively seek out African scholars to publish their research and policymakers will hasten to apply their results?If you are working in this space, we invite you to let us know what you think! We will soon launch an online survey for students, faculty, research professionals, journal editors, and funders to share their thoughts on the current state of inclusion in the evidence ecosystem and ways to improve it. Please watch for the official announcement of this online survey later this month. The results of the survey and other CIDR research will be released in 2024.It Takes a Village to Raise a Researcher was originally published in CEGA on Medium, where people are continuing the conversation by highlighting and responding to this story.