Limiting Disaster and Impact

Studies indicate that Pokhara valley has fragile geology. Besides, the situation has been jeopardized due to fast growing urbanization leading to the pressure on the fragile geology. Quarrying of sands and stones along the rivers and beds, and over extraction of ground waters also add pressure to fragility. Thus, such disasters can be prevented if the concerned government authorities in active participation of the community people develop the urban and rural land use plans, implement and periodically review them. As a result, infrastructure building measures are strictly followed. There will be limited quarrying and people will maintain greenery that help bind soil erosion. Community awareness is another important dimension of limiting the disaster vulnerability. Communities also may know early symptoms of disaster which they can disseminate among other members. They can strengthen their early warning system; share their local expertise with government and other stakeholders to mitigate the impact of disaster. As an individual, I would be interested to strengthen the local early warning system through studying the early warnings and disseminating to the communities.


http://epaper.thehimalayantimes.com/PUBLICATIONS/THT/THT/2013/12/15/ArticleHtmls/PEOPLESPEAK-Inevitable-natural-disasters-15122013008007.shtml?Mode=1

My few learning from American Evaluation Association (AEA) conference 2013

Here, I’m sharing some points from the American Evaluation Association (AEA) Conference that was held from 16 to 20 October 2013, including the professional development workshop on the last day. The event was held at Washington DC, USA.

First of all, I’d like to thank the Mercy Corps and AEA for accepting my paper for presentation and providing financial support to attend the event. Without those and other logistic supports, the participation was not possible.

Remaining, I’ll try to briefly share my learning on the event and from some of the presentations that I attended. The AEA conference 2013 was the huge event that I’d ever seen or attended. More than 37 hundred participants presented papers that were categorized under close to nine hundred sessions. The link takes to the detailed list of sessions and the abstracts of the papers therein: http://www.americanevaluation.org/search13/allschedule.asp. The email of the presenters is also provided to facilitate the communication in case a paper is of interest to the reader. Besides, another link compiles the list of participants that also lead to the presentation of the participant: http://www.americanevaluation.org/search13/choosepresenter.asp.

I presented a paper in the session entitled ‘Around the World: Cost Experiences and Insights’. The paper is entitled as Estimating Social Return on Investment of Social Protection Programs:
Piloting an Approach in Irrigation Canal Improvement Project in Far-western Hills of Nepal. The paper concludes that irrigation scheme is a strong investment to social benefit of target communities. However, the high benefit cost ratio (BCR) poses whether irrigation scheme alone is attributed to increased production. The cost of other interventions (such as training, demonstration, seed distribution) needs to be considered for their attribution and contribution to the increased production. Participants raised their interests to know on what basis discount rate of 10% and time horizon of five years were assumed to estimate BCR. It was clarified that those were assumed based on the standard used by Asian Development Bank (1009) and other studies undertaken in disaster risk reduction in Nepal (White and Rorick 2010 and Willenbockel 2011). There were some suggestion to undertake the sensitivity analysis  by altering the discount rate and time of benefit to see at what levels the cost of intervention outweighs the net present value. The abstract of my paper that I submitted at the time of call is available at the link: http://www.americanevaluation.org/search13/session.asp?sessionid=2037&presenterid=0.

Jon Kurtz, Mercy Corps Director for Research and Learning, Lara Hilton, an Evaluation Expert (PhD scholar at Claremont Graduate University) and I presented jointly a paper entitled Connecting program and social science theory in a Nepalese microfinance program evaluation. The paper is based on the baseline evaluation of PROMISE project. The purpose of the evaluation is to examine who accesses and benefits most from VSLAs, thus answering the question of whether they constitute an effective vehicle for social protection of very vulnerable groups. Besides, it will contribute to answering the broader question of: Are the lack of formalization and capacity of existing savings and loan groups in rural Nepal major constraints to their abilities to contribute to improving their members’ livelihoods and abilities to cope with risks and shocks? During presentation, it was emphasized that Mercy Corps relies increasingly on program theory-based evaluation to increase knowledge about effectiveness of its intervention strategies. The theoretical approach to move beyond the current practice of performance monitoring evaluation was discussed. Participants were curious to know whether there was any difference between eastern and western districts. Abstract of the paper submitted to the organization is posted with a different title: What They Don't Tell You: Connecting Program and Social Science Theory, in a Nepalese Microfinance Program Evaluation at:  http://www.americanevaluation.org/search13/session.asp?sessionid=2145&presenterid=1795.

Jon Kurtz, Mercy Corps Director for Research and Learning presented a paper entitled Mercy Corps' Contingency Approach to Measuring Resilience. The paper highlighted the methods applied to generate evidence on what contributes to strengthening resilience in the arid lands of Africa. The main methods are: analysis of post-shock cross-sectional data to predict resilience; quasi-experimental impact evaluation designs to determine attributable program effects on resilience outcomes; and ex-post program evaluations to understand how target populations recover and manage subsequent shocks.

Scott Chaplowe, International Federation of Red Cross and Red Crescent Societies (IFRC) presented a paper on Counting People Reached – Reality versus Fiction. Harry Carr from Catholic Relief Services presented the paper on Tracking Beneficiary and Service Delivery - Lessons From Catholic Relief Services. Michael Wallace, Mercy Corps Senior Monitoring, Evaluation and Learning Advisor had facilitated the discussion. In the pursuit of demonstrating accountability, measuring both output and outcome level results have been the pressing need. However, measuring direct versus indirect people-reached, as well as avoiding double-counting are contested. The papers discuss the challenges in the development and roll-out of the Monitoring System. IFRC has the system of counting only once the direct recipients of services within regardless of how may services they receive. They only count indirect recipients when a credible approximation can be made. The paper suggests to count Households to determine the number of people reached, based on the credible evidence that household has received service and based on reliable statistic of the average household size. The purpose of collecting people reached data is an important organizational decision.

Jane Buckley, Cornell University and Thomas Archibald Cornell Office for Research on Evaluation facilitated the workshop on Evaluation Thinking (ET). They define ET as a cognitive process motivated by inquisitiveness and a belief in the value of evidence. The process includes: 1) identifying assumptions, 2) posing thoughtful questions, 3) pursuing deeper understanding through reflection and perspective taking, and, 4) making informed decisions in preparation for action. In the development context, the interventions are undertaken with the assumption that they produce short-run to long-run results. For e.g. irrigation canal is constructed with the assumption that it will increase access to irrigation facility to cultivate crops. With irrigation facilities, the farmers are assumed to increase the production of the crops that they have been cultivating, or the farmers are assumed to select those varieties of the same crop that are potential to bear higher production in irrigated condition or the farmers are assumed to change their cropping pattern to introduce new crops that are highly producing and fetching good market price. With increased production, the farmers are expected to increase the value of production from their crops. Besides, the farmers are assumed to meet their own food requirements and sale surplus. These sorts of assumptions will help come up with evaluation questions: does irrigation intervention increase land under irrigation? Have farmers increased their production with the crops they have been producing for years? Have farmers changed the variety of the same crop? Have farmers introduced new crop? Have farmers increased the monetary value of the crops?. The question will help establish claims: irrigation intervention increases area under irrigation, area under irrigation increases choice of crops that are suitable for irrigated conditions. There, come the parts of what approach, methodology and tools will be applied to measures the variables to establish these claims: discussion, survey, or what else. Thus, evaluative thinking is based on identifying the assumption and believing on the value of evidence to establish the claims.

Overall, AEA conference 2013 was a giant forum. My learnings are few drops of water from the ocean.

Estimating Social Return on Investment of Social Protection Programs: Piloting an Approach in Irrigation Canal Improvement Project in the Far-western Hills of Nepal

Development organizations are asking where, when, and how much to invest in order to maximize their impact. Social return on investment is an attempt that values social benefit relative to the resources invested. This paper introduces a calculator that computes three measures namely discounted present value (DPV), net present value (NPV), and benefit cost ratio (BCR). Piloted in a food security project, the calculator measures the value of production through irrigation schemes. Two assumptions were made to estimate the measures: first, a discount rate of 10% per year and second, five years of benefit duration for irrigation canal improvement.

Eight clusters located in five village development committees in two districts of Baitadi and Doti were selected. 40 households, five from each cluster selected using a random sampling technique, were surveyed in June 2012. Irrigation schemes with the project cost of US$ 13.9 thousand out of a total of US$ 16.9 thousand increased production from 32 hectares of land in the entire clusters under pilot. To estimate the monetary value of the benefits in the clusters, the findings from 40 sampled household respondents were used. The survey data revealed an overall increase of 24% in the total production of paddy, wheat, maize, potato, lentil and soybean after intervention. With the increasing market price of commodities, the value of production increased by 45% per year. Given 32 hectares of land irrigated, the DPV and NPV were estimated at US$ 0.45 million and US$ 0.43 million respectively. Given the project cost, the benefit cost ratio was 32, suggesting that the benefit from the irrigation improvement was 32 times greater than the investment over five years. Thus, it was concluded that the irrigation canal improvement was positive investment in the targeted communities. 

High BCR poses whether irrigation scheme alone is attributed to the increased production of crops. There were several other interventions undertaken in the communities in order to improve the agricultural productivity, which were not accounted. Thus, the future studies need to consider the project logic and also the costs of other interventions that are intended to improve crop, for estimating the more realistic returns. Besides, it will be worth considering more measures highlighting different aspects of an investment, such as an Internal Rate of Return (IRR), to the calculators in order to illustrate a more nuanced picture of the investment.

The Evaluation Conclave 2013: What I Heard and Learned

Around three hundred evaluators and development professionals from 20 countries attended the four-day event that Community of Evaluators (CoE), South Asia/ Nepal began in Kathmandu on 26 February 2013 with the support from International Development Research Centre (IDRC), Rockefeller Foundation, among others. This is the second event organized by CoE.
Evaluator experts and development professionals shed light on the issues and challenges concerning evaluation of development interventions. Katherine Hay, a consultant with IDRC who had been instrumental for the institutional development of CoE emphasized that evaluation should be able to "measure what the [target communities] treasure". Giving background to need of addressing gender based violence, she emphasized that GDP along will not suffice to compare the human development in different countries. Marco Segone shared that EvalPartners, a forum of evaluators is growing with the interest and partnership of development organizations and universities. Vice Chairperson of National Planning Commission of Nepal shed light that the event would add a milestone to improving the evaluation policies and practices.


There were thematic panels and workshop that provided fora for learning and sharing the experiences. AEA 2010 awardee Evaluation Veteran, Jim Rugh shed light on real word evaluation based on his book. He shed light that intent of the organization that commission evaluation and level of rigor sought matter in guiding the scope of evaluation. Sometime, the evaluation ToR limits to evaluating the real world. He emphasized that donors are most often confined to commissioning project completion evaluation. Rather, they need to be interest to commission post-project evaluation as well. Because, in many cases the control groups are found doing better than intervened after the termination of the project. He viewed that in absence of baseline data, secondary sources, project records, recall, key informant or participatory methods could be imperative to generate information indicative of the baseline status.


The Evaluation Conclave 2013 ended on 1 March 2013 amidst the evaluators and development professionals, some of whom were the surgeon of Paris Declaration and others young graduates looking future in the field of evaluation in international development.
The event remained exciting with the video conference with Michael Q. Paton who had inspiring presentations covered: 1) globalization of professional network of evaluators; 2) new framework and criteria; 3) importance of joint evaluation; 4) multiple and mixed methods: triangulation and synthesis; 5) visual data; 6) evaluation capacity development; 8) seeing through a complexity lens and 9) process use.


"Ask Them! and They Can Do It!!" were two inspirations of Robert Chambers, supported by Mallika Samaranayake, to the lovers of participatory approach.
Two books and three discussion papers were launched in the event. "Who Counts?: The Power of Participatory Statistics", written by Jeremy Holland with an afterward by Robert Chambers; and "Evaluating Communication for Development: A Framework for Social Change" by June Lennie and Jo Tacchi drew the attention of the participants. Likewise, Discussion Papers 1) Quasi-Experimental Designs for Impact Evaluation, 2) Scaling-up of Social Innovations: The 4D Model and 3) Monitoring and Evaluation of Non-Communicable Diseases: Some Issues published by Sambodhi Research and Communication, New Delhi are expected to be the good learning materials in evaluation.
 
Conclusively, the event shed light that the evaluators need to market their expertise and learning for better evaluation policies, bye-in by international development decision makers and contributing to better living of the marginalized people. Institutional and capacity building are important dimensions in that direction. Getting evaluators and development professionals' membership to Voluntary Organizations of Professionals for Evaluation (VOPEs) could be an entry point for cross-learning and sharing. International Organization for Cooperation in Evaluation (http://ioce.net) is the global federation of VOPEs. The ethics of evaluators and international development agencies commissioning evaluation are always important for "real world" evaluation. There are tremendous number of tools and techniques available in the market. However, "Cutting one's coat according to the cloth" has and will always be there for appropriately selecting the tools and techniques in the given complexity, interrelatedness and holistic context.

The organizers had put almost all their efforts to make it remarkable. Although noting is cent percent perfect, availability of all presentation materials either in paper form or through uploading in the web could instantly provide reading materials to the participants. Besides, some declarations at the length of the event could have been better for effective evaluation policies and practices.

It is posted at:
http://evaluationconclave2013.org/reflections/