《简洁大方ppt模板.ppt》由会员分享，可在线阅读，更多相关《简洁大方ppt模板.ppt（33页珍藏版）》请在三一文库上搜索。
1、,2011年11月,Monitoring and Evaluating ODA,This presentation will cover Why monitor? Why evaluate? Monitoring in the World Bank, other multi-lateral development banks, bilateral donors Evaluation in the World Bank, other multi-laterals, bilateral donors,What is monitoring?,Definition of monitoring: a c
2、ontinuous process that uses systematic collection of data about progress in an on-going development project, for use by managers and other stake-holders,Why monitor?,To improve implementation of on-going projects; must provide practical information To assess whether development objectives likely to
3、be met To identify systematic issues and risks across projects, by country, by sector, and overall To provide input for ex post evaluation,What is Evaluation?,Definition of (ex post) evaluation: assessment of a completed project, program, or policy, its design, implementation, and results.,Why evalu
4、ate and who cares?,Why? Two reasons: To learn from experience To provide accountability Who uses (or should use) evaluations: Boards of Directors (MDBs) and Ministers and Directors General for oversight; ODA managers for policies and strategies; ODA staff for program and project designs, ways of doi
5、ng business; Client countries, external partners, and the public for policies, aligning programs, research,M Approves 280 loans and credits/year; active loan portfolio : 1,400 in 120 countries Bank also provides grants and other non-lending services (diagnostic work) Bank has total administrative bu
6、dget of $2 billion Most other MDBs commit $1 6 billion/year Bilateral ODA : $200 million (Greece, Luxembourg, New Zealand, Portugal) - $25 billion/year (USA) Bilateral agencies have administrative budgets of $11 million (Luxembourg) to $700 million (Japan),Monitoring in the World Bank,Every on-going
7、 lending operation is monitored by visits from Bank staff about twice a year; Implementation status and results (ISR) report issued twice each year: Progress on implementation Progress in meeting development objectives Problem projects identified Portfolio at risk reports separate methodology update
8、d monthly Other quality control activities include quality at entry reviews; quality of supervision,Evaluation: the topics,Self evaluation Independent evaluation: why it matters Tools: projects; sectors; countries; etc. Coverage: how many evaluations Resources/budget Main challenge for evaluation Di
9、sclosure Other Evaluation Issues,Self-Evaluation in the Bank,Self evaluation in the Bank: Every project has an implementation completion report, done by staff of responsible unit (about 280/year) Every new country assistance strategy now has completion report covering previous country assistance str
10、ategy, done by staff of responsible unit (about 20/year),Self-evaluation in other donors,Multi-lateral institutions: almost all MDBs have requirement for self-evaluation, although in some MDBs (Asian DB, EBRD, EIB), the evaluation department is involved Bilateral donors: some require for projects ov
11、er given value, others dont; SDC (Swiss): external evaluators do self-evaluation; GTZ (German) has innovative eVAL (electronic) system.,Independent evaluation,What is independence? The extent to which the evaluation unit: Reports directly to a Board or other governing unit; Has staff, budget, and fu
12、nctions independent from management; not involved with management of ODA Behaves independently by issuing objective, impartial, and balanced reports,Independent evaluation,In the World Bank, independent evaluation, since 1973, has looked like this:,Independent evaluation,In bilateral donors, indepen
13、dent evaluation has evolved to look like this:,Independent evaluation,Why independence matters: Allows for objective, unbiased evaluations Evaluation findings are not tailored to managements wishes Independence lends credibility to findings Independence difficult to achieve and maintain,Evaluation T
14、ools: World Bank,Project evaluations Implementation Completion Report (ICR) reviews Project performance assessment reviews (PPARs) Country evaluations CAS Completion Report (CASCR) Reviews Country assistance evaluations (CAEs) Sector, thematic, global evaluations Process evaluations Corporate evalua
15、tions (ARDE, AROE),Evaluation Tools: interactions,Synergies among different evaluation tools can provide valuable input,Evaluation Coverage: World Bank,Evaluation coverage,In most MDBs, full range of evaluations: project, country, sector, thematic evaluations In most bilaterals, few project evaluati
16、ons; some country level; most focus on themes or programs Most bilaterals produces 3-10 evaluations/year DFID produces 20-25 reports/yr; Potential weakness of country and thematic evaluations without underpinnings of project evaluations to use as input,Resources for evaluation: MDBs,Resources vary,
17、but as percent of total administrative budget, fairly similar:,Resources for evaluation: bilaterals,Bilateral donor agencies are more likely to hire external consultants for virtually all evaluations (UK, Norway, Denmark, Switzerland, Australia) Based on numbers from a few bilaterals (DFID, NORAD, S
18、DC), evaluation budget is 2% of total administrative budget; figures of 0.5 2% have been cited by others. Smaller donors (Greece, New Zealand) are still setting up their evaluation units,Main challenge for evaluation,How to make evaluations useful Most managers, staff focused on future, not on past
19、Evaluation must / should be: Timely Operationally relevant: avoid generalizations, platitudes Easily accessible: avoid long reports Constructive: good practice should be rewarded Widely disseminated and disclosed,Disclosure of evaluation reports,World Bank: public disclosure of all: PPARs CAEs Secto
20、r, thematic, global evaluations Process evaluations Corporate evaluations EBRD, ADB: selective public disclosure: evaluations involve private sector Bilaterals: majority disclose, but varies: UK, Norway full disclosure; Germany selective disclosure,Disclosure,For World Bank: reports and information
21、on evaluation tools, approaches, on website: http:/worldbank.org and scroll down to right-had side of home page, or: http:/www.worldbank.org/ieg For other donors: check their websites,Other issues in evaluation,Evaluation approach Use of performance indicators Sources of information Ratings Processe
22、s Partnerships Other challenges,Evaluation Approach,Within context of results-based management, Bank uses objectives-based approach, focus on outcomes and impact,Evaluation Approach,Bank assesses: Outcome (relevance, efficacy, efficiency) Use of performance indicators for outputs, outcomes, impact R
23、isk to Development Bank and Borrower Performance,Sources of information for evaluation,Baseline data in project appraisal documents (sometimes) Monitoring reports should have information on performance indicators Data bases for project area; sector; country at beginning of project, end of project Ot
24、her country-level indeces (IMF financial statistics, Doing Business surveys; World Bank Indicators of corruption surveys; UNDP human development indicators) Surveys Interviews, focus groups,Use of Ratings,World Bank uses ratings on project outcomes and outcomes of country assistance. No ratings on s
25、ector, thematic, other evaluations Other MDBs use ratings for project outcomes; not for country assistance Bilaterals: some use ratings, only for projects; some do not use any ratings,Evaluation Approach: ratings,Project Ratings in World Bank: Outcomes: 6 point scale (HS - HU) Risks to Development:
26、4 point scale (H - N) Bank and Borrower performance: 4 point scale (HS to HU) Pros and cons of rating Pro: provides data and accountability Con: takes focus away from lesson learning and richness of evaluation,Processes,Independence does not mean isolation MDBs and bilateral donors incorporate comme
27、nts from operational managers and staff and from stakeholders in client countries into evaluation reports: Banks initiating papers for country, thematic, global evaluations sent to management for comment and to Board of Directors for information; IMFs initiating paper for all evaluations posted on I
28、MF website and public comment invited; wide consultations throughout process.,Partnerships in evaluation,Country and sector evaluations sometimes involve partnerships with other MDBs and/or bilateral donors (joint or parallel evaluations) Opportunistic, not systematic, partnerships Pros and cons of
29、evaluation partnerships Pros: wider coverage, other viewpoints Cons: more costly and more time required; may not agree on outcomes; may affect timeliness of final report,Other challenges,Data difficult to obtain, or different sources have conflicting data Often difficult or impossible to have a control group (especially in sector and country evaluations) Beneficiary surveys can be costly, low response rate Problem of attribution (cause-effect) difficult or impossible to establish,