|1. Project Data:
ICR Review Date Posted:
|Id National Program For Community Empowerment In Rural Areas
Project Costs(US $M)
|C4385, L7505, L7666
Loan/Credit (US $M)
Cofinancing (US $M)
Board Approval Date
|Sub-national government administration (25%), Irrigation and drainage (20%), Water supply (20%), Roads and highways (20%), Primary education (15%)|
|Decentralization (23% - P)
Rural policies and institutions (22% - P)
Participation and civic engagement (22% - P)
Social safety nets (22% - P)
Rural services and infrastructure (11% - S)|
||ICR Review Coordinator:
||Robert Mark Lacey
|2. Project Objectives and Components:|
a. Objectives:The project development objectives as stated in the Project Appraisal Document (PAD, p.5) are as follows: “villagers in PNPM-KDP [the National Program for Community Empowerment Kecamatan (sub-district) Development Project] locations benefit from improved socio-economic and local governance conditions.” The project is focused on rural areas and is designated by the acronym PNPM-Rural.
The project’s activities were a sub-set of the National Program for Community Empowerment [PNPM], the overall goal of which is to reduce poverty, and to empower Indonesia’s diverse rural and urban communities to actively participate in development.
The statement of objectives in the Financing Agreement varies from that in the PAD: “The objective of the Project is to contribute to the realization of the PNPM’s overall development goals, namely, to reduce poverty and improve local-level governance in rural areas in Indonesia through the provision of investment resources to support productive proposals developed by communities, using a participatory planning process.”
This ICR Review is based on the statement of objectives in the PAD, since it is more monitorable.
On April 14, 2009, the Board approved Additional Financing in the form of an IBRD loan of US$300 million. In essence, the statement of objectives used in the Project Paper (PP) for the Additional Financing (AF) loan remained unchanged compared to those in the PAD (the objectives in the main body of the PAD referred to “PNPM-KDP [Kecamatan Development Project] locations” whereas the Project Paper (p. 2) refers to “villagers in these rural locations”) (p. 2). The Additional Financing increased the target number of participating sub-districts from 2,000 to 4,000, and the geographic scope of the project was expanded to cover all rural sub-districts in Indonesia (ICR, p. 4). Since there was no essential change in the development objectives, this Review will not undertake a split evaluation.
The combined project funded by the original loan and credit plus the Additional Financing Loan is known as PNPM Rural 1 and 2.
b. Were the project objectives/key associated outcome targets revised during implementation?
If yes, did the Board approve the revised objectives/key associated outcome targets?
Date of Board Approval: 04/14/2009
c. Components:The Project comprised four components:
1. Block grants to Kecamatan (Sub-districts) (US$1,633.6 million or about 88 percent of total Project Baseline Cost at appraisal and US$1,901.6 million at closure). The block grants, which were to be disbursed "directly from the Special Account to collective community accounts, without entering the inter-governmental transfer system” (PAD, p. 62), aimed to support; (i) investment grants (US$1,566.3 million or about 84% of total Project cost at appraisal) to finance small-scale economic and social infrastructure sub-projects developed through participatory community planning as well as the GenerasiSehatdan Cerdas [Healthy and Bright Generation] Conditional Cash Transfer (CCT) pilot program aimed at accelerating attainment of the Millennium Development Goals related to health and education; and (ii) sub-district planning grants to support development planning at the village and sub-district levels, particularly by training “a cadre of self-selected social and technical facilitators” of the planning process.
2. Facilitation and training (US$131.9 million or about 7 percent of total Project Baseline Cost at appraisal and US$116.9 million at closure) to finance salaries and operational expenses of social and technical facilitators who were to strengthen the capacity of local government institutions, in accordance with decentralization laws, by training technical staff of local government councils, including village and sub-district fora, village and sub-district executives and district parliaments.
3. Implementation Support and Technical Assistance (US$79.4 million or about 4 percent of total Project Baseline Cost at appraisal and US$71.9 million at closure) to finance technical assistance, field oversight, and local-level coordination to support project administration at the national, provincial and district levels.
4. Support for Project Management (US$14.2 million or about 0.8 percent of total Project Baseline Cost at appraisal and US$8.4 million at closure) to finance “several critical national-level activities needed to manage the national program” (PAD, p. 7). These included training and study tours for “key PMD staff” as well as electronic record keeping for the Directorate General of Village Community Empowerment (PMD), within the Ministry of Home Affairs (MOHA).
d. Comments on Project Cost, Financing, Borrower Contribution, and DatesProject Cost
At appraisal, the total Project cost was estimated at US$1,859.3 million, The Project was approved by the Board in May 2008 and became effective in July 2008. The Project scope was revised in 2009 to “support expansion of the PNPM-Rural Project from its current geographical coverage of 2,864 rural sub-districts (kecamatan) to approximately 4,371 total rural sub-districts in 2009” (the targets in the main body of the PP for AF, p. 1, are different from the (downward rounded) ones in the Results Framework of the PP for AF, p. 20). However, according to the PP for AF, the revised total Project cost remained the same at US$1,859.3 million. According to the ICR, the total actual Project cost, i.e. total of World Bank, Borrower (and IFAD) funding disbursed under the Project, amounted to US$2,098.7 million or 113 percent of the above appraisal estimate (ICR, p. 26).
Financing (see Table 1 below)
The ICR (p. 26) lists the estimated Project financing at appraisal (by funding source) and the actual/last estimate of Project financing (by funding source). The latter represents the total disbursements for both the Original and Additional Financing (by funding source). There are several errors and inconsistencies in the tables in the ICR (p. 26), such as the “Appraisal Estimate” column not adding up to the stated total amount (see Section 15 below). The combined project funded by the original loan and credit plus the Additional Financing Loan (column 4 in Table 1) is known as PNPM Rural 1 and 2.
Table 1: Original, Revised and Actual Financing by Source of Financing (in US$ million)
|column 1||column 2||column 3||column 4||column 5||column 6|
|Source of Financing||Original Financing (PNPM-Rural 1)||Additional Financing (PNPM-Rural 2)||Original Financing plus Additional Financing ||Actual disbursement/|
counterpart contribution (column 5) as % of OF+AF (column 4)
|122% if compared to reduced counterpart contribution (column 5 as % of column 4), and 94% if compared to original counterpart contribution (column 5 as % of column 2)|
In 2008, the Board approved financing in the amount of US$231.2 million (or about 12 percent of total Project cost), consisting of an IDA credit (US$190 million) and an IBRD loan (US$41.19 million). Additional IBRD Financing in the form of a US$300 million loan was sought and approved in 2009.
Other External Financing
PNPM-Rural cofinancing from IFAD in the amount of US$68.5 million equivalent was scheduled for 2009 (PAD, p. 26 and PP for AF, pp. 3 and 6). According to the ICR, "“in November 2008 The International Fund for Agricultural Development (IFAD) agreed with the [Government] to provide a loan of SDR 42.03 million plus a TA grant of SDR 0.25 million for a co-financing operation that is directly linked to PNPM-Rural 1&2, with the Bank named in the project financing agreement as “cooperating institution” to “administer” the loan and grant." Of this IFAD cofinancing, US$44.6 million had been disbursed by project closure. However, the closing date of the IFAD loan and grant is September, 2016. The PP for AF (p. 6) further mentioned that the PNPM Support Facility (a multi-donor Trust Fund administered by the World Bank), provided management and technical support, especially in the areas of monitoring and evaluation, supervision, pilot programs, financial management, policy guidance and coordination of donor support. At the time of Additional Financing approval, commitments of US$60 million had been made to this World Bank-administered Trust Fund by the Royal Embassy of the Netherlands, the Danish International Development Agency (DANIDA); the UK’s Department for International Development (DFID), and Australia (AusAID). However, this so-called PNPM Support Facility supports a variety of programs under the PNPM umbrella, and the Project Paper did not specify the support for this project.
According to the PAD, the Borrower was to finance US$1,628.1 million (or about 88 percent of total Project cost), comprised of central, local government and beneficiary contributions.The counterpart contribution (comprising central and local governments’ as well as beneficiaries’ contributions) was reduced to US$1,259.6 million at the time of Additional Financing approval (see Table 1 above). The ICR did not explain why this was the case. According to Annex 1 of the ICR (p. 26), the actual Borrower contribution, comprised of central government, local government and beneficiary contributions, amounted to US$1,535.6,or about 122 percent of the reduced estimate, and about 94 percent of the original estimate.
A six month extension of the Project closing date from June 30, 2011 to December 31, 2011 was approved, together with the Additional Financing, in April 2009, in order “to allow for full disbursement of all funds and final reporting on project impact” (PP for AF, p. 1).
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:Rating: High
According to the Progress Report (2011) for the 2009-2012 Country Partnership Strategy (CPS) for Indonesia, the PNMP was considered the “centerpiece of Bank engagement” and its long-term relationship with Indonesia, in collaboration with other development partners (p. 14). Similarly, the CPS Progress Report referred to the PNPM as the Government’s flagship poverty reduction program (p. 45) and one of the World Bank’s key instruments for assisting the Government in achieving reduced extreme poverty headcounts. (The latter was a country development goal under the CPS’ Core Engagement Area 3: Community Development and Social Protection [CPS Progress Report, p. 34]). Community Development and Social Protection was the third of five core areas of engagement of the World Bank Group in Indonesia.
b. Relevance of Design:Rating: Substantial
The causal chain outlined in the PAD (p. 16) between project activities and the first part of the development objectives, i.e. improved socio-economic conditions, was convincing. Improved access to infrastructure and revenue generation were to be supported, for example, through the construction of roads that would: (i) provide access to previously isolated villages, improve access to markets, town centers, education and health facilities, and clean water supply; and (ii) increase business opportunities and employment for villagers. In addition, the Conditional Cash Transfer pilot activities under Component 1 aimed to improve utilization of health and education facilities further.
The causal chain between Project activities and the second part of the development objectives, i.e. improved local governance, was less clear. As recognized in the PAD (p. 41) good local governance encompasses “less easily quantifiable goals of empowerment, accountability, and active participation.” The PAD implicitly defined local governance as “responsible institutional structures that enable bottom-up planning and provide for development oversight at the local level” (PAD, p. 56). However, it did not specify a concrete set of activities, outputs and intermediate outcomes that could be expected to support this objective. As pointed out in the 2012 governance review (Woodhouse, A. (2012); Governance Review of PNPM Rural; Community Level Analysis; Jakarta: PNPM Support Facility), local governance is affected by: (i) social accountability mechanisms; (ii) financial management; (iii) facilitators; (iv) complaints-handling mechanisms; and (v) management information systems & reporting. Several of these elements were addressed in the Results Framework in the PAD, for example, quality of participation as a proxy for social accountability; local governance satisfaction, and audits. Nonetheless, the PAD did not contain a concrete set of activities that could be expected to lead to the attainment of the final intended outcome.
|4. Achievement of Objectives (Efficacy) :|
1: Villagers in PNPM-Rural locations benefit from improved socio-economic
1.1. Improved Household Expenditure rates
- The project created mostly temporary employment of nearly 3.8 million workers in the labor-intensive construction of infrastructure sub-projects and provided access to basic social and infrastructure services, including roads, water supply, schools and health clinics financed through block grants under Component 1 (see Table 2 below).
- Under the sub-heading of poverty alleviation, the ICR (p. 15) references a 2009/10 Impact Evaluation, which “showed increased per capita household consumption of about 9% on average in PNPM areas and consumption increases of 19% in the poorest 20% of the sub-districts.” It is not clear whether this cited Impact Evaluation was a follow up study to the referenced 2008 Baseline Study, which would have entailed interviewing the same households as the ones included in the 2008 Baseline Study. The 2008 study utilized a propensity score matching methodology to construct the counterfactual. The ICR does not report changes in per capita consumption in treatment locations compared to the control locations.
- According to a 2011 PNPM publication (p. 2), for the period 2007-2009, which overlapped with the last phase of the PNPM’s predecessor, the Kecamatan (sub-district) Development Project, per capita consumption in PNPM locations increased by 5 percentage points compared to control locations and poor households in PNPM locations “were 2-3 percentage points more likely to move out of poverty in comparison to control areas” (PNPM Mandiri: What Works, What Doesn’t and What’s Next. Jakarta: PNPM Support Facility. Accessed on August 4, 2012).
1.2 Improved Access to economic and social infrastructure
Outputs of infrastructure investments
- The project funded construction/rehabilitation of about 48,500 infrastructure “activities,” about 8,700 schools and nearly 3,500 health clinics. In addition, about 30,000 (existing) microcredit groups were funded (ICR, p. 15). Table 2 below provides a breakdown of completed sub-projects by sector (ICR, p. 28f.)
Table 2: Types of Sub-projects completed in 2008 and 2009
|Type of Sub-project||2008||2009||Number of activities|
|Length of Roads (km)||12,900||18,105||31,005|
|Water Supply Systems (units)||1,674||2,296||3,970|
|Public Sanitation / Ablution Blocks (units)||666||906||1,572|
|Irrigation Systems (units)||2,296||3,332||5,628|
|New Markets (units)||228||320||548|
|Rehabilitation of Markets||-||52||52|
|Dock / Jetties (units)||114||-||114|
|Other Infrastructure (units)||3,131||1,172||4,303|
|Green sub-projects (units) ||282||452||734|
|Schools (new construction and rehabilitation)||3,426||5,288||8,714|
|Health Clinics (new construction and rehabilitation)||1,421||2,054||3,475|
- The utilization of rehabilitated and newly constructed health clinics was enabled by project-financed staff training, equipment, and sanitation and nutrition programs for health facilities (ICR, p. 29). The ICR does not state the number of incremental users of these health facilities as a result of Project investments.
- School furniture, teaching materials, and a total of 342 scholarship “packets” were financed (ICR, p. 29).
Outcomes of infrastructure investments
- Access to economic and social infrastructure improved significantly during the project’s lifetime, and the revised coverage target of 4,000 sub-districts was exceeded by almost 10 percent, based on the target of 2,600 in the Results Framework (ICR. p. 15). According to the 2011 PAD for PNPM 4 (p. 2), the PNPM-Rural 1 covered 2,600 Kecamatan (sub-districts) in 2008, and the PNPM-Rural 2 supported 4,258 sub-districts in 2009. The PNPM-Rural 1 and 2 combined (the project under review) thus achieved a significant increase in access compared to the 1,800 sub-districts covered by the last Project in the KDP series between 2005-2009.
- The ICR noted that the outcomes “are not exclusively attributable to PNPM 1 and 2, but also to the successor and overlapping PNPM-Rural 3 loan,” which was implemented between mid-2010 and the end of 2012. However, the overlap appeared to be minimal, since by September 2010, i.e. when disbursements of PNPM-Rural3 began,98.5 % of the PNPM 1 and 2 loan and credit funds had already been disbursed (ICR, p. 9).
- According to a 420 sub-project sample assessed during “an ongoing 2012 Technical Evaluation” nearly two-thirds of interviewed beneficiaries were reportedly highly satisfied “regarding functionality and utilization of infrastructure,” and about one-quarter of interviewed beneficiaries reported average satisfaction.
- While the Project enabled a significant number of students to access education, it is not clear how 342 “packets” benefited more than 42,000 students, and how the latter “were able to complete nine years of basic education,” given the Project’s total “life time” of four years (ICR, p. 29).
Outputs of the Generasi (Healthy and Bright Generation) Conditional Cash Transfer
- 340 schools were constructed or rehabilitated; financial support (transportation subsidies, scholarships, uniforms) was provided for more than 200,000 students; textbooks were distributed to 380,000 students; and financial support given for recruiting 2,600 contract teachers/assistants (ICR, p. 32).
- Supplementary feeding and nutrition counseling was provided to nearly 830,000 mothers and infants; iron supplements supplied to about 270,000 pregnant mothers; 120,000 children were immunized; Vitamin A was given to more than 281,000 children; post-natal care was provided to 97,000 women, and stipends for 34,000 community health volunteers were disbursed (ICR, p. 32).
Outcomes of the Generasi (Healthy and Bright Generation) Conditional Cash Transfer
- The Generasi Conditional Cash Transfer pilot program was intended to improve health and education indicators in 130 Kecamatan (sub-districts) in five provinces (ICR, p. 16). Table 3 shows that the pilot program achieved or exceeded its targets.
Table 3 Results of the Generasi Conditional Cash Transfer program
*The ICR reports “increases of 10-11 [percentage points] in immunization coverage for 12-23 month-olds, in prenatal care visits, and in deliveries assisted by trained professionals relative to the baseline values” (ICR, p. 16).
|Health||2005* baseline (%)||2010 target (%)||Actual (2010)|
|Immunization coverage for |
|38||48||48 (ICR, p. 16)*|
|Prenatal care visits||56||66||66 (ICR, p. 16)*|
|Deliveries assisted by trained professionals||40||50||50 (ICR, p. 16)*|
|Education|| || || |
|Increased primary school enrollment rates||96.5||97||91.5*** |
|Increased junior high school enrollment rates||57||72||72% (ICR, p. 16)|
** The ICR states an increase of “3.5 percentage points from the baseline value of 88%" (ICR, p. 6). This baseline value is inconsistent with the baseline value included in the PNPM-Generasi-related outcome indicators (ICR, p. 3).
- Childhood malnutrition was reduced by 2.2 percentage points, and severe stunting of infants declined by 6.6 percentage points during the 30-month study period (ICR, p. 16).
- The ICR does not comment on the linkages and potential synergies between the health and education activities carried out under the Generasi Conditional Cash Transfer and the investments in education and health facilities under the sub-project part of Component 1 of the project (see Table 2 above).
2. Villagers in PNPM-Rural locations benefit from improved local governance conditions.
The sub-objective of villagers benefiting from improved local governance conditions was supported by the following outputs:
- A total of nearly 31.3 million community members participated in sub-project planning, implementation and monitoring activities, 45 percent of whom were women, and 57 percent of whom were poor. The ICR notes that the community participation data may be overestimated due to potential double counting (ICR. p. 33).
- A total of about 11,400 consultants and facilitators, representing about 90 percent of the intended number, were employed by the project in 2009. About 9,800 facilitators were deployed at sub-district level. The ICR does not mention whether the consultants and facilitators remained in place for the remainder of the project.
- A total of 2,155 complaints of alleged misuse of funds and power were registered by community members, facilitators, the audit agency, NGOs, and the media in 2008-2009. A total of nearly 1,700 persons, more than 70 percent of whom were community members, were found culpable of wrongdoing; 76 percent of the diverted funds were recovered. The ICR does not mention complaints data for the remainder of the project.
The project did not bring about any notable changes in the way that local governments function, although according to the ICR, the establishment of the complaints mechanism and audits did advance "demand side" governance.
The ICR reports (p. 15) that while there is evidence of enhanced governance with regard to project activities, the impact on general village governance and development was low, and "there was limited impact on overall local governance in Indonesia" (ICR, P. 17).
With regard to project activities, the following results were achieved:
- Corruption. According to the ICR, the complaints handling system, supplemented by increased random audits carried out by the (internal) National Government Audit Agency was "effective in uncovering fraud, diversion of funds, violation of procedures and abuse of authority...only 0.3% of total block grants are known to be affected by fraud and corruption. Of this amount, 57% has been recovered and returned to communities, and 43% remains under litigation" (ICR, p. 13) -- although, according to the Governance Review of the PNPM-Rural, the low rates of corruption under-reported the true extent of the problem (Woodhouse, p. 4). A “tougher regime of sanctions” resulted in 80 jail sentences, the dismissal of about 300 facilitators, and the suspension of disbursements in 3 percent of districts and 4 percent sub-districts. However, It is unclear why this sanctioning regime was only implemented in 2010, given that more than 90 percent of the IDA/IBRD proceeds had already been disbursed by the end of 2009 (ICR, p. 27).
- Social accountability and transparency. According to the PNPM-Rural governance review (Woodhouse, 2012), the project had positive impacts on social accountability and transparency within the project but they did not spill over to the broader organizational culture, mainly due to the mixed quality of facilitation. The ICR suggests that a “tendency of requirement satisfaction” and “routine implementation” contributed to shortcomings in the effectiveness of facilitation as it may have led community members to simply follow PNPM procedures without embracing the underlying principles (ICR, p.38). In addition, shortcomings were observed in the quality of facilitator training and difficulty in filling facilitator position, due to a shortage of qualified facilitators (ICR, p. 8).
- Participation. The block grant program’s community-driven development approach was based on community participation in sub-project planning and monitoring. The governance review (Woodhouse, 2012) concluded that participation rates were still high but the quality of participation was variable. Women and the poor were included in the process, yet the quality of their participation in decision-making “can still be improved” (ICR, p. 18
The ICR assessed efficiency at closure based on four distinct aspects: (i) the economic rate of return (ERR) on infrastructure investments; (ii) the size of the income or consumption multiplier on PNPM block grant funds; (iii) the construction cost of PNPM sub-projects relative to estimated costs of local government procured construction of similar infrastructure items; and (iv) a technical evaluation of the quality of the infrastructure (ICR, p. 39).
Economic Rate of Return
No cost benefit analysis was carried out at appraisal, given that the PNPM used the same technical standards and designs as previous KDP operations, the economic analyses of which therefore provided a useful proxy measure of likely returns to PNPM for small-scale infrastructure projects (PAD, p. 84). The PAD referred to several economic analyses undertaken under the KDP, whose estimated weighted average ERRs varied from 51.4 percent for a sample of 113 sub-projects in 2004, to 60.1 percent for a sample of 41 KDP-1 sub-projects (PAD, p. 84).
A replication of the 2004/5 ERR study with a small sample of 48 infrastructure projects was carried out in 2012 (it is unclear how this sample was selected). The results suggest rates of return of between 35 and 75 percent for recent road/bridge, irrigation/drainage, or clean water supply sub-projects (ICR, p.16), which are comparable with the 2004/5 results, which indicated ERRs of between 39 and 68 percent for the same types of sub-projects (ICR, p. 39).
Income or Consumption Multiplier
A 2010 PNPM-Rural impact evaluation funded by the PNPM Support Facility concluded that the recorded 9 percent growth in per capita consumption rates among PNPM households versus 5.3 percent growth among matched household samples represents a significant return on project investment. According to the ICR, “using a conservative estimate of 5.3% growth from the matched household sample, the yearly impact is 3.3 times the amount of the block grants invested” (ICR, p. 17). Regarding the income multiplier from PNPM-Rural funds, the 2012 economic analysis replication suggests an average income multiplier of 1.3 for the same types of sub-projects. This is again broadly in line with the 2004/5 results that indicated an average income multiplier of 1.2” (ICR, p. 39). The ICR did not explain sufficiently how these figures were calculated.
Comparative Construction Cost
Regarding the costs of construction, the 2004/5 ERR study showed that the cost of community infrastructure sub-projects implemented under the PNPM-Rural’s predecessor project was 30 to 50 percent lower than those executed by local contractors. The 2012 ERR replication suggests cost savings for recently built PNPM-Rural sub-projects under this operation to be in the 35 to 40 percent range (ICR, p. 39).
The 2012 ERR replication found that 75 percent of the (relatively small) sub-project sample was rated as "good" or "excellent" compared to a 94 percent "good" or "excellent" quality rating obtained by the 2004/5 KDP study” (ICR, p. 39). A recent larger-scale technical evaluation of PNPM infrastructure (no reference provided in the ICR) rated about 68 percent of about 350 inspected sub-projects of "sufficient" construction quality and 27% “slightly below specifications” (ICR, p. 39).
The shortcomings in the design and performance of the MIS system, coupled with the less than satisfactory performance of the National Management Consultant compromised the quality of field oversight and reporting (ICR, p. 23). In addition, administrative inefficiencies were caused by delays in filling Bank senior staff positions in the supervision team; procurement delays, particularly in contracting consultants responsible for overseeing implementation at the sub-national level; and in recruiting and training facilitators responsible for ensuring meaningful participation in project-supported communities (ICR, p. 23 and Section 11 below). The project did not experience any significant cost overruns. There were some procurement delays (see Section 8b). The project closed six months later than originally planned.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated