|1. Project Data:
ICR Review Date Posted:
|Second Urban Poverty Project (upp2)
Project Costs(US $M)
|C3658, L4664, L7752
Loan/Credit (US $M)
Cofinancing (US $M)
Board Approval Date
|Health (27%), Roads and highways (27%), General education sector (27%), Other social services (13%), Sub-national government administration (6%)|
|Urban services and housing for the poor (23% - P)
Municipal governance and institution building (22% - P)
Health system performance (22% - P)
Participation and civic engagement (22% - P)
analysis and monitoring (11% - S)|
||ICR Review Coordinator:
|Robert Mark Lacey
||George T. K. Pitman
|2. Project Objectives and Components:|
a. Objectives:The project under review is the second in a series of urban poverty projects in Indonesia. The first such project became effective in 1999 and closed in 2004. The second operation became effective in 2002 and closed in 2010. A third project was approved in 2005, and was implemented in parallel to this project, closing in 2011.
According to the Project Appraisal Document (PAD, page 2) and the Development Credit Agreement (page 13), the project development objectives of this second operation in the series are:
" (i) to establish or support representative and accountable community organizations that are able to provide services to the urban poor and increase the voice of the poor in public decision making; (ii) to make local government more responsive to the needs of the poor through increased cooperation with community organizations; and (iii) to improve services for the urban poor (financial services, social services and infrastructure).”
In April 2007, Additional Financing in the amount of US$135.5 million was approved. The Project Paper (page 3) states that “changes to the existing project include: (a) geographic expansion to new areas; (b) adoption of a more rigorous anti-corruption action plan; (c) strengthened financial management, procurement, and disbursement arrangements; and (d) improved training program for consultants and facilitators, including modules on community-based disaster management. These revisions are consistent with the project’s objectives, which will not change, and will improve the effectiveness of project implementation.”
b. Were the project objectives/key associated outcome targets revised during implementation?
c. Components:There were four components:
A. Community development and local government capacity building (US$17.08 million at appraisal, US$29.24 million at closure). This component provided technical advisory services to carry out community
development and local government capacity building activities, including: (a) the formation (or confirmation) of an elected body of representatives known as the Board of Community Trustees (hereafter Trustees); (b) the formulation of a community development plan for each project kelurahan (ward with a community organization), using a transparent and participatory process, and (c) assistance to Trustees on how to form associations (Trustee Fora)
and to local governments to build capacity for working with Trustees and Trustee Fora. The component was to involve a guided socialization process where facilitators were trained to work directly with communities in project kelurahan and with local government officials at provincial, district and municipal levels.
B. Kelurahan grants (US$58.82 million at appraisal, US$325.93 million at closure). This component was to finance block grants to participating kelurahan to finance activities identified in their community development plans (CDPs), including: (a) specific high-priority infrastructure investments, (b) competitive proposals from community groups consistent with the priorities in their CDPs, (c) microcredit loans for community groups, and (d) social safety programs for the benefit of the poorest and most vulnerable groups or individuals. Kelurahan grants would not finance activities on the project’s negative list (such as religious buildings or government offices). Each participating kelurahan would receive a one-time block grant of between IDR 150 million and IDR 500 million (US$16,600 to US$55,400) to finance investments in one of more of the above categories, in accordance with their CDPs.
C. Poverty alleviation partnership grants (US$35.23 million at appraisal, US$88.35 million at closure). The purpose of this component was to encourage partnerships between local government and communities and to institutionalize a consultative process between the partners for future activities undertaken by local governments. It was to provide local governments with access to matching grants (the poverty alleviation partnership grants) to finance poverty alleviation sub-projects that are: (a) too big to be financed by the kelurahan grants or that require local government involvement, (b) located in more than one kelurahan, (iii) not on the negative list for kelurahan grants, and (iv) jointly prepared, proposed and implemented by Community Organization (Badan Keswayadan Masyarakat) in collaboration with local government departments. Both participating local governments and eligible sub-projects were to be selected on a competitive basis.
D. Implementation support (US$9.73 million at appraisal, US$58.49 million at closure). This component was to support project management by a Project Management Unit in the Ministry of Public Works. The Unit would hire consultants and facilitators to assist in project implementation. Technical assistance would be provided through national management consultants at the central level, and oversight consultant teams at the province level. The oversight consultants were to have offices in the participating district governments, and facilitators and community cadres at the kelurahan level. National management and oversight consultants would also assist the Borrower with monitoring. Evaluation would be done separately by evaluation consultants independent from the national management consultants.
US$5.83 million at appraisal and US$5.34 million at closure were not allocated by component.
The components were not revised, but the scope, costs, and Bank financing of the activities were increased and funded by the Additional Financing.
In April 2005, the project was restructured so that it could provide emergency assistance and reconstruction grants to the urban poor affected by the 2004 tsunami and earthquake. In October 2005, an agreement was signed to channel an additional US$1.26 million through the project from the Japan Social Development Fund for this purpose. However, because the Multi-Donor Trust Fund established for disaster relief was already deploying resources through other, dedicated disaster-relief projects, these activities were not implemented under the project, and the Japanese grant was cancelled. “The restructuring was never actually implemented or integrated into the project, and the project essentially continued as though the restructuring had not taken place” (ICR, page 5).
d. Comments on Project Cost, Financing, Borrower Contribution, and DatesProject cost.
Total project cost at closure was US$507.65 million (including an IBRD front-end fee of US$300,000), about four times the appraisal estimate of US$126.99 million. The greater part of the increase was in components B and C – the Kelurahan grants and the poverty alleviation partnership grants, although the greatest proportional increase was in component D, implementation support, which rose by over five times from US$9.73 million to US$58.49 million.
From 2006 onwards, within component B, the proportion of Kelurahan grants allocated for microcredit loans for community groups was restricted to 20%, and this was subsequently reduced to 10% from 2007 onwards. This was in response to the relatively poor performance of these loans during the first three years of implementation (ICR, page 5).
The only external financing for the project came from IBRD and IDA. IBRD funds were provided through two Loans. The first, IBRD-46640, was approved n June 11 2002. In July 2009, this was converted into IBRD 77520. At the time of the conversion, only US$2.48 million had been disbursed. However, the remaining US$27 million, which had become Loan 77520, had all been disbursed by project closure. At the same time as the approval of the original IBRD Loan, on June 11, 2002, the Board approved an IDA Credit of US$70.5 million. The actual amount disbursed from this Credit was US$81.14 million, thanks to the appreciation of the SDR against the US dollar during the life of the project. The Additional Financing of US$135.5 million, approved on May 22, 2007, was entirely from IDA resources. Again, thanks to exchange rate appreciation, the dollar value of disbursements from the Additional Financing was US$139.63 million. Nearly all the extra dollar funds resulting from exchange rate movements were absorbed, and only US$0.3 million was cancelled at project closure. There was also a Japanese Trust Fund of US$1.26 million (TF55389, approved on January 24, 2006), which was to finance a pilot housing program for vulnerable communities . According to the Operations Portal, this Trust Fund was closed in November 2007, without any request for disbursement having been received before the Grant period had expired. There is no mention of this Trust Fund in the ICR.
Contributions of the Government and communities were, respectively, US$140.09 million and US$109.88 million, many times the appraisal estimates of US$13.29 million from the Government and US$13.71 from the communities. This reflected in large part the expansion of the program to national scale.
Three extensions to the closing date were granted, for a total of 30 months. The first, six month extension (from June 30, 2008 to December 31, 2008) was authorized at the signing of the Additional Financing in April 2007, because it was anticipated that the Government would need additional time to on-lend an extra US$135.5 million to a much larger number of kelurahan than originally envisaged. A second extension of twelve months to December 31, 2009, was to allow time to disburse funds to kelurahan that had not received block grants because of delays in national government budget execution and mid-year budget revisions, as well as difficulties in mobilizing and retaining facilitators (partly because of delays in salary payments). A third, and final, extension, also of twelve months, to December 31, 2010, was granted in part to enable the Government to utilize part of the project proceeds to support urban neighborhoods in West Sumatra in their recovery from the earthquake that hit the province in September, 2009. However, the extension was also required because of recurring delays in budget execution, budget revision and in the mobilization of facilitators to assist the expanded number of beneficiary communities. The ICR does not specify the dates on which the last two extensions were granted. The project team subsequently stated that the approvals were on December 19 and 28, 2008, respectively.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:High.
- Project objectives were, and remain, relevant to government policy. In 2006, the Government launched the first nationwide poverty reduction program, comprising two pillars: (a) the National Program for Community Empowerment (Program Nasional Pemberdayaan Masyarakat), and (b) the Conditional Cash Transfer Program targeting poor communities. The key objectives of the policy were to reduce poverty from 18% in 2007 to 8% in 2009, and unemployment from 10% to 5% over the same period. These efforts would be achieved through nationwide community-driven-development and labor-intensive activities. The Additional Financing provided through this project in 2007 extended Program implementation in about 7,300 kelurahans in all 33 provinces. The project was, therefore, also highly relevant to the Government’s pursuit of its ongoing strategy of community-driven-development, as initiated by the 2000-2004 National Development Plan. The Government remains firmly committed to such endeavors, and has repeatedly stated that it will continue to implement its national community-driven-development program at least until 2014.
- The project’s development objectives are highly relevant to the World Bank Group's 2009-2012 Country Partnership Strategy (CPS) for Indonesia, current at closure. Community development and social protection is one of the five thematic areas that were expected to form the core of the Bank’s engagement. The CPS (pages 26-27) states the Bank: “will continue to support Government in the design and effective expansion of programs that promote more inclusive growth and social protection, and that also enhance the accountability of elected officials and service providers…….The CPS approach inherent in the partnership between the Government and the WBG is built around the National Community Empowerment Program (PNPM-Mandiri), which is based on the well-performing Kecamatan Development Project (KDP) and the Urban Poverty Project (UPP) models — programs in which the World Bank Group has had a long-standing engagement.”
- The project directly addressed two of three priority areas of the Bank’s 2001-03 Country Assistance Strategy (current at appraisal): (i) sustaining economic recovery and promoting broad-based growth, and (ii) delivering better public services for the poor.
- The continued relevance of project development objectives is affirmed by the request for Bank support for follow-on operations such as the National Program for Community Empowerment in Urban Areas.
b. Relevance of Design:Substantial.
Although the PAD does not include a Results Framework as this would be currently understood, the project’s development objectives were clearly stated, and there is a logical causal chain between the activities to be supported, the outputs expected and the intended outcomes. The components were well designed, realistic, and not overly complex. They were adapted from proven approaches developed in the First Urban Poverty Project. The first objective – to establish or support representative and accountable community organizations that are able to provide services to the urban poor and increase the voice of the poor in public decision making – would be directly supported by the capacity and institution-building activities in component A and by the national and local level consultants and facilitators financed by component D. Efforts to attain the second objective -- to make local government more responsive to the needs of the poor through increased cooperation with community organizations – would be underpinned by the inclusion of the poverty alleviation partnership grants which would avoid by-passing established local governments and hence hampering the development of productive and sustainable relationships with communities. The third objective – to improve services for the urban poor – would be supported by the kelurahan and partnership grants. However, one notable design shortcoming was the absence in the PAD of targets for the outcome indicators. This was corrected in the Project Paper for the Additional Financing.
|4. Achievement of Objectives (Efficacy) :|
The project development objectives were (i) to establish or support representative and accountable community organizations that are able to provide services to the urban poor and increase the voice of the poor in public decision making; (ii) to make local government more responsive to the needs of the poor through increased cooperation with community organizations; and (iii) to improve services for the urban poor (financial services, social services and infrastructure).
This project followed the First Urban Poverty Project, and was implemented in parallel with the Third Urban Poverty Project. All three operations had broadly the same aims. Attribution of results to individual projects is, however, facilitated by the fact that geographical coverage was different in each case (see ICR, page 6, PAD, page 7, and page 27 of the PAD for the Third Urban Poverty Project), although the third project did also provide some additional support for 660 Boards of Trustees already selected under the first and second operations.
(i) To establish or support representative and accountable community organizations that are able to provide services to the urban poor and increase the voice of the poor in public decision making. Substantial.
- 99% (target 100%) of all participating kelurahans completed and ratified community development plans for the provision of services to the urban poor, and 67% of all plans were implemented.
- 99% (target 90%) of beneficiary communities, local authorities and other targeted areas received technical support from national management consultants, oversight consultant teams and other facilitators.
- 87% (target 90%) of the oversight consultant teams provided accurate and timely data on the operations of kelurahans, and cooperation with local governments through the management information system.
(ii) To make local government more responsive to the needs of the poor through increased cooperation with community organizations. Substantial.
- The target of at least 5,092 (or 70%) of the Boards of Community Trustees operating in a representative, effective and participatory manner was exceeded (ICR, page 20), based on (i) their composition, participation and voting rates, (ii) women’s participation, (iii) complaint handling, and (iv) their success in identifying infrastructure priorities, creating community development plans, and carrying out the required investments. The evidence for this is based on the following outcome and intermediate outcome indicators which were monitored in the project’s management information system:
- 19% of the population (kelurahan members) participated in community discussions during project implementation (target 30%).
- 2% of the adult population voted in the final stage of Board of Trustee elections (target 3%).
- 44 community volunteers were recruited per kelurahan (target 25).
- 66% of the kelurahan population was aware of the project and of its objectives (target 25%).
- 22% of those elected to Boards of Trustees were women (target 20%).
- The percentage of kelurahan populations aware of community development plans is not yet available (it is awaiting the results of the government survey); the target was 25%.
- The following three indicators were added at the time of the approval of the Additional Financing:
- 42% of women and other vulnerable groups participated (target 30%).
- 35% of the adult population voted in Board of Trustee elections at the neighborhood level (target 30%).
- 99% of Boards of Trustees completed annual financial audits (target 44%).
- According to a Government survey made available after ICR completion, 73% of the Community Organizations (Badan Keswayadan Masyarakat – BKM) are regarded representative, effective, and operate in a participatory manner (the target was 70%).
- While the community organizations met all quantitative targets of voter turnout and women representation, thereby suggesting that they were generally representative and accountable, the ICR also notes (page 18) that the average participation level of kelurahan members as a whole (poor and non-poor) was lower than expected (19% against 30%). Moreover, some of the targets (for example, 2% of the adult population voting in the final stages of BKM elections) seem less than ambitious.
- There is no evidence concerning handling of complaints and ensuring good governance, although the ICR states on a number of occasions that this worked well throughout implementation.
- 40 poverty alleviation partnership grants (PAPG) cities were selected to participate in the project (target 30).
- 40 PAPG selection committees were formed (target 30).
- The number PAPG sub-projects completed per city was 81 compared to a target of 40.
- The Central Government has issued a policy paper for long term poverty alleviation.
- 57% of participating local governments formed Trustee Fora (target 40%).
(iii) To improve services for the urban poor (financial services, social services and infrastructure). Substantial.
- The contribution of local government cofinancing in poverty alleviation partnership grant-supported initiatives was 43% (target 25%). This cofinancing was funded from the local governments’ own budgets, and enabled considerable leverage of the Bank’s contribution. It reflects in part changes in financing arrangements. Since 2008, provincial, district and municipal level governments chosen to participate in the project are required to finance kelurahan grants from their own budgetary resources. Since 2009, they are also required to finance at least 50% of PAPG grants from their own budgets (ICR, page 5).
- In addition to funding investments, 117 local governments provided their own resources for project monitoring, partnership with kelurahans, channeling resources to support community development plan implementation etc. (the final target, was revised upwards after the availability of Additional Financing, was 78 local governments).
- Three indicators – (i) the percentage of local government staff in cities aware of the poverty alleviation partnership grant program (target 30%); (ii) the number of poor people among direct beneficiaries of grant services as a percentage of the poor in grant cities (target 1.6%); and (iii) the percentage of kelurahan populations in grant cities aware of the grant program (target 30%) – are pending the results of the Government’s survey.
Source: ICR, page 18.
- As a result of the Additional Financing, the project was extended to new areas and coverage was expanded from 2,227 to 7,273 kelurahans.
- About 58% of kelurahan grants were invested in physical infrastructure, primarily in roads, bridges, irrigation, drainage and water supply as shown in the following table:
|Type of infrastructure||Unit measure||Original project||Additional Financing||Total|
|Water supply facilities||unit||10,679||39,229||49,908|
|Markets and shops||unit||95||1,449||1,544|
|Water disposal facilities||unit||5,957||33,353||39,310|
|Community health facilities||unit||1,116||5,138||6,254|
- About 27% of the kelurahan grants were on-lent as micro-credit loans. The remaining 15% of the kelurahan grants financed social infrastructure and services. Grants for microcredit loans and social services were channeled to the poorest residents of the project’s kelurahans (ICR, page 11).
- 56% (target 35%) of all poor households in kelurahan covered by the project benefited from infrastructure grants, and 53% (target 10%) of poor families received social grant assistance.
- 61% of grant recipients (target 30%) were women.
- The project provided 19% (target 15%) of all poor households in its coverage area with access to microcredit through revolving funds.
- According to an impact evaluation survey conducted in three rounds between 2004 and 2007, the project resulted in an improvement in access to adequate sanitation, in particular for the poor. The majority of persons who received project-funded credit, although already having access to credit, were able to obtain lower interest rates thanks to the project. However, the survey was unable to identify a statistically significant impact of the project on the welfare of the population living in project areas. The survey is not included in the ICR, nor is there any information on its methodology.
- No similar survey could be conducted after 2007, since nearly all kelurahan were covered by the nationwide Community Empowerment Program – known as PNPM - and a control group could no longer be constructed. The results of a Government survey, based on an assessment of the project’s key performance indicators in 2011, were still awaited when the ICR was completed.
- The project team subsequently made the results of the Government survey available to IEG. According to the survey: (i) at least 37% of revolving fund recipients have increased their income level, compared to the 2010 target of 20%; (ii) 73% of the Community Organizations (Badan Keswayadan Masyarakat – BKM) are representative, effective, and operate in a participatory manner (the target was 70%); and (iii) 75% of the kelurahan population is aware of the project and its objectives (the target was 25%).
- According to the ICR (page 20), 92% of beneficiaries expressed satisfaction with the improved level of services provided, exceeding the target of 80%.
- The micro-credits demonstrate only moderate levels of sustainability – the percentage of revolving funds with repayment rates in excess of 90% was 49% (target 70%). In response to the relatively poor performance of microcredit loans prior to the availability of the Additional Financing (2003-2006), the allocation of kelurahan grants to the revolving funds was restricted to no more than 10% from 2007 onwards. As a result, the share of kelurahan grants allocated to microcredit loans dropped from 36.6% in the original project to 11.7% in the Additional Financing.
Given the low cost of the sub-projects financed by the project and the substantial economic benefits of infrastructure investments in locations where infrastructure is largely underdeveloped, the ICR did not deem it useful to estimate indicators such as net present values or economic or financial rates of return.
Instead, efficiency was gauged by “unit rate norms” (ICR, Annex 3, page 23). These are described as “highly favorable for community-executed infrastructure projects.” Supporting evidence was drawn, not from the project under review, but rather from three other sources. First, a 2005 study by Indonesia’s National Planning Board is cited, which found that “the average construction cost of community-driven-development projects was 40% lower than those done by contractors of local governments. The cost savings were mainly attributed to the high level of voluntary labor contributed by residents in community-driven development projects.” Second, small-scale infrastructure constructed by communities in a related Bank-financed project (the Community-Based Settlement Rehabilitation and Reconstruction Project for Nanggroe Aceh Darussalam and Nias) was found to have cost 23% less than similar projects undertaken by local governments. Tendering of materials by communities reduced their costs by 12%. Third, a November 2011 review of Indonesia’s National Community Empowerment Program (Urban) by the Rand Corporation “commented favorably on the cost-effectiveness of infrastructure projects undertaken financed by UPP2 [the Second Urban Poverty Project, under review] and similar projects.
The study was not able to assess in detail the cost-effectiveness of UPP2-financed projects because (i) budget data was deemed insufficient or questionable, (ii) counterfactuals were not available, and (iii) the study team lacked suitable measures of cost- effectiveness (in terms of impacts)” (ICR, Annex 3, page 23). However, according to the project team, “the conclusions of these PNPM-wide surveys also apply to UPP2 because the UPP2 loan supported precisely the same type of infrastructure projects (which were developed based on the same approaches) as projects financed from other funding sources for PNPM. In fact at the time of UPP2 completion, it has been considered as part of the overall PNPM urban. The studies under PNPM [are] consistent with UPP2. Qualitative study completed after ICR confirm this assumption take by the Team for ICR.”
The average size of kelurahan grant per beneficiary kelurahan was US$44,813 at closure, compared to an appraisal estimate of US$26,412. Total project cost per beneficiary kelurahan was US$69,799 at closure, compared to an appraisal estimate of US$57,022. The project team subsequently stated: “The increase in kelurahan grants does not necessarily represent an increase in cost, because the project finances priority needs of community, and additional grants finances more activities. It is not cost overrun. Extra cost increases the benefits, because the grants financed much more small-scale infrastructure––the economic benefits of which are documented in Annex 3 of the ICR––than was envisaged at the time of appraisal (especially roads).”
Implementation support costs (component D) increased by proportionally more than the project cost as a whole. According to the project team, “higher than foreseen implementation costs also do not necessarily indicate that that the project’s efficiency was modest. The number of facilitators (and their salaries) were deliberately increased in order to improve the quality of the grant-financed project and increase the number and ensures sustainability. Stated differently, the implementation cost was increased in order to generate additional benefits and significantly more output. The net effect is this measure is not automatically a reduction in cost efficiency.” Such increases were justified by persistent problems with the recruitment and retention of facilitators. According to the ICR (page 15), “many of the implementation issues could have been avoided if more funds and attention were allocated to training of facilitators.”
The project’s closing date was extended for a total of 30 months. The first extension (6 months was granted at the time of approval of the Additional Financing to make allowance for the enlarged scope of the project. The second extension (12 months) was related entirely to inefficiencies – delays in budget execution, mid-year budget revisions, and difficulties in mobilizing facilitators. The third extension (12 months) enabled the allocation of part of the project proceeds to support urban neighborhoods in West Sumatra recover from the September 2009 earthquake, but also accommodated some of the same inefficiencies that resulted in the second extension.
On balance, efficiency is rated substantial.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated