Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Third Urban Poverty Project


  
1. Project Data:   
ICR Review Date Posted:
11/20/2013   
Country:
Indonesia
PROJ ID:
P084583
Appraisal
Actual
Project Name:
Third Urban Poverty Project
Project Costs(US $M)
 186.1  208.5
L/C Number:
C4063, L4779, L7759
Loan/Credit (US $M)
 138.7  137.98
Sector Board:
Urban Development
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  05/17/2005
 
 
Closing Date
03/31/2011 03/31/2011
Sector(s):
Other social services (32%), General transportation sector (29%), General water sanitation and flood protection sector (13%), Health (13%), General education sector (13%)
Theme(s):
Urban services and housing for the poor (29% - P) Participation and civic engagement (29% - P) Other urban development (14% - S) Municipal governance and institution building (14% - S) Social safety nets (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Kavita Mathur
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:

The project under review is the third in a series of Urban Poverty Projects in Indonesia and is an expansion of the preceding second phase project to 14 additional provinces. The First Urban Poverty Project (UPP1, which became effective in 1999 and closed in 2002) was followed by the Second Urban Poverty Project (UPP2, which became effective in 2002 and closed in 2010). While the first urban poverty project addressed urban poverty in the aftermath of the Asian financial crisis, the second project was designed as a vehicle to mainstream Government’s urban poverty reduction program and address a wider number of urban centers throughout the country. The third urban poverty project (the project under review) was used to make this a national program and to provide additional institutional support to strengthen the community organizations in the project areas included in the two previous projects.

The development objective of the third phase was “to provide improved services for the urban poor and strengthen community and government institutions for responsive service delivery” as stated in the Project Appraisal Document (p. 6) and the Development Credit Agreement (p. 21).

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
1. Community Development and Local Government Capacity Building (Planned US$11.4 million Actual US$12.5 million). This component would provide technical advisory services to carry out community development and local government capacity building activities, including:
(a) the formation (or confirmation) of an elected body of representatives known as Badan Keswadayaan Masyarakat(Board of Community Trustees);

(b) the formulation of a community development plan for each project kelurahan (urban neighborhoods, the lowest level of administration in Indonesia), using a transparent and participatory process;

(c) organizing and assisting community groups (Kelompok Swadaya Masyarakat) who would submit proposals to the Trustees to utilize project resources and implement programs identified in the community development plan; and

(d) assistance to Trustees on how to form associations, to local governments to build capacity for working with the Trustees and their associations, and the preparation of city level plans for implementation of regional poverty alleviation strategy programs.

Facilitators were to be trained to work directly with communities in project kelurahan and with local government officials at the provincial and kabupaten/kota levels in a guided socialization process.

2: Kelurahan Grants (appraisal estimate US$93.8 million, actual US$96.0 million). Block grants to participating kelurahan to finance activities identified in the community development plan, including:

(a) specific high-priority infrastructure investments,

(b) competitive proposals from community groups consistent with the priorities in the community development plan,

(c) revolving fund loans for community groups, and

(d) social safety programs for the benefit of the poorest and most vulnerable groups or individuals.

Kelurahan grants would not finance activities on the project’s negative list (such as religious buildings or government offices). Each participating kelurahan would receive a one-time block grant of IDR 200-500 million (US$24,000 to US$59,000) to finance investments in one or more of the above categories, in accordance with their community development plans. Kelurahan grants would be disbursed in three tranches to ensure proper fund use and management, and to encourage better outcomes.

3: Poverty Alleviation Partnership Grant (appraisal estimate US$64.9 million, actual US$83.5 million). The purpose of this component was to encourage partnerships between local government and communities and to institutionalize a consultative process between the two partners for future activities undertaken by local governments. It would provide 30 local governments with access to matching grants (the Poverty Alleviation Partnership Grant) to finance poverty alleviation sub-projects that are: (i) too big to be financed by the kelurahan grants (in the range of IDR 30-200 million or U$3,500 to US$24,000) or that require local government involvement, (ii) located in more than one kelurahan, (iii) not on the negative list for kelurahan grants, and (iv) jointly prepared, proposed and implemented by the Trustees in collaboration with local government departments. Both local governments and eligible subprojects would be selected on a competitive basis.

4: Implementation Support (appraisal estimate US$13.0 million, actual US$14.3 million). The project would be managed by a Project Management Unit in the Ministry of Public Works. The project management unit would hire consultants and facilitators to assist in project implementation. Technical assistance would be provided through national management consultants at the central level, and oversight consultant (OC) teams at the provincial level, with oversight consultants’ offices in the participating district governments, and facilitators and community cadres at the kelurahan level.

It was intended that the project would provide additional grants to 660 kelurahan supported in the two previous Urban Poverty Projects locations where the Boards of Community Trustees were functioning well. During implementation this number was reduced to 273, of which ten were located in areas not previously covered by either of the two prior urban poverty projects. However, each of the 273 kelurahan was to receive a much higher additional grant (IDR 1 billion, to be disbursed in three tranches) than envisaged in the PAD (IDR 150 million).

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Cost: The project’s final cost of US$208.5 million (including an IBRD front end fee of US$0.34 million and an unallocated designation of US$1.92 million) was about 12%, above the appraisal estimate. The cost increase was primarily due to the fact that the Borrower spent a higher amount on the poverty alleviation partnership grant than expected. The number of local governments participating in that component was increased to 70 while the initial target was 40. Further, local governments financed more than 40% of the cost of sub-projects from their own budgets, thereby leveraging the use of Bank financing by a considerable margin.

Financing: The project was financed by both a Bank Credit and a Bank Loan. The initial IDA Credit for US$71.4 million increased in value to US$72.0 million because of the depreciation of the US dollar against Special Drawing Rights. At project completion actual disbursements were US$71.4 million and US$0.6 million was cancelled.

The IBRD Loan for US$67.3 million was initially approved in May 2005, and became effective in October 2005 after which US$66.6 million was disbursed and the balance, US$0.68 million, was cancelled.

In total, the Bank financed US$137.98 million. The total differs from that in the ICR (page 17) because it reflects the Bank’s finalization of the Credit after project closure.

Borrower contribution: The Borrower’s actual contribution was US$42.0 million, much higher than the appraisal estimate of US$23.8 milion. The communities' actual contribution was US$25.0 million, a little higher than the appraisal estimate of US$23.1 million.

Dates: The project closed on March 31, 2011 as scheduled.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Rated high.
The project objectives are relevant to the "Core Engagement 3 - Community Development and Social Protection" pillar of the Country Partnership Strategy for FY 2009-2012. The Bank aimed at improving the socio-economic and local-level governance conditions of the urban and rural poor through wider implementation of poverty reduction and community empowerment programs.

Project objectives are relevant to government strategy. In August 2006 the Government of Indonesia launched the first nationwide poverty reduction program, comprising two pillars: (a) the National Program for Community Empowerment (Program Nasional Pemberdayaan Masyarakat); and (b) the Conditional Cash Transfer Program targeting poor communities.

The key objectives of the policy were to reduce poverty from 18% in 2007 to 8% by 2009 and to cut the unemployment rate from 10% in 2006 to 5% by 2009. These efforts would be achieved through nationwide community-driven-development and labor intensive activities, which began in 2007 and will run through 2015. The additional financing provided through the Second Urban Poverty Project in 2007 covered implementation of the program in about 7,300 kelurahans in all 33 provinces. This included areas covered by this Third Urban Poverty Project. In total, the National Program for Community Empowerment planned to expand all urban poverty project operations from 7,273 to 7,869 kelurahans by 2008, and to 10,791 by 2009; it also aimed to further strengthen the kelurahans supported by this third project. The continued relevance of project objectives is affirmed by the request for Bank support regarding follow-on projects such as the National Program for Community Empowerment in Urban Areas and the Third National Program for Community Empowerment in Urban Areas.

b. Relevance of Design:
Rated substantial.
The statement of objectives was clear and there is a logical causal chain between the activities to be supported, the outputs expected and the intended outcomes. They were adapted from proven approaches developed in the First and Second Urban Poverty Projects. The objective "to provide improved services for the urban poor and strengthen community and government institutions for responsive service delivery" was supported by the second component (Kelurahan Grants) and the first component (community development and local government capacity building). It was expected that project activities would strengthen the link between the local governments and the communities which would lead to more responsive service delivery.

The main shortcoming of the project design was that while the project design supported the strengthening of the capacity of the communities to organize and plan for high priority development projects, the project did not provide these same communities with any programs to develop their capacity to effectively operate and maintain the newly constructed infrastructure. As the ICR indicates on page 12, some of the sub-projects financed through the project grants were of low technical quality especially for relatively complex items such as housing and water supply systems. The report also indicates that some of the communities relied on the facilitators financed by the project due to their limited capabilities in operations and maintenance. The project team clarified that technical training was provided to over 100,000 community members in the operation and maintenance of infrastructure.


4. Achievement of Objectives (Efficacy) :


Although all three Urban Poverty Projects had broadly the same aim, attribution of results to individual projects is facilitated by the fact that geographical coverage was different in each case. However, there was some overlap. The intended beneficiaries of this project consisted of two groups:

(i) some 8.4 million persons (of which an estimated 3.55 million were classified as poor) living in about 1,736 kelurahan (urban neighborhoods) in 78 districts (kabupaten and kota) in 15 provinces throughout Indonesia that were previously not covered by either UPP1 or UPP2; and

(ii) approximately 3.1 million persons living in 660 kelurahan already covered by one of these two projects.

To provide improved services for the urban poor: substantial.

Outputs

  • About 49% of kelurahan grants were invested in physical infrastructure, followed by revolving loan funds (27%) and grants for social infrastructure and services (24%). Most of the grants for physical infrastructure was invested in roads (1,516 km of village roads, 44% of total cost), followed by: drainage (753 km of drainage, 20% of total cost), water supply (19,900 clean water units, 11% of total cost), small bridges, sanitation and house rehabilitation (20% of total cost), irrigation canals, markets, shops, electricity connections (5% of total project cost).
  • About 117 participating cities were funded by UPP2 and UPP 3 Poverty Alleviation Partnership Grants (PAPG) instead of a target of 70. On average about 80 sub-projects were completed in each participating city. About 58% of sub-project financing was provided by Local Government, the private sector, and others, which was higher than the target of 50%. Communities contributed 14% of the financing. Most of PAPG investments were used for physical infrastructure, particularly roads, drainage, houses rehabilitation, education facilities, water supply, and irrigation.
  • Social programs covering aid for the elderly, scholarships for students, skills trainings, etc, were received by about 550,000 beneficiaries of which 59% were reportedly poor urban households in the participating kelurahans.
  • The project provided revolving funds to 32,881 community groups of which about 46% were women. However, only 45% of the community groups with revolving funds had repayment rates over 90%.

Intermediate Outcomes
  • 53% of poor households received Poverty Alleviation Partnership Grants against a target of 60%.
  • 50% of poor households in the targeted kelurahan received benefits and improved services (p. 35).
  • 74% of poor households used project infrastructure against a target of 75%
  • 48% of poor households had access to micro credit against a target of 40%
  • 59% of poor urban households received grant assistance against a target of 15%

Outcome

The ICR does not provide outcome indicators because the quantitative and qualitative study to measure the outcome and results of the project was still ongoing when the ICR was completed. New information from the Final Quantitative Evaluation Report (provided by the project team to IEG) show that targets for nearly all indicators were met or exceeded. About 90% of the poor households in the targeted kelurahan received benefits and improved services from the project compared to the Year 5 2010 target of 50%. Improved services from road access were cited most frequently as the major benefit from the project - about 95% of the respondents noted improved access to markets. The second most frequently cited improved service was loan access. More than 80% household who had applied for revolving fund stated that loan procedure is simple, loan requirement is simple, loan processing time is short, and loan improves household income. About 38% of the revolving fund recipients indicated increase in their income level. The report also found that 88% of the BKMs formed were representative, effective, and operate in a participatory manner compared to the Year 5 2010 target of 70%.

To strengthen community and government institutions for responsive service delivery: Substantial

Outputs

The capacity building activities conducted by the project through trainings, focus group discussions, workshops, etc, have enabled the Local Governments to prepare the Regional Poverty Alleviation Strategies for each city. Also, 70 Regional Poverty Alleviation Strategy papers were completed against a target of 40.

The project supported the establishment kelurahan organizations, 100% of the participating local governments formed the Community Board of Trustees (BKMs) against a target of 80%, to provide services to the urban poor and increase the voice of the poor in public decision-making. According to the ICR (p. 18), the vast majority of the BKMs were established through democratic election processes that involved about 35% adult voters, which is higher than the targeted value of 30%. About 86% of the BKMs completed financial audits against a target of 80%.

The community organizations established under the project included 22% of women as elected members. The ICR reports (p. 18) that over 35% of female and vulnerable groups (compared to the target of 30%) were participating in discussions about the project. The vast majority (99%) of all participating kelurahans completed and ratified community development plans for the provision of services to the urban poor.

Outcomes

The project was successful in encouraging partnership between Local Governments and community organizations participating in the Poverty Alleviation Partnership Grants (PAPG) to plan and invest in community infrastructure and social programs as evidenced by: (a) the number of Local Governments participating in PAPG was much higher than expected (70 instead of a target of 40), (b) 30 Local Governments financed more than 40% of the cost of sub-projects from their own budgets, thereby leveraging the use of Bank financing by a considerable margin, (c) about 35 Local Governments are replicating the PAPG approach in their Kota/Kabupaten, and (d) the PAPG mechanism is being considered by the Local Government for implementing its neighborhood upgrading program for the poorest and slum kelurahan starting in 2012.


5. Efficiency:

As the sub-projects were not pre-identified, no cost-benefit analysis or cost-effectiveness analysis was carried out at appraisal.

The ICR did not calculate an ex-post economic rate of return. The ICR noted (p. 20) that the average construction cost of the community driven projects was 40 percent lower than those done by contractors of local governments. The cost savings were mainly attributed to the high level of voluntary labor contributed by residents in community driven development projects. This is based on a 2005 study by the National Planning Board, Finding of Post Construction Economic Impact Analysis Study for community driven-development programs. The Quantitative Evaluation Report confirms that the infrastructure built under the project was cheaper than that built by the Government.

However, the ICR notes (p. 12) that these cost savings are, in a few places, offset by other factors, including: (i) low technical quality (especially for relatively complex infrastructure, such as housing units and water supply systems), (ii) limited capabilities for operations and maintenance, and (iii) heavy reliance on facilitators, which resulted in relatively high project implementation costs.

There was poor compliance with Financial Management (FM) procedures regarding the use of revolving funds and late payments. According to the Project Team, the FM shortcomings did not lead to any known misuse of funds or unresolved complaints about the use of funds. According to the Final Quantitative Evaluation Report, the number of BKM with Audited Financial Report increased from 27 in 2007 to 63 in 2011.

There was a high turnover of facilitators, late arrival of socialization materials, and lengthy delays in consultant recruitment (even though the Bank already provided the implementing agency with a short list during loan negotiations). Project activities were frequently interrupted or postponed because of avoidable delays in budget execution and budget revisions. Despite this, the project closed on time.

While the implementing agency invested in several qualitative evaluations (e.g. on infrastructure, community development plans, women’s participation, local government, community volunteers, etc.) their implementation was delayed.

The overall efficiency of the project is rated substantial.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The relevance of objectives is rated high and relevance of design is rated substantial. The efficacy of the first objective -- to provide improved services for the urban poor -- is rated substantial.The second objective -- to strengthen community and government institutions for responsive service delivery -- is also rated substantial since the urban poor were successfully targeted and the capacity of beneficiaries, local and urban governments to plan together was enhanced. Efficiency is rated substantial. Overall outcome is assessed as satisfactory.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:


Technical: The investments financed by the project are generally simple in nature, but a potential problem concerns the ability of the communities to identify technicians with sufficient skill and ability to provide the needed operational support and maintenance. The ICR reports that some sub-projects were of low technical quality (especially for relatively complex infrastructure, such as housing units and water supply systems). The technical risk is assessed as moderate.

Financial: The risks associated with financial management and the ability of the communities to monitor the use of the funds provided in the future are moderate. The ICR indicated that the management of the revolving fund loans was less than satisfactory throughout the project as the revolving funds were often unable to comply with the financial management guidelines. This risk is associated with the technical risk as communities may not have access to qualified accountants to manage and control the funds allocated to operation and maintenance. The project team clarified that the grants for revolving funds were substantially lower than grants for infrastructure or social services (only 8% of the project cost) and the decision of both Bank and Borrower to phase them out of CDD-type programs - the risk is assessed as moderate. An additional financial risk concerns the ability of the communities to raise the funds needed to purchase spare parts and to properly operate and maintain the investments financed by the project.

Political: Concerns about elite capture, and the ability of local governments and communities being willing to work together, and about government interference, still exist. The ICR reports that the procedures established by the project regarding fraud and corruption proved to be effective in combating corruption and these procedures may be sustained given the support from government at the highest level.

The risk of the withdrawal of central and local government support for the community driven development approach is negligible to low.

Overall, the risk to development outcome rating is rated as moderate.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

The project was a geographical expansion of the Second Urban Poverty Project (UPP2), which was still under implementation at the time of appraisal. While the lessons learned from the Second Urban Poverty Project could not be incorporated in the Third Urban Project since the findings from the impact evaluations were unavailable, the Bank team was able to incorporate learning from the two previous projects based on the implementation experience, and the management information system. Safeguards were appropriately identified. These were: (i) provision of stronger support to local governments; (ii) allocation of more grants per kelurahan to well-performing BKMs in UPP1 and UPP2 locations; and (iii) inclusion of easily measurable targets. Risk identification was also adequate. The PAD rated the overall risk for the project as modest with the only risk element rated above either negligible or modest as the “willingness of local governments and kelurahan organizations to work together” which was rated as "substantial". Other risks included that of political interference in either the establishment of the ward level community organizations (BKM) or the risk of the BKM’s being co-opted by the local elite. Concerns were also raised regarding the financial management. While these risks appear reasonable, the mitigation measures were generally simple and focused on capacity building. Since this was the third project in this series, the overall risk rating of modest appears to be reasonable.

There were weaknesses in M&E design (the performance indicators were largely output based and there was no baseline estimates).

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:

The Bank mobilized a multi-disciplinary supervision team with expertise in project management, financial management, procurement, monitoring, and safeguards. The project team typically fielded two full supervision missions per year in addition to many shorter site visits between missions. Virtually all team members were based in the country office, which allowed for much closer and more continuous supervision. All core members of the Bank’s task team remained unchanged over the course of eight years, allowing for continuous learning and long-term relationships with government counterparts.

Intensive supervision enabled the Bank to identify and proactively address key issues adversely affecting the achievement of the project development objective at an early stage, notably delays in budget execution and revision, consultant mobilization, facilitator recruitment, implementation of the Poverty Alleviation Partnership Grants, and limited compliance with financial management procedures for the revolving funds.

The Bank team systematically visited over 100 villages per year. In addition, the Bank team developed various innovations in supervision including a website, a complaint handling mechanism and a teleconference system with project operators and the communities.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Government of Indonesia displayed a strong commitment to the project as evidenced by its 2000-2004 National Development plan to guide the project, and support its long-term strategy for poverty alleviation. As further evidence of its ownership of the project, the Government established an Inter-Departmental Team consisting of the Coordinating Ministry of Social Welfare, the National Planning Agency, the Ministry of Public Works, the Ministry of Finance, and the Ministry of Home Affairs, to improve coordination at the central level.

The Government created a unified management structure for the three urban poverty projects which enabled the projects to be placed within the new National Community Empowerment Program in Urban Areas. It created a separate National Management Consultant for Poverty Alleviation Partnership Grants and the Neighborhood Development Program in view of the special attention that these components required.

In 2006, the Ministry of Finance changed a regulation on fund channeling that allowed the Government to channel kelurahan grants through a commercial bank. Initially, the implementation of this regulation led to delays in fund disbursement, but ultimately had positive impacts on the project's financial management and transparency in the flow of funds.

The Government contribution to the project increased from US$23.8 milion to US$42.0 million, a 76% increase.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:

The implementing agency was the Ministry of Public Works and the Project Management Unit was housed in the Ministry. The Ministry of Public Works took steps to modify implementation arrangements where needed and responded to complaints or suspicions about the misuse of funds. The implementing agency also created a detailed master schedule, which helped the Government reduce avoidable delays during project start up and implementation.

From the start of implementation, the implementing agency and the Bank were willing to modify the design of the project in response to changing circumstances. The design of the block grant component and flexibility in its implementation also allowed the communities to put the available funds where they mattered most to them. Anti-corruption measures such as the complaints handling unit were introduced to report incidences of suspected corruption. As of 31 December 2010, the project had resolved 7,384 of 7,394 reported complaints. A total of 121 cases of corruption were reported, accounting for an estimated misuse of funds of approximately US$ 310,342 equivalent. The ICR reports (p. 19) that the Project Management Unit had recovered all of these funds.

There were, however, several shortcomings that stronger management might have been able to correct, such as poor compliance with financial management procedures regarding the revolving funds, late payments, the high turnover of facilitators, late arrival of socialization materials, and lengthy delays in consultant recruitment (even though the Bank already provided the implementing agency with a short list during loan negotiations).

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

At the beginning of the implementation of Second Urban Poverty Project (in September 2003), the Government requested the Bank to assist with the development of a web-based management information system that would cover both the First and Second projects and future urban poverty projects. The management information system was to be regularly updated and included arrangements that would make it readily accessible by the public through a website.

The project design included a large number of indicators that were realistic and measurable, although most of them measured outputs or intermediate outcomes. The project under review targeted mostly new geographic areas, and there were no baseline data. The independent National Management Consultant was responsible for the monitoring of the project.

b. M&E Implementation:

The design of the management information system took much longer than expected, and it began to generate actual values of indicators only in early 2006. The outputs of the system were publicly available on the National Community Empowerment Program website (http://www.p2kp.org) in the form of downloadable Microsoft Excel and PDF files, most of which were updated on a monthly basis. From 2007 onward, the system also published information on contracts financed by the project and payments of contractors. The content of the files is in Indonesian, and was presented in easy-to-understand and properly labeled tables.

Initially, the lack of reliable information limited the system’s usefulness as a transparent M&E tool. To address this shortcoming, the implementing agency agreed to assign the responsibility for data accuracy and validity to the team leader of the consultants tasked with data collection, and delayed payment until data was brought up to the required standard. Together with spot check monitoring, these measures improved the quality of the information, but some problems with data collection persisted until project closure.

a. M&E Utilization:

Once the Management Information System system was fully operational, the data collected and the monitoring of progress helped the project to: (a) identify and remedy quality problems and to incorporate better controls as the project progressed, (b) fine-tune the targeting of resources to the areas with the greatest needs and to help achieve the greatest impacts, and (c) increase transparency, strengthen public confidence in the openness and fairness of the project, and demonstrate in tangible terms the benefits provided to communities and households.

At the end of project implementation, the project’s website not only provided access to data on all three of the urban poverty projects, but also on all follow-up projects. The system was adopted for other projects in Ministry of Public Works.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
The project was assigned Environmental Category "B" and four safeguards policies were triggered: OP4.01 (Environmental Assessment), OP 4.12 (Involuntary Resettlement), OP 4.20 (Indigenous People), and OP 4.11 (Cultural Property).
The PAD noted that due to the small size of the sub-projects financed by the project (under US$2,000 on average), any adverse environmental impacts would be expected to be limited, highly localized and readily mitigated through the project’s negative list or through standard operating procedures and guidelines (PAD Annex 10 a). Also, sub-projects were not expected to require involuntary resettlement or significant amount of land acquisition. In the event that land acquisition were necessary, a Land Acquisition and Resettlement Policy Framework was developed (PAD Annex 10 a).

The ICR noted (p. 9) that as expected at appraisal, the adverse social and environmental impacts of sub-projects activities were very small. However, the recording of potential social and environmental impacts remained poor, and required significant improvement in the final stages of project implementation. The ICR provided no further information regarding how any environmental impacts were addressed or if any sub-projects resulted in any environmental problems. Further, the ICR provided no information regarding the level of compliance with the other safeguards policies that were triggered.

According to the project team, all four safeguards were rated "Satisfactory" throughout project implementation.

b. Fiduciary Compliance:
The ICR reports that the project improved the financial reporting capability of the Project Management Unit through the Management Information System and independent financial audits. At the district, provincial and central level, project implementation units used the Government’s computerized accounting system. At the village level, community board of trustees used manual accounting systems.
The project encountered problems with financial management throughout implementation, primarily because revolving funds were often unable to comply with financial management guidelines, especially those related to safekeeping and the dissemination of financial reports. These problems were exacerbated by difficulties in finding qualified facilitators. According to the project team, the performance ratings for financial management were generally moderately satisfactory. The project was audited by Badan Pengawasan Keuangan dan Pembangunan, a Government auditing agency. In addition all villages are required to have an independent public accountant to audit the accounts of the boards of trustees. The auditor's opinions of the project’s financial reports were unqualified.

The ICR was silent regarding issues involving governance of procurement. Project activities were postponed because of lengthy delays in consultant mobilization and facilitator recruitment (even though this was the third project using similar implementing arrangements, and the Bank had already provided the implementing agency with a short list during loan negotiations). To minimize procurement risks where works and goods were procured by communities, a series of mutually reinforcing measures were put in place. These included: (i) an extensive complaints handling system, (ii) the publication of the consultants’ invoices in the project's public website, (iii) regular spot checks, (iv) independent audits, (v) intensive Bank supervision, (vi) disbursements of funds to communities only after verification of financial management performance of the community board of trustees and community development plan by facilitators and a community board of trustee cosignatory, and (vii) oversight by consultant teams at district and national level.

According to the project team, procurement performance was rated as moderately satisfactory due to delays at Central Government level in the processing and in document preparation. There were no reported cases of misprocurement.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:
None reported.



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Satisfactory
 
Risk to Development Outcome:
Negligible to Low
Moderate
The risks associated with financial management and the ability of the communities to monitor the use of the funds provided in the future are moderate..  
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Moderately Satisfactory
There were moderate shortcomings in Implementing Agency performance (see section 9b above). 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
Adapted from the ICR:
  • Community driven development projects can result in substantial cost savings. The ICR indicated that the unit costs of small-scale infrastructure were about 40% lower than similar work undertaken by local government contractors, partly because of the high level of voluntary labor contributed by residents.
  • For a natural disaster prone country like Indonesia, the community driven development projects can facilitate natural disaster preparedness and management. The Government benefited from using the project for an immediate response to organize communities and develop community systems for recovery and reconstruction after the 2006 Jogjakarta earthquake. The facilitators were used by the Government to assist in assessing the damage and the communities’ needs.

IEG Lesson:
  • Community participation is necessary but not sufficient to ensure sustainability of improved infrastructure. The experience from this project suggests that in some cases the sub-projects are of low technical quality standards, and communities capacity for operations and maintenance is limited. Greater attention to strengthen the technical capacity of communities would enable them to operate and maintain their assets and to harmonize planned new investments with maintenance programs and budgets.
  • Developing trust and confidence in public officials at the local government level is essential in promoting effective civic participation
  • Clear communication of the distinct roles and functions of government and civil society and the way in which they are brought together in a participatory process is essential to bring about recognition and legitimacy of the adopted bottom-up approach of development process.

14. Assessment Recommended?

Yes
Why?
To verify ratings and to draw lessons from the three urban poverty projects.

15. Comments on Quality of ICR:

The quality of the evidence and analysis is good and the ICR is internally consistent. The ICR provides relevant facts and data about the performance of the project and the report reads well. The lessons drawn are based upon the project's experience. The report is results oriented and not a narrative of implementation experience. There are some shortcomings: (a) the report provides little information regarding compliance with safeguard and fiduciary policies, and (b) a more candid discussion of the sustainability issues associated with the sub-projects financed by the project would have been useful.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions