Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Tocantins Sustainable Regional Development Project


  
1. Project Data:   
ICR Review Date Posted:
01/28/2014   
Country:
Brazil
PROJ ID:
P060573
Appraisal
Actual
Project Name:
Tocantins Sustainable Regional Development Project
Project Costs(US $M)
 100  112
L/C Number:
L7208
Loan/Credit (US $M)
 60  60
Sector Board:
Transport
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  12/09/2003
 
 
Closing Date
12/31/2009 10/31/2011
Sector(s):
Sub-national government administration (50%), Roads and highways (50%)
Theme(s):
Rural services and infrastructure (25% - P) Municipal governance and institution building (25% - P) Environmental policies and institutions (24% - P) Participation and civic engagement (13% - S) Decentralization (13% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Kavita Mathur
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:

The project development objectives in the Project Appraisal Document (p. 4) were:”to contribute to raising living standards, reducing inequalities and regional disparities by improving access to markets, job opportunities, infrastructure and social services for rural communities in Tocantins' poorest regions, while ensuring sustainable use of natural resources and the protection of fragile ecosystems.”

The objectives according to the Loan Agreement (Schedule 2, p. 24) were: “to assist the Borrower in raising living standards of its rural poor population and in reducing inequalities and regional disparities among its inhabitants, by improving access to markets, job opportunities and infrastructure and social services for rural communities in the Borrower’s poorest regions, while ensuring sustainable use of natural resources and the protection of fragile ecosystems.”

The objectives in the Loan Agreement will be used as the benchmark for this evaluation.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

The cost table by components in Annex 1 of the ICR is incomplete, the figures just refer to the Bank Loan amount. The project included following three components:

1. Participatory planning and management of regional and municipal development (appraisal US$6.5 million, actual US$4.1 million) to: (a) establish and strengthen mechanisms for local civil society participation; (b) set-up, in each region of the project, regional units of the Planning Secretariat (SEPLAN) to support the local and regional planning and management capacity; and (c) strengthen the state's capacity to undertake multi-sectoral state-level planning, administer public programs and monitor/evaluate the performance of such programs.

2. Environmental management (appraisal US$10.2 million, actual US$19.30 million) to: (a) strengthen the state's land management capacity; (b) consolidate the state's environmental protection system; and (c) support the regulation and promotion of sustainable land use.

3. Rural transport improvement (appraisal US$42.7 million, actual US$36 million) to: (a) strengthen the state's road management capacity; (b) define in a participatory fashion and improve, to all-weather condition, about 6,000 km of priority road sections of the main municipal road networks and improve about 120 km of high-priority state feeder roads; (c) promote cost-effective rural transport services; and (d) provide support to the project coordination unit.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project Cost: Annex 1(a) of the ICR refers only to the Bank Loan amount. According to the project team the actual project cost was US$112 million.

Financing: The loan of US$ 60.0 million was fully disbursed. There were no other external sources of financing.

Borrower Contribution: The project team confirmed that the Borrower's actual contribution was US$52.0 million and not US$ 23.0 million as reported in Annex 1 (b) of the ICR. This was higher than the appraisal commitment of US$ 40 million.

Dates: On August 13, 2009, the project closing date was extended by one year from December 31, 2009, to December 31, 2010, to allow minor adjustments to the loan amount distribution and to use the unallocated category which constituted 11.5% of the loan amount. On December 16, 2010, the closing date was further extended by 10 months, until October 31, 2011 to allow the project implementation to take advantage of the dry period for completing physical works.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Rated high.
The objectives are relevant to the Bank’s Country Assistance Strategy and to Brazil's priorities. Both the 2008-2011 and 2012-2015 Country Partnership Strategies (CPSs) aim for increased efforts towards a more equitable and sustainable economy. The project directly addresses two of the four objectives of the later CPS, namely to enhance equity and focus on environmental sustainability. The objectives are also relevant to the State of Tocantins’ Pluri-Annual Plan focused on building core infrastructure that supports the lagging regions and aims to promote social inclusion.

b. Relevance of Design:
Rated modest.
By improving the rural roads and promoting cost-effective rural transport services, it is plausible that the project could contribute to improving the access to markets, job opportunities, and social services for rural communities. However, the linkage to raising living standards, reducing inequalities and regional disparities is not spelt out in the results framework.

The project's environmental management component included activities to consolidate the state's environmental protection system and support the regulation and promotion of sustainable land use. These activities are directly linked to the project objective of ensuring sustainable use of natural resources and the protection of fragile ecosystems.


4. Achievement of Objectives (Efficacy) :

The project's objectives - (i) raising living standards of Tocantins' rural poor population, and (ii) reducing inequalities and regional disparities among its inhabitants are assessed below. Outputs are common to both objectives.
Outputs
  • The project improved 144 km of roads compared to the target of 120 km. However, the proposed output based rehabilitation and maintenance contracts were not implemented.
  • The project supported the construction of bridges and culverts.
  • The project financed the State Water Master Plan, a Hydro-geological Water Resources Master Plan and river water flow studies, and then operationalized recommendations through five river basin action plans and through creation of basin management committees.
  • With project support, the Borrower carried out four zonings, including 3 thematic and 1 geographic, which establish measures for sustainable development with a view to ensuring that economic development activities take account of environment and natural resources protection and of biodiversity conservation.
  • The project funded a large number of activities such as municipal development agendas and regional development agendas. 67 municipalities had finalized their municipal development agendas by project closure. During the initial stage, there were delays in the process of establishing regional development agendas. However, all four regions targeted by the project had prepared regional development agendas by project closure. The new four year State Development Agenda is based on a series of 9 regional development agendas.
  • None of the six protected areas which were to be established under the project was set up. Preparatory activities for 18 areas were finalized. The ICR reports (p. 31) that the establishment of the protected areas was delayed by lengthy processes of consultation and community mobilization, political decisions, and handling of social issues to manage stakeholders expectations.
  • The project planned to create four inter-municipal consortia. None was established.
  • Numerous studies were carried out of the State fauna and flora, and program of biodiversity conservation were developed; preparatory work was carried out for legislation for the protection of fauna and the formulation of a State policy on fishing.

Outcomes

(i) Raising living standards of Tocantins rural poor population: rated substantial.

The following evidence on intermediate outcomes is obtained for the most part from two Impact Evaluation surveys carried out by the Borrower, the first in 2003-2005 (before the start of the civil works), and the second in July, 2011, four months before project closure. The results of the surveys are reported in the ICR.

Intermediate outcomes

  • In the southeast, 92% of the households surveyed stated having all-weather access to a health facility, against 68% before the project.
  • Travel time to health facilities was reduced. For example, in the Southeast region, the travel time to the nearest hospital was reduced from 2 hours to 1 hour and 30 minutes. In the north, the travel time to hospital was reduced from 1 hour 52 minutes to 1 hour 25 minutes.
  • In the Southeast region, 94% of the households surveyed stated having improved access to education facilities, versus 79% before the project. The travel time to a preschool decreased marginally from 29 to 27 minutes. The time to reach a secondary school was reduced from 45 to 32 minutes. In the North region, 79% of the households surveyed stated having improved access to education facilities, versus 66% before the project. The travel time to pre-school was reduced from 21 to 13 minutes, and from 44 to 14 minutes for secondary school. In both regions, the travel time to the closest elementary school was not reduced.
  • The surveys indicate that the project improved access to markets. In the Southeast region 18% of the production (the production does not only refer to crops but to any type of product) was destined to other municipalities after the project against 7% before the project. In the North region, 100% of the traded production was destined to other municipalities after the Project against 26% before the project. This is calculated based on the share of cash crops in total agricultural production. The ICR reports (p. 28) that data were not available to estimate the impact of road improvements on commodity prices.
  • A household survey showed that in the Southeast, the average distance to the municipal center decreased from 38.5 to 22.5 km, while in the North the same average distance decreased from 18.5 to 10 km.
  • A survey carried out by the Road Agency on 27 private firms contracted for civil works estimated that about 2600 temporary jobs were created, of which 84% were local and regional (55% beneficiary municipalities and 29% in neighboring municipalities) and 16% were jobs generated in the firm's headquarters. The duration of these job assignments varied between 4 to 8 months in the field and between 6 to 10 months in headquarters.
  • The project attempted the formation of social capital through Rural Producers Associations. In southeast, the 11 Rural Producers Associations increased their average membership from 35 to 43 per Association. However, in the North, the number of Associations of Rural Producers declined from 20 to 11, and average membership decreased from 90 to 72.
  • The project supported participatory mechanisms which relied on extensive consultations in each municipality on their municipal development agendas. This helped the communities in prioritizing investments on municipal roads. The beneficiaries voted on roads that the Government would improve that best responded to their needs. However, “the project inadvertently created unfulfilled expectations that could have been avoided with clearer communication and outreach” (ICR, page 23).

Final Outcomes

The ICR provides no evidence of improved living standards of the rural poor in Tocantins. It is assumed that the road improvements and their intermediate outcomes listed above would have raised living standards but the link is not made explicit and no indications are provided that living standards did in fact increase.

However, the project team subsequently provided IEG with the results of an Impact Evaluation which was not available in final form at the completion of the ICR. The project team reports that the methodology used in the Evaluation underwent a thorough peer review process in Fiscal Year 2013/2014, and that the Evaluation is expected to be published under the auspices of DEC. According to this Evaluation, the project's rural road interventions contributed to the creation of about 220 jobs in the interviewed rural communities. The project also had a likely positive impact on agriculture employment in two of the four regions (+17%) and in public service delivery in the other two. The Evaluation estimates that the monthly household income increased by R$100 or about US$50, and that while household income has tended to increase regardless of the project, the increase was larger where the project was implemented.

The project team further reported that the trends of a human development index (the IFDM or Indice FIRJAN de desenvolvimento humano, a composite municipal index based on health, education and income indicators) across the municipalities of Tocantins show an 9% growth in the municipalities of Eastern Tocantins (where the project was implemented) between 2005 and 2010, compared to a 6% growth in Western Tocantins.

According to the project team, while the health impacts are inconclusive, the educational impact of improved roads was positive in two of the four Project target regions (Bico do Papagaio and Southeast) corresponding to the regions where the surveys had the most relevant samples of control group. The number of girls attending school has increased by 0.03 girls per household (respectively about 0.05-0.06 girls per household) in these regions, a figure considered to be statistically reliable.

(ii) Reducing inequalities and regional disparities among the inhabitants of Tocantins: rated modest.

The ICR did not include any evidence on either intermediate or final outcomes related to this objective. It reports that the project generated employment opportunities, infrastructure and social services for rural communities in Tocantins' poorest regions that might have led to reducing inequalities. A bottom-up approach was adopted to foster rural communities empowerment in local development through participatory planning, and promoting better social inclusion and enhanced appropriation of economic opportunities through accompanying measures to local development initiatives. However, no evidence is provided of reductions in either income inequalities or regional disparities in income and wealth.

According to the project team, the improved rural road connectivity changed people’s transport modal choice. More people came to use public buses and individual cars after the rural road improvement. Numerically, households that used public buses to go to school have increased by 3% after the project compared to the control group. Use of individual cars to access nearest populated place, municipal center, nearest hospital and school has increased by 4-8% (values varied by each destination). Individual car ownership has also increased by 4%. Similarly ownership of bicycles increased by 8%.


5. Efficiency:


The ex-post Economic Rate of Return (ERR) was 20.2% compared to the appraisal estimate of 18.5%. The ex-post economic analysis, which used the same methodology as at appraisal, focused on project roads (which accounted for 60% of total project cost). Investment costs were taken from the actual civil works contracts. No maintenance costs were included on the grounds that “structures put in place are expected to remain useful for decades, with basically no resources being needed for maintenance, an important factor considering the general lack of capacity of the municipalities to undertake such maintenance” (ICR, page 19). The benefits consisted of the estimated increase in producer surplus resulting from the project investments. The improvements to all-weather condition of the municipal roads were expected to increase this surplus through: (i) reduction of overall production costs, notably, reductions in transport costs, decrease in stocks, and reduction of losses related to transport and stocking; (ii) productivity gains, which varied according to the product; (iii) increased size of cultivated areas; and (iv) faster changes towards more productive agriculture in areas where available land is scarce (ICR p. 34).

Principal among the issues associated with this methodology are, first, it is unclear how the area of influence of the roads was estimated; second, it appears to attribute the whole increase in producer surplus in the areas of influence to the project-financed road improvements; third, the sources for the municipal-level agricultural production, price and cost estimates prior to and after the project are not clear, and there is no discussion of their reliability; and fourth, estimates purport to show a considerable increase in agricultural production, and hence of generated traffic on the rural roads. Despite this, no estimate of maintenance costs is included.

The project experienced a delay of one year and ten months due for the most part to operational and administrative inefficiencies. Given these, and the methodological issues with the cost-benefit analysis, efficiency is rated modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
18.5%
70%
ICR estimate:
Yes
20.2%
70%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of objectives is rated high, and that of design modest. The efficacy of the first objective - raising living standards of Tocantins rural poor population was substantial, based on the results of an Impact Evaluation. There is modest evidence concerning the attainment of the second objective - reducing inequalities and regional disparities among its inhabitants. Project efficiency is rated modest. Overall outcome is assessed as moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:


The risk of policy reversal is low. The ICR reports (p. 18) that at the State level, the State officials acknowledge the benefits of decentralization, strengthening of municipal administrations and empowerment of local populations. The project team informed IEG that since the closing of the loan (about two years) the Government has recently awarded the contracts of municipal road improvement for the pending six municipalities which could not be financed under the project for timing and budget reasons during project implementation (for an additional cost of approximately US$2 million, fully financed with local funds in spite of the challenging budgetary stringency in the State of Tocantins). This indicates State commitment to the initiatives supported by the project beyond government cycles.

The risk of inadequate maintenance is moderate. The ICR appears to assume that maintenance of bridges and culverts would be unnecessary. The project team subsequently stated that the resources needed to maintain the network have been reduced, as those structures made of concrete are far more durable than the fragile structures of before-the-project (often made of wood and requiring regular ad-hoc maintenance). For other structures, the ICR acknowledges that maintenance would be critical. The municipalities lack capacity to carry out maintenance. The Inter-Municipal Consortia, which were intended to foster cooperation between municipalities in maintaining infrastructure, were not formed. Only the State road agency at Federal level continues to provide support to the municipalities in maintenance of their networks. It is not clear from the ICR how maintenance would be funded. According to the project team, maintenance needs on these unpaved roads are mainly limited to grading and re-gravelling over two or three years, depending on the traffic levels. The maintenance activities are easier to carry out given the improvement of the key structures. However, the team acknowledges that road maintenance funding is,and will remain, a challenge across Brazil.

The project supported some strengthening of the state's environmental management capacity though it is unclear how this will be sustained and further improved. According to the ICR, the participatory approach supported by the project led to greater engagement of the local population in the planning process. Overall, the risk to the project’s environmental and social achievements of the project is estimated as moderate.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
On the positive side:

  • During preparation, there were close and collaborative efforts between the Bank and Borrower in developing multi-sector approach to improving living standards in the poor rural regions of Tocantins (ICR p. 20).
  • Risks relating to fluctuating Government commitment, lack of coordination among different players and lack of availability of counterpart funds were appropriately identified and mitigation measures were included in the risk matrix. During implementation, some unidentified risks materialized, including the financial crisis and the impeachment of the State's Governor in 2009. Although both had significant repercussions on project implementation, neither could have been foreseen at the preparation stage.
  • Safeguards identification was satisfactory and detailed analysis was conducted for potential environmental and social impacts during project preparation.

However:
  • The project was complex - it included 67 municipalities, diverse communities and involved multiple institutions (several state secretariats and agencies). The ICR (p. 10) states that "the complexity of the Project design appears as one of the main factors that might have caused implementation delays and the dispersion of Borrower’s efforts.
  • Some activities, such as the creation of the conservation areas required lengthy processes and involved broad social commitment to an extent not foreseen or allowed for in design. The ICR reports (p. 5) that the implementation challenges increased as a result of the wide scope of the project's design "particularly given the overall limited amount allocated to the project (US$ 100 million, out of which US$ 60 million from the Bank loan), and the large contribution expected of the State (40%) in a context of limited available resources at the State level (financial, human resources, capacity etc.)".
  • M&E design was weak (see section 10 below for details).

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
Regular supervision missions, with a mid-term review, were carried out, on average twice per year. The ICR reports (p. 5) that project management was challenging as the works involved multiple small contracts (67 contracts averaging only US$ 0.65 million each). Over 200 procurement prior reviews were carried out to foster quality of the bidding processes. The Bank team was pro-active in supporting the introduction of environment protection policies, territorial development and management, and road management. Compliance with Bank safeguards policies was monitored through site visits. Supervision did not, however, attempt to improve the weak results framework (see section 10 below).

The project team confirmed that after project closing the State of Tocantins has gradually put in place output-based contractual approaches for the maintenance of more than one third of the state network. According to the team, this result was the outcome of the ongoing dialogue since the start of the project. This effort builds on similar efforts carried out at the Federal level and in the States of Rio Grande do Sul and So Paulo since the early 2000s.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Government complied with the project's legal agreements and with Bank procedures. However, commitment varied during project implementation, and was affected by the international financial crisis and frequent changes in the State's Government (there were four different Administrations during implementation).
Implementation got off to a slow to start, mostly as the result of a gradual takeover by a newly elected state Government. There were delays in counterpart funding in the initial stages.

The Governor was impeached in June 2009, which led to a general slowdown in the State's public administration. Also in 2009, a severe fiscal constraint was imposed on the State as a result of the austerity measures adopted nationally in the wake of the international financial crisis.

The new Government which was elected in September 2009, following the impeachment, had a short term mandate (it was only there for 15 months as the electoral cycle imposed new elections in October 2010) and the commitment to the project was weak. Once again there were difficulties in mobilizing counterpart funds and delays in implementation.

The new Government that was elected in 2010 was more strongly committed to the project. As a consequence, implementation accelerated and the final counterpart contribution was US$ 52 million, higher than the planned US$ 40 million.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
Five implementing agencies are cited in the ICR Data Sheet. although there is no evaluation of the performance of four of them in the text of the ICR. However, according to information subsequently provided by the project team, there was in fact only one implementing agency -- the Project Management Unit under the Secretariat of Infrastructure. The Unit was responsible for all aspects of fiduciary (financial management, procurement etc.) and safeguards management. According to the project team, "this centralized model of implementation was critical in turning a complex operation more manageable and to expedite implementation, particularly when considering the more than 200 procurement processes involved in project execution." The ICR provides little information on the Unit's performance except to state (page 22) that "general institutional management and effective capacity" were "adequate."

The ICR reports the following implementation problems, most of which would appear to be outside the control of the Project Management Unit: (i) initial preparatory activities were slow to start due to lack of State Government commitment; (ii) there were 67 contracts, averaging only US$0.65 million each, increased the volume and complexity of procurement processes; (iii) works were interrupted every year during the rainy season which spanned over 6 months; and (iv) frequent changes in the State Government were associated with shortfalls in counterpart funding.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
Outcome indicators were characterized by: (a) lack of clear definition; (b) lack of baseline values; and (c) difficulty in measurement. There were no intermediate or final indicators to measure the attainment of the second objective i.e "reduction in inequalities and regional disparities". The output indicators were pragmatic and easily measurable, although the output targets were overly optimistic (ICR p. 8). The project design included an Impact Evaluation based on previous experience of rural development projects in Northeast Brazil and Vietnam (for example, the Rural Alleviation Projects of the Northeast of Brazil and the Impact Evaluation of the Vietnam Transport Project). The Evaluation is discussed below.

b. M&E Implementation:

No revision of the outcome indicators was undertaken during the mid-term review because implementation delays meant that only few activities had been undertaken at that time. The ICR reports that the M&E relied on the Impact Evaluation undertaken by the Borrower. The Impact Evaluation was carried out through two separate surveys administered before and after the project, and targeted at both household and local community levels. About 1492 households in beneficiary municipalities and 14 community associations in all four project regions participated in the first survey. The second survey was carried out in July 2011, shortly before project closure, in two of the four regions where works had been completed, representing 1005 households and 11 communities (the survey was limited to two regions as works had began in the two other regions only a few months before project closure, allowing insufficient time for households to ascertain the benefits of the works). The surveys covered 10 main topics (education, health, employment, resources and availability, domestic production activities, credit, community activities, transport, living standards). They were complemented by qualitative questions on perceptions of the Project's impact among beneficiaries.

It is not clear from the ICR if the Impact Evaluation included a counterfactual - i.e. a comparison group to estimate what would have happened to the project participants without the project. This was subsequently clarified by the project team, which provided IEG with the final draft of the Impact Evaluation. According to the methodology section, "In this study, these communities with uncompleted or postponed works were defined as a control group because (i) these were caused mainly due to the financial difficulties in paying counterpart funds for the works, which was little linked with socio-economic or geographical characteristics of households/municipalities, resulting in little endogeneity on choosing the control group, and (ii) all communities surveyed in the evaluation, including both control and treatment groups, were once selected through the public consultation for receiving the works at the beginning of the project, which would mitigate the risk on comparability between two groups". As noted in Section 4 above, the project team reports that the methodology used in the Evaluation was subjected to a thorough peer review.

a. M&E Utilization:
The ICR suggests (p. 9) that more could have been done to institutionalize monitoring and evaluation practices within the State. Nevertheless, it considers the Impact Evaluation to have been “an important achievement, which is expected to provide valuable information to inform the preparation of new operations within the State and Brazil.”

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:
The project was classified as Category “A” for Environmental Assessment purposes, and the following five safeguard policies were triggered:
OP 4.01 Environmental Assessment. According to the PAD (p. 27) an Environmental Assessment was completed and made available for public consultation in October, 2002. Environmental impacts from civil works were expected to be relatively minor, as physical activities under the project were basically concerned with improvement of existing road sections in the state network, and very localized improvements of road infrastructure in the municipal network. The monitoring and evaluation of environmental impacts was carried out by the Road Agency's environmental team in collaboration with the state's Environmental Agency. The ICR reports (p. 10) that "the monitoring of environmental impacts by the Road Agency was thorough and ensured good quality and overall environmental compliance with Bank safeguard requirements.”.

OP 4.01 Natural Habitats. The ICR (p. 10) states that changes in agricultural production patterns as a result of the project may have a cumulative impact on natural habitats. However, it notes that this impact is hard to assess at this stage, is likely to be limited to areas affected (three to five kilometers from the roads), and would be mitigated by the environmental protection measures built into project design.

OPN 11.03 Cultural Property. According to the project team no artifacts were found.

OD 4.20 Indigenous Peoples. An indigenous people development framework was completed and made available for public consultation in October, 2002. The ICR reports (p. 10) that no indigenous people were affected by the project investments.

OP 4.12 Involuntary Resettlement. A resettlement framework was completed and made available for public consultation in October, 2002. However, there was no involuntary resettlement or land acquisition.

b. Fiduciary Compliance:
Financial Management: The ICR reports that "fiduciary compliance was adequate despite the extended number of contracts (over 200) handled under the Project and rising challenges encountered in terms of procurement throughout implementation" (p. 10). There is no discussion in the ICR about auditing of project accounts, either internal or external. The project team stated that audits were undertaken on a yearly basis by external auditors, as required by Bank Policy and no notable issue were identified. There were regular (at least yearly) World Bank financial management supervision missions.
Procurement: The ICR reports (p. 10) that prior procurement reviews were carried for over 200 processes to foster quality of the bidding processes. The ICR does not explain why these reviews were considered necessary or the extent to which Bank's procurement policies were complied with. There were no reported cases of mis-procurement. The project team clarified that all procurement processes were subject to prior review, and all Bank procurement policies were fully complied with.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:
None.



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Negligible to Low
Moderate
There is a moderate risk that maintenance of project-enhanced infrastructure will be under-funded. 
Bank Performance:
Satisfactory
Moderately Satisfactory
Although, the Quality of Supervision is rated moderately satisfactory, there were significant shortcomings in Quality At Entry. The ICR acknowledges that the complexity of the Project design was one of the main factors causing implementation delays. M&E design was weak. 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Unsatisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following lessons are taken from the ICR with some adaptation of language:
  • In multi-sector operations conducted in a politically unstable environment, limiting the number of entities involved and recognizing and allowing for political risk can mitigate the impact of changes in administration on implementation.
  • For multi-sector operations, clear and functional implementation arrangements, including a central coordination mechanism (such as a project implementation unit), are critical for efficiency in project management. In this case, no such central coordination existed.
  • Effective monitoring and evaluation requires that outcome indicators should be simple, measurable and with a direct link to both the project financed activities and to the objectives. Opportunities such as mid-term reviews or restructuring can be used to improve and update the results framework.
  • Although participatory mechanisms can help to foster local ownership, they can also create unfulfilled expectations that could be avoided or mitigated through sufficient outreach and clearer communication.

14. Assessment Recommended?

Yes
Why?
To verify project ratings and examine the impact of road improvements on the rural economy.

15. Comments on Quality of ICR:

The ICR’s assessment of the project's implementation experience is incomplete. There are two major shortcomings: (i) the lack of almost any discussion of, and evidence concerning, the degree of attainment of the two stated objectives; and (ii) five entities are mistakenly listed in the Data Sheet as implementing agencies, and the discussion of implementing agency performance is inadequate. Other shortcomings are: (a) the lack of any discussion of the auditing of project accounts; (b) incomplete cost table in Annex 1, where the figures refer only to the Bank Loan amount; and (c) the absence of a list of the abbreviations and acronyms used in the text.

a. Quality of ICR Rating: Unsatisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions