Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Road Rehabilitation And Maintenance Project


  
1. Project Data:   
ICR Review Date Posted:
06/19/2013   
Country:
Bolivia
PROJ ID:
P068968
Appraisal
Actual
Project Name:
Road Rehabilitation And Maintenance Project
Project Costs(US $M)
 284.0  451.0
L/C Number:
C3630
Loan/Credit (US $M)
 98.3  88.0
Sector Board:
Transport
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  04/16/2002
 
 
Closing Date
12/31/2007 06/30/2011
Sector(s):
Roads and highways (97%), Central government administration (3%)
Theme(s):
Infrastructure services for private sector development (40% - P) Rural services and infrastructure (20% - S) Other public sector governance (20% - S) Public expenditure financial management and procurement (20% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Ramachandra Jammi
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The objectives of the project, as stated in the Development Credit Agreement (DCA, page 22) and the Project Appraisal Document (PAD, page 2) are “to improve road transitability and accessibility, through the rehabilitation of key segments of the national and secondary road networks and strengthen the country’s capacity to manage its road assets.”

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
Component 1: Rehabilitation of about 353 km priority segments of the national road network (at appraisal: US$36.4 million; at completion: US$42.5 million). This component would finance the rehabilitation of two critical links of the primary paved network: 165 km of the Calamarca-San Pedro section located on National Route 1, a main corridor for export and import that connects the four regions of the country and links La Paz to Oruro; and 188 km of the Boyuibe-Yacuiba section located on National Route 9, linking Santa Cruz to the borders with Argentina and Paraguay, and providing continuity to the Abapo-Camiri highway whose rehabilitation was financed under a previous Credit (3235-BO). The works would essentially consist in the repair of all damaged areas, including the reconstruction of the most severely deteriorated pavement sections and shoulders, followed by the application of an asphalt concrete overlay of about 5 cm. thickness, with adequate horizontal marking and vertical signs.

Component 2: Pilot of rehabilitation of about 200 km of secondary roads (at appraisal: US$5.7 million; at completion: US$2.5 million). This component would be implemented in four selected Departments: La Paz, Oruro, Chuquisaca, and Tarija. Works would consist essentially in restoring the drainage system along key segments that link to the main economic corridors network, including the construction of small to medium-size bridges, followed by the re-gravelling of the surface course, at an estimated maximum cost of about US$ 25,000/km.

Component 3: Road maintenance and Resurfacing Program on the national road network (at appraisal: US$211.8 million; at completion: US$393.1 million). This component sought to expand and rationalize the practice of routine and periodic maintenance over the primary road network, by financing a portion of the costs incurred in the periodic maintenance sub-component, while the funding of routine maintenance activities would be absorbed by the resources of the Road Maintenance National Account or Cuenta Nacional de Conservación Vial (CNCV). A tentative program of about 600 km/year of periodic maintenance over the 5-year implementation period of the project was initially contemplated at appraisal, at a total estimated cost of about US$ 108 million (about US$ 36,000/km), of which US$ 34.3 million would be financed by the Credit. Resurfacing works would comprise surface-dressing and/or thin asphalt concrete overlays on sections belonging to the paved portions of the primary network, or re-gravelling and drainage system restoration on sections belonging to the unpaved portion of that network. The implementation of the component was expected to have a decisive impact on the riding quality of the national network, helping to increase the percent of road in good condition from about 19% at project start to a projected 41% at project end.

Component 4: Technical Assistance (at appraisal: US$28.3 million; at completion: US$10.2 million). Under this component, the project aimed to finance consultant’s assignments, equipment, and training in the following key areas:
(a) A Road safety initiative aimed at: (i) developing an institutional framework that would permit the proper management of road traffic safety issues such as the creation of an entity such as a National Road Safety Council; (ii) establishing an Accident Information System , concentrating initially in the two Departments that are most severely affected by road accidents, namely La Paz and Santa Cruz; and, (iii) carrying out a systematic auditing of new and/or existing roads to assess their vulnerability to accidents and recommend appropriate mitigating measures;
(b) Pre-investment studies, aimed to support, through the procurement of specialized consulting firms, the preparation of engineering studies for the rehabilitation and periodic maintenance programs, as well as for the pilot program of rehabilitation of secondary roads to be undertaken by the SEPCAMs (Servicio Prefectural de Caminos or Prefectural Road Agencies);
(c) Supervision of the rehabilitation and resurfacing works, including the pilot sub-component,, through the procurement of consulting services; and,
(d) Technical and financial audits, related to the use of the CNCV resources, the pilot program of rehabilitation of secondary roads, the accounts of the SNC (Servicio Nacional de Caminos) and ultimately the impact assessment of the implementation of the project.

Component 5. Road Sector Management and Institution Building (at appraisal: US$ 1.8 million; at completion: US$2.8 million). The activities financed under this component focused on three key areas:
(a) Maintenance programming; including the training in the use of the Highway Development and Maintenance Model (HDM) to help better plan, design and prioritize multi-year maintenance programs for the road network, as well as the carrying out of network condition and traffic surveys (including the procurement of specialized equipment);
(b) Procurement and contract management, through the development of adequate information systems, training and acquisition of appropriate equipment; and;
(c) Contract maintenance, to support the analyses, monitoring of and comparison between the current modalities of unit-price maintenance contracts while exploring the possibility of using performance-based systems, as well as the strengthening of the coordination between SNC and Vice Ministry of Transport in what relates to overall planning and definition of road investments.

On December 10, 2007, the Bank agreed to revise the project description to incorporate a new Category for emergency rehabilitation works located in the Beni region, due to severe flooding that affected, in February 2007, the transitability and structural integrity of four unpaved road sections of the primary network, totaling about 181 km (raising of embankments, spot re-graveling, and improvement in the drainage system). The target for rehabilitating national roads was increased from 353 kilometers to 533 kilometers to provide for 180 km of emergency works. Since such works fall within the broad category of road rehabilitation that the project originally intended to finance, the development objective remained unchanged.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: The final project cost of US$453 million was 46% higher than appraised cost of US$284 million. The difference in project cost was largely due to the emergency works in the Beni, for which the Credit Agreement was amended to reallocate funds among categories of disbursement, essentially from uncommitted contingency funds. In addition, expenditures were incurred for routine maintenance between 2008 and 2011 (beyond the original closing date) and financed with local funds. Actual periodic maintenance costs incurred amounted to US$61 million (for about 1,330 km) compared to an appraisal estimate of US$ 108 million (for 3,000 km).

Financing: Due to the weakening of the US dollar against Special Drawing Rights in which the Credit was nominated, the dollar value of the Credit rose from US$89.4 million to US$98.3 million. Of this, the Bank disbursed US$88.7 million and the balance of US$ 9.6 million was cancelled towards the end of the project. There were no other external sources of funding.
Borrower contribution. According to the ICR (Annex 1, Table b), the total Borrower contribution to the project was US$364 million, over 75% higher than the appraisal estimate of US$207 million. This was due to higher expenditures than foreseen on both routine maintenance and emergency works.
Dates: The closing date of the credit was extended twice: first for two years from December 31, 2007 to December 31, 2009 to provide for additional emergency rehabilitation works arising from exceptional rainfalls and extensive flooding particularly in the Beni region; and next, for an additional eighteen months till June 30, 2011, to allow for more progress in implementation. Among the reasons that led to the overall delay were: (a) social and civil disturbances, including road blockades, strikes and demonstrations that swept the nation, particularly from 2003 to 2006 which led to long gaps in the supply of diesel fuel in the country; (b) changes in world prices, between 2003 and 2007, notably for crude oil and its by-products (diesel, asphalt, lubricating oils) that caused delays in the execution of the overlays and a dramatic increase in the cost of works items that use asphalt products; and (c) poor performance of local contractors, due to their lack of equipment, lack of working capital, weak financial capacity and internal organization.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Bolivia has a territory of one million square kilometers and a small population base of about 8 million, and its road network is characterized by large distances between cities, low traffic levels, high transport costs, and high accident rates. At appraisal, the density of the road network and the percentage of road that was paved were among the lowest in the region: about 55km/1,000 square kilometers and 7 percent compared to 116 km/1,000 square kilometers and 20 percent respectively for the Latin America Region.

The Government perceived the state of the road sector as the key bottleneck to helping local producers become more competitive and to open market opportunities for the rural regions. For several years prior to project appraisal, the road sector had been the central focus of the government’s investment program, with over a third of the investment budget being devoted to it. However, the bulk of the road investment program was centered on new construction and paving programs that were not supported by adequate economic analyses and prioritization policies, while routine and periodic maintenance activities were severely underfunded.

Against this background, the project’s objectives were relevant in addressing the objectives of improving of road transitability and accessibility, and strengthening the country’s capacity to manage its road assets.

The project objectives were also aligned with the strategies relating to road infrastructure and economic development outlined in the Country Assistance Strategy (CAS) of 1998 and the Poverty Reduction Strategy Paper (PRSP) of 2001. The objectives remain consistent with the Bank’s latest Interim Strategy Note presented to the Board in June 2009, for fiscal years 2010 and 2011. On the part of the Government, the National Development Plan of June 2006 featured a strategy for road development and maintenance that sought the physical integration of the country in support of the productive sector, as well as provisions for pursuing the strengthening of democratic institutions and reducing poverty. This project was a follow-up to a previous loan that was granted by the Bank in 1999 for the restoration of a crucial link of the primary network, the Abapo-Camiri Highway.

The relevance of project objectives is rated high.

b. Relevance of Design:
The causal chain between the activities financed, the expected outputs, and the achievement of the development objectives is clear and convincing. To meet the development objectives, the project focused on (i) resolving the poor condition of the national road network, (ii) the lack of a sustainable mechanism for the financing of road maintenance, and (iii) the weak institutional capacity of the national and departmental road agencies. On the first front, restoring the structural integrity and riding quality of those severely damaged and heavily trafficked portions of the primary network would help reduce road user costs while allowing total traffic and commercial vehicles to continue increasing at a sustainable rate, thus facilitating the development of productive activities in their areas of influence. On the second front, establishing and financing a maintenance account would help build a platform for a more sustainable financing mechanism. On the third front, the pilot for rehabilitation for the secondary road network, as well as the broader technical assistance and institutional strengthening components of the project, would help overhaul the national and departmental road agencies, and bring about reforms in the management of road assets in Bolivia. The works would not only help increase the benefits accruing from the rehabilitation of segments belonging to the primary network, thus facilitating transport from low-income communities, but also serve as an incentive for the Prefecturas to engage in adequate road management practices under the National Road Agency or Servicio Nacional de Caminos (SNC) supervision and guidance.

The implementation of the component on road maintenance and resurfacing was expected to have a decisive impact on the riding quality of the national network, helping to increase the percent of road length in good condition from about 19% at project start to a projected 41% at project end.

Also, the project addressed the critical issue of road safety by initiating the definition and establishment of an adequate institutional framework.

The relevance of project design is rated high.


4. Achievement of Objectives (Efficacy) :

(a) to improve road transitability and accessibility of key segments of the national and secondary road networks. Rated Substantial.

Outputs: The project outputs that contributed to road quality at project completion included rehabilitation: completed for two key roads totaling 353 kilometers as planned, albeit with some less-than-optimal results regarding the surface quality for one of the roads, from Boyuibe to Yacuiba; resurfacing: this activity achieved only about 44 percent of its original target: at closing, about 384 km of the primary paved network had been resurfaced and about 947 km of the primary unpaved network had been re-graveled with substantial improvements in the drainage system; routine: maintenance: compared to the 80 percent appraisal target (representing about 8,000 km), 100 percent of the primary road network, i.e., about 16,000 km had been subjected to adequate routing maintenance. (the primary road network had increased from about 11,858 km to about 16,000 km during the project period due to transfers from the Prefecturas to the National Road Agency)

Outcomes

The percentage of the country’s paved primary roads in good condition was 59% in 2008 and improved to 84% percent in 2010 (there was no target specified for the paved portion, as explained in the first bulleted point below). The percentage in poor condition in 2008 was 5.5 percent. (Good and poor condition for these results are defined in terms of the International Roughness Index (IRI) measure being respectively less than 4 meters and more than 6 meters per kilometer). The ICR notes the following limitations/features of the results for road quality:
  • Comparability of baseline and achievements: The results pertain to the paved portion of the primary road network (the project team notes that 30% of the Primary Road network of 16,000 kilometers was paved as of 2011). The condition of the unpaved portion remains to be assessed. The baseline percentage of primary roads in good condition (19%) and the target (40%), which were set at project appraisal, covered both the paved and unpaved portions of the network. Also, the notion of “good condition” at appraisal was based on somewhat ill-defined visual and subjective surveys. Therefore comparison between baseline and achievements is hindered.
  • Attribution: In addition to the rehabilitation and resurfacing works financed under this project, the country’s roads also benefited from restoration operations financed with local and/or other donors’ funds, and to the aggressive upgrading and paving programs implemented by the Road Agency over the last 5 to 6 years of project duration. Overall, this project is judged to have made a major contribution to improving the condition of the country's paved primary roads through the scale of its investments in physical improvements as well as its provision of technical assistance.
  • Data sources: The percentages of primary network roads in good and poor condition in 2008 was derived from a survey by the ABC (National Roads Agency) Planning Department covering about 2,000 km of the primary paved network (i.e., about 50 percent of the total); and in 2010 was assessed by an international consulting firm on the totality of the paved portion of the national network of about 5.500 km. (The length of the primary paved network increased from about 3,500 km to nearly 5,500 km between 2002 and 2011)

The ICR notes that the improvement in the road network resulted in a large increase in freight and bus traffic along the two main links that were rehabilitated. However, it is likely that some of this growth can be attributed to the generally higher rates of economic growth experienced by Bolivia over the project period, compared to the years immediately preceding project appraisal. The results for passenger tariffs were mixed while the freight tariffs increased, even though fuel prices have remained generally unchanged over the project period.
  • Freight traffic: The number of trucks per day for the Calamarca-Oruro and Boyuibe-Yacuiba segments increased to 983/day and 476/day respectively, representing an increase of 71% and 24% over the corresponding revised targets established in December 2007.
  • Passenger traffic: The number of buses per day for the Patacamaya-Caracollo and Villomontes-Yacuiba segments increased to 876/day and 370/day respectively, representing an increase of 69% and 381% over the corresponding revised targets established in December 2007, indicating that the road improvements helped address a huge suppressed demand .
  • Freight tariff: Freight tariff for the La Paz-Orura and Boyuibe-Yacuiba segments at US$44.2 per ton and US$33.7 per ton, respectively was 33% and 8% higher than the revised targets set in 2007.
  • Passenger tariff: Passenger tariff per person between La Paz and Oruro (15.32 Bolivianos) was marginally higher than the target of 15 Bolivianos; and significantly lower for Santa Cruz to Yacuiba, at 37 Bolivianos against the target of 50 Bolivianos.

(b) to strengthen the Borrower's institutional capacity to manage road assets. Rated Modest.

Outputs/Intermediate Outcomes

National Road Agency’s capacity for planning, contract management, procurement and financial management: Capacity in these areas was strengthened although there were some moderate shortfalls. There was some progress in installing quality control management systems at the SNC (Servicio Nacional de Caminos), and ISO (International Standards Organization) 9001certification was in progress. A multi-year program of interventions for the mostly unpaved road network was established using the Highway Development and Management (HDM) model. The project also helped to procure an important stock of laboratory and road survey equipment to strengthen the capacity of the SEPCAM (Servicio Prefectural de Caminos or Prefectural Road Agencies) to carry out project design and supervision as well as network surveys. Network-based road maintenance plans and budgets were partially achieved in the Prefecturas of La Paz and Oruro out of four targeted prefecturas. The National Road Agency established a system of merit-based appointment of managerial and technical staff, and appropriate budgeting of salaries, but this progress was not maintained by its successor, the Bolivian Road Administration (known by its Spanish acronym ABC). A pilot rehabilitation of about 200 km of roads on the secondary network was below expectations. In Oruro, only two bridges were constructed while contracts for the rehabilitation of a 15 km-long gravel road and for the improvement of an 87 km-long unpaved road in Chuquisaca, made little progress due to weak financial and technical capacity of the part of contractors. In the Prefectura of La Paz, no civil works were undertaken.

Coordination between the National Roads agency and Prefecturas: There was little progress in developing an effective collaboration between the ABC and the four selected Prefecturas towards developing and implementing network-based road maintenance plans. Although formal agreements were signed between the national and departmental road agencies, there was not much implementation.

Road Safety Initiative: The Road Safety Initiative that aimed to reduce accident rates in Bolivia began well during the 2002-2006 period, but was dismantled soon after the new administration took office at the end of 2006. It was restarted between 2007 and 2010, with assistance from the Nordic Development Fund. Within the framework of the creation of a National Road Safety Strategy, a Supreme Decree (No. 29293), promulgated in 2007, approved the National Road Safety Plan and helped create the Inter-institutional Council of Road Safety. Also, a manual on traffic control including road horizontal marking and vertical signs was developed in 2008 with the financial assistance of the Andean Development Corporation. However, the project fell short of achieving the development and implementation of an Accident Information System in the Departments of La Paz and Santa Cruz, because of the poor performance of the consultant assigned to the task.

Outcome

Increased use of micro-enterprises with maintenance contracts for routine maintenance: The maintenance of the entire national road network (16,000 kilometers at project completion) is now carried out with micro-enterprises and unit-price maintenance contracts against a target of 90% of the 11,818 kilometer network at project commencement. Some 50 unit-price contracts covering individual lengths of about 300 km are currently in the hands of small to medium-size firms that carry out the recurrent maintenance of the pavements (pothole patching, crack sealing, grading of unpaved roads etc.), while some 486 micro-enterprises under the supervision of ABC regional engineers are taking care of miscellaneous activities, such as bush clearing and the cleaning of culverts and other drainage system features. The ICR notes that routine maintenance is fully funded though no specific numbers are provided. Periodic maintenance needs are only partially covered by sustained resources, mainly road user charges.

5. Efficiency:

The Economic Internal Rate of Return (EIRR) for (i) rehabilitation and (ii) for overlay resurfacing were 56% and 47% at completion, compared to 27% and >30% at appraisal respectively. The share of total project cost of rehabilitation and overlay resurfacing was respectively 17% and 74% at appraisal; and 10% and 87% at completion. The ex-post economic analysis was carried out using the Highway Design Model (HDM) model for the two main links rehabilitated under the first component of the project (Calamarca-Oruro and Botuibe-Yacuiba) and for a representative sample of the periodic maintenance (asphalt concrete overlays/resurfacing) executed over the paved portion of the primary network.

The higher rate of return obtained ex-post for the rehabilitation component is due to a combination of three factors: (i) the lower final cost of the works as compared to appraisal; (ii) the increase in traffic volumes that occurred since the appraisal; and (iii) the 20 year analysis period taken in the ex-post evaluation, compared to the 10-year period taken ex-ante (that failed to capture all the benefits of the project). A comparison for the periodic maintenance sub-component (which according to the PAD (page 13) mainly covered resurfacing activities) is difficult to make because the appraisal figure for the rate of return is not explicit (above 30%). However, the ex-post analysis confirmed that despite the increase in the cost of the overlays and because of an increase in traffic volumes since the appraisal, the average internal economic rate of return for that sub-component remains high at 47%

A comparison between the unit costs estimated at appraisal and those finally achieved for the main sub-components of works financed under the project shows that the rehabilitation of the two important links of primary paved network cost less than expected at appraisal (because of the small number of variation orders from design modifications prior to the dramatic increase in the cost of asphalt in 2005-2006). However, the other sub-components and particularly the resurfacing of the primary paved network suffered an important increase in cost. Such overruns were due to: (i) significant variation orders and increases in the quantities of items of works, resulting from modifications to the original designs arising from shortcomings in financial management, procurement capacity and oversight of consultants and contractors, and (ii) the surge in the cost of asphalt during a protracted execution period of the works, from 2004 to 2011. On balance, the increase in the unit cost of the physical sub-components of the project relative to the original contract values amounted to about 29 percent.

Other factors that affected efficiency (also discussed under borrower performance) included delays in restructuring the implementing agency (the National Road Agency, SNC) in 2006 that resulted in one-and-a-half years of project inactivity; and frequent turnover of the implementing agency leadership. Also, as explained in section 2d, the project experienced a total time overrun of three and a half years due mainly to additional emergency rehabilitation works arising from exceptional rainfalls and extensive flooding in early 2007; and social and civil disturbances during 2003 to 2006.

On balance, efficiency is rated substantial.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
27%
90%
ICR estimate:
Yes
47%
90%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

Relevance of both the project’s objectives and its design is rated high. The efficacy of the objective relating to transitability and accessibility is rated substantial due to improvements in road quality and an increase in freight and passenger traffic. The second objective relating to the capacity of the road agency is rated modest due to some shortcomings. Efficiency is rated substantial. Overall outcome is rated moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

Budgetary resources allocated to ABC (National Roads Agency) for road maintenance and rehabilitation have doubled in nominal terms from US$115 million in 2002 to US$238 million in 2008. The rolling five-year program of investments for 2011-2015 should allow for a total annual budget of about US$ 500 million that would be used to: (i) increase by 40% the length of the paved network; (ii) rehabilitate and/or resurface about 2000 to 3000 km of existing paved roads; (iii) routinely maintain the entire primary road network, including patching of all potholes and improving the horizontal markings and vertical signs over nearly 7,500 km; and (iv) grade and re-gravel the unpaved portion of the national road system.

The Bank continues to provide support to Bolivia through the National Roads and Airport Infrastructure Project (2011-2016) which involves rehabilitation/upgrading of a key segment of the national primary network: the 114 kilometer San Buenaventura-Ixiamas road. In addition to the physical component the project design includes a sustained institutional strengthening of ABC, including extensive training in contracting, procurement and financial management.

However, the ICR adds that despite these positive aspects, sustainability may be at risk if the government pursues and applies the policies that it has recently embraced, including: (i) the empowerment of ABC’s regional offices, by transferring to them the responsibility for design, supervision, and contract management of road programs, while there is no evidence that regional capacities currently exist; (ii) the re-activation of a force-account unit to substitute private firms in breach of their contracts; and (iii) the reduction in earmarked funding from the gas/petrol taxes for routine maintenance (that is, to the Road Maintenance National Account). Additionally, ABC’s staffing and general institutional weaknesses (as discussed in section 9b below) need to be addressed, if the gains from the project are to be sustained. The huge growth in traffic (especially truck traffic, which was way over forecasts, represents a considerable repressed demand. This will also increase routine and periodic maintenance requirements.

Overall, the risk to development outcome is rated significant.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
Preparation and appraisal were thorough and responsive to the country’s needs, and some of the project risks were correctly diagnosed. The concerns of the communities located in the areas of influence of the project were taken into account through a social assessment carried out by a specialized consultant.

The project appropriately identified several risks and included measures in the project design to mitigate them: (i) delay in implementation due to frequent turnover of managerial staff the project executing agencies (appraisal was delayed till institutional reforms in the civil service were well underway); (ii) the possibility that institutional reforms would not take root (a strategic development plan was undertaken in collaboration with a wide spectrum of stakeholders to help internalize the strategies and concepts); (iii) a lack of commitment and enforcement of a sustainable financing mechanism by the central government; (iv) weak financial and execution capacity of local contractors, (the last two risks were to be mitigated through financial support to CNCV (Cuenta Nacional de Conservación Vial ) and by enhancing the quality of sub-project design, bidding documents, and works supervision.). The mitigating measures were on the right lines, but did not go far enough to address the considerable risks faced by the project.

The project design required SNC to have a project management and coordinating unit in place throughout project implementation and this unit was initially set up with a team including a civil engineer with international experience in managing road projects. As indicated by the project team (email dated May 22), the political developments in the country in 2005-06 led to a major restructuring of the national road agency, during which technically qualified staff was removed from the unit, a development that could not have been anticipated by the Bank at the design stage.

The choice of some project performance indicators could have been improved. For example, the road surface condition assessment for paved and unpaved roads was bundled together; and the choice of transport tariffs as an indicator appears predicated on the simple assumption that riding quality of the network is the single most important or unique factor that impacts tariffs. The project team states that developing capacity for comprehensive road inventory and objective pavement surface surveys (and improving upon visual inspection of paved and unpaved roads bundled together) was indeed one of the issues that the project tried to address.

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:
The Bank performed its supervision responsibility well from project effectiveness to completion. Supervision missions were regular and timely, and became more intense and frequent between 2005 and 2010 when the project was at risk, focusing on implementation problems and their solutions. The Bank displayed flexibility in granting two extensions of the closing date and in keeping the project open for a long time, thus allowing it to produce tangible physical achievements. Also, the Bank showed intense engagement with the Administration when it confronted the highly sensitive reform of the road agency (among others, this was due to the opposition from the new administration in the Ministry of Finance to the reform in staff salaries and incentives; and lack of commitment and enforcement of a sustainable financing mechanism by the Central Government), in particular with the swift deployment of a consultant with ample experience in the restructuring of road sector entities. Immediately after, the Bank again provided support in restructuring the Credit Agreement to include an additional emergency sub-component to counteract the impact of the natural disaster that hit the country in early 2007. Finally, to the Bank’s credit, the training and the application of the HDM model was well received by the Borrower, and in particular the assistance provided in developing the multi-year program of road maintenance for the Prefecture of La Paz.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Government had a clear concept of its development objectives in the highway sector and the assistance expected from the Bank. However, it failed to recognize the limitations of its capacity to implement the project and underestimated the complexity of institution building. Furthermore, and despite several instances when the Bank's supervision missions called the Borrower’s attention to the problem, adequate local funding needed to carry out the periodic maintenance component remained an intractable problem throughout project implementation.


The highly politicized institutional restructuring process of the implementing agency (the National Road Agency, SNC) following the change-over of the administration in 2006 hindered decision-making and resulted in, at least, one-and-a-half year of project inactivity. Indeed, soon after the new administration took office in January 2006, the intention to liquidate the SNC was announced, and through the promulgation of Law 3507 and the enactment of four regulatory decrees in October-November 2006, the new National Road Agency, the Administradora Boliviana de Carreteras (ABC) was created. Because of the slow progress in the institutional transformation process, the project remained practically idle until June 2007. Also, the agenda for reform in staff salaries and incentives met stern opposition from the Ministry of Finance of the administration that took over in 2006.

The lower-than-expected level of counterpart funds assigned to the Road Maintenance National Account, and insufficient contributions recovered from fuel taxes and tolls, adversely impacted the scope and implementation of the periodic maintenance of the national highway system.

Government Performance Rating: Unsatisfactory

b. Implementing Agency Performance:
The original implementing agency, SNC, was liquidated and was replaced by ABC in 2006, four years after project commencement. Weak institutional capacity in both SNC and ABC translated into: (i) poor financial management of the works contracts, including delays in payments to contractors; (ii) lack of procurement staffing and management capacity that caused delays in bidding and awarding contracts, with frequently late requests for Bank’s no-objections; (iii) inadequate monitoring and evaluation of the road network condition and traffic volumes; and (iv) inefficient oversight of consultants, both at the design and supervision stage, that caused a considerable quantity of project modifications and variation orders, during contracts execution.

Additionally, staffing quality inadequacies, as well as high rotation and turnover both at the national and departmental level, hindered coordination between the various agencies, and affected technical support that had to be provided by the national road agency to the Prefectural Road Agencies. Since the creation of ABC, the General Director has changed four times between 2006 and 2010, and the Head of the Road Maintenance Department has changed six times. However, despite all the constraints, the Road Agency undertook an aggressive upgrading and paving programs over the last 5 to 6 years of project duration that was reflected in the outcomes as discussed in section 4.

Nevertheless, the executing agency performed well with respect to the routine maintenance sub-component of the project.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The outcome indicators for M&E were not defined or spelt out in clear terms, which made it difficult to measure and compare them across time periods. The key outcome indicator for the transitability of the national road network was expressed in terms of “percent of roads in good condition” without initially defining the meaning of “good condition” and without discriminating between the paved and unpaved portions of that network. Likewise, the improvement in the transitability and accessibility of the two segments to be rehabilitated was expressed in terms of “an increase in the daily volume of trucks and buses”, but without clear definition of duration, location and period of the traffic surveys. Also, the sector-related outcome indicator, intended to measure the improvement in transport interconnection across Bolivian regions, was expressed in terms of “freight and passenger tariffs per ton or per passenger” without a clear description of the methodology to be used to estimate those tariffs, and without considering that such tariffs are not solely dependent on road surface condition but also on external factors that are beyond the control of the project, such as the volatility in the prices of fuel, diesel and vehicle spare parts, and the degree of competition in the transport services industry. Baseline data was specified for the outcome and output/intermediate outcome indicators. The National Road Agency (SNC and later ABC) was responsible for M&E.

b. M&E Implementation:
During project implementation, the National Road Agency failed to carry out annual road network condition surveys systematically. Therefore, the improvements in the riding quality resulting from the rehabilitation and resurfacing works under the project could not be properly monitored. The only reliable road surface condition surveys were carried out first in 2008, covering 50 percent of the length of the paved primary network, and at the very end of the project, in 2010, by an international consulting firm which made an objective inventory of the condition of the entire national road network. Likewise, albeit to a smaller extent, traffic data that had been systematically collected between 1991 and 2003 was interrupted in 2004 and 2005, as well as between 2007 and 2009. For the earlier period, these interruptions in regular data collection were most likely a result of the social-political upheaval at the time, while the latter was because of the institutional transition from SNC to ABC The project was instrumental in the preparation and public dissemination of annual report on operational and financial performance of the Road Maintenance National Account between 2002-2010.

a. M&E Utilization:
Given the shortcomings in its design and implementation, there was little scope for the M&E mechanism to be utilized as an effective feedback mechanism to correct deviations in expected outputs and outcomes from project activities

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
The project was classified as an Environmental Assessment Category ‘B’ project and triggered the Environmental Assessment (OP 4.01) and Indigenous Peoples’ (OP 4.20) safeguard polices. Because the project would finance the rehabilitation and/or resurfacing of roads within their existing right-of way, including the overall routine maintenance of the national network, no displacement of people was expected, and no significant negative environmental impacts were anticipated. However, during appraisal and throughout project implementation, the National Road Agency carried out adequate screening analysis of all sub-projects financed under the Credit Agreement to ensure compliance with the Bank’s guidelines. Those screenings were based on an environmental Manual that was prepared especially to that end, including for the use of contractors executing the works. In addition, social assessment studies were also carried out for the key rehabilitation sub-projects, indicating ample acceptance of such works by the communities located in their areas of influence, as in the case of the Guarani and the Weenhayek indigenous communities located near the Boyuibe-Yacuiba segment. Throughout project implementation, supervision reports mentioned no significant safeguard policies or fiduciary issues, and the rating for "compliance with Bank’s policies in that regard was always satisfactory." as stated in the ICR.

b. Fiduciary Compliance:
Financial management. At appraisal, an assessment of the financial management capacity of the Road Agency (SNC) concluded that it satisfied the Bank’s minimum requirements. In addition, in November 2002, the establishment of a computerized financial management system fulfilled one of the conditions for project effectiveness. Nevertheless, throughout implementation, the financial management capacity of the national road agency was consistently rated as only moderately satisfactory, with shortcomings including late payments to contractors, and untimely submission of periodic financial reports and annual audit reports. The project team confirmed that there were no qualified audits

Procurement. During project preparation, in 2000-2001, a procurement assessment of the National Road Agency described SNC as a high-risk institution. This assessment was not based on any major procurement issues that had taken place in the past, but essentially because of the very slow rate of implementation in the preparation of bidding documents, evaluation of proposals, and award of contracts, generally due to frequent staff reshuffling and their lack of expertise. In 2002, an action plan designed to enhance the procurement function at the SNC and subsequently ABC, including intensive training and hiring of competent personnel, helped improve the procurement capacity to a level acceptable to the Bank to proceed with project implementation. There were no cases of misprocurement.

c. Unintended Impacts (positive or negative):

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Moderate
Significant
Concerns relating to institutional capacity in the National Road Agency's regional offices; option to revert to use of force-account in case of nonperformance of private contractors; and reduction in earmarked funding for routine maintenance. 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Unsatisfactory
Shortcomings arising from the highly politicized institution restructuring process; weak institutional capacity of the national roads agency  
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following lessons are adapted from the ICR.
  • Institution-building efforts should not be limited to an ‘enclave’ in the implementing agency, in isolation from the larger institutional and country context. In this project, institution building was largely restricted to one Ministry, in isolation from the general country context limiting the scope for improving capacity in road administration units across the country.
  • The scope and complexity of a project should be commensurate with the existing institutional capacity, and a realistic estimate of how much this capacity might improve in response to project interventions. This project took on several activities on the physical improvement and institutional fronts, having taken an optimistic view of the pace of institutional improvement within the government and the highway agency leading to less than expected outcomes.

14. Assessment Recommended?

Yes
Why?
This project has been carried out in a difficult and evolving institutional environment, and with many physical and financial challenges. Such situations are frequently to be found in transport and other projects in several client countries. Deeper analysis of the project experience and an update of the sustainability of the project outcomes can be of use to others.

15. Comments on Quality of ICR:

The ICR is written in an informative and analytical manner. It made a good effort to put together information and evidence for its arguments given the relatively weak framework for data collection. The ICR discussion devotes adequate attention to the project outcomes, alongside the discussion of outputs and intermediate outcomes. Some ratings were optimistic, but the lessons drawn are balanced and objective. The length of the ICR is reasonable. Overall, the ICR quality is rated satisfactory.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions