Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - National Highway Asset Management


  
1. Project Data:   
ICR Review Date Posted:
08/15/2011   
PROJ ID:
P088153
Appraisal
Actual
Project Name:
National Highway Asset Management
Project Costs(US $M)
 308.1  697.8
Country:
Argentina
Loan/Credit (US $M)
 200.0  197.9
Sector, Major Sect.:
: Central government administration, Roads and highways
Cofinancing (US $M)
 0  0
Theme(s):
Infrastructure services for private sector development (50% - P) Injuries and non-communicable diseases (25% - S) Administrative and civil service reform (25% - S)      
L/C Numbers:
L7242
Board Approval (FY)
  2004
Partners involved
 
Closing Date
12/31/2008 08/31/2010
         
Evaluator: Panel Reviewer: ICR Review Coordinator: Division:  
Kavita Mathur
Ridley Nelson IEG ICR Review 1 IEGPS1

2. Project Objectives and Components:

a. Objectives:
This project was the first operation of a two stage APL supporting a two-phase, multi-year program. According to the Project Appraisal Document (PAD, page 12), "The overall purpose of the program is to gradually consolidate an efficient road network management strategy, bringing about the the necessary resources to preserve the national road network in the long term."

According to thePAD (page 12), the project's development objectives (PDO) were to:
(a) preserve the condition of vital road assets, through the gradual expansion of performance based contracts for the rehabilitation and maintenance of the non-concessioned primary paved network; and (b) strengthen road sector management through carrying out a renewal program of the National Highways Directorate (DNV) to revitalize its role in the sector by: (i) reinforcing its human resource base, (ii) consolidating its transformation into a results oriented organization accountable for specific outputs; (iii) introducing a systematic approach to bridge management and (iv) enhancing road safety.

The PDO according to the Loan Agreement were to:

(a) further preserve the non-Concessioned national road network, through the gradual expansion of the Performance Based Rehabilitation and Maintenance Contracts (CREMA) in said network; and

(b) strengthen DNV’s road sector management capacity.

This review uses the PDO as stated in the PAD as they are more monitorable.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components (or Key Conditions in the case if Adjustment Loans):
The project was formulated as a two-phased Adaptable Program Loan (APL) to be executed over a total period of six years (2004 to 2010), the first phase (APL1) from 2004 to 2008 and the second phase (APL2) from 2006 to 2010.
Component 1 – Rehabilitation and Maintenance through performance-based contracts (Appraisal cost US$286.1 million, Actual Cost US$682 million). This component would finance the implementation of CREMA contracts.

Component 2 – Bridge Restoration (Appraisal cost US$10.7 million, Actual cost US$6.6 million). This component would finance works on five bridges pre-selected by DNV due to their critical status and urgent need of intervention. Works would include construction, reconstruction, replacement, widening and rehabilitation activities. Also, a comprehensive bridge management tool to systematically prioritize bridge interventions would be implemented, and engineering designs for future bridge restoration projects prioritized through this tool would be financed under the project.

Component 3- Road Safety (Appraisal cost US$5.8 million, Actual cost US$6.5 million). This component would finance: (i) the design and implementation of a large-scale road safety pilot program in four corridors of the national road network and the preparation of engineering designs for the upgrading works in the proposed corridors; and (ii) road safety engineering interventions (including engineering designs) on six identified critical spots or intersections of the road network across the country, as well as the horizontal pavement marking on approximately 310 km.

Component 4- Institutional Renewal (Appraisal cost of US$3.5 million, Actual cost US$1.74 million). The project would provide technical assistance to: (i) identify DNV’s needs in terms of human resources and implement a short-run strengthening plan; (ii) facilitate DNV’s technological modernization; (iii) streamline time-consuming administrative processes; (iv) strengthen DNV’s capacity for addressing environmental issues; (v) design transparency mechanisms to disseminate DNV’s performance, (vi) analyze options for ensuring long term financial sustainability in the road sector and stable flows of funds for road maintenance, and (vii) strengthen the Project Coordination Unit (PCU) to assist DNV in the implementation of the project.

Revised Components. No revisions were made to the original components of the project.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Costs and Financing: The actual total project cost were US$697.8 million. This was more than double the appraisal estimate of US$308.1 million. The ICR reports that the financing gap of US$ 293.6 million was financed through additional funds from the Treasury and from the follow-up project (APL2) Bank Loan. The increase in costs was related mainly to the changes in world market and prices (particularly increase in the cost of petrol and associated by products such as asphalt and fuels). The cost increase for road works was persistent during project implementation, averaging about 1.3% per month. Between January 2004 and December 2009 the cost of road works in Argentina increased by about 90%.
Borrower Contribution: The Borrower contribution almost doubled from US$108.1 million to US$206.3 million.

Dates: The closing date was extended twice to facilitate implementation of the institutional and bridge restoration components.


3. Relevance of Objectives & Design:

Objectives: rated high. The project objectives supported the main pillars of the 2004 CAS: social inclusion and improved governance. The CAS also emphasized that one of the building blocks for sustained economic growth was maintaining and improving key infrastructure assets that facilitated regional development and the inclusion of Argentina in a global economy. Expanding the CREMA program to other regions of the country was expected to decrease disparities in terms of transport investment by targeting poorer regions and provinces. The project objectives also focussed on initiatives to improve governance, accountability and transparency in the roads sector. The project objectives remained relevant throughout the life of the project and are consistent with the latest 2010 Country Partnership Strategy which aimed at upgrading Argentina’s infrastructure to address potential bottlenecks to competitiveness.
Design: rated high. The project incorporated lessons from previous Bank-financed operations. Problems of cost overruns and delays in project execution that were in the past attributed to inadequate or untimely designs or variation orders commonly observed on traditional contracts were expected to be addressed through the expanded use of the CREMA system. To promote institutional development activities the project design focused on a few critical achievable goals and allowed for additional flexibility to accommodate evolving needs through the selection of the APL lending instrument. A range of risks were identified during project preparation and appropriate mitigation measures were considered. For example the risk relating to the Government’s ability to expand the CREMA program due to fiscal constraints was to be mitigated by defining expansion targets according to historic budget allocations and adapting standards of rehabilitation to the conditions of different segments of the network, as well as by supporting a road sector financing study. The risk of DNV’s lack of support and/or ownership for the institutional renewal process was to be mitigated by designing a gradual agenda, preparing and implementing Twinning Arrangements, and designating high-level staff and a steering committee responsible for coordinating the process. The monitoring system and safeguards assessment was adequate.

4. Achievement of Objectives (Efficacy) :

Preserving the condition of vital road assets, through the gradual expansion of performance based contracts for the rehabilitation and maintenance of the non-concessioned primary paved network: substantial.
The number of kilometers of CREMA contracts in execution at project closing was 12,664 km. This represented 89 percent of the project target. Of these 12,664 km, 5,163 km (or 40.7 percent) were located in the poorest provinces of the country, thus meeting the appraisal target of 40%.

The rehabilitation and maintenance works carried out under the project helped in preventing further deterioration of the non-concessioned network. At appraisal, the target for the International Roughness Index (IRI) was to be below 4 throughout the project implementation period. The target was not only met but the average IRI declined from 3.6 in 2006 to 3.1 in 2009. The better-than-expected results are due to the thicker overlays used for the locally financed CREMA contracts. However, this reduced the total length of roads rehabilitated under the locally-financed CREMA system from 1,192 km to 1,126 km.

The project also reduced the maintenance backlog from 16.1 percent in 2004 to 5.4 percent at the end of 2009.

A large majority of the overall road network in the country was in "good" or "excellent" condition:
2004 (%)
2009 (%)
% of roads in Good condition (IRI<4)
83.8
94.6
% of roads in Excellent condition (IRI<3)
24.8
44.4
% of roads in Poor condition (IRI>5
4.9
0.5

Strengthening road sector management through carrying out a renewal program of the National Highways Directorate (DNV): modest.

As the ICR points out (page 12), "Progress achieved under the objective to renew and revitalize DNV’s role in the road sector has been mixed, even though a number of important actions have been carried out during the final years of project implementation to address some critical issues identified at appraisal. About 70 percent of the original scope of the Action Plan has been achieved, with some of the activities shifted to the second phase of the program." A number of the actions foreseen in the PAD (Annex 4) for this first phase project were carried out only partially or not at all, and have been shifted to the now ongoing APL 2. These actions concern especially the implementation of the different strategies and plans based on the diagnoses that were completed under this project. There were, nonetheless, some achievements which will facilitate the further implementation of the program, including a detailed diagnosis of DNV's current staff capabilities, training of the staff in road planning, road safety and winter maintenance, partial web site publication of DNV's performance indicators, the design of a bridge management system and its implementation in six provinces, and the improvement of four road safety black spots. In addition, an Environmental Manual was updated and training provided to DNV’s environmental unit staff.

Despite the modest achievement of the second objective, overall progress towards attaining the goals of the two-phase program was satisfactory, and the targets for physical improvement of the network's maintenance and preservation were fully met or exceeded.


5. Efficiency (not applicable to DPLs):

CREMA Component. The ICR calculated the ex-post ERR for the CREMA component using the Bank's HDM model. The ERRs are 48% compared to the appraisal weighted estimate of 40%. Although, the actual cost of the CREMA program almost doubled compared to the appraisal estimate, the ex-post ERRs are higher due to the following reasons:
(a) the thicker and greater coverage of overlays used for the locally financed CREMA, as compared to the appraisal design, resulted in much lower roughness value after rehabilitation (typically 1.8 IRI). This resulted in a much lower vehicle operating cost than initially anticipated; and

(ii) The cost of operating a vehicle in Argentina (costs of new vehicles, tires, spare parts, fuel, and wages) increased by a much larger factor (between 150 and 200 percent), and as a result the net benefits accrued from carrying out the project, as compared to the without-project case, were also much larger than anticipated at appraisal.

According to the study comparing the unit costs of the rehabilitation works of the CREMA to those obtained from the traditional unit-price contracts in Argentina, the study found that CREMA contracts are on an average 15 percent lower than traditional contracts.

Bridge Restoration component. The five bridges actually rehabilitated under the project (which were about 1% of the total project cost) were different from those identified at appraisal. The ex-post ERR is 21% compared to an appraisal estimate of 38%. The decline in economic returns is explained by the increase in the unit cost of rehabilitation works, from US$ 11, 200/linear meter to US$ 26,800/linear meter and from modifications in the design standards.

Overall, efficiency is rated substantial.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:



Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
Yes
40%
98%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The objectives set out at the time of appraisal remained highly relevant not only throughout the project implementation period but also to the pillars of the most recent Country Partnership Strategy. Design relevance was also high. The project was largely executed as planned and substantially achieved the development objective of preserving the condition of vital road assets, albeit with some delays. Despite the significant increase in project costs that was largely due to price increases and enhanced rehabilitation standards, the economic efficiency of the project-financed investments remained substantial. Overall, outcome is rated satisfactory.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating :

The CREMA model has been an important tool to maintain roads in good condition. However, there are some factors that could jeopardize its effectiveness and the risk to development outcome is rated moderate. These risks include: (i) lack of timely preparation and tender of new bids; and (ii) DNV’s tendency to adopt enhanced rehabilitation standards and unrealistic routine maintenance unit costs in the CREMAs financed by the government. The ICR reports that adopting unreasonable quantities for maintenance activities such as pothole patching and crack sealing could add up to about US$ 2 million for a typical 150-km long CREMA contract.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

Quality at Entry: Preparation was generally thorough and the design of the components to address the institutional weaknesses and to improve road safety were good. Lessons learned from previous operations were duly incorporated in the project design and appropriate measures to mitigate the risks identified were incorporated. However, there were some minor shortcomings at appraisal namely: (i) the overoptimistic targets regarding the pace at which the institutional renewal plan for DNV could be implemented; and (ii) the lack of a specific and well-defined target regarding the number of kilometers of new CREMA to be implemented during the first phase of the program.
Supervision: On an average two supervision missions per year were carried out. Also there was a mid-term review and ISR's were appropriately rated. The Bank found a pattern of higher than expected bid prices for the ten CREMA contracts and a limited price variation among the bids and decided to carry out independent technical audits of the CREMA contracts under execution. Following the review the Government decided to cancel the bidding process and postpone all subsequent bidding. This led to the design and execution of a comprehensive action plan which helped to resolve the procurement problem. During IEG's discussion with the task team, it was noted that the team, through a lot of effort and diligence, managed to convince the Minister of Economy and the Minster of Infrastructure that Bank financed CREMA contracts were cost effective and that both Bank financed and DNV financed CREMA contracts should have the same standards. In the ongoing APL 2, these standards are being applied. The main shortcoming of supervision was that the focus on the implementation of the first PDO contributed to the relative lack of progress on the second until a late stage of project implementation.

a. Ensuring Quality-at-Entry: Satisfactory

b. Quality of Supervision: Satisfactory

c. Overall Bank Performance: Satisfactory

9. Assessment of Borrower Performance:

Government Performance. The Government demonstrated its commitment to the objectives of the project and met its counterpart funding obligations. It increased the budget allocations to DNV throughout project implementation with annual allocations being five times the historical level in dollar terms. This helped in mitigating the effect of inflation on the cost of road works. In response to higher than expected bid prices for the ten CREMA contracts and a limited price variation among the bids the Government decided to cancel the bidding process and postpone all subsequent biddings under the project until an in-depth analysis was carried out. The Government took timely actions to process the approval of the second phase of the project (APL2).
Implementing Agency Performance. During project preparation, DNV's performance was satisfactory. It carried out surveys that enabled the design of the first round of CREMA bids. DNV adequately implemented the Action Plan designed to address the procurement issues raised by the 2005 in-depth analysis of the bidding process. DNV also carried out the annual road condition surveys to monitor the progress and achievement of the M&E key indicators.

DNV adopted an overly conservative overlay design that resulted in higher costs of routine maintenance for the CREMA contracts financed with Treasury funds. The standards for Bank financed CREMA contracts were overlay thickness of about 4 to 5 cm and routine maintenance (unit costs US$ 4,000 and US$ 5,000/km/year) compared to 6 to 7 cm for the Treasury financed CREMA contracts (unit cost US$7,000/km/year to 10,000/km/yr). The higher design led to reduced km covered by the Treasury financed CREMA contracts.

According to the mid-term review (November 2006) DNV had concentrated most of its efforts on solving the procurement and implementation issues affecting the project’s main component (the CREMA) while lagging behind on other smaller components. For example the rehabilitation of 5 bridges had not started; consulting firms for the design of the road safety corridors had not been procured, the bidding documents for the works on the critical black spots had not been finalized, the human resources study had not been awarded, and Code of Ethics and development of road user satisfaction indices had not started. However, the ICR reports that by project closing, most activities were executed in a satisfactory manner, including the preparation and formal adoption on November 16, 2010, of a Code of Ethics for the institution.

a. Government Performance: Satisfactory

b. Implementing Agency Performance: Satisfactory

c. Overall Borrower Performance: Satisfactory

10. M&E Design, Implementation, & Utilization:

M&E Design. M&E was appropriately designed to monitor the progress toward achieving the project development objectives. To first PDO relating to the preservation of the condition of vital road assets was to be monitored through: (a) the number of kilometers of CREMA contracts tendered, awarded and in execution under the project; and (ii) roads in good condition as measured by “the International Roughness Index (IRI)”. The target was to keep IRI below 4 throughout project implementation. The second PDO relating to the strengthening of the road sector management was to be monitored by specific implementation schedules that would specify defined products and due dates for completion. This included implementation of the Institutional Renewal Dated Action Plan. There were no indicators for measuring the level of results orientation.
M&E Implementation. During project implementation, appropriate data was collected to monitor progress, such as the annual traffic and road network condition surveys (including the measurement of roughness - IRI) using reliable methods and equipment. To measure the progress made under the institutional strengthening component, semi-annual progress reports were prepared by DNV. The ICR reports that the data collected was used to inform management on issues and/or on progress achieved under each component, and helped decision-makers take appropriate actions aimed at ensuring successful implementation of the project.

M&E Utilization. There is limited evidence on utilization. Information gathered was coordinated and tracked through supervision and quarterly reports. The data were utilized by the Bank management to take appropriate actions during project implementation.

a. M&E Quality Rating: Substantial

11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts):

Safeguards. At appraisal, the project was categorized as “Category B” because the main civil works consisted of rehabilitation and maintenance of existing roads, and no significant negative environmental and social impacts were expected. An Environmental Assessment of the project was carried out as per Bank guidelines. The Environmental Unit within DNV was responsible for the mitigation of potential risks. No resettlement was expected and no social safeguard policy was triggered. The ICR reports that during project implementation the project remained in compliance with the Bank’s safeguards policies. However, no details were provided.
Fiduciary. The ICR reports that during implementation the financial management of the project remained in compliance with Bank procedures. No information was given about audit findings.

Unintended Impacts. No unintended impacts are reported in the ICR.



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Satisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Satisfactory
 
Quality of ICR:
 
Satisfactory
 

13. Lessons:

The main lessons adapted from the ICR are:
  1. The preliminary designs for Performance Based Rehabilitation and Maintenance (CREMA) Contracts need to have rehabilitation and maintenance standards commensurate with surface and traffic characteristics of the network. This would avoid unnecessarily over-designed solutions that could jeopardize cost-efficiency.
  2. In addition to regular supervision, independent technical audits are a valuable tool for monitoring the performance of contractors. During the execution of the CREMA program, the technical audits can: (i) verify that works are being executed in accordance with the contractual program of activities in terms of progress and quality; (ii) detect and report, through additional testing any deficiency or non-compliance with specifications; and (iii) recommend any measures or actions deemed necessary to improve the design, costing, and execution of the CREMA contracts.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is of satisfactory quality. It gives a generally adequate description of the progress of the project. The economic analysis is quite thorough. However, no information was given about audit findings and no details were provided on the safeguards implementation experience.

a. Quality of ICR Rating: Satisfactory

(ES-Rev5-Jul/06)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions