Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Road Sector Program Support Project


  
1. Project Data:   
ICR Review Date Posted:
04/29/2014   
Country:
Moldova
PROJ ID:
P100929
Appraisal
Actual
Project Name:
Road Sector Program Support Project
Project Costs(US $M)
 48.7  72.2
L/C Number:
C4283
Loan/Credit (US $M)
 16.0  5.0
Sector Board:
Transport
Cofinancing (US $M)
 32.7  62.2
Cofinanciers:
EBRD; EIB
Board Approval Date
  03/29/2007
 
 
Closing Date
06/30/2011 06/30/2011
Sector(s):
Roads and highways (81%), Central government administration (19%)
Theme(s):
Trade facilitation and market access (33% - P) Infrastructure services for private sector development (33% - P) Injuries and non-communicable diseases (17% - S) Administrative and civil service reform (17% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Victoria Alexeeva
Soniya Carvalho Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:


    The statement of objectives in the Project Appraisal Document (PAD) and the Financing Agreement (FA) is essentially the same:

    FA: "to reduce road transport costs for road users in the road network covered under the project by improving the condition and quality of said network and its management".

    PAD: "reduce road transport costs for road users in Moldova, by improving the condition and quality of its road network and the way it is managed".

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

Component 1: Road Network Recovery (appraisal cost US$44 million, US$13 million of Bank financing; actual cost US$67.2 million) included carrying out rehabilitation works on selected road sections along the main North- South road corridor between Balti, Sangerei, Orhei, Chisinau, Hincesti and Comrat.

Component 2: Institutional Support (appraisal cost US$4.7 million, US$3 million of Bank financing; actual cost US$5 million) included (i) support to improve the Recipient’s capacity to plan, manage and maintain the network of national roads in Moldova, including activities such as the development of a reliable road maintenance financing mechanism, improvements to the system of road maintenance execution and contracting, and implementation of a system designed to curb the circulation of overload trucks; (ii) support to strengthen the capacity of the Project Implementing Entity to efficiently manage and maintain the roads under its responsibility and to manage investment programs for road rehabilitation, reconstruction and construction.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project cost: Total project cost was US$72.2 million at closure, 33% higher than the appraisal estimate of US$48.7 million due to the increased scope of engineering work and associated costs.

Financing: The World Bank Group contribution consisted of a credit in the amount of US$16 million, which was reduced to US$5 million at closure; US$11 million were cancelled due misprocurement (see section 11b below). The appraised amount of co-financing was 25 million (US$ 32.7 million equivalent) divided equally between the European Bank for Reconstruction and Development (EBRD) and the European Investment Bank (EIB) (12.5 million each). The amount increased with additional 35 million during implementation (17.5 million each), out of which 12.5 million of EBRD's loan were cancelled due to misprocurement. At closure, the actual amount of co-financing was 47.5 million (US$62.2 million equivalent).

Borrower contribution: There was no Borrower contribution.

Dates: The project had a two- year delay in implementation caused by misprocurement. The Bank-funded portion of the project closed as appraised on June 30, 2011; the EBRD and EIB completed the project main civil works in 2013. During restructuring of December 2009 caused by mis-procurement, US$11 million of Bank credit were cancelled and US$2 million left under Component 1 were re-allocated to Component 2. From 2009 till credit closure in 2011, the Bank provided institutional support and no longer financed civil works. Project performance was assessed on the basis of works funded by other external partners as part of the wider road sector program.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High.
At the time of appraisal, about 67% of national roads and more than 75% of local roads were classified as poor in Moldova. Poor road infrastructure was hurting Moldova's economy, and the Government's resources allocated for the country's road network were not adequate. The project development objectives were consistent with the focus on transport infrastructure enhancement in the Government of Moldova's Land Transport Infrastructure Strategy (LTIS 2008-2017) that included a Strategy for Road Infrastructure Recovery and a prioritized 10-year Road Sector Investment and Expenditure Plan (together called the Road Sector Program). The objectives were also relevant to the World Bank Group's FY09-12 Country Strategy for Moldova, strategic goals of which included an improved management of the road network. The objectives were also in line with the main priorities of the Country Strategy FY05-08 at appraisal, the key goals of which included promoting conditions for economic stability and growth.

b. Relevance of Design:
Substantial.
The statement of development objectives was clear. The project results framework indicated a clear causal chain between the activities financed by the project and the outputs and outcomes related to the attainment of the development objectives. For example, with regard to physical investments in road works (Component 1), the activities were expected to reduce vehicle operating costs and increase the percentage of national roads in good condition. These, reinforced by strengthened institutional capacity of the public sector road agency, were intended to lead to improved management of the country's road network.


4. Achievement of Objectives (Efficacy) :


Reduced road transport costs for road users by improving the condition and quality of the road network and its management. Substantial.
    Outputs
      • 297 lane-km of road rehabilitation and improvement were completed with funding from EBRD and EIB. The target of 400 lane-km was not met due to the greater scope of engineering interventions than anticipated at appraisal. In many cases, full reconstruction or rehabilitation was undertaken whereas during appraisal it was estimated that only heavy periodic maintenance would be done (p.27). Technical audits confirmed that road works complied with contractual specifications.
      • Technical support was provided for a coherent Road Sector Program, including a Road Sector Strategy and a prioritized Road Sector Investment plan, which are a part of Land Transport Infrastructure Strategy adopted for the period of 2008-2017.
      • Technical assistance was carried out in defining and implementing road sector reforms, in particular related to road maintenance management and axle load control.
      • Technical assistance for building a pipeline of new road investment projects was carried out, i.e., feasibility, design, environmental and social studies for 650 km of priority road sections.
      • Studies on road maintenance reforms were completed.
      • SRA staff received training in environmental and social safeguards, procurement and financial management with lasting benefits to their professional skills and to the capacity of the SRA as an institution. Following the International Financial Institution (IFI) contract monitoring requirements, SRA started to use supervision engineers in its internally funded projects.
      • Implementation arrangements were set up for externally funded road projects. The State Road Administration (SRA) established a unified system for managing road investments, i.e, accounting and financial management, procurement, and contract management.

    Outcomes
      • Vehicle operating costs were reduced by 11% surpassing the target of 6% by 2011.
      • There was an increase in the percentage of national roads in good condition from 7.3% to 9.1% achieving the target.
      • The target of road asset value increase was achieved, with a value of US$2.96 billion in 2011.
      • Increasing the funds for maintenance was a condition for road sector funding from the World Bank and other external partners. The Government complied with the commitment through the amended Road Law in 2010 allocating 50% of the fuel excise tax revenue to the Road Fund; 65% and 80% targets for 2011 and 2012 were also reached.
      • The introduction of an axle load control system was successful in reducing truck overloading: the percentage of overloaded trucks fell from 31% to 9% over 2009- 2011. Thirteen mobile axle load weighting platforms, procured under the State budget, were operating across the country by 2011. The Ministry was planning to expand the system in 2014 with a purchase of 13 new systems and by increasing monitoring to 24 hours with three shifts in place. To better control for corruption risk, some technical improvements were subsequently made to the system that included installation of 26 cameras on the mobile units to stream live picture to a new operational control center in Chisinau.
      • The most important concrete reform action resulting from the road maintenance reforms studies was the consolidation of the existing 39 road maintenance companies into 12 companies.
      • The Government’s Treasury department was applying the financial controls introduced to this project to other externally funded projects. SRA started using supervision engineers on internally funded contracts.
      • The Bank-led reform dialogue and coordination efforts through the institutional support component reassured the external partners that their planned road sector investments would be implemented within a more robust institutional framework. By 2013, the external partners in the road sector committed a total of US$460 million in road sector funding, in addition to US$5 million provided by the Bank.

    5. Efficiency:


    The economic rate of return (ERR) for 6 completed road sections with a total length of 297-lane km is estimated to be 25.5% at closure, with a net present value (NPV) of US$ 106.4 million; the ERRs for individual road links varied from 17.1% to 60.1%. At appraisal, a cost-benefit analysis carried out for 400-lane km of main and secondary roads resulted in an NPV of US$177 million; ERRs varied from 30% to 240%.

    The ex-post analysis was based on the actual traffic for 2013, road user costs savings (transport user benefits from using the new roads), and project economic cost. The Highway Development and Management Model (HDM-4) was used to calculate the respective road user costs and economic rates of return.

    The actual ERR for all roads are lower than estimated at the appraisal stage due to higher civil works costs. At appraisal it was estimated that costs would range from US$ 150,000 to US$ 220,000 per km but the average cost was US$760,000 per km and the range was between US$ 430,000 and US$ 1.34 million in 2013 prices. In many cases, full reconstruction or rehabilitation was undertaken whereas during appraisal it was estimated that only heavy periodic maintenance would be done.

    The Bank-funded institutional support component of US$5 million was disbursed and closed as planned with the achievement of all the outputs.

    The declaration of mis-procurement of two civil works contracts (one to be funded by IDA and the other by EBRD) reduced the efficiency of project supported civil works resulting in a delay of two years-– while the Bank- funded portion of the project closed as appraised on June 30, 2011, the EBRD and EIB completed the project’s main civil works in 2013.

    Efficiency is assessed as modest.

    a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


    Rate Available?
    Point Value
    Coverage/Scope*
    Appraisal:
    No
    %
    %
    ICR estimate:
    Yes
    25.5%
    93%

    * Refers to percent of total project cost for which ERR/FRR was calculated

    6. Outcome:


    The project’s development objective was highly relevant to the challenges faced by the road sector in Moldova. Relevance of design is rated substantial. Along with the reductions in road transport costs achieved under the project, the Government increased its allocations for maintenance of the road sector. Efficiency is rated modest due to a two-year delay in the EBRD and EIB part of the project’s main civil works caused by misprocurement.

    a. Outcome Rating: Moderately Satisfactory

    7. Rationale for Risk to Development Outcome Rating:

  • The project succeeded in generating reform momentum which is now being sustained by the Government as well as the external partners. The Government has implemented all the required reform measures since project beginning and stays committed to reform, which is also a condition of continued disbursement of remaining IFI funds. The total amount of additional external funding mobilized for Moldova’s road sector was US$460 million in 2013.
    • The Bank's leading role on dialogue and reform efforts on behalf of all external partners in the road sector have been assumed by the Millennium Challenge Corporation (MCC) who are doing reportedly an excellent job in taking forward the reform agenda.
    • Road maintenance allocations are now less vulnerable to politics because of the new Road Fund legislation of December 2009, which established clear allocation mechanisms. The government is fully meeting its commitments to the road fund which stands at five times the level envisaged under the original project.
    • The risk of political interference, however, has not completely diminished. EBRD experiences procurement issues on M3 section, which were not specified by the ICR (p.18). The IFIs have to continue careful procurement monitoring with adequate anti-corruption measures.

      a. Risk to Development Outcome Rating: Moderate

  • 8. Assessment of Bank Performance:

    a. Quality at entry:

    The Road Sector Program Support Project (RSPSP) was the first World Bank supported operation in Moldova’s transport sector. At appraisal, it was the second externally financed road project since the country’s independence in 1991. The first project on road rehabilitation financed by the European Bank for Reconstruction and Development (EBRD) was cancelled prematurely in 1998 because of contract mismanagement by the Government. As Moldova’s International Monetary Fund (IMF) program included a requirement for a certain percentage of grants or concessionary lending in all new external loan packages, the Bank participation was necessary to secure funding from EBRD and EIB, which could not provide stand-alone loans.

    The Anti-Corruption Action Plan (ACAP) for the project was built in part on the lessons learned from the cancelled EBRD road project. The risk of public sector corruption during procurement was assessed as being high in the PAD. The strong controls in place did not eventually prevent the political interference from occurring but were instrumental in staving off the corruption attempt. A “system failure” was not to blame, but rather a single case of high level political interference.

    During project design and preparation, the government was actively consulted to help create strong ownership and trust. Implementation arrangements through the State Road Administration, rather than a separately created project implementation unit, ensured a more lasting development impact. Preparation was quick; it took seven months from the start of formal project preparation to approval. A Project Preparation Facility (PPF) was used to fund feasibility studies, engineering designs, and environmental and social studies for the road sections to be rehabilitated under the project; international consulting firms were hired for this purpose. Road designs and bidding documents were ready by the time of project effectiveness.

    Quality-at-Entry Rating: Satisfactory

    b. Quality of supervision:

    The Bank led the reform dialogue with the Government and coordinated this effort with other IFIs, including joint missions every six months. The Bank’s proactive role in maintaining the reform momentum helped leverage funding from the partners, which had included specific reform measures in loan and grant conditions, such as maintenance spending requirements. The Bank team also brought the IMF on-board over the establishment of a road fund and associated funds earmarking.

      When the problems surfaced with procurement of the first rehabilitation contract and politically motivated interference emerged, the team emphasized that the Bank’s procurement guidelines must be followed and the bid evaluation committee’s recommendation could not be overturn. The ICR reports that there were no major shortcomings in the Bank’s handling of the situation trying to protect the integrity of the procurement process (p.20). EBRD and EIB did not start funding the cancelled sections until the political leadership suspected of interference changed two years after the declaration of mis-procurement. No changes were made to the anti-corruption guidelines or procedures which worked as planned alerting stakeholders to the political interference.

      The ICR judges that the project indicators should have been changed after restructuring to better reflect the situation on the ground (p.20).

      Quality of Supervision Rating: Moderately Satisfactory

      Overall Bank Performance Rating: Moderately Satisfactory

      9. Assessment of Borrower Performance:

      a. Government Performance:

      The Government met its loan covenants and other obligations satisfactorily; it changed the Road Fund Law, increased road maintenance funding as per requirements of the new legislation, introduced the axle load control system and provided all tax exemptions and permits for road works as requested.

        The satisfactory Government efforts, however, were overshadowed by its role behind the perceived political interference into the bidding process for road rehabilitation works that resulted in the partial credit/loan cancellations. There was an attempt to disqualify the firm to which the bid evaluation committee had proposed the award of two contracts. The political instruction was to award one contract to a firm that had given a considerably higher bid price. The Bank declared mis-procurement and cancelled the corresponding part of the IDA credit (US$11 million) in December 18, 2009, after several unsuccessful attempts by the Bank (through visits and letters by Europe and Central Asia Region (ECA) senior management) to convince the Government to award the contracts in line with the Bank’s procurement procedures. Finally, the politicians suspected of involvement in the interference were no longer in power after two years of the declaration of misprocurement.

        Government Performance Rating: Highly Unsatisfactory

        b. Implementing Agency Performance:

        The project was implemented by the State Road Administration (SRA) that performed its duties satisfactorily (p. 21). The agency was initially slow and required capacity development at the beginning of project implementation in such areas as procurement, financial management and safeguards. During the events leading to mis-procurement, SRA showed its reliability as a project implementation agency by bringing the political interference to the World Bank’s and other partners’ attention. Due to this and the progress made on the institutional component, the Bank and its partners were confident that project implementation could continue with SRA despite the partial credit/loan cancellations. SRA deserves credit for trying to uphold the integrity of the procurement process despite heavy political pressure when reportedly threatened with job termination and even imprisonment.

        Implementing Agency Performance Rating: Satisfactory

        Overall Borrower Performance Rating: Moderately Satisfactory

        10. M&E Design, Implementation, & Utilization:

        a. M&E Design:

        The project's indicators were appropriately linked with the intermediate objectives. The key performance indicators for the project development objective included: (i) reduction of vehicle operating costs; (ii) increased percentage of national roads in good condition; and (iii) increased road asset values for the road network. The second indicator does not, however, specify the target for an increase of percentage of national roads in good condition. The third indicator on road asset valuation was not based on one methodology (i.e., the end value was to be recorded with a new methodology devised by consultants) and could have been revised during restructuring, as noted by the ICR (p.9). A WB core indicator measuring the length of non-rural road rehabilitation was added in 2010 as part of the mandatory retrofitting of monitoring.

          The ICR finds that in case of the intermediate level indicators, the targets could have been better defined and more measurable (p.7,9).They were adequate in scope but could have been improved if measured gradual progress. Also, a road safety indicator could also have been considered since Component 1 incorporated specific road safety features (i.e., installation of guardrails and traffic calming measures).

          The State Road Administration (SRA)was responsible for managing and operating the M&E system.

          b. M&E Implementation:

          SRA used its existing data collection and analytical tools to carry out the ex-post economic evaluation and road condition surveys required for PDO monitoring. Intermediate outcome indicators were monitored using reports and data provided by SRA together with technical auditors.

          a. M&E Utilization:

          The M&E system was used to support recommendations and inform management decisions during the course of implementation.

          M&E Quality Rating: Substantial

          11. Other Issues:

          a. Safeguards:
          This was a Category “B” project that triggered one safeguard policy – OP4.01 Environmental Assessment.

          The ICR reports that "all environmental and social safeguards were handled satisfactorily" (p.10). A Sector Environmental Assessment (SEA) and a Social Assessment were carried out. All project civil works were to be implemented within the existing right-of-way on all road sections without any need for land or asset acquisition.

          The environmental protection standards for national roads construction were revised with support from a Bank's Safeguards Specialist. A new environmental handbook for roads was prepared in coordination with the Ministry of Environment. Public awareness materials were distributed about the revised standards and guidance. Throughout the TA activities, a local environmental consultant recruited by the Bank supported the SRA staff who are now able to handle the environmental aspects of road projects.

          After the World Bank’s participation in physical road works was cancelled, no environmental supervision of civil works by Bank staff was carried out. However, EBRD and EIB environmental specialists continued to cover environmental safeguards for their respective road sections. The Environmental Management Plans prepared for the original Bank-financed sections were still used when EIB and EBRD decided to substitute their funding for the sections that were planned to be funded by the Bank.

          b. Fiduciary Compliance:
          Procurement: Mis-procurement was declared on December 18, 2009. A joint bidding process for the three road contracts under Component 1 (to be funded by IDA, EBRD and EIB) was launched in February 2008. The SRA’s bid evaluation committee evaluated the bids and recommended the award of two lots to one firm and one lot to another firm. However, the high-level Road Sector Steering Committee countered the recommendation by saying that the price of one of the recommended firm’s was too low and that at least one lot should go to another bidder whose bid was twice as high. There were clear indications that the recommendation by the Steering Committee was motivated by instructions from the highest political level. The Bank declared mis-procurement and cancelled the corresponding part of the IDA credit (US$11 million) in December 18, 2009, after several unsuccessful attempts by the Bank (through visits and letters by Europe and Central Asia Region (ECA) senior management) to convince the Government to award the contracts in line with the Bank’s procurement procedures. Through this process EBRD also cancelled €12.5 million from its loan. After the mis-procurement, procurement consisted only in consulting services; the associated bidding processes were transparent and fair, and all procurement activities were carried out in accordance with World Bank guidelines (p.10).

          Financial management. SRA submitted quarterly Financial Management (FM) reports following the required Bank format. The Bank’s FM staff found the reports acceptable. The project and SRA financial statements were audited annually by independent audit firm selected on competitive and eligibility basis. The audit reports were submitted initially with significant delays (up to five-six months), which were later fixed with all audit reports being submitted on time. The project account audit reports were unqualified. The SRA entity audits, however, revealed a series of internal control weaknesses. The audit opinion was therefore undermined by issues related to road asset values, such as: (i) possession of assets (roads) in operational management without adequate supporting documentation confirming their historic value, and (ii) a lack of assessment methodology for recoverable road asset values. Several measures were taken to address the auditors' concerns (for instance, road asset valuation).

          c. Unintended Impacts (positive or negative):
          The cancellation of parts of the Bank credit and EBRD loan as a result of mis-procurement signaled that political interference will be detected and repelled by these international agencies.

          d. Other:



          12. Ratings:

          ICR
          IEG Review
          Reason for Disagreement/Comments
          Outcome:
          Moderately Satisfactory
          Moderately Satisfactory
           
          Risk to Development Outcome:
          Negligible to Low
          Moderate
          The risk of political interference has not completely diminished. 
          Bank Performance:
          Moderately Satisfactory
          Moderately Satisfactory
           
          Borrower Performance:
          Moderately Satisfactory
          Moderately Satisfactory
           
          Quality of ICR:
           
          Satisfactory
           
          NOTES:
          - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
          - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

          13. Lessons:

          The ICR has identified a number of lessons, of which the most important are listed below with some adaptation:
            • Relatively small Bank's investment could help leverage significant external resources with a programmatic approach. Under this project, EBRD and EIB funding could not be raised without some amount of concessional financing provided by IDA credit. The Bank team support to the development of a long-term road sector investment strategy and implementation arrangements provided a foundation for the external partners to select the road sector for their investments.
            • Coordination with development agencies makes operations more efficient for a greater development impact. This project formalized cooperation among the IFIs. During project preparation, the PAD was partly written by EBRD and EIB staff, who also used much of the World Bank’s PAD text for their own project documentation. The Bank took the lead in organizing joint preparation and supervision missions and communicating shared positions.
            • In a joint project design, attribution of program success to specific institutions is difficult. The PDO indicators in this project were not limited to the results of planned Bank-funded road sections but also included sections financed from EBRD and EIB.

          14. Assessment Recommended?

          No

          15. Comments on Quality of ICR:

          The ICR is concise and outcome-oriented. The quality of evidence is adequate. There is a good discussion of M&E, including suggestions for improvement. The lessons are evidence-based. The cost tables in Annex 1 had some discrepancies.

          a. Quality of ICR Rating: Satisfactory

          (ICRR-Rev6INV-Jun-2011)
          © 2012 The World Bank Group, All Rights Reserved. Terms and Conditions