Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Road Sector Investment Program Support


  
1. Project Data:   
ICR Review Date Posted:
11/03/2005   
PROJ ID:
P003236
Appraisal
Actual
Project Name:
Road Sector Investment Program Support
Project Costs(US $M)
 384.2  466.2
Country:
Zambia
Loan/Credit (US $M)
 70.0  69.2
Sector, Major Sect.:
Central government administration, Vocational training, Roads and highways, General transportation sector,
Law and justice and public administration; Education; Transportation; Transportation
Cofinancing (US $M)
 62.6  212.0
L/C Number:
C2993      
   
Board Approval (FY)
  98
Partners involved
DANIDA, Govt. of Norway, Other bilateral donors 
Closing Date
03/31/2003 03/31/2005
         
Evaluator: Panel Reviewer: Group Manager: Group:  
Peter Nigel Freeman
Ridley Nelson Alain A. Barbu OEDSG

2. Project Objectives and Components:

a. Objectives
The overall goal was to facilitate economic growth and diversification, particularly in the agricultural sector, through appropriate investments in road infrastructure and through a sustainable system for the financing and management of the road network. The Government of Zambia's (GOZ) road sector development program (ROADSIP) was planned to cover a ten year period.

Specific objectives were:

  1. To reform the road sector policy and institutional framework, including legislative changes and strengthening and refocusing sector institutions to enhance capacity, efficiency in the use of resources and improved planning and implementation;
  2. To strengthen road sector financing, including increasing and diversifying road user charges to the Road Fund to ensure 70% coverage of periodic and routine maintenance costs of the core road network by 2002;
  3. To strengthen the local construction and consulting industry, including creating greater opportunities for the private sector to participate in road design, construction and maintenance, and achieving a target to create 14,000 new jobs by 2002;
  4. To address the road maintenance and rehabilitation backlog through a priority investment program, with a specific target of increasing the proportion of main roads in good condition to about 15% of the road network by 2002;
  5. To establish a pilot program for community road management in at least three districts to test approaches for sustainable improvements in maintenance and to promote increased usage of non-motorized transport.
No revisions were made to the project objectives.

b. Components (or Key Conditions in the case of Adjustment Loans):
Civil works; maintenance and rehabilitation of 22,000 km of the core road network, including accessibility improvements on 17,000 km of feeder roads (US$347.3 million, 90.4% of appraisal costs; final cost US$ 428.6 million);
Community accessibility program; 70 community-based projects and improvements to 500 km of feeder roads. The use of intermediate means of transport to be encouraged, (US$ 6.3 million, 1.6% of appraisal costs; final cost US$ 3.4 million);
Construction industry development program; included technical assistance and training (US$ 2.8 million, 0.7% of appraisal costs; final cost US$ 5.4 million);
Institutional capacity building; included policy support, implementation and technical support, and institutional development across the full span of road management (US$ 7.0 million, 1.8% of appraisal costs; final cost US$ 8.2 million);
Engineering Services (US$ 20.8 million, 5.5% of appraisal costs; final cost US$ 20.6 million).

c. Comments on Project Cost, Financing, Borrower Contribution, and Dates
The Bank credit was fully disbursed, but counterpart funding fell short of expectations by US$ 66.6 million. This was made up by a greater share of funding from other donors and the program was expanded by nearly 18 percent. The project had to be extended twice from March 2003, the original closure date, first to March 2004 and, finally, to March 2005. The main reasons were the lack of local capacity in the construction industry, which impeded implementation progress, coupled with the irregular provision of counterpart funding. This project was regarded as a first phase.


3. Relevance of Objectives & Design:

The project objectives were highly relevant to the 1996 CAS which aimed to assist the government to create a stable macroeconomic environment, promote private sector development and make the public sector more efficient. To this end the government laid out its sector strategy in a Letter of Sector Policy which supported the establishment of a clear institutional framework for the road sector and a robust and sustainable financing mechanism for roads. These objectives were equally valid at project completion. The project was generally well-designed except for a lack of realism about a few target output indicators that did not take into account the problem of dealing with the backlog of maintenance which had built up over the years.

4. Achievement of Objectives (Efficacy) :

To reform the road sector policy and institutional framework, (Substantially achieved)
GOZ adopted a National Transport Policy and established supporting legislation through the National Road Fund Act, the Public Reform Act and the Road Traffic Act. The establishment of a Road Fund Agency, a Road Development Agency and a Road Transport and Safety Agency was carried out and the functions of these agencies were defined. Equipment was procured to enhance road safety and an Environmental Management Unit was established. However a Highway Management System did not function according to expectations and a Road Safety Action Plan was compiled, but its actions were not effected due to a lack of capacity. Staff training was also carried out. The targets for the reduction of accidents were not met by project closure - in fact they were worse than predicted. It remains to be seen whether the new institutions will be exempted from civil service conditions and be enabled to pay higher salaries and recruit on the open market. Although this was the intention, when the ICR was prepared this still had not actually happened.
To strengthen road sector financing; (Modestly achieved).
The target set at appraisal was that by the end of the project 70% of the total funding requirements for routine and periodic maintenance would be covered by the Road Fund. At project closure a disappointing 23% was achieved. While this is ameliorated to some extent because the shortfall was made up by budgetary and donor support, the intention was to create a sustainable flow of funds through a Road Fund. Given the difficulties with the flow of counterpart funds during the project and the apparent lack of revenue from other road user charges planned, this is a cause of concern, especially when the recent dramatic escalation in oil prices will put pressure on governments not to increase fuel levies. This issue, which is pivotal to sustainability is to be pursued further through the Road Rehabilitation and Maintenance Project. At the start of the project it was estimated that the Road Fund would be able to finance routine maintenance of the core road network to the extent of about 6,700 km/year; the target for the end of the project was 15,700 km/year. The amount actually achieved was 10,246 km or 65 percent of planned expenditure.
To strengthen the local construction and consulting industry; (Substantially achieved)
During project implementation 50,000 new employment opportunities were created, exceeding the appraisal target of 40,000. Most jobs could be sustainable as the new Road Development Agency will be outsourcing maintenance works on performance based contracts to small and medium scale contractors. This sustainability, however, is contingent on an assured supply of maintenance funding (see above). Of the 300 contractors targeted for routine and periodic maintenance training 265 or 88 percent received training, while the number of local civil engineering consulting firms increased from 10 to 15. The National Council for Construction (NCC) , however, was unable to address all the constraints on local contractors to effectively exploiting all the opportunities created under the project. It is noteworthy that the elapsed time between submission of invoices and payment was 14 days against a target of 10 days. A study was nevertheless undertaken of the construction industry in Zambia and its recommendations are enabling NCC to draw up a strategic plan for the further development of the industry.
To address the road maintenance and rehabilitation backlog through a priority investment program; (Substantially achieved)
The target to increase the proportion of main roads in good condition to about 45 percent was exceeded; at the end of the project the extent of such roads in good condition was between 57 and 60 percent. The target for feeder roads in good condition was 15 percent, but only 7 percent was achieved. This shortfall was largely due to a change in the design approach. Whereas, the plan had been to focus primarily on spot improvements, in the event the deteriorated condition of many of the road sections dictated that often full improvement had to be the solution. This meant that the available funds were insufficient. The target of 6,600 km set at appraisal for periodic maintenance was only 27 percent achieved, but the planned target of roads to be rehabilitated of 1,500 km was exceeded by 109 percent because both GOZ and other donors contributed more to funding than estimated. Similarly feeder road upgrading target of 2,600 km was exceeded by 139 percent.
To establish a pilot program for community road management in at least three districts to test approaches for sustainable improvements in maintenance and to promote increased usage of non-motorized transport. (Substantially achieved)
Pilot community-based road management programs were established in three districts in which communities participated in the improvement of community roads and were trained in basic maintenance skills. Because of the success of these pilots, the program was extended to encompass another six districts. However, GOZ has yet to put in place a coherent structure to replicate and extend the continuity of the schemes on a wider basis. A further pilot in respect of the support and promotion of non-motorized transport means was undertaken in the district of Mpongwe. Later this was extended to four other districts. A Trust Fund has been set up to sustain these pilots and there are clear benefits from its introduction. The major shortcoming, however, remains the very low level of loan repayments.

In summary, this was a very comprehensive program and, although there were some shortcomings, on balance substantial progress was achieved.

5. Efficiency:

The ERR was calculated using actual construction costs, life cycle maintenance costs over a 20 year design period, and savings in vehicle operating costs (VOC) before and after road improvement. The savings arise from VOCs before and after road improvement, adjusted for each year of the road design period. Costs and benefits are taken over a 20 year project life and a discount rate of 12% is used for the opportunity cost of capital. This results in an ERR of 30% on completion in comparison to 17.25% at SAR, based on imputed costs. While this is an excellent return, it also presupposes that the roads will be maintained properly during their design life. Historically, this has not been a valid assumption, so the effectiveness of the new regulatory and institutional strengthening measures will be very important.
6. M&E Design, Implementation, & Utilization:

The M&E system for the project was ineffective. There was no baseline information on most of the project activities, except for major roads and this adversely affected the comparability, usability and integrity of the data generated on the project. Only now under the follow-up Road Rehabilitation and Maintenance Project is this matter being addressed. With regard to the pilot on non-motorized transport there was no baseline information on poverty in the selected district for the study.
7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative):

No safeguard issues or unintended impacts.

8. Ratings:
ICR
OED Review
Reason for Disagreement/Comments
Outcome: 
SatisfactorySatisfactory
Institutional Dev.: 
SubstantialSubstantial
Sustainability: 
LikelyNon-evaluableWhile sustainability does appear possible at project closure, there is still doubt about the extent and continuity of the flow of funds as well as the robustness of the capacity and autonomy of the new institutions. It will be important to follow up and ensure that the efforts put into this project are not diminished over time.
Bank Perf.: 
SatisfactorySatisfactory
Borrower Perf.: 
SatisfactorySatisfactory
Quality of ICR: 
Satisfactory

NOTES:
- When insufficient information is provided by the Bank for OED to arrive at a clear rating, OED will downgrade the relevant ratings as warranted beginning July 1, 2006.
- ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness.

9. Lessons:

It is acknowledged that some of these lessons are derived from the ICR.
  1. An M& E system and baseline information for project activities should be introduced early in a project to enhance project management, including tracking implementation progress, and quantifying the impact of the project on the economy and the stakeholders.
  2. Target output indicators should be realistic and attainable, especially when a backlog of maintenance needs has built up over a long time period.
  3. Design and implementation of institutional reforms is a lengthy process and demands a sustained dialogue among all stakeholders.
  4. Small focused infrastructure improvements can have significant local benefits for which communities are prepared to help fund improvements, either in cash or kind.
  5. Spot accessibility improvements to feeder roads, especially at river crossings, will often have substantial returns.

10. Assessment Recommended?  Yes

          Why?  To assess the sustainability of the institutional strengthening measures including road condition and the revenue streams to the Road Fund directly from government and from other sources. Also to ascertain whether further progress has been made on the community pilot projects.

11. Comments on Quality of ICR:

The ICR is sufficiently detailed to convey general performance and to be able to assess ratings; it is also candid where shortcomings are evident. The links between the costs and percentages in the text and the figures shown in the tables in Annex 2 are difficult to follow. In particular, although the project comprised a high proportion of parallel donor funding, there is very little information about how this money was spent and what these (nameless) partners thought of the project. It is also difficult to reconcile the amounts estimated to be needed for the Road Fund in the SAR in comparison to the amounts given in Annex 1 of the ICR.

(ES-Rev4-Jul/05)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions