Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Ma-rural Roads


  
1. Project Data:   
ICR Review Date Posted:
06/05/2013   
Country:
Morocco
PROJ ID:
P082754
Appraisal
Actual
Project Name:
Ma-rural Roads
Project Costs(US $M)
 43.7  50.9
L/C Number:
L4747
Loan/Credit (US $M)
 36.9  43.1
Sector Board:
Transport
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  06/29/2004
 
 
Closing Date
06/30/2010 06/30/2011
Sector(s):
Roads and highways (97%), Central government administration (3%)
Theme(s):
Rural services and infrastructure (67% - P) Injuries and non-communicable diseases (33% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Kavita Mathur
John R. Heath Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The objective of the Project as stated in the Loan Agreement was" to increase the level of road access by rural populations, especially in the Borrower’s more disadvantaged provinces in the Project Area, through the implementation of national program of rural roads development and measures aiming at strengthening the Borrower’s institutional capacity" (Schedule II, p. 12). The objective in the Project Appraisal Document and the Loan Agreement are identical.

The project included the following outcome indicators:

  • A National Rural Road Accessibility Index (NRRAI), defined as the percentage of population living within one kilometer from an all-weather was included. The target for this indicator was to improve from 45 percent in 2002 to 67 percent in 2010.
  • An Accessibility Differential Indicator (ADI), to quantify the accessibility gap between the ten provinces. The target for this indicator was to improve from 0.38 in 2002 to 0.60 in 2010. This measure was designed for this project
  • A Transport Service Improvement Indicator (TSII), a measure to assess the impact of the improved roads on the quality of inter-city transport services for passengers.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
(i) Rural Roads Component (appraisal estimate US$ 42.0 million, actual cost US$ 50.2 million): Rehabilitation and/or upgrading of 625 km of rural roads included in the Second National Program of Rural Roads (NPRR-2). The roads under the NPRR-2 were chosen based on: (a) their contribution to improving rural access in a cost-effective manner, or having satisfactory economic returns in the case of higher traffic roads, (b) a strong participatory process involving the provinces and the rural municipalities (communes), (c) acceptable design standards including road safety, and (d) appropriate environmental, resettlement and social safeguards. (ii) Institutional Support Component (appraisal estimate US$ 1.4 million, actual cost US$ 0.5 million): Consultant services to: (a) develop a system to monitor and evaluate the achievement of accessibility targets and execution of the NPRR-2 (as well as other road improvements outside this program), at the provincial and communal level, and disseminate periodic reports on the NPRR-2 progress and achievement of targets, in order to facilitate sectoral integration and strengthen participation of local governments and communities; (b) formulate specific proposals that would help prepare for an eventual take over by provinces and communes of the management of the local rural roads; and (c) strengthen the capabilities of the Directorate of Roads and Road Traffic (DRCR) for social and environmental assessments.

Front-end fee IBRD (appraisal estimate US$ 0.37 million, actual cost US$ 0.16 million)

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost and Financing: The Loan Agreement was amended to reallocate the original funds under categories “Goods” and “Unallocated”, as well as, the balance remaining under category “Consultants‟ and other services” to category “Works”. The works incurred cost overruns due to unanticipated increases in the costs of key inputs (mainly oil, bitumen, and construction materials). The reduction of the loan amount allocated to consultants services was made possible by the Borrower's decisions to: (i) use their own budget to fund selected activities, particularly those related to the formulation of proposals for the management of local rural roads by the provinces and communes, and (ii) benefit from a grant of the European Investment Bank (EIB) to strengthen the capabilities of the Directorate of Roads for social and environmental assessments.
Borrower Contribution: The Borrower’s actual contribution of US$ 7.8 million was slightly more than the US$ 6.9 million estimated at appraisal.

Dates: The project closed on June 30, 2011, one year after the original closing date to allow for full disbursement of the loan. As mention above in the project cost and financing, there was reallocation of funds between different categories.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
The Project development objectives are fully consistent with the current development priorities of the Government. The key goal of the Government's 2020 Rural Development Strategy is to improve access of rural dwellers to basic transport infrastructure. The 2020 Rural Development Strategy has been adopted by the Bank Country Partnership Strategy (CPS) for the period FY10-13 under its second pillar: service delivery to citizens. The Bank's strategy aims to extend rural network coverage to increase rural population's access to all weather roads. The Bank would also facilitate setting up maintenance measures for unclassified rural roads network. The relevance of objectives is rated High.

b. Relevance of Design:
The project objectives were clearly formulated. Part of the project objective was to serve ‘more disadvantaged provinces.’ The project design accomplished this by, first, building on the 2002 inventory of rural roads which showed which areas were least accessible, and second, by giving priority to roads that reduced the difference in accessibility between the most and the least accessible provinces (PAD, p. 28).” As the PDO refers to a ‘national program’ the design details of the project needed to specify the role of central and local governments in financing and implementation (different from a CDD project with a rural roads component—a design alternative that was explicitly rejected, see PAD, p. 7). "Provincial offices of the Ministry of Works and Transport signed Specific Agreements with the provinces/communes to ensure that the latter ‘provide necessary rights of way, contribute to the roads financing, and guarantee their future maintenance’ with no works started until the respective Agreements were signed (PAD, p. 8).” This was critical for (a) achieving the project objective and (b) minimizing the risk to development outcome. A monitoring and evaluation system was developed and put in place from the launching of the project. The Project's results framework clearly laid out the causal chain between project components, intermediate outputs and final outcomes.
The relevance of design is rated High.


4. Achievement of Objectives (Efficacy) :

Increase the level of road access by rural populations, especially in more disadvantaged provinces in the project area: substantial.
According to the ICR, the project identified poor and isolated communities as its primary beneficiaries building on rural roads impact evaluations and stakeholder surveys conducted during the first National Program for Rural Roads. Following outputs were delivered:

The project rehabilitated/upgraded 520 kilometers (31 links in 17 provinces) of rural roads compared to the appraisal estimate of 625 kilometers. The shortfall was due to substantial increase in the cost of key inputs specifically, oil-based and energy intensive inputs, including fuel, asphalt and cement. According to the Project team, all links were in poor/isolated communities from several provinces for a balanced distribution.

The capacity of the Directorate of Roads was enhanced through: (a) preparation of a manual for design and appraisal of rural roads; (b) training workshops on social and environmental assessments for both the Directorate of Roads (including its provincial and regional offices) and consultants; and (c) development of an accessibility monitoring and evaluation system.

A study on maintenance strategies for all road networks was carried out. According to the Project Team, the study recommended adopting a functional classification to enable the Directorate of Roads to focus on the management of the main national roads while preparing for the devolution of the local roads to the provinces and communes. Based on the results of this study and in response to the insistence of the financiers of the Second National Program of Rural Roads, the Directorate of Roads took the initiative to request the establishment of an inter-ministerial committee to prepare an action plan for the maintenance of the local rural roads.

Outcomes:

The percentage of rural population with access to an all-weather road increased from about 50 percent in 2005 to 70 percent in 2010 against a target of 67 percent.

The accessibility differential between provinces measured by the ratio of the accessibility of the ten less served provinces above the one of the ten best served provinces has been reduced (from a ratio of 0.43 in 2005 to a ratio of 0.63 in 2010 against a target of 0.60).

The costs for goods and passenger transport on completed roads open to traffic for more than two years decreased on an average by about 40% (based on surveys carried out in 2010 on a sample of roads).


5. Efficiency:

At appraisal, an Economic Rate of Return (ERR) was carried out for 12 roads (selected for the first year with total length of 185 kilometers). The ERR was estimated at 30%. The analysis was done according to the Road Economic Decision (RED) model developed by the World Bank for the economic evaluation of investments and maintenance for low-volume roads. The RED model uses road user costs (such as vehicle operating costs, passenger time costs, and accident cost).
The ex-post ERR for 31 rural roads funded by the project (total length of 520 kilometers) ranged from 12.7 percent to 57.9, with average ERR of 25.1%. Lower ERR was due to substantial increase in the cost of key inputs (specifically oil based and energy intensive inputs, including fuel, asphalt and cement. Cost increased by about 40%.

A cost-effectiveness analysis of very low-volume roads was also done by calculating the total investment cost per beneficiary person. Considering only persons living in villages (douars)of more than 50 households, US$ 490 per beneficiary was estimated as a reasonable maximum cost effectiveness at appraisal. The threshold was updated to US$ 620 to account for the increased costs.

The efficiency is rated substantial.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
30%
32%
ICR estimate:
Yes
25.1%
98%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The outcome is Satisfactory due to high relevance of objectives and design, substantial efficacy, and substantial efficiency.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The risk to development outcome rating is rated "negligible to low" for the following reasons:
  • For the classified network, the maintenance requirements for the roads constructed under the project are included in the budget of the Directorate of Roads. Furthermore, the Directorate of Roads completed a study on road maintenance strategies and at project closing was finalizing a National Program of Road Rehabilitation to clear the backlog of road maintenance. This program (planned to be supported by the Bank under the national road asset management and network modernization), would be implemented over a period of five years and would include the maintenance and rehabilitation of the classified roads.
  • For Provincial roads, maintenance risk was mitigated through the signing of agreements between the Provincial offices of the Ministry of Works and Transport and the provinces/communes to ensure that the local governments are committed to provide financing for future maintenance (PAD, p. 8).
  • For the unclassified and communal rural roads, the maintenance would be met by the communes, provinces and regions in accordance with the action plan established by the inter-ministerial committee. It is not clear in the ICR whether the communes have the technical and financial means to tackle maintenance promptly and adequately.
  • The Directorate of Roads (DoR) is also preparing a road sector development plan based on four pillars: (i) placing maintenance at the core of DoR's priorities; (ii) undertaking a roads reclassification after their rehabilitation to keep under the DoR's jurisdiction the main roads and transfer to the regions, provinces and communes the roads servicing local or rural traffic to accompany the government decentralization program; (iii) adding new capacity on strategic missing links for a regionally balanced economic development; and (iv) renewing the Directorate of Roads business processes including assistance to the regions and provinces for the management, development and maintenance of their roads.

    a. Risk to Development Outcome Rating: Negligible to Low

8. Assessment of Bank Performance:

a. Quality at entry:
The project supported the Government's Second National Program of Rural Roads (NPRR-2) and drew upon the experiences of other donor supported rural road projects in Morocco. The project design took into account two lessons from the previous Highways project in Morocco (Loan 3901 closed June 30, 2001). First, the need for rural roads projects to define objectives based on accessibility. Second, the difficulties of carrying out monitoring and evaluation (M&E). The project therefore paid special attention to this function. Risk assessment was carried out at appraisal, and appropriate mitigation measures were proposed. The project preparation included a road inventory that guided the selection of specific investments (PAD, p. 28); the implementing agency (DRCR) was well chosen, because it had experience from past projects; the design relevance was rated high. The monitoring and evaluation framework was adequate.

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:
The Bank team interacted continuously with the Directorate of Roads throughout the project implementation through the establishment of a dedicated unit to manage the overall rural roads program. The Implementation Progress and ratings were adequately documented in the aide memoirs and reflected in the Implementation Status Results and Reports (ISRs). The ICR reports that the supervision budget was adequate and effectively used. Supervision missions included procurement and financial management specialists and contributed to the overall supervision effort by a systematic and comprehensive review of procurement and financial management aspects. An environment specialist performed two dedicated supervision missions including field visits in several provinces. However, only during the last two years, a social development specialist reviewed the projects social aspects, and reported on the overall compliance and highlighted the need for the Directorate of Roads to reinforce its capacity for monitoring and reporting.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Ministry of Equipment and Transport (MoET) remained committed to the compliance with the NPRR-2 strategic directions stated at the project onset. It secured the Ministry of Finance support to the update of the financing plan of the NPRR2. The Directorate of Roads also remained committed to the project. After the launch of the project the Directorate of Roads realized that it would not be able to effectively manage a program of the size and importance of the NPRR2 without a dedicated
unit for a close implementation monitoring. It then established a Program Management Unit staffed by
qualified engineers and succeeded to mobilize a team of international technical assistants funded by the
European Investment Bank as part of its financing of the NPRR-2. The Government’s contribution to funding was larger than expected, an indication of its commitment.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:
The implementing agency was the Directorate of Roads of the Ministry of Equipment and Transport, including its provincial offices, its Center for Road Studies and Research (CNER) and the Program Management Unit (PMU).
The ICR reports that after a slow progress at the launch of the Project due to the delayed establishment and staffing of the PMU, revision of detailed designs and validation of the monitoring and Evaluation with the National Center for Road Study and Research (CNER), the implementing agencies performance improved substantially. During the last two years, the PMU made appreciable efforts to improve its record keeping and reporting on social safeguards related matters.

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The M&E design was satisfactory and included both output and outcome indicators. The outcome indicators were:
A National Rural Road Accessibility Index (NRRAI), defined as the percentage of population living within one kilometer from an all-weather was included. The target for this indicator was to improve from 45 percent in 2002 (most recent road inventory) to 67 percent in 2010, (80 percent in 2015 when the NPRR-2 would be completed) and 60 percent by the Mid-Term Review).

An Accessibility Differential Indicator (ADI), to quantify the accessibility gap between the ten provinces (representing about 20 percent of Morocco’s 48 provinces) with the lowest accessibility, and the ten provinces with the highest accessibility. The target for this indicator was to improve from 0.38 in 2002 to 0.60 in 2010 (0.65 in 2015 and 0.50 at Mid-Term). This measure was designed for this project

A Transport Service Improvement Indicator (TSII), a measure to assess the impact of the improved roads on the quality of inter-city transport services for passengers. In order to monitor this indicator, a sample of roads would be selected every year. It was envisaged that by the end of the project, at least 80 percent of all sample roads that will have been open to traffic for two or more years would show an improvement in the quality of service, as reflected in higher service frequency, more comfortable vehicles, or lower rates.

The design of the M&E system benefited from the field surveys of rural road conditions and accessibility carried out by the Directorate of Roads for the first National Program of Rural Roads launched in 1995 and used a rural road accessibility database updated in 2005 including demographic and road network information for each of Morocco‟s provinces. The first two indicators are well designed. The third indicator "TSII" encompasses a number of factors and may be difficult to monitor.

b. M&E Implementation:
The Center for Roads Studies (CNER) was responsible for the project M&E. More time than anticipated was used to train and mobilize all provincial offices of the Directorate to timely collect the required data and properly report on implementation progress. After a validation of an accessibility baseline based on the results of a national population census carried out in 2004 the CNER started issuing in 2007 yearly reports on rural roads accessibility and progress of the NPRR2. It is not clear from the ICR if the "TSII" was monitored effectively.

a. M&E Utilization:
Not discussed in the ICR.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
Environmental Safeguards: The project was assigned Environmental Assessment category B. An operational manual on environmental procedures, including an environmental management plan was prepared. The manual established the process and institutional requirements for environmental management during the various project stages.
The ICR reports that the compliance with OP 4.01 on environmental assessment was satisfactory and no environmental safeguard issues arose during implementation. Environmental issues addressed during the implementation of project were mainly construction related and limited to the roads sites; these included borrow area operation and redevelopment, controlling erosion and slope stabilization, as well as disposal of wastes and other construction material. The ICR further reports that adequate environmental monitoring was performed during construction of rural roads by the provincial and regional directorates of roads and visits from an environmental specialist from the Program Management Unit in charge of the NPRR-2.

The technical assistance team funded by the European Investment Bank as part to its support to the NPRR2 delivered training sessions on environmental management of road projects to improve the Directorate of Roads environmental capacities and carried out field visits focusing on environmental aspects.

Social Safeguards: According to the Project Appraisal Document, the project would not entail resettlement of population since the roads rehabilitated/upgraded under the project are on existing right of ways and there will not be significant changes in road characteristics such as width or alignment. Minor land expropriation would be required and OP 4.12 Involuntary Resettlement was triggered. The DRCR prepared a Resettlement Policy Framework.

The ICR reports that the compliance with OP 4.12 on Involuntary Resettlement was satisfactory. Works carried out remained within the existing rights of way or required small tracts of land for necessary realignments. The Resettlement Policy Framework, consistent with both Moroccan law and Bank safeguard policies, defined the land acquisition and compensation required for Rural Roads projects throughout the Project areas

b. Fiduciary Compliance:
The ICR reports that the Financial Management was satisfactory and in compliance with the Bank procedures. Budgeting, accounting, flow of funds, annual year-end financial reporting and submission of audit reports were found acceptable by Bank supervision missions. The submission of Interim Financial Reports was initially delayed, but improved during the last two years. The internal control system was adequate, but the internal audit function was not yet established and the Inspection Générale des Finances identified deficiencies in financial information system although these weaknesses were not material.
The ICR reports that the Procurement management was satisfactory and in compliance with Bank procedures. A few delays have been noted during supervision missions with recommendations made by the Bank to address them. No issues of Fraud and collusion were detected or uncovered during implementation.

c. Unintended Impacts (positive or negative):
None.

d. Other:
None.



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Satisfactory
 
Risk to Development Outcome:
Negligible to Low
Negligible to Low
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following lessons are taken from the ICR with some adaptation.
  • An adequate monitoring and evaluation system with well-defined and measurable indicators can play an integral role in successful implementation of the project. The "measure of accessibility" to all weather roads by the rural rural population was well defined and measurable. It was understood and internalized by all stakeholders as a result of the dissemination of yearly accessibility reports to all provinces.
  • Well stated technical standards are essential to the quality and sustainability of infrastructure investments. The preparation and dissemination of the manual for design and appraisal of rural roads projects facilitated quality and ensured conformity among the designs prepared by different consultants. The manual also integrated lessons learned from technical audits of the rural roads constructed under the first national program of rural roads.

IEG Lesson: To ensure adequate financing and maintenance of the non-classified rural roads, these roads should be included in national plans and the linkage between these roads and the broader network should be emphasized, rather than a piecemeal approach through separate CDD-type projects.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The quality of the ICR is satisfactory. Adequate evidence was provided to support the analysis and the report was candid. Lessons were appropriately identified.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions