Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Mizoram State Roads Project


  
1. Project Data:   
ICR Review Date Posted:
04/30/2013   
Country:
India
PROJ ID:
P069889
Appraisal
Actual
Project Name:
Mizoram State Roads Project
Project Costs(US $M)
 70.0  117.0
L/C Number:
C3618
Loan/Credit (US $M)
 60.0  103.1
Sector Board:
Transport
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  03/14/2002
 
 
Closing Date
12/31/2007 12/31/2010
Sector(s):
Roads and highways (95%), Central government administration (5%)
Theme(s):
Infrastructure services for private sector development (100% - P)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Ramachandra Jammi
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The project development objective, which is stated identically in both the Project Appraisal Document (PAD) and the Development Credit Agreement, is “ to improve the management and carrying capacity of the Mizoram core state road network.”

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
1. Improvement (widening and strengthening) of about 184 km of state roads; (at appraisal: US$36.8 million, at completion: US$87.0 million).

2. Land acquisition, utility relocation, implementation of the Resettlement and Indigenous Peoples Development Plan, Environmental Management Plan and Environmental and Social Management Plan for the civil works components (at appraisal: US$2.3 million; at completion: US$4.0 million).
3. Rehabilitation and maintenance of 518 km of state roads in two phases, Phase I: 278 km and Phase II: 240 km. The works were designed to reduce the backlog of periodic maintenance and comprised overlaying, resealing and minor rehabilitation (at appraisal: US$17.7 million; at completion: US$11.4 million - Phase I only).
4. Design, supervision, and technical advisory services for civil works; (at appraisal: US$5.7 million; at completion: US$11.0 million).
5. Institutional strengthening consisting of implementation of Technical Assistance, training, equipment, and pre-investment studies to help implement the Institutional Strengthening Action Plan. This included improved coordination between government agencies and other parties, streamlining of overhead costs and improvement of the Public Works Department’s capability to manage the Mizoram road network including the implementation of a comprehensive Road Management System; (at appraisal: US$3.0 million; at completion: US$2.6 million).
6. Road safety engineering measures and the implementation of a road safety plan; (at appraisal: US$1.0 million; at completion US$1.0 million).

The development objective was not revised during project implementation. However, Phase II of the Rehabilitation and Maintenance Component was dropped (thus reducing the scope of the component from 520 km to 297 km) due to higher unit costs than estimated at appraisal; a shortfall in funding; and delays in implementation. This was acknowledged by the Bank’s supervision/Mid Term Review (MTR) mission in January, 2004 (Aide-Memoire, Jan 16-28, 2004), The formal restructuring was undertaken at the second additional credit (Sep. 1, 2010). The portion of the road length dropped from the project was expected to be funded by other Government of India programs.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project cost. The final project cost (US$117 million) was nearly 70 percent higher than that at appraisal (US$70 million). The increase in costs was due mainly to a higher cost of works and construction materials, such as bitumen, steel and cement; price escalation from delays; and the extra cost of engineering supervision.
Financing. The increased cost was financed by an increase in borrower contribution of US$4.45 million and by two separate amounts of additional Bank financing, one of US$18 million in January 2007, and the other of US$13 million in September 2010. The final amount disbursed by the Bank (US$103.1 million) differs from the amount shown in the Credit Agreement because of a substantial depreciation of the rupee against the US dollar. There was no other external financing.
Borrower contribution. At US$14.45 million, the Borrower’s contribution was nearly 45% higher than the US$10 million planned at appraisal.
Dates. The original credit closing date was extended by a total of 36 months from December 31 2007 to December 31 2010 spread over three separate extensions (18 months in December 2007, two interim extensions totaling six months in June and October 2009, and a further 12 months in December 2009). The implementation arrangements remained unchanged. The extensions were necessitated by the slow pace of implementation of civil works, which in turn were due to several reasons including slow mobilization of contractors, heavier and longer monsoon rains, and shortage of construction materials and skilled manpower.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
The objective was highly relevant to the socioeconomic development of the north-eastern part of India. This was because of the sub-region’s geographical remoteness from the rest of India, hilly topography and extended transport and logistical links.
The Bank's Country Assistance Strategy for India at that time focused on strengthening the enabling environment for development and growth through the promotion of the private sector. The project's goal of providing high quality infrastructure and more efficient management of the road transport sector was thus in line with this strategy. The shortcomings of the road transport system in India are also a key feature of the Country Partnership Strategy for 2009-2012, which focuses on improving infrastructure to help sustain rapid economic growth to achieve poverty reduction objectives.

Mizoram’s state road network of 6,840 km of single lane roads (largely unpaved) provide the only means of transport for goods and people within the state. Poor road quality and inadequate capacity to manage them resulted in a low level of service to road users who endured inadequate riding quality, slow travel speeds (10-15 kilometers per hour) and high vehicle operating costs. Given that access to a reliable transportation network is a precondition for a competitive economy, Mizoram state's development agenda attaches an overriding importance to improving the road network to facilitate access to all parts of the state and beyond.

Relevance of the project objective is rated high.

b. Relevance of Design:
The project design appropriately included a range of physical works (widening, strengthening, and rehabilitation/maintenance of state roads) to support the sub-objective of improving the carrying capacity of the road network. The project design also provided for measures to improve the state's institutional capacity for the roads sector, including processes for managing the road network in the long term (planning, financial accounting, inventory management, performance-based contracting, inter-agency co-ordination, training for upgrading technical and other skills etc). The results framework thus provides a logical causal chain linking the physical components of the project with intermediate and final outcomes. However, while the nexus between the institutional components and intended intermediate results is adequately made, the ultimate impact on institutional and financial capacity is less clear from the results framework.

Relevance of the project design is rated substantial.


4. Achievement of Objectives (Efficacy) :

To improve the management of the Mizoram core state road network. Rated substantial.

The state government established an Institutional Strengthening Group with the responsibility of mainstreaming the project’s Institutional Strengthening Action Plan across the Public Works Department. The ICR states that the Department gained substantially from these reforms and established procurement, contract management and management information systems. The ICR also states that the Department is now independently able to plan and monitor programs of larger size; "and makes decisions based on resource allocation in accordance with technical and economic analysis, rather than local politics." The project team adds (email dated Jan 22, 2013) that “In the sector’s view, the PWD has undergone a remarkable transformation through the project. It is now responsible for planning of the road network under the ambitious North-East Multi-Model Transport Project and for management of the National Highways (this responsibility was earlier with the national Border Roads Organization, which has a generally much higher level of capacity). The PWD is responsible for planning, design, and execution of road works ranging from $ 10 – 110 million.

The Department administrative costs were reduced to 15 percent of the total state road sector budget, significantly better than the target of 20 percent.

The ICR, supplemented by additional information by the project team (email dated Jan 22, 2013) details the following outputs:

Provision for road maintenance: The ICR states) that “the state government has been unable to increase funding for sustained maintenance due to lack of fiscal space (even though some additional central government funds are becoming available over a four year period). The actual expenditure on road maintenance is only about 35 percent of the amount estimated to be required to keep the roads in good condition, and is resulting in a growing backlog of maintenance needs.” The project team clarifies (email dated Jan 22, 2013) that “both the current road sector funding (INR 5,730 million) and maintenance funding (INR 449 million) for FY 12-13 have seen a double digit increase from the pre-project level. The available maintenance funding fully meets the needs of routine maintenance and 45% funding requirements for routine and periodic maintenance combined for the entire road network. Notwithstanding the shortfall for the entire road network, the available funding would be adequate to meet the 80 percent maintenance target for cores state road network considering that (i) Mizoram has received about INR 2,000 million from the Planning Commission for the rehabilitation of the core road network and there are many other planned funds available for major maintenance of the core state road network to make-up for any shortfall; and (iii) the maintenance funding framework in Mizoram, as elsewhere in India, has substantially improved through the grants recommended by the 13th Finance Commission (INR 329 million for Mizoram in FY 12-13) and the focus is now shifting from more funds to their effective utilization. There has been no increase in the maintenance backlog for cores state road network.

Road Fund: The ICR states that “although the Road Fund legislation has been approved by the state government and board members have been appointed, the fund is not in operation which is a serious shortcoming.” The project team clarifies (email dated Jan 22, 2013) that the project only set a target of completing a feasibility study of establishing the Road Fund. After completing the feasibility study, the state government set up the Road Fund and completed the key steps of drafting the Road Fund Rules and Road Fund Act, and submitted the Mizoram Road Fund Act to the state assembly for approval and appointed the Board members. The next step is the operationalization of the Fund.

Road contracting industry: The ICR states that "specific measures in assisting the modernization of the local contracting industry – which was relatively inexperienced – were identified but not followed through." The project team clarifies (email dated Jan 22, 2013) that the project introduced the process of modernizing the construction industry in Mizoram by introducing modern road construction equipment, improved technologies and material specifications, better management of social and environment impacts during construction, international contracting and dispute resolution systems, better resource planning and contract management, and on-the-job training of contractors’ staff. The project has also supported development of area-wide road maintenance contracts which are now being initiated in two districts (Aizawl and Serchhip). The contractors are now executing road works ranging from $ 10 million to $ 100 million (compared to less than $ 1 million before the project).

Road Management System: The ICR states that a computerized Road Management System is in place but not yet implemented. The project team adds (email dated Jan 22, 2013, that the system requires some further refinements to realistically predict the damages in hill roads and define suitable maintenance treatments for the same. The PWD has established a good road inventory and condition database of its road network, and built in-house capacity to undertake road inventory and condition surveys using modern instrumented vehicles . The PWD is already using the available data in the planning and prioritization of road improvement and maintenance works.

Information technology systems for business processes: The ICR states that "the Public Works Department has procured hardware and software for an information technology system to enhance efficiency in its business processes, but the system is not yet finalized." Noting developments since project completion, the project team adds (email dt Jan 22, 2013) that with the project-funded assistance for hardware and software, the PWD now has a fully-developed information and management system (PWIMS) having modules including Accounting, Cost-Estimation, Human Resources Management. In terms of usage, the software is being piloted in three divisions of PWD. The design cell of PWD is already using the software for engineering designs of roads and bridges to be funded under various programs. About 50 PWD engineers have been trained in the use of these software packages. The IT systems procured for Quality Management System are also fully being utilized in the divisional office

Planning and quality control: The state government enacted the Mizoram Highway Act (2002); and prepared a Road Development Policy (2002) and Rural Roads Master Plan. Planning and Quality Assurance Units were established and made functional. Selected offices in Aizawl (the state capital) acquired ISO 9001* quality certification in 2010. The Public Works Department intends to scale up ISO 9001 to cover the entire department over two years following project completion. Environmental and social policies and guidelines were finalized and adopted organization-wide by the Department.

Training: In respect of training, the project team (email dt Jan 22, 2013) states that a flexible and need-based approach was followed by catering to various training needs as soon as those were identified during project implementation . About 200 personnel were trained through various national and overseas technical programs; about 90% of them are from the Engineering discipline and the remaining from Accounts and IT departments.

Road Safety: A number of road safety engineering measures have been completed. However, the quality of the improvements and the effectiveness of their implementation were not measured.

*The ISO 9000 is a family of standards, published by the International Organization for Standardization, and designed to help organizations meet the needs of customers and other stakeholders while complying with statutory and regulatory requirements.


To improve the carrying capacity of the Mizoram core state road network. Rated substantial

The project team reports (email dated Jan 22, 2013) that, overall, "all the 471 km of roads (for which the project was finally responsible after mid-term restructuring) have been fully completed, have fulfilled their impact on the social and economic development of this remote region and are currently carrying about two times the traffic compared to appraisal projections."

The scope of the maintenance component was reduced from 518 to 297 km (57 percent of original plan), through a formal restructuring following the mid-term review. The project team adds that the remaining 221 km roads have already been rehabilitated or are being rehabilitated under other programs financed by the Asian Development Bank or the central/state governments.

The upgrading of the road has reduced the distance between the state capital (Aizawl) and the second important town (Lunglei) by about 70 km and the travel time by more than 3 hours. The cost to passenger travel from Lunglei to Aizawl has come down from INR 300 (US$6.72) to INR 270 (US$6.05) in maxi cabs despite significant increases in the cost of fuel. Similarly, the 28 km project road between Aizawl and Aibawk has seen a reduction of INR 5 per passenger by bus/maxi cabs.

The project widened and strengthened 174 km of state roads were against a target of 184 km (95 percent). The project team adds (email dated Jan 22, 2013) that the same that “the same objective of upgrading the road between Aizawl to Lunglei was achieved more efficiently with 174 km length compared to the PAD estimate of 184 km due to better engineering, which saved both the construction and travel costs associated with the extra 10 km.” This led to an estimated reduction in travel time of 65 percent against a target of 30 percent, and substantial savings on road user costs on the improved portions of the core state road network. The portion of the core state road network in good condition (International Roughness Index [IRI]< 4 meters/kilometer) increased from zero to the targeted 50 percent. The portion of core state road network in poor condition (IRI > 12meters/kilometer) remained almost static at ten percent, against a target of five percent (ICR page 11, para 3.2.5). The task team clarifies in its memo dated Jan 22, 2013 that "there was no target set to reduce the percentage of CSRN in bad condition to 5 percent."

A meeting was organized at Aizawl on May 20, 2010 to interact with beneficiaries. Overall, attendees expressed satisfaction with the completed operation. However, attendance consisted mainly of Public Works Department employees; road users were not represented. The attendees raised issues concerning contracting capacity, road safety, environmental issues, and road management capacity as key factors affecting the sector. Separate informal interaction with village councils and some prominent residents of Lunglei confirmed a general feeling of satisfaction with the eventual outcomes.


5. Efficiency:

The Economic Rate of Return (ERR) at project completion for the project’s widening & strengthening component – which accounted for 74 percent of the final project cost - was 17 percent, which was marginally higher than at appraisal (15 percent). To the extent possible, the same methodology and assumptions used during the appraisal stage were used at completion (i.e. in accordance with the Indian Roads Congress guideline SP-30), but replacing forecast values with actual values where appropriate. The ICR states that although the civil works costs more than doubled and the time overrun was 36 months (a 60 percent increase over the original 5 years), these were compensated by the growth in traffic (86 percent). The assumptions for traffic growth rates from 2011 onwards are lower than the actual growth rate experienced in the previous last 10 years, making the ERR projection conservative.

The benefit-cost ratio is lower at completion for most of the sections where information is available. However, with the increase in the volume of traffic in the project roads (from a predicted annual average growth rate of 8.5 percent at appraisal to an average of 15.4 percent at project completion) has helped keep the ERR above twelve percent for most links compared to the overall 18 percent estimated at appraisal.

The highway model used for ERR calculations assumes that there will be additional and timely maintenance to make up for the greater volumes of traffic.

Supervision consultant’s costs increased sharply, despite the fact that the Bank also provided technical advice to the Public Works Department, because of the construction delays. According to the ICR (page 6, para 2.2.4) final figures show that supervision costs were nearly 35 percent of the civil works costs. The project team clarifies (email dt Jan 22, 2013) that the correct figure is not 35 percent but about 11.1 percent even after including the cost of engineering design ($ 11 million for the total civil works costs of $ 99 million).

The task team adds that several features that are specific to the project - the remote and difficult terrain, narrow time window for construction due to local weather patterns, and the need to retain manpower during lean periods – were inherently responsible for higher supervision costs than for projects in less demanding circumstances. Even if other efficiency-related factors that raised supervision costs were kept under control, the costs would be expected to be significantly higher than the average. The overall 223 percent increase in the state road improvement component (from US$39 million to US$87 million) is attributed to the extra supervision costs and the delays in implementation and weak contract management, apart from factor's outside the project's control - increases in the global price of inputs (especially bitumen, steel, diesel fuel and cement), and the need for additional quantities. As already noted, there were time overruns of three years and a cost overrun as well. The project team ads (email dt Jan 2, 2013) that the actual unit costs of INR 25.1 million/km and INR 1.9 million/km, for road improvement and road rehabilitation, respectively, are of the same order of similar road projects in the state.

Other issues mentioned in the ICR that had a bearing on efficiency included shortcomings in designs that were partly based on visual observations and not on actual field conditions; inability to divert traffic during construction, slowing down on-going activities; time and cost overrun in resettlement and rehabilitation; lack of coordination with the field staff; inability to take timely actions with respect to poorly performing works contracts; a lack of adequate quality control over site works; inadequate contractor capacity, poor contract planning and monitoring, and insufficient plant, equipment and manpower at the sites.

Taking all these factors into consideration, efficiency is rated modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
17%
90%
ICR estimate:
Yes
15%
90%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The relevance of the project objectives is high, while the relevance of project design is rated substantial. Efficacy is rated substantial based on the level of achievement of both physical and institutional outputs/outcomes. Efficiency is also rated modest taking into account the large time and cost overruns experienced by the project. Overall outcome of the project is rated moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The most important risks to development outcome arises from the availability of funds for ongoing maintenance, and from sustaining the improvements in institutional capacity from the project. Moreover, traffic growth on the upgraded project roads has been much higher than anticipated as have also the diverted traffic volumes during construction. This has put greater stress on the roads than anticipated and they are likely to require more maintenance far sooner than planned. It is unclear that there will be sufficient available funds for this purpose.
The ICR reports that the state government appears to be committed to the institutional reforms, as evidenced by its persistence with other adopted sector policies and the senior management's understanding of the issues.

The ICR states that, “until recently, the actual expenditure on road maintenance was only about 35 percent of the estimated needs to keep the roads in good condition and is resulting in a growing backlog of maintenance needs. This raises doubts concerning the sustainability of the project investments as well any future investments that are made for the core road network. However, as stated in the their email date Jan 22, 2013 (also see section 4 above) the project team indicates that “the available maintenance funding fully meets the needs of routine maintenance and 45% funding requirements for routine and periodic maintenance combined for the entire road network. Notwithstanding the shortfall for the entire road network, the available funding would be adequate to meet the 80 percent maintenance target for cores state road network.”

Risk to development outcome is rated moderate.


a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:
This operation was the first significant transport project in one of the least developed, remotest and hilliest regions of India, and considerable effort was made in preparing the officials and stakeholders for what was involved in undertaking a large scale project. Based on state road projects in other parts of India, the project team was able to incorporate into the design well tested approaches involving international best practice encompassing economic analysis, quality assurance systems, engineering know-how, public accountability and business practices, procedures and processes. ‘Soft’ components were incorporated into the design to strengthen government organization, as well as technical and business capacity for economically sound road planning, budgeting and accountability. Recognizing the sensitivity of the project to the environment and affected people the design included a sound and extensive participatory process. An HIV/AIDs awareness program was also included which had a positive reception leading to a second campaign with a wider coverage. The project was prepared in a participatory manner with extensive involvement of key stakeholders, including local and district government units, community leaders, trucking associations, nongovernmental organizations (NGOs), and project affected people including farmers, villagers and roadside residents.

However, design called for the rapid expansion of the institutional capacity for contract and project management in line with the scheduled pace of project implementation. This was over-ambitious given the known weaknesses in capacity. Design also lacked specific measures to build the necessary capacity including how the project would assist in the modernization of the local contracting industry, other than providing stable funding for road works contracts. The project team adds (email Jan 22, 2013) that the project design made adequate provisions by using private sector capacity services for engineering designs and supervision and technical assistance, fully recognizing the limited capacity of the Borrower.

The PAD mentioned a number of lessons taken into account from previous operational experience which included paying attention to comprehensive and quality preparation of engineering, social, environmental, and institutional aspects; and strict prequalification and selection of contractors and consultants. However, but some mistakes in the design were still repeated, such as not emphasizing sufficiently the importance of site readiness for the timely commencement of civil works, and not stipulating thorough screening processes for selecting of contractors and consultants. Moreover, the project team underestimated the local consultants’ understanding of some project specific risks; in particular those associated with the preparation of quality hill road designs.

Due to an inherent design flaw, a significant portion of the civil works had to be redesigned and led to project delays. Because certain physical works under the rehabilitation and maintenance components were completed much in advance of the up gradation works, these segments were instantly used by traffic as detours and caused unanticipated damage. The ICR states that the risk mapping also did not take sufficient account of some other key risks associated with civil works construction in the local environment, such as, unavailability of suitable construction material, the relatively small window for physical construction during the year, a long material and equipment supply chain, environmental risks associated with fragile geological exposed cut surfaces, and the sequencing of physical works across components. The project team (email dated January 22, 2013) adds that even so, the project design adequately included mitigation measures for the risks such as soil stabilization and other modified technical specifications to use local material; improved work programs to account for the small window for physical construction; better resource planning and use of PWD’s equipment considering supply chain issues; and bio-engineering measures to mitigate the environmental risks.

The project significantly underestimated the physical and price contingencies (which the ICR suggests were pegged at only two percent in each case to fit within the available IDA credit envelope), and this partly explained the need for subsequent additional financing. The Project team (email Jan 22, 2013) clarifies that the central government allows only small contingencies to exercise better fiscal discipline as 90 percent funds were central government grants to Mizoram.

Quality at entry is rated moderately unsatisfactory.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
Supervision missions took place twice a year on average, which the ICR consider to have been adequate. The supervision teams brought to bear relevant skills in a cost-effective manner, relying where possible on country office staff. The Mid-term review took place 30 months after the project effectiveness (June 3, 2002), and took note of the slow pace and increased cost of civil works. It expressed concern at the slow progress of all four civil works contracts, and in particular the Phase I road improvement contract that had been awarded 18 months earlier, but in which physical progress, at the time of the Review, was only 7.5 percent. The review also recognized the need for additional counterpart funds, as well as a plan to fund the maintenance and rehabilitation of Phase II roads. Following the review, the maintenance component was scaled down, and the Phase II maintenance activities were excluded, mainly due to the availability of alternative sources of financing.

Steps could have been taken to monitor civil works supervision costs more closely and contain them where appropriate. However, the quality of the permanent pavement works at project closure was generally satisfactory.

The project team could also have insisted on better M&E reporting during implementation, especially for the Institutional Strengthening Action Program activities, and could have followed up more assertively to ensure that proper monitoring took place. The Bank has played a role in assisting the state government to focus on the importance of maintenance and building institutional capacity.

With regard to safeguards, the Bank was able to ensure that the project ultimately conformed to the guidelines in respect of consultation and transparency (see Section 11 below). The team created goodwill with the affected communities by introducing small sub projects to provide soccer pitches and small ponds.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The state government demonstrated a strong level of commitment and ownership toward the project design, preparation and implementation through a number of actions including: (i) full agreement with the Institutional Strengthening Action Plan; (ii) endorsement of the road development policy and adoption of a roads master plan outlining the state’s commitment to improve both management and funding of the road sector; (iii) maintaining a high degree of continuity in its project management team; and (iv) providing adequate counterpart funds through the budget.


According to the ICR, at project completion, the state government did not provide adequate resources recommended in the master plan for maintenance of the road network, including the project roads, "which has put a question mark on the medium to long term sustainability of the project assets." However subsequent developments (as per email Jan 22, 2013) appear to have made adequate provision for maintaining the core road network, and improved the availability for maintenance funds for the rest of the network (see section 4).

A planned stakeholders workshop to cover issues of road performance and management was unsuccessful and no result was forthcoming.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:
The ICR reports that this complex project proved to be “a huge challenge from design to construction” for the Public Works Department, the implementing agency. The Department recognized its limited capacity for handling the contracts under this project which were the largest it had encountered thus far.

In terms of institutional capacity, the Department and the Project Implementation Unit, which was housed in the Public Works Department, faced a lack of skilled manpower. There was not enough capacity to monitor and closely supervise the consultants and contractors for the project. The mobilization of contractors, plant, staff, and equipment was slow.

The Department and the Unit faced constraints from heavy annual monsoon rains during a few years which held up work. In respect of procurement they could not attract enough competition among bidders due to the difficult terrain in which they would have had to operate. Materials and natural aggregates for road construction were expensive and difficult to obtain in required quantities and quality. The project experienced shortages and disruptions of major commodities such as bitumen, oil, cement, steel and diesel fuel.

The engineering designs prepared by the consultant for Phase I of the road improvement component were assessed to be deficient close to the commencement of implementation. . The design consultant did very little to remedy some of the many design issues that arose during construction of works. Furthermore, the design and supervision consultant disagreed, and this created issues of coordination and continuity.

At the Mid-term review, the issue of inadequate documentation related to Resettlement and Rehabilitation disbursement was also highlighted. A detailed post hoc verification of project affected persons undertaken in July 2006 indicated several gaps in documentation. Although no financial anomalies were detected, this was nevertheless a cause for concern and the project subsequently improved its resettlement documentation and processes. The social and environmental aspects, though satisfactory in the end, experienced problems in execution. .

Nevertheless, the Public Works Department pro-actively supported the proposed road sector reforms and took the initiative to acquire and demonstrate new capabilities in handling some large new civil projects. Initially, the Project Implementation Unit was too lenient with contractors who did not meet their obligations, reflecting its inexperience. Both the Department and the Unit evolved to a considerable extent over the project implementation period. Encouraged by the Bank, the Department also embraced the practice of greater transparency, through ‘clean’ procurement and consultation with road users, other government branches and civil society.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The M&E design included several appropriate outcome indicators and intermediate outcome indicators to monitor progress and the extent of achievement of the PDO during implementation. However, some indicators listed in the PDO indicators matrix such as economic rate of return or delivering works on time are not in the nature of output/outcome indicators.

The main outcome indicators were travel time on improved sections; share of state road network in good condition; periodic maintenance backlog; and Public Works Department administrative costs as a percentage of sector budget. Intermediate outcome and output indicators included improved road capacity and quality; establishment and proper functioning of planning, quality assurance, and Environmental and Social Management Units within the Department by project completion; number of effective training days; improved allocation and provision of adequate funding for the sector; a functioning road management system in use in Public Works Department headquarters and some field offices; and a study on the Road fund.

Provision was made for the Public Works Department, responsible for implementing the M&E system, to be assisted by consultants in various aspects of data collection. Baseline data were specified for all output and outcome indicators.

b. M&E Implementation:
The ICR states that the performance indicators were monitored and evaluated periodically by the supervision missions of the Bank in collaboration with the Public Works Department. However, the Institutional Strengthening Action Plan results were monitored poorly or in some cases (such as training days) not at all. The performance of consultants in assisting with M&E was not always fully satisfactory. At the Mid-Term Review, the maintenance component was scaled down and the Phase II maintenance activities were excluded. Nevertheless, the performance indicators were not modified at the time of the Review with the exception of the extent of roads to be maintained under Phase I which was revised downwards by 40 percent. However, the project team’s email dated Jan 22, 2013 states that the quarterly/monthly progress indicators for all project activities including institutional development activities were established and closely monitored.

a. M&E Utilization:
The project monitored indicators for outcomes, intermediate results and outputs, which included baseline data at appraisal. The indicators were sufficiently appropriate for broad decision making and resource allocation purposes. The data were used in monitoring progress as well as in tracking outputs and outcomes.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:
The project was classified as environmental category “A” (full assessment) because the hilly terrain posed issues in respect of erosion, drainage and disposal of soil, and because the road corridor passed through an area of rich biodiversity. There were also several social issues surrounding land acquisition, compensation and relocation of affected people, community land ownership and land dependency. The Bank’s safeguard policies were triggered for Environmental Assessment (OP 4.01), Natural Habitats (OP 4.04), Involuntary Resettlement (OP 4.12), Indigenous Peoples (OP 4.20), and Cultural Property (OPN 11.03). The ICR reports that an Environmental Management Plan (which also covered Natural Habitats and Cultural Property), a Resettlement Action Plan, and an Indigenous Peoples’ Action Plan were prepared and disclosed to the public.

The ICR states that Environmental and Social Safeguard compliance overall is rated satisfactory, with environmental safeguards being fully satisfactory and social safeguards satisfactory but with qualifications. This is consistent with the project’s final Supervision and Status Report dated Nov 10, 2010, which states that overall safeguard issues were moderately satisfactory with no pending issues. Individually, performance is rated satisfactory for Natural Habitats (OP 4.04), Involuntary Resettlement (OP 4.12), Indigenous Peoples (OP 4.20), and Cultural Property (OPN 11.03), and moderately satisfactory for Environmental Assessment (OP 4.01). Further details are stated in the ICR as below.

Environmental Safeguards: The project led to the setting up of an Environmental Management Unit in the Public Works Department, training of local engineers and the preparation of environmental management guidelines. Hillside cutting for a major length of the project route posed serious issues of stability of hill slopes given the immature geology of the project area. For slope protection, a combination of conventional techniques (such as retaining walls and stone pitching) and bio-engineering techniques (such as terracing hill slopes with locally available bamboo) was used. Although “cut and fill” measures utilized 40 percent of the soil, about 0.7 million cubic meters of debris still required disposal. There were initial delays in implementation of environmental management plans due to capacity limitations and harsh climatic and geological conditions. However, the ICR reports (page 7) that the Public Works Department took appropriate measures to address institutional capacity and implement the Environmental Management Plan. Also according to the ICR, extensive community consultations proved to be effective in implementing Environmental Management Plan measures. Some soil was compacted and used to establish soccer pitches for the recreation of local residents. The goodwill thus created was extended further through the creation of small roadside retaining structures at five sites to provide ponds to give the communities easier access to water. The ICR states that the performance of environmental safeguards varied during preparation and implementation of the project from Satisfactory to Moderately Unsatisfactory, but the overall rating was satisfactory.

Social Safeguards: The total land acquired under the project was 104.97 hectares and involved 1,717 project affected families (the estimate in the PAD was 1,504). This includes additional land acquisition of 11,238 square meters to provide adjustments in designs. At appraisal, the cost of land acquisition, compensation and related costs was estimated at US$2.25 million. The final cost was US$3.22 million. The ICR reports (page 8) that the majority of the project resettlement activity was completed on time (for Phase 1 roads during 2005 and for Phase 2 roads by October 2006). An end-term independent monitoring and evaluation of the implementation of the Resettlement Plan and Indigenous Peoples Development Plan was completed for the project in June 2008 by an external consultant. It was found that institutional arrangements for Plan implementation were effected as agreed. NGOs implemented the Resettlement and Indigenous Peoples’ Plan with the assistance of the Project Implementation Unit. The NGO team identified for the project during Phase I, however, had to be replaced by another team for Phase II due to poor performance. Implementation of the Resettlement Plan and Indigenous Peoples Development Plan found to be acceptable to the Bank, as per the task team’s memo to the implementing Agency dated December 2, 2008. Regarding the end term evaluation report, the timely payment of resettlement and rehabilitation assistance and compensation played an important role in Plan implementation. Project affected families under both the phases reported in an ex post survey that the assistance/compensation money was timely (72 percent), and adequate (62 percent). These reflect noticeable dissatisfaction rates, especially for the adequacy of compensation (38 percent).

The ICR reports that the overall status of social safeguards at project closure was reasonably satisfactory with no critical issues pending, but some areas were flagged – such as loss of access to properties, and the safety of buildings and losses of crops and trees due to hill excavation - where procedures could be further improved over time. Safeguard compliance benefitted from robust design and supervision and went beyond mere compliance through listening to and then assisting the affected communities with small low cost projects.

Construction-Induced Social Impacts: The project had to devote substantial attention to address the issues of loss of access to properties, and the safety of buildings due to hill excavation and careless dumping downhill resulting in losses of crops and trees. Several complaints were received by the Public Works Department in this regard. Moreover, due to substantial delays in completing the road works, the construction sites remained open for a much longer duration than anticipated and the work activities accordingly caused extended inconvenience to roadside inhabitants outlined in Aide Memoires and M&E study. These issues were, however addressed during implementation, and, as stated above, there were no pending issues at project completion.

b. Fiduciary Compliance:
Procurement. The ICR reports that procurement of works, goods and consulting services was carried out in accordance with the Bank's guidelines, And that, overall, the procurement process was carried out without any irregularities.

A major works contract was cancelled and rebid by splitting it into three smaller contract packages. The Bank assisted where it could by encouraging on-the-job procurement training. Considerable delays in implementation and higher than anticipated costs on several contracts contributed to the need for additional financing and two extensions of the project closing date. No cases of mis-procurement were reported.

Financial management, The Project Implementation Unit appointed a qualified chartered accountant in May 2002. Submission of quarterly Financial Monitoring Reports introduced from 2007 onwards was regular although there were some delays in submitting them to the Bank. Annual external project audits by the Auditor-General were regular and the audit reports were then submitted to the Bank, albeit with some delays. However there were persistent delays in addressing the external auditor’s observations. A risk-based internal audit, aimed at strengthening internal controls, was introduced from March 2009. After the approval of the first additional financing, use of standardized off-the-shelf accounting software (Tally) was implemented to overcome the inadequacies of the Public Works Department-developed project financial management software. The ICR reports that there were no outstanding qualified audits at project completion. However, some suggestions by the internal auditors for further strengthening internal control arrangements remained unimplemented due to delays as the project neared closure.

The project’s contract management was weak, which contributed to considerable delays in obtaining the Bank’s no objections to extension of time and price variations. These delays were repeatedly highlighted by the Auditor-General’s audit reports and followed up by the Bank for resolution.

During the Mid-term review, the issue of inadequate documentation related to resettlement and rehabilitation compensation disbursement was highlighted. A detailed verification of project affected persons undertaken in July 2006 indicated gaps in documentation, but no financial anomalies were detected. The project subsequently improved its resettlement and rehabilitation documentation and processes.

c. Unintended Impacts (positive or negative):
An initial HIV/AIDS awareness program planned as part of the R&IPDP received a positive reception locally, and accordingly a second HIV/AIDS awareness campaign with a wider sweep was carried out at an additional cost of US$0.6 million.

d. Other:
The project team (email Jan 22, 2013) provides the following assessments of its own, as well as a survey undertaken ba the ‘Bank’s external team” in 2009: Project roads are giving a big boost to the economy of the region. Land values along the new roads have increased substantially. Agricultural incomes have also increased and farmers are now able to take their produce by road to Aizawl where they fetch higher prices. The sale of two wheelers and second-hand cars has increased, and workshops that repair light vehicles are being established. In the weaving town of Thenzawl, women weavers who earlier sold their distinctive fabrics to middlemen in Aizawl have now begun to open small stalls along the road where they sell to customers directly, increasing their earnings.



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Significant
Moderate
According to the project team, since project completion, the Government has increased the provision of maintenance funds for the core state road network and the rest of the road network.  
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following important lessons from the project experience are adapted from those mentioned in the ICR:

Institutional change in a complex sector such as transport should be approached gradually over a medium term time-frame. The experience from this project suggests that the Bank should commit support for a commensurate time period if it wants to assure sustainable results in institutional development in the transport sector. A program approach with triggers to move from one stage to the next may be more appropriate than a single operation. In time it should be possible to transition from a project implementation unit to a more stable integrated departmental solution for project management.

Project design should take into account the client's resources for implementation and supervision in designing a complex project. The inclusion of too many components in this project negatively affected implementation, because both the client and the Bank had limited resources for effective supervision and implementation.

The set of performance indicators for a project should be restricted to those that are most objective and measurable. The experience from this project shows that when institutions have limited capacity to begin with, it may be prudent to limit the number of components. In such a situation, fewer, more measurable monitoring indicators and a well-designed evaluation system will better assess the achievement of project outcomes.

14. Assessment Recommended?

Yes
Why?
Given the substantial new information and assessments presented by the region through its email dated January 22, 2013, it would be useful to assess important outcomes from the project a few years from now. Such an assessment can be usefully combined with similar state-level roads projects in India that have been completed recently.

15. Comments on Quality of ICR:

The ICR made an informative and analytical examination of the project's experience. It has tried to marshal considerable information and evidence for its arguments. The ICR discussion pays adequate attention to the project outcomes, alongside the discussion of outputs and intermediate outcomes. Shortcomings in project design and implementation are acknowledged in a frank manner throughout the narrative. The length of the ICR is reasonable. Overall, the ICR quality is rated satisfactory.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions