Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Rural Roads Project


  
1. Project Data:   
ICR Review Date Posted:
06/21/2013   
Country:
India
PROJ ID:
P077977
Appraisal
Actual
Project Name:
Rural Roads Project
Project Costs(US $M)
 685.9  708.6
L/C Number:
C3987, L4753
Loan/Credit (US $M)
 400.0  411.2
Sector Board:
Transport
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  09/23/2004
 
 
Closing Date
03/31/2010 03/31/2012
Sector(s):
Roads and highways (95%), Sub-national government administration (5%)
Theme(s):
Rural services and infrastructure (100% - P)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Kavita Mathur
Roy Gilbert Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
To achieve broader and more sustainable access to markets and social services in Participating Districts (PAD p. 5 and Loan Agreement Schedule 2, p.19)

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
1. New Connection and Upgrading of Core Rural Road Networks (appraisal estimate US$430.9

million, actual cost not available).

This includes new construction and upgrading of 9,900 km of the core rural road network in four states (Himachal Pradesh, Jharkhand, Rajasthan, and Uttar Pradesh) to provide all-weather road access to habitations in identified project districts.

This component would also finance Consultants in each state to undertake independent technical examination of Bank funded works and also to verify: (i) proper application of environmental and social and economic screening procedures for the selection of rural road sub-projects; (ii) detailed design are in compliance with agreed technical standards as well as stipulated environmental and social management measures; (iii) the quality of bidding documents is satisfactory and that procurement is undertaken in conformity with agreed procedures; (iv) compliance of actual works with contract conditions and quality assurance procedures as well as agreed environmental and social management measures; and (v) expenditures under the Credit/Loan have been made for the purpose intended (financial review).

Wherever roads require compensatory afforestation to meet the provisions of any Forest Department clearance, State governments will deposit funds to undertake this afforestation prior to the clearance becoming effective.

2. Periodic and Routine Maintenance of Core Rural Road Network in Project Areas (appraisal estimate US$, financed by State Governments, actual cost not available).

This includes an annual implementation of periodic and routine maintenance program in every district where the Bank is providing additional investment for a total of about 100,000 km roads.

3. Institutional Development (appraisal estimate US$13.56 million, actual cost not available):

This includes institutional development at both national and state level.

National Level:

(a) Technical assistance and training to Ministry of Rural Development, and the National Quality Monitors to build capacity in various areas, and

(b) Setting up of a poverty impact and rural road user satisfaction monitoring system.

State level:

(a) Technical assistance to each state to: (i) develop and establish a simple Road Management System at the headquarters and in district field offices; (ii) prepare annual maintenance programs and support the implementation by the road agency of these programs, including the use of performance based contracting, on the core road network; (iii) recommend and help implement on a pilot basis a framework for transferring ownership of non-core rural roads to Panchayat Raj Institutions; and (iv) transfer skills and procedures to an adequate number of staff in the road agency to sustain the use of the Road Management System and continuing implementation of maintenance.

(b) Procurement by each State of material and quality control testing equipment, IT and associated office equipment - for use by the implementing agencies.

(c) Staff training within the main rural road agency plus the local contracting industry.

4. Incremental Operting Cost (appraisal estimate US$4.4 million, financed by State Governments, actual cost not available).

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: The appraisal estimate was US$685.9 million, but the ICR reports (p. 28) these appraisal costs as US$409.87 million. The ICR does not provide actual project cost. Actual projects costs had to be taken from the project portal, which amounted to US$708.6. Instead of total actual project cost it provides figures for the Bank's contribution only.
Financing: At appraisal IDA committed US$300.0 million, actual amount was US$311.7 million. IBRD commitment at appraisal was US$100.0 million, actual amount was US$99.5. The Credit and Loan were fully disbursed and the differences in appraisal and actual amounts are due to exchange rate fluctuations.

Borrower Contribution: The appraisal estimate was US$285.9 million. The ICR does not provide information about the actual borrower counterpart contribution. According to the Project Portal, the actual Borrower Contribution is US$309.06 million.

Date: The closing date of the project was extended by two years from March 31, 2010 to March 31, 2012 to: (i) consolidate the institutional development initiatives; (ii) complete the ongoing and additional rural roads works reallocated to Uttar Pradesh (US$27 million) and Rajasthan (US$9 million); and (iii) undertake further capacity building activities planned for rural roads agencies.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Rated substantial.
The project objectives remained consistent with the most recent Country Assistance Strategy for the period FY 09-12 which aimed at achieving rapid, inclusive growth, ensuring that development is sustainable, and increasing the effectiveness of service delivery. To achieve rapid, inclusive growth the Strategy aimed to assist in removing infrastructure and skills constraints to growth in both urban and rural areas.

Exclusion has geographic and social dimensions. Of the 300 million people who lived below the official poverty line in 2005, nearly 60% were in the seven so-called "lagging" (or low-income) states- Bihar, Chhattisgarh, Jharkhand, Himachal Pradesh, Orissa, Rajasthan, and Uttar Pradesh. Over 600,000 villages are not connected to a road.

The project objectives were in line with the Government's priority of addressing the problem of poor rural accessibility and improving planning and management of rural roads. The Prime Ministers Rural Road Program (Pradhan Mantri Gram Sadak Yojana, PMGSY) was launched in late 2000 and the project is part of the program. In 2005, PMGSY incorporated the Bharat Nirman targets for road connectivity. These targets are to connect all habitations of more than 1000 people (500 in case of hilly, desert, and tribal areas) by the year 2009 and include the upgrading of 194,000 km of rural roads by the year 2010. This is also reflected in India's Eleventh Five Year Plan (2007-2012).

b. Relevance of Design:
Rated substantial.
The project objectives were clearly stated. There was a logical link between project objectives and project activities. The project design was based on the Comprehensive New-Connectivity Priority List framework and also extensive public participation. The project included new construction and upgrading of core rural network in four states which had poor connectivity, There is a strong linkage that new and improved rural roads would result in sustainable access to markets and social services.


4. Achievement of Objectives (Efficacy) :

Achieve broader and more sustainable access to markets and social services in participating districts: rated modest.
Outputs
  • By project closure the physical works for new construction were completed. About 9625 kms of rural roads that provided better access for 78 participating districts (or 97% of the target of 9,900 kms) were completed.
  • The Bank supported institutional development through: (a) the development of a pavement design manual; (b) independent reviews of engineering designs through technical examiners; and (c) the preparation of a rural roads manual by the Ministry of Roads and Development with separate technical specifications for rural roads. However, most of the implementation remained as per conventional designs in all the states, and the use of cost effective optimal designs employing locally available material was inadequate.
  • The project provided training in maintenance management systems, quality control, environment management and e-procurement that would help ensure that the access benefits of the project roads continue. The ICR (p. 8) reports that the project's training and capacity building activities have been mainstreamed. Except for Jharkhand, simple maintenance management systems were developed by Rajasthan, Himachal Pradesh, and Uttar Pradesh, and these were used to prepare the annual maintenance plans for their respective states.


Outcomes

The project contributed to increased access to markets and social services due to the construction of roads. Overall, about 93% of eligible habitations were connected compared to the target of 60%. (Note: According to the PAD (p. 12) the Prime Minister's Rural Roads Program (PMGSY) guidelines required preparation of core network plans, which ensure one all-weather road connection from each habitation to nearby market centers. The roads under the project will be selected from these core network plans). About 50% of the routes were in good condition in 2012 compared to a target of 43%. Routine maintenance funding is not fully assured raising concern about the extent to which the 'sustainable access' objective will be achieved (see Section 7).

Table: Percentage of eligible habitations with all-weather access to social services and markets.
Eligible habitations are defined as villages with more than 1000 people (500 in case of hilly, desert, and tribal areas).

Baseline
Target
Actual
Uttar Pradesh
50%
55%
97%
Himachal Pradesh
40%
60%
78%
Jharkhand
35%
60%
69%
Rajasthan
40%
65%
86%

Note: The actual figures are at state level and include domestic (PMGSY) funds. The ICR does not provide figures for the roads funded by the project.

To evaluate the potential economic and social benefits of the program, the Bank undertook a study to estimate the economic and social benefits of providing all weather access roads to different villages in the four selected States: Uttar Pradesh, Rajasthan, Himachal Pradesh, and Jharkhand. The baseline survey (date not provided in the ICR) covered 11490 households within 750 habitations across 33 districts. The road user satisfaction was assessed on the basis of four major factors: reliability, transit time, connectivity, and user- friendliness and assessed on three levels viz. “low”, “medium” and “High”. Overall the satisfaction level was found high (ICR p.iii) (The ICR does not provide specific numbers). According to the ICR (ICR p. 20), a methodology for undertaking poverty impact and road user satisfaction surveys has been developed by the Ministry of Rural Development for the Prime Minister’s Rural Roads program in various states in addition to the participating states. The Bank contributed to this methodology by participating in various meetings and providing comments.

The ICR lists the following findings from the survey:

  • increase in child vaccination by 8% due to improved access to health centers;
  • increase in number of female patients visiting private doctors by 8%;
  • increase in population in income category of "more than Rupees 10,000 per month" by 3.46%;
  • increase in area cultivated by 12%;
  • reduction in travel time.

There is an attribution issue i.e. the improvements listed above are likely but cannot be completely linked to improved roads.

5. Efficiency:

The ex-post Economic Rate of Return (ERR) was 15%, below the appraisal estimates of 19% (see table below for th four states). This was because of increased construction cost and time over run. The ERR calculation assumed 12% cost of capital and reinvestment rate. Cost escalation was very high in Himachal Pradesh (34.5%) followed by Uttar Pradesh (7.6%). The ERR analysis at appraisal and ICR preparation does not include counterpart financing (ICR p. 18).
The project closed after a two years delay due the the following reasons: (i) to consolidate the institutional development initiatives; (ii) to complete the ongoing and additional rural roads works reallocated to Uttar Pradesh (US$27 million) and Rajasthan (US$9 million); and (iii) to undertake further capacity building activities planned for rural roads agencies.
Completed Road Length (Km)
Ex-ante ERR
Ex-post ERR
Himachal Pradesh
984
21%
11%
Jharkhand
126
18%
17%
Rajasthan
6,287
17%
15%
Uttar Pradesh
2,227.9
25%
18%
Total
9624.9
19%
15%

The ICR does not provide the total actual project cost which are available in the project portal. No other measure of efficiency is discussed in the ICR.

Overall, project efficiency is rated modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
19%
80%
ICR estimate:
Yes
15%
85%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The relevance of the project objective and relevance of design were substantial. The project efficacy and efficiency were modest, partly due to lack of sufficient evidence. There was inadequate evidence on some of the intended outcomes (e.g., access to social services) and also on attribution of the reported outcomes to the project.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

  • Routine maintenance funding is not fully assured. In respect of maintenance of the core road network in the participating districts of the States concerned, the implementation experience has been mixed. Only after the Ministry of Rural Development stipulated to the States that the construction funds would be released only if the States would specifically earmark funds for maintenance, did the situation start improving. Initially, maintenance of core road network was neglected and it received attention mainly towards the last two years of the project. The ICR reports that the routine maintenance for five years (i.e. until 2017) is funded under the 13th Finance Commission which approved USD 4.2 billion for rural roads. The routine maintenance will be carried out by the same contractor who constructed the roads under the project. However, the ICR does not provide information on who will guarantee and supervise the contractor responsible for maintenance. The major source of maintenance funding is expected from diesel cess. Currently, a cess (or tax) equal to Rs .0.75 per litre of diesel is set apart for development of rural roads. The ICR does not state whether this amount is adequate.
  • Allocation of maintenance funds based on a transparent Maintenance Management System. All states except for Jharkhand developed simple maintenance management systems (MMS) to prepare annual maintenance plans and ICR reports (p 8) that Himachal Pradesh is allocating maintenance funds based on the MMS. The ICR maintenance management system in Uttar Pradesh and Rajasthan is in need of updating the data, which raises questions about the extent to which the system is actually being used on a regular basis in these states.

    a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
The project built on the Government of India rural connectivity program. The objectives were highly relevant to country priority and the Bank's Country Assistance Strategy. The project was proposed as a first in several credits/loans to the Government to support the rural roads program. The project design was technically sound, it benefitted from the experience gained from the implementation of the Government Rural Roads Program (PMGSY) in other states. Safeguards assessment was adequate (see section 11 below). M&E design was weak (see section 10 below) as the performance indicators referred to the performance of the program as a whole in respective states rather than performance in project districts. The project sought to facilitate sector planning and management reforms in the four participating states. An Environmental and Social Management Framework was prepared. The framework would provide overall guidance for the integration of social issues at each stage of the project cycle. Some risks were underestimated: (i) rural road staff design road works poorly; and (ii) delayed or improper procurement by rural road agencies. In the case of Jharkhand, the planned construction of roads was substantially reduced because weak capacity and resulting delays, which should have been foreseen. Another risk which was not identified during preparation was the reluctance of states to use Bank funds when funds could also be accessed through the normal Prime Minister's Rural Roads Program, which had fewer requirements in terms of safeguard and fiduciary compliance. Several states were not able to meet the additional requirements of the project. Limited capacity of the implementing agency and the National Rural Roads Development Agency was also a risk that was not suitably identified and mitigated. However, the Technical Assistance sub-project on Economic Reform provided additional capacity building support which helped mitigate this risk to some extent.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:
A total of seventeen supervision missions were carries out, on an average twice in a fiscal year. In FY 2009 there were three supervision missions. Supervision of the project was complicated as it included four states that were geographically spread out. The supervision missions identified issues which were likely to affect the progress towards the achievement of the project development objective. With the help of closer monitoring and hand holding by the Bank, some states showed substantial improvement. According to the ICR supervision of fiduciary and safeguard aspects was satisfactory (ICR p. 23). However, earlier re-allocation of funds from Jharkhand to Uttar Pradesh and Rajasthan would have made a project closing date extension unnecessary. Also, while reallocating the funds, corresponding targets were not revised, which made it difficult to monitor the performance of different states.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Government performance during project preparation was satisfactory. The project was envisaged to provide funds, additional to existing Government of India transfers, to four states - Himachal Pradesh, Jharkhand, Rajasthan and Uttar Pradesh. These states were selected because of low level of connectivity within the states. The Government was fully committed to the project. Detailed guidelines were prepared by the Ministry of Rural Development for planning, standard and specifications, and procurement.
The Government (Ministry of Rural Development and the National Rural Roads Development Agency) provided close monitoring throughout the course of the project, to ensure that lagging states came up to a satisfactory level whenever slow decision-making threatened to hamper the progress of certain activities. For instance, the initial slow procurement in states such as Uttar Pradesh and Himachal Pradesh as well as Jharkhand, poor compliance with ESMF, and slow roll out of the computerized Financial Management System (FMS) were met with increased pressure and resources by NRRDA, which undertook state-by-state monitoring reviews, as well as efforts at further training, improved documentation and stricter enforcement.

The Government counterpart funding was not reported in the ICR.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:
Of the four states, the performance of the States Road Agencies of Rajasthan and Himachal Pradesh was satisfactory. Uttar Pradesh faced initial issues with respect to frequent changes in senior management, inadequate staffing and weak coordination amongst agencies, but managed to overcome these with the help of pro-active support from the Bank. Some project works were initially delayed in the participating districts of Jharkhand, Himachal Pradesh and Uttar Pradesh due to delays in the preparation of Detailed Project report and lack of availability of contractors to the qualifications and experience imposed by the World Bank.
The implementing capacity of Jharkhand state was unsatisfactory right from the beginning of project implementation. Capacity and willingness to implement the program and resolve implementation issues in a timely manner remained a challenge. Compliance with fiduciary requirements and adequate ownership and staffing for the same remained an issue. Eventually, in September 2009, proceeds earmarked for Jharkhand were re-allocated to Rajasthan and Uttar Pradesh.

There was weak implementation of the Environment and Social Management Framework in the state of Uttar Pradesh. In 2006, the Bank conducted complete audit of social safeguards preparation and implementation for all project roads and developed a time bound action plan for retrospective corrective measures (ICR p. 13). A series of training programmes were conducted at the Divisional level for field officers through Third Party consultants. The Government of Uttar Pradesh also prepared a documentary video on social safeguards for wide dissemination and capacity building on social safeguards. Post Mid-Term Review, these measures resulted in substantial improvement in safeguard compliance in Uttar Pradesh. Rajasthan and Himachal Pradesh performed well on social safeguards implementation (ICR p. 13).

There were initial delays in the institutional strengthening and maintenance components. Despite these initial delays, by project closing, all activities under this component were completed.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The achievement of project objectives was to be measured through the following indicators:

  • percentage of eligible habitations in project areas with all-weather access to social services and markets;
  • percentage of through routes in the core rural road network in participating districts in fair or better condition;
  • and level of stakeholder satisfaction with rural road network in participating districts.

Baseline figures were provided but the date of the baseline survey is not given in the ICR. The performance indicators were general. They combined access to markets and social services and were not dis-aggregated to the type of social services like health centers, or schools. Given that the project development objectives referred to the performance of selected districts, the performance indicators should ideally have been designed to measure performance in selected districts rather than the program as a whole in participating states. Thus there was a basic mismatch between the M&E indicators and the likely results of the project itself. Another shortcoming of the M&E framework was that the indicators did not capture all the expected outcomes of the institutional development component.

Also, the performance indicators referred to the performance of the program as a whole in respective states rather than performance in project districts (the project covered 60% of the districts in participating states). The monitoring framework did not reflect outcomes relating to the institutional development components.

Another shortcoming of the M&E framework was that the indicators did not capture all the expected outcomes of the Institutional Development component, for which only one indicator i.e. percentage of rural road works prioritized using maintenance management system, was designed. The other anticipated outcomes from this exercise, for instance initiatives on quality assurance and capacity building, transfer of ownership of non-core rural roads to Panchayat Raj Institutions was not captured.

b. M&E Implementation:
The level of road user satisfaction with the project roads and poverty impacts of the Prime Ministers Rural Roads Program roads was assessed through a comprehensive nation wide study covering a total ten sample states including three participating states of this project - Himachal Pradesh, Uttar Pradesh, and Rajasthan.
A methodology for undertaking poverty impact and road user satisfaction surveys was developed by the Ministry of Rural Development. The Bank contributed to this methodology by participating in various meetings and providing comments.

The baseline survey covered 11490 households within 750 habitations across 33 districts. The road user satisfaction was assessed on the basis of four major factors - reliability, transit time, connectivity, and user- friendliness and assessed on three levels, “low”, “medium” and “High”. Overall the satisfaction level was found high (ICR p.iii).

a. M&E Utilization:
Not discussed in the ICR.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:
The project was assigned Environment Category "A". Following safeguards policies were triggered: Environmental Assessment, Natural Habitats, Cultural Property, Involuntary Resettlement, Indigenous Peoples, and Forests.
The ICR reports that environmental and social safeguard compliance was overall satisfactory (ICR p. 12).

Environment Safeguards

The Government at appraisal had carried out an environment assessment in the four project states and formulated an Environment and Social Management Framework (ESMF) to address environmental issues in this project. A stand-alone document called Environmental Codes of Practice was prepared covering aspects such as sub-project selection and planning, construction camp management, top-soil conservation, debris management, public consultation, drainage and worker safety. This approach helped minimize the need for sub-project level Environmental Assessments and Environmental Management Plans (EMPs) by mainstreaming environmental issues in the selection, planning, design and construction stages of the project.

The project also introduced slope protection and disposal of debris, which otherwise are responsible for creating a series of direct and indirect environmental and social impacts, particularly in hilly terrain. Efforts were made by Himachal Pradesh State Rural Road Development Agency to prevent damage to properties and water sources through protection works, proper management of spoils/debris, and provision of adequate drainage works.

Forests: The ICR does not report on compliance with the Forests safeguards. It reports (p. 15) that In almost all the participating states, some roads, especially in the initial phases, were delayed due to lack of forest clearances. In subsequent phases, with insistence of having a prior forestry clearance in hand before the bids are invited/awarded, such situations were almost completely avoided.

Natural Habitats: The ICR does not report on compliance with the Natural Habitats safeguards. The ICR reports (ICR p. 12) that since the construction of rural roads was on the existing alignments, the extent of impact on environmental features such as fertile farmland, orchards, trees, sacred groves, water bodies and religious structures was minimal.

Cultural Property: The ICR does not report on compliance with the Cultural Property safeguards.

Social Safeguards

Involuntary Resettlement: The Government at appraisal had carried out a social assessment in all the project states through an independent consulting agency and formulated an Environment and Social Management Framework - Resettlement Policy and Framework to address social and resettlement issues in this project. The assessment brought out minimal land requirements for road widening and community willingness to volunteer for land donation. The Framework adopted voluntary land donation approach.

The Bank closely monitored the various safeguards measures particularly for preparing social safeguard documentation and land donations. Following the identification of serious deficiencies in safeguard preparation in project Detailed Project report in Uttar Pradesh in 2006, the Bank conducted complete audit of social safeguards preparation and implementation for all project roads and developed a time bound action plan for retrospective corrective measures (ICR p. 13). A series of training programmes were conducted at the Divisional level for field officers through Third Party consultants. The Government of Uttar Pradesh also prepared a documentary video on social safeguards for wide dissemination and capacity building on social safeguards. Post Mid-Term Review, these measures resulted in substantial improvement in safeguard compliance in Uttar Pradesh. Rajasthan and Himachal Pradesh performed well on social safeguards implementation (ICR p.13).

Indigenous Peoples: The ICR does not report on compliance with the Indigenous Peoples safeguards.

b. Fiduciary Compliance:
The ICR rates fiduciary compliance as moderately satisfactory (ICR p.11) with procurement and financial management also rated moderately satisfactory.
The Prime Ministers Rural Road Program's financial management systems were under development at the time the project was prepared. The Bank contributed towards the design of the system combining the features of traditional works department accounting practices with a double entry accounting system to prepare monthly and annual financial statements. An Online Monitoring Management and Accounting System was developed. The public citizen interface facility allowed access to updated state wise financial reports.

The computerized Receipts and Payments module of the Online Monitoring Management and Accounting System was slower than anticipated by the Ministry of Rural Development. This was due to lack of reliable web connections in some districts and limited staff capacity.

Internal and external audit reports found weaknesses with respect to delays in Bank reconciliations, inadequate controls over fund authorization, stale bank guarantees, and non-settlement of advances. The states took remedial measures to address these weaknesses, however, the results varied across states. For the Program as a whole, revised guidelines for selection of Chartered Accountant firms for external audit were prepared.

An Interim Performance Review carried out by the Bank suggested mitigation measures for weak contract management – namely, independent procurement, financial management, and contract management reviews; regular inspections by the senior officers of the road agency and SRRDAs; and improving the quality of engineering designs, field investigations, and safeguard aspects to avoid subsequent delays during execution.

The National Rural Roads Development Agency was responsible for supervising the day-to-day operation of the Program's financial management system. The Agencies' oversight of the financial management arrangements at the state level was limited to review of the annual audit reports, troubleshooting on Online Monitoring Management and Accounting System and providing training on an “as required” basis.

Procurement: moderately satisfactory.

The procurement capacity varied considerably between the states. Rajasthan demonstrated adequate procurement capacity. Procurement in Jharkhand remained slow throughout the project and no procurement was done during the last two years of the project.

ICR (p. 11) reports that all goods, works and services were generally procured in accordance with the provisions contained in Bank Guidelines except some procedural errors noticed in procurement of works. The factors responsible for inadequate competition in bidding were limited availability of qualified contractors, outdated cost estimates and reluctance of the states to award contracts above the estimated price even if the price obtained against the tender was competitive and reasonable compared to market rates.

Different qualification criteria and procurement procedures for the project compared to those used in the overall Program also created problems in states such as Jharkhand and Uttar Pradesh where the level of competition was very low and many of the works had to be subjected to repeated re-invitation of bids.

A Procurement Manual was finalized at the time of the writing of the ICR. It is part of the on-going second rural roads project.

c. Unintended Impacts (positive or negative):

d. Other:
None.



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Satisfactory
The relevance of the project objective and relevance of design were substantial. The project efficacy and efficiency were modest, partly due to lack of sufficient evidence. There was inadequate evidence on some of the intended outcomes (e.g., access to social services) and also on attribution of the reported outcomes to the project. 
Risk to Development Outcome:
Moderate
Significant
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following lessons are drawn from the ICR (pp. 25 - 27), with adaptation:
  • In case of large Government programs such as the Prime Minister's Rural Roads Program (PMGSY), where the Bank’s contribution is small relative to the overall size of the program, it is important to agree on harmonized standards for fiduciary and safeguard requirements across Program and Bank funds. This is essential to avoid the issues that arose with respect to, for instance, procurement on this project, where Bank guidelines were different from those under PMGSY.
  • In cases where the project is to be implemented in different states/regions with wide variation in implementation capacity, the design of the projects should build in some flexibility in the level of support required by different states in order to tailor the technical assistance program to the needs of each state. To support the development of capacity at state level, there is a need to institutionalize training activities within existing national and state institutions like state technical agencies and other academic institutions engaged in the project.
  • To achieve comprehensive community participation, social accountability systems through models such as Community Score Cards, Social Audit, etc. can be useful. The project, though well designed for community participation for the ‘project planning’ cycle through tools such as information dissemination, collaboration for participatory mapping of the impacts, etc. had limited design for ‘community participation and supervision’ during the stage of construction.

The Borrower identified following important lessons:
  • The Environmental and Social Management Framework codes of practices and mitigation measures should become an integral part of the Detailed Project Report and tender documents.
  • There is need for a more critical analysis of the strengths and weaknesses of each individual State Rural Road Development Agency so that the institutional strengthening and hand-holding support is geared to their specific needs.
  • Field staff along with staff of the National Rural Roads Development Agency and the Ministry of Rural Development also needs to be exposed to good national/international practices in project planning, design and execution.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

There is inadequate evidence on some outcomes and issues of attribution are not fully discussed. The total project cost was missing from the ICR (it had to be taken from the project portal). The ICR does not also provide the actual borrower contribution and costs by component. Compliance with some of the safeguards is not separately discussed.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions