Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Uttar Pradesh Water Sector Restructuring Project


  
1. Project Data:   
ICR Review Date Posted:
04/30/2014   
Country:
India
PROJ ID:
P050647
Appraisal
Actual
Project Name:
Uttar Pradesh Water Sector Restructuring Project
Project Costs(US $M)
 US$173.70  US$162.52
L/C Number:
C3602
Loan/Credit (US $M)
 US$149.20  US$130.99
Sector Board:
Agriculture and Rural Development
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  02/19/2002
 
 
Closing Date
10/31/2007 10/31/2011
Sector(s):
Irrigation and drainage (35%), Sub-national government administration (34%), Agricultural extension and research (15%), Other social services (15%), Renewable energy (1%)
Theme(s):
Water resource management (25% - P) Other rural development (25% - P) Rural services and infrastructure (25% - P) Administrative and civil service reform (25% - P)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
John R. Eriksson
Robert Mark Lacey Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:
The project objectives stated in the Development Credit Agreement (beginning of Schedule 2) and in the Project Appraisal Document (p.2) are virtually identical. The Credit Agreement formulation is:


    "to assist Uttar Pradesh: (i) to set up an enabling institutional and policy framework for water sector reform for integrated water resources management; and (ii) to initiate irrigation and drainage sub-sector reforms to increase and sustain water and agricultural productivity."

IEG will use this formulation of the project objectives..

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
There were six components:
A. Creation of Apex Water Institutions and Strengthening Social and Environmentally Sustainable Multi-sectoral Water Resources Planning, Allocation and Management Capacity (appraisal: US$10.22 million; actual: US$11.05 million) To support the establishment of State apex water institutions as proposed in the State Water Policy. Capacity to be enhanced through provision of computer hardware, software and connectivity, and consultants' services.
* State Water Resources Agency: To develop environmentally and socially sustainable inter-sectoral water allocation and optimal water resources management. It would also serve as the technical secretariat for the State Water Board.
* State Water Resources Data and Analysis Center: To collate, verify, analyze and disseminate information on all aspects of integrated water resources management and assistance in basin planning.
* State Water Tariff Regulatory Commission: To be the central agency to review and monitor water sector costs and revenues, and to rationalize water user fees, i.e., to set fees to enable sector institutions to be financially self-sustaining. The Uttar Pradesh (UP) Water Management Regulatory Commission Act of 2008 authorized the establishment of a Commission with the same name as the Act, which according to the ICR Results Framework Analysis (RFA) and the project team, was to have the same function as the planned State Water Tariff Regulatory Commission.
* Ghagra-Gomti Basin Development and Management Entity: To be a multi-sectoral and representative body to help refine and implement environmentally and socially sustainable water resources development and management plans for the Basin.

B. Irrigation Department Reform, Capacity Building and Business Process Re-engineering (appraisal: US$32.10 million; actual: US$21.37 million) To initiate measures to improve cost-effectiveness and streamline the manner in which irrigation and drainage investments were planned, implemented, managed, operated and monitored, involving the following activities:
* Institutional reform -- a program to reduce Department establishment costs through a phased Voluntary Retirement Scheme, including consultation, training, and establishment of a task force to, among other things, monitor reform studies.
* Department capacity-building, including modernization of the technical, administrative and managerial capacity through the provision of technical advisory services, training and study tours to equip it for modem irrigation management in partnership with farmer clients.
* Business process re-engineering, involving establishment of a management information system and supporting databases and associated analytical and communication facilities to modernize and streamline UP Irrigation Department business processes.

C. Piloting Reform Options in Water Resources Management (appraisal: US$7.47 million; actual: US$1.30 million) To apply integrated water resources management supported by demonstration pilots in selected decentralized basin development and management institutions.
* Creation and strengthening of Jaunpur and Imamganj Branch Sub-basin Development and Management Boards
through support for infrastructure, technical advisory services and operating costs.
* Decision support systems development through information systems and modeling for socially and environmentally sustainable optimal water resources management in the selected sub-basins.
* Rainwater harvesting through improvement of community-managed village ponds through technical
advisory services and conservation and management of other water bodies in the selected sub-basins.
* Piloting canal-based small hydro development in the Imamganj Branch sub-basin with participatory management.
* The ICR indicates that rehabilitation of the Haidergarh Branch canal (Cultivatable Command Area of 20,287 ha) was added after the Imamganj Branch canal (22,000 ha) was omitted due to a restriction from the Government of India to do survey and design work in an area close to the border with Nepal (p.4).

D. Piloting Reform Options in Irrigation and Drainage Operations (appraisal: US$ 80.38 million; actual: US$114.70 million) This component was to involve the rehabilitation and modernization of the irrigation and drainage systems in about 300,000 has, so as to build confidence and generate lessons in mainstreaming the reforms. It was to apply the concepts and provide the necessary investments for (i) reliable delivery of water measured and supplied on an appropriate volumetric basis in the irrigation systems to improve system performance, cost-recovery and accountability of the service provider; (ii) effective drainage network operation and management and conjunctive use to assist in effective water management (iii) an outcome-oriented approach with integrated sustainable agricultural intensification and diversification and private sector participation to complement effective water management; and (iv) unbundled operational management of the irrigation and drainage systems [i.e., divide responsibilities according to functions, such as investment, operations and maintenance] , including participatory user management and private sector participation. Water user associations were to be formed and supported to become operational. Water delivery systems ranging from civil works and related equipment to measuring devices were to be financed, as well as extension, study tours and training to support agricultural diversification.

E. Feasibility Studies and Preparation Activities for the Next Phase (appraisal: US$ 4.96 million; actual: US$0.07 million) To include initiating topographic surveys and environmental, social and other assessments and preparation of feasibility studies for activities to be undertaken in the proposed second project in the UP Water Sector Reform Program. Lessons learned during early implementation of the project would be reflected in this preparatory work.

F. Project Activities Coordination (appraisal: $US6.02 million; actual: $US14.07 million) To be coordinated by a multi-disciplinary Project Activities Core Team. Component to assist Team in facilitating and guiding implementation and monitoring all project activities, ensuring synergy and coordination among participating activities and agencies.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Cost
The actual project cost was US$162.52 million, compared with an estimated cost at appraisal of US$173.70 million.
The difference in total cost figures masks substantial variation among components. The largest shifts in absolute terms were for Component D (Piloting Reform Options in Irrigation and Drainage Operations), which increased by US$34.32 million, and Component B (Irrigation Department Reform), which decreased by US$10.73 million. Components C (Piloting Reform Options in Water Resources Management) and E (Feasibility Studies and Preparation Activities for the Next Phase) declined respectively by 83% to US$1.30 million and by almost 100% to US$.07 million. However, Component F (Project Activities Coordination) increased by 134% to US$14.07 million While the ICR provides considerable detail on activities and accomplishments by sub-component (especially in Annex 2), it does not explain the differences between appraised and actual cost by component.

Financing
Of the original IDA Credit of US$149.20 million equivalent, US$40 million was reallocated to meet emergency needs in South India as a result of the Indian Ocean Tsunami of December 2004. Despite this, US$130.99 million equivalent had been disbursed at closure thanks to the depreciation of the US Dollar against Special Drawing Rights. There was an undisbursed amount of US$5.1 million at project closure that was canceled. There were no other external sources of financing.

Borrower Contribution
The Borrower (the State Government of Uttar Pradesh) contributed US$30 million equivalent, 25% more than the US$24 million equivalent anticipated at appraisal.

Dates
The closing date was extended three times for a total of four years to October 31, 2011. These extensions are attributed mainly to significant delays in contracting and procurement, especially in the early years of the project. One reason for this was a lack of suitable large contractors, which resulted in contracts having to be rebid and/or broken down into smaller lots. Most of the construction contracts did not start until 2007, shortly before the original closing date of October 31, 2007, and some did not start until 2008. Total implementation time was almost double the original four-and-a-half year estimate.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial.
The project’s objectives are relevant to the six water sector reforms to be supported by the Bank according to its Country Strategy for the Republic of India for Fiscal Years 2009-2012: enabling water policies and institutional and legal frameworks to improve stewardship of water resources; integrating surface water with groundwater through inter-sectoral basin approaches; restructuring public sector institutions; promoting participatory delivery mechanisms, with a focus on user service, accountability and extending service to the poor; introducing modern management practices and technology applications, including improved operations and maintenance; and enabling sustainability of resource management and service delivery through improved financial management, including rational tariffs or user charges that cover operating and maintenance costs. Water resource management also features in the Bank's 2010 Country Strategy Progress Report, which mentions watershed development interventions in three states, including Uttar Pradesh.

Water resource management is not among the explicit priorities of the Government of India’s Eleventh Five Year Plan (2007-2012). However, Plan goals such as a 4% average annual agriculture growth rate, and by 2011/12, an increase in forest and tree cover by 5% and treatment of all urban waste water in order to clean river waters, would need enhanced water resource management as a precondition of their achievement. Specifically in Uttar Pradesh, the Plan expects that agriculture would play an important role in alleviating poverty. The project’s objectives of reform and modernization of irrigation are relevant given the impact that irrigation has on agricultural productivity and growth.

b. Relevance of Design:
Substantial.
The dual emphasis of project design on reforms and infrastructure investment was relevant to the two project development objectives. Three project-supported reforms – introducing multi-disciplinary, multi-sector and transparent approaches into the management of the State's water resources; devolving decision-making to Sub-basin Board levels; and rationalizing water tariff-setting through establishing and capacity-building of a State Water Tariff Regulatory Commission – were relevant to the first development objective. Four other project-supported reforms – improving financial sustainability; enhancing the skill mix and technical competence of Irrigation Department personnel; enabling the introduction of participatory irrigation management (through Water User Associations); and initiating private sector participation in the operation and maintenance of irrigation and drainage infrastructure – were relevant to the second development objective.

Neither the PAD design nor the Results Framework Analysis in the ICR Data Sheet provides a clear, logical results chain linking inputs to outputs, intermediate outcomes, outcomes and project development objectives.


4. Achievement of Objectives (Efficacy) :

Objective 1: Set up an enabling institutional and policy framework for water sector reform in the state for integrated water resources management. Modest.
Outputs
  • The UP Water Management Regulatory Commission Act was passed in 2008; A Commission and Chairman with 2 of 4 members had been appointed by project closure.
  • The Participatory Irrigation Management Act (promoting Water User Associations) was passed in 2009 and associated regulations promulgated in 2010. 805 Water User Associations had been formed at minor canals by project closure, compared to the target of 600 Associations.
  • A State Water Resources Agency and State Water Resources Data Analysis Center were established. The ICR (p. 8) reports that these organizations satisfactorily performed their functions of collecting, analyzing and disseminating water resources data.
  • The Jaunpur Branch Sub-basin Development and Management Board was established.
  • Increased Irrigation Department efficiency was to be achieved, inter alia, by reducing staff numbers. However, the reduction was from 84,699 in 2002 to 77,073 at project closure, compared with a target of 45,910, so that the actual decrease was only 20% of the target. The reason for this was that the planned government-run Voluntary Retirement Scheme was not adopted.
  • A planned formalization of linkages between the agriculture and water sectors and the Government Departments concerned did not take place. Although a Memorandum of Understanding was signed to build synergy between these Departments, it was not implemented due to delays in improving agricultural infrastructure and a disagreement between the State Government and the Bank on agricultural marketing reforms
  • A Development and Management Plan was prepared for the Ghagra-Gomti Basin, but no entity had been established to implement the Plan as intended by project closure.
  • A Management Information System (MIS) was developed and in partial use in the Irrigation Department at project closure. According to the ICR (pp.8-9), the use of the MIS was not fully effective by closure.
  • An attempt to involve the private sector in growing sugar cane resulted in some pilots, but did not succeed in bringing about full private-public partnerships in the management of irrigation infrastructure.
Outcomes
  • Most of the water resources management institutions called for in the PAD (pp.13-14) were set up, but there were delays and weaknesses in making them operational and in implementing policies. The number of Water User Associations exceeded the target, but this was accomplished with a four year delay.
  • The Water Management Regulatory Commission had not implemented rationalized tariff setting by project closure so that no mechanism was in place to enable water institutions to become financially sustainable.
  • Basin level and sub-basin level integrated water resources management plans were developed, but only at the sub-basin level was an entity (Jaunpur) established. There is no indication of the effectiveness of this entity beyond a statement that it is "working" and that "resource management decisions are being taken" (ICR, p. 21). However, because of excessive silt content, the feeder canal at the head of the Jaunpur branch canal was closed, on average, for 166 days a year during 2007-2010, compared to an annual average of 97 days during 2002-2005.

Objective 2: Initiate irrigation and drainage sub-sector reforms in the state to increase and sustain water and agricultural productivity. Substantial.
Outputs
  • 343,040 hectares (ha) were rehabilitated with irrigation and drainage infrastructure, with water flow control and rehabilitation targets "largely" met by project closure. However, "improved measurement and more rehabilitation are still needed" (ICR Data Sheet Results Framework).
  • 180.400 ha were irrigated by canal compared to a target of 215,564 ha and a baseline of 117,427 irrigated ha in 2002.
  • Rainwater harvesting was to be undertaken as a part of integrated water resources management through community-managed village ponds. Although it is mentioned in several ICR annexes, the magnitude of achievement is unclear.
  • The small hydro pilots that were planned did not materialize due to lack of assured water supply in the Imamganj Branch canal, which was in any event omitted from the project.
Outcomes
  • Water productivity for wet season (kharif) rice production is stated to have increased from an average of 68 ha per million cubic meters of water (Mm3) in 2002 (0.14 kg of rice per m3) to 324 ha/Mm3 in 2010 (1.3 kg of rice per m3) and for dry season (rabi) wheat production from 129 ha/Mm3 in 2002 (0.33 kg of wheat per m3) to 292 ha/Mm3 in 2010 (1.1 kg of wheat per m3).
  • Agricultural productivity is also stated to have increased as a result of project interventions. Agricultural yields in areas benefiting from the project are estimated to be 29% higher than those outside the project for paddy, 30% higher for wheat, 40% higher for pulses and 54% higher for mustard. Cropping intensity in the project area has increased by about 10% more than outside the project. The drainage improvements added almost an additional 20,000 ha of previously unproductive lands.
  • At the time of the ICR, available data indicated that there had been negligible crop diversification. But additional data based on satellite imagery recently provided by the team strengthens the finding that agricultural productivity has increased as a result of greater multiple cropping made possible by the project. The replacement of water logged areas with multiple cropping is also consistent with greater water productivity
  • These results tend to offset earlier attempts to measure water and agricultural productivity increases and determine their attribution to the project that were based on PACT Team in-house data collected in 2008/09 on irrigated area, cropping intensity, crop yields and technology adoption. Significant project interventions occurred after 2008/2009, and a planned survey in 2011 at the end of the project was not conducted. Moreover, the control group areas included non-irrigated areas, whereas a more valid comparison would have been with non-project irrigated areas.
  • Sustainability of these results could be an issue, either technically or financially. Due to delays in improving irrigation infrastructure and maintaining water flow rates, much of the large amount of silt entering the system was not removed (for example, the capacity of the Haidergarth branch canal is only about 40%-65% of design levels due to siltation). For the amount of silt that was removed, adequate disposal sites were not found. A planned study to identify ways to deal with the silt problem never materialized.
  • Some minor canals had reverted to poor condition prior to project closure due to lack of maintenance and inadequate upkeep by Water User Associations. Most Associations did not have the skills or resources to operate and maintain canals adequately and the requisite training was not provided (ICR, pp.10 and15). About 28% of one Command area remained water logged at project closure.
  • The need for maintenance and replacement of aging equipment had not been fully taken into account. Computers purchased in 2006 are now getting to the end of their useful life. Appreciable numbers of automatic piezometers (only 134 of the 500 automatic recording piezometers were reported working at project closure), weather stations and acoustic flow-measuring devices are non-functional in some places and contracts for their maintenance are not in place.
  • There is little evidence of financial sustainability. Without Government budget support to Associations, there is a risk that canals would continue to deteriorate. As noted above, the Water Management Regulatory Commission had not implemented rationalized tariff setting by project closure so that no mechanism was in place to enable water institutions to become financially sustainable.

5. Efficiency:

While an economic rate of return (ERR) of 17.6 % estimated at closure was above the standard assumption of an opportunity cost of capital of 12%, it was significantly below the estimate at appraisal of 24.8%. This resulted from some irrigated areas missing the targets projected at appraisal (ICR, p.11). Other factors bearing on the ex post ERR included: 40% less than anticipated diversification into higher value crops; wheat and mustard productivity being less than their respective targets; and considerable delays in project procurement, construction and rehabilitation, which delayed the stream of expected benefits. Still not completed at closure were works for the Haidergarh Branch sub-basin (ICR, p.5).
The ICR does not explicitly indicate the coverage or scope of the ERRs but the reference to "irrigated areas" suggests that ERR estimates were made for Components C and D; this was subsequently confirmed by the project team. Accordingly, percentages of "Coverage/Scope" shown below are estimates based on the share in total project cost of Components C and D (about 62% at appraisal and 71% at closure).

Construction quality was highly variable, and in some cases seriously deficient. This contributed to the persistent problems of siltation and water-logging.

The closing date of the project was extended three times and implementation took over nine-and-a-half years to complete rather than the five-and-a-half years planned. The project team confirms that these extensions resulted from considerable delays in procurement and construction that affected the project from the beginning, as well as high turnover in leadership, especially of the Implementing Agency (the Project Activities Core Team) and slow staffing up of the Water Management Regulatory Commission. According to the Borrower's ICR (ICR, Annex 5), “.... the project in initial stage was delayed. Engaging the contractor/consultancy for survey and design should have been done at the outset, which was delayed by more than two years. Rehabilitation of canal and drainage works was delayed by more than three years. Time lines could not be controlled. The cause of delay can be attributed to lack of monitoring, [lack of] experienced and trained staff and experts, frequent changes at senior management level, attrition of engineers and staff, frequent transfers, dual charges, unfilled vacancies.”

Efficiency is rated modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
24.8%
62.2%
ICR estimate:
Yes
17.6%
71.4%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The relevance of the project objectives and design are rated substantial. There were significant gaps and limitations in the achievement of the first project objective, which is rated modest, but the achievements recently reported by the team lead to a substantial rating for the second project objective..Substantial contracting, procurement and staffing delays, as well as poor quality of construction, lead to a modest rating of efficiency. Overall outcome is assessed as moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The main risks to development outcome are the following:
  • Political. Delays in appointing and/or high turnover of the leadership of two key institutions, the UP Water Management Regulatory Commission and the Project Activities Core Team, indicate a need for stronger political backing from the State Government than was apparent during project implementation. The risk that sufficient backing will not materialize is significant.
  • Institutional. The Water Management Regulatory Commission, the State Water Resources Agency and the State Water Resources Data Analysis Center have all been established, but there is a risk to the effectiveness of their regulatory and monitoring roles due to the uncertainty of contracted staff and vacancies that remain unfilled. The Water Resources Agency and the Data Analysis Center were to be established as independent entities under the Regulatory Commission but remained in the State Irrigation Department. In the absence of continued budgetary support, the Water User Associations could lose their enthusiasm and lack resources for operation and the capacity to maintain their canals and construct field channels. The overall institutional risk is significant.
  • Technical. Operation and maintenance of the improved irrigation and drainage infrastructure by the Irrigation Department continued at project closure to be inadequate. As noted in Section 4 above, continued siltation in the irrigation system could undermine rehabilitated system performance, and the main canal in one project sub-basin is operating at half its designed capacity. Piezometers, computers and other sophisticated equipment supplied by the project have not been adequately maintained or updated due to inadequate funding. The technical risk is assessed as significant.
  • Financial. The Water User Associations are not financially viable and are reliant on budgetary support from the State Government. There are also financial management issues regarding the Associations. An audit of approximately 400 Associations raised questions about the adequacy and integrity of their financial procedures (ICR, p.14). The Water Management Regulatory Commission had not implemented rationalized tariff setting by project closure, so that no mechanism was in place to enable water institutions to become financially sustainable. The financial risk is rated significant.

    a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
The ICR reports that 15 "critical" political, institutional, technical, financial and social risks were identified at appraisal (pp.4-5) and that mitigation measures were put in place and "executed as effectively as possible." In most cases, these measures were insufficient to mitigate the impact on the project when the risks materialized. Moreover, the Bank seriously underestimated the time required for award of important contracts. The ICR argues (p.13) that it is "difficult to see how better these risks could have been avoided, other than by not having the project.” Given the significant nature of the issues which arose, and the fact that most of the problems were still unresolved at closure, this statement calls into question whether the preconditions for a successful and sustainable project were present at entry.
Implementation arrangements were inadequate. The new project implementing agency, the Project Activities Core Team (PACT) suffered from extremely high turnover of its leadership Chairman position, for which the incumbent changed 19 times during the nine years of project implementation (over twice a year on average). This appeared to stem, at least in part, from lack of clarity in the respective roles of the PACT Chairman vis-a-vis those of the Principal Secretary of the Irrigation Department. Although the responsibility of Government, this could have been addressed at the preparation stage.

There were significant weaknesses in M&E design (see Section 10 below).

According to the PAD, a full Environmental Assessment was carried out for this category “A” project, although the adequacy of institutional arrangements for the implementation of safeguards policies is questionable (see Section 11a below). Reasonably adequate preparation appears to have been made for financial management, but the procurement capacity of the implementing agency was insufficiently strengthened (see Section 11b below).

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
There were 19 implementation supervision reports prior to the ICR mission. However, the Bank’s ability to supervise the project adequately was undermined by the high turnover of incumbents of the post of Chairman of the Project Activities Core Team. This caused frequent unforeseen delays along with other issues resulting from inconsistent leadership. The Bank had frequent consultations with Core Team counterparts to mitigate these delays. During implementation the Bank recommended that the Core Team utilize a consultancy to carry out an impact assessment of project results for further discussion (see Section 10-b below). By extending the Credit closing date by four years the Bank enabled modest achievement of the project's development objectives.

Consistent with IEG/OPCS Harmonized Criteria, overall Bank Performance is rated Moderately Satisfactory.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The State Government passed some relevant pieces of policy and legislation, including the State Water Policy Act during project preparation, the Water Management Regulatory Commission Act in 2008 and the Participatory Irrigation Management Act in 2009. A sub-basin board was formed (Jaunpur Branch) but the basin level entity (Ghagra-Gomti) was not. However, there was a lapse in Government commitment during the early years, owing in large measure to political uncertainty following state elections in April 2002. A State Water Management Regulatory Commission was established but with quite limited functionality, leadership and staffing. There was no progress toward rationalized tariff-setting. The State Water Resources Agency and Data Analysis Center were established but were located in the Irrigation Department rather than as independent entitles under the Water Management Regulatory Commission, as intended. Continuity of leadership of the Project Activities Core Team was a serious issue, with 19 Project Directors over the nine year project duration (ICR, pp. 5,13,14). The substantial delays in making key decisions during project implementation may be attributed largely to this lack of continuity in leadership.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:
The main implementing agency, the Project Activities Core Team, oversaw the creation of over 800 Water User Associations, establishment of the Management Information System, and the rehabilitation and modernization of the irrigation and drainage systems. These accomplishments were carried out, according to the project team, by Core Team staff with the support of intensive Bank supervision, notwithstanding the extremely rapid turnover at the Project Director (Chairman) level. However, procurement, both by the Core Team and by the Irrigation Department field staff was very slow, aggravated by frequent transfers of staff, dual charges and unfilled vacancies. The Procurement Post Review found that contract management and understanding of Bank procurement procedures were weak at the Core Team and Irrigation Department levels. Follow-up actions relating to Procurement Post Review findings were tardy with regard to major issues relating to inappropriate application of post-qualification criteria and bad quality road construction. The project did not publish award decisions on its web site. At the time of project closure, claims submitted by a contractor had not been resolved, despite arbitration and ongoing court proceedings. The quality of construction, particularly concrete and brick lining of canals, although supervised by the Irrigation Department and an international consultant, was criticized. Although design consultants made recommendations concerning exclusion of silt from the head of the feeder canal, the Irrigation Department, according to the ICR, did nothing to implement them (pp.14, 31).

Internal audit arrangements suffered from delays in receipt of audit reports and a lack of attention to audit observations. These were not only financial management matters, but also reflected overall management deficiencies. An audit of some 400 Water User Associations found instances where advances to Associations remained unadjusted in accounts at division level; supporting documents could not be found; and Associations were not maintaining independent books of accounts. Irregularities were also found in the accounts of participating Non-Governmental Organizations.

Implementing Agency Performance Rating: Moderately Unsatisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The Implementing Agency, the Project Activities Core Team (PACT), had responsibility for M&E. The Activities Coordination Component of the Project (Component F: actual cost US$14.07 million) was, among other things, to assist PACT in "monitoring all project activities" (ICR, p.3).

The “Project Design Summary” of the PAD (Annex 1, pp. 30-42) proposed an M&E framework with 14 output and 13 outcome/impact indicators. The framework presents a number of issues:

  • In several cases, the output indicator is the same as the outcome/impact indicator.
  • Five output indicators had no corresponding outcome indicator. For instance, "Development of canal-based hydros, conservation & development of water-bodies and rainwater harvesting" is explicitly mentioned as an output, but the link to any outcome is not obvious.
  • Four outcome indicators had no corresponding output indicator.
  • There are no outcome indicators for the two-sub-basin development plans and entities, which represents an important gap since the ICR puts considerable emphasis on the sub-basin sub-components.
  • One output indicator should be classified as an outcome indicator (" Increase in irrigation & cropping intensities & diversification to cash crops").
  • Some terms such as "operationalization," "rationalized," "modernized," and "agricultural intensification," are not defined.
  • Quantitative or qualitative metrics are often insufficient to enable objective measurement of outcome achievement.

The PAD (pp. 28 and 30) also proposed an M&E process for assessing environmental factors and social development, including monitoring government-provided services, social status, environmental conditions and other parameters as part of a system of benchmarking and regular monitoring of canal systems. Water User Associations were to be monitored with attention to participation in decision-making, equity, and improved livelihoods through surveys, including comparable and transparent reporting. The information would be integrated to the extent possible into a spatial framework (using a Geographic Information System - GIS) and fed into an overall Management Information System (MIS) and Decision Support Systems. Quantitative baseline data were to be collected prior to implementation (pp.20-22).

b. M&E Implementation:
Monitoring of financial progress was satisfactory, but monitoring of physical and socio-economic progress was problematic. The system designed at appraisal does not appear to have been implemented. For example, a key result of modernizing the irrigation system was to have been reliable delivery of water. The introduction of new measuring gates at the heads of minor canals was therefore an important step, but measurements in a number of branch canals were not satisfactorily carried out. Monitoring of agricultural benefits during implementation was deficient. Two consultancies were assigned to evaluate project impact, the first during calendar year 2007 and the second from August 2009 to March 2010. Sample household surveys on 43 representative minor canals were conducted by the consultants. The ICR reports, without explanation, that the surveys "were not suitable for use in this ICR.” At another juncture it is reported that, in 2009, PACT organized baseline, initial and final impact assessment studies to be undertaken as part of a broader project impact evaluation in 2009. But the implementation of these studies was considered deficient, lacking adequate guidance and supervision, which resulted in limitations to data coverage, quality and analysis. The approach was insufficiently rigorous, and did not include identification of a valid control group, which would have facilitated a clearer assessment of impacts that could be attributed to project interventions. Given these inadequacies, the ICR team in 2011 instead estimated project benefits based on PACT Team in-house data collected in 2008/09 on irrigated area, cropping intensity, crop yields and technology adoption (ICR, pp.5-6). An impact survey planned for 2011 was not undertaken (ICR, p.39).

a. M&E Utilization:
Monitoring of physical and financial progress was used in regular project reports. Monitoring of groundwater levels throughout the project area was carried out since 2005, utilizing automatic piezometers supplied by the project, although only 134 of the 500 were reported working at project closure. Some information collected by the Management Information System was used for management but that collected by the irrigation and water management module was not used. Data collected by the MIS was not used to evaluate project performance.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
The project was classified as Category “A” for Environmental Assessment purposes. The PAD (p. 30) reports that, as well as Environmental Assessment (OP 4.01), Pest Management (OP 4.09) and Projects on International Waterways (OP 7.50) were triggered.

Compliance with the Environmental Safeguard was of concern during project implementation as a result of inadequate removal of silt deposited in canals. Relevant clauses were inserted in contracts to ensure satisfactory disposal of the silt. This resulted in improved disposal of silt in one major branch canal (Jaunpur) but not in another (Haidergarh). The ICR (p. 9) states that, at project closure, "the large amount of silt entering the system is a cause for concern" (p.9). Other identified issues included soil fertility and nutrient management, lack of attention to drainage, lack of diversification and little modernization of tools and techniques. Neither the ICR nor the project team indicated whether the Bank's policy on environmental safeguards had been complied with.

Regarding Pest Management, the Project Activities Core Team developed, with the assistance of three experts, an "eco-friendly pest management" strategy in 2006, and the ICR reports "improved practice" in this regard (p.66). Identified issues included limited knowledge of integrated pest management. No special assessment of this safeguard was conducted, and there is no clear statement of compliance in the ICR.

Regarding International Waterways, a waiver for notification of riparian states was provided by the Bank’s Regional Vice-President.

b. Fiduciary Compliance:
The financial management rating in supervision reports ranged from Satisfactory to Moderately Satisfactory for most of implementation. Receipt of internal audit reports was usually delayed by one to four months. The financial management arrangements generally resulted in timely availability of funds, but there were some shortcomings: (a) persistent weaknesses in internal controls (e.g. advances to Water User Associations remained unadjusted for long periods of time; weak contract management; senior management not paying adequate attention to audit findings; weak monitoring of nongovernmental organizations); (b) responses to financial management weaknesses identified by the Bank were delayed in many instances; and (c) corrective measures to resolve financial management weaknesses tended not to be effective (ICR, p.7). The ICR provides no indication as to whether external audits were qualified or whether they complied with Bank financial management policies.

Throughout the project there were substantial delays due to slow procurement. The procurement post review found that contract management and understanding of Bank procurement procedures were weak at the implementing agency and the division levels of the Irrigation Department. Follow-up actions relating to procurement post-review findings were slow with regard to major issues such as inappropriate application of post-qualification criteria and bad quality road construction. The project did not publish award decisions on its web site. The quality of construction, particularly concrete and brick lining of canals, despite the presence of third party consultants, was called into question. No cases of misprocurement were reported.

c. Unintended Impacts (positive or negative):
A lowering of the groundwater level appears to have taken place in the project areas. It is not clear if this was due to improved drainage, less rainfall, less irrigation, more pumping of groundwater, or a combination of all of these.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
The relevance of the project objectives and design are rated substantial. There were significant gaps and limitations in the achievement of the first project objective, which is rated modest, but the achievements recently reported by the team lead to a substantial rating for the second project objective..Substantial contracting, procurement and staffing delays, as well as poor quality of construction, led to a modest rating of efficiency. Overall outcome is assessed as moderately satisfactory.  
Risk to Development Outcome:
Moderate
Significant
At closure there were significant political, institutional, technical, financial and social risks to development outcome, including siltation of canals and weak leadership of the Core Team. Financial threats to outcome included no progress in rationalizing the water tariff structure and apparent fragility of the Water User Associations. 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
There were significant shortcomings in Quality at Entry. But consistent with the IEG/OPCS Harmonized Criteria, overall Bank Performance is rated Moderately Satisfactory. 
Borrower Performance:
Moderately Satisfactory
Moderately Unsatisfactory
There were significant performance shortcomings in both Government and implementing agency performance. 
Quality of ICR:
 
Unsatisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following lessons are taken from the ICR with some adaptation:
  • Strong Water User Associations (WUAs) can help to ensure sustainability of irrigation rehabilitation, provided they are strengthened and consulted at the outset. It is important to include water users in the design and implementation of irrigation system rehabilitation, to strengthen ownership and impart management skills and conflict resolution. This begins with eliciting their input at the design stage, which will decrease the risk that illegal cuts will be made after the designs are implemented, compromising the ability of the system to deliver the proper allocation of water to all the outlets. This experience should also increase the likelihood that Associations will be effective in assuming responsibility for Operations and Maintenance (O&M). Success would normally require a substantial long-term effort, beyond limited training. Provision needs to be made for continued strengthening of Association members and especially of newly elected officers.
  • Institutional effectiveness requires consistent leadership at the chairman or director level. Frequent changes of senior management and key staff can hinder efficient implementation. The more consistency that can be achieved at leadership level, the better performance the project will likely achieve. But the costs and benefits of creating an independent or quasi-independent project implementation unit need to be carefully weighed.
  • A benefit or impact evaluation needs to be well-designed in advance and adequately resourced. The intensive phase of evaluation of project benefits for the implementation completion report will need to be carried out during the last year of the project and should ensure that a suitable “without project” area is included in the survey to act as a control. However, the design of such an evaluation should be part of project preparation and a baseline survey should be undertaken before implementation in order to permit a "before-after" comparison of benefits.

14. Assessment Recommended?

Yes
Why?
To verify the ratings.

15. Comments on Quality of ICR:

Although this was a "Category A" project, the ICR does not state explicitly whether the Bank's policy on environmental safeguards was complied with. The Results Framework Analysis in the Data Sheet lacks a clear differentiation between outputs, intermediate outcomes and outcomes. Three of the five results indicators are outputs rather than outcomes. Other gaps and inconsistencies include: several references to a "procurement post review" without a date; insufficient information about what entity(ies) was (were) responsible for implementing M&E; several unclear sentences and phrases; the discussions of Bank and Borrower Performance are brief and contain little information and analysis; several important sub-objectives are mentioned in the PAD but are missing from the ICR, including improved equity in distribution of incomes among stakeholders in the selected sub-basins, agricultural intensification, and agricultural diversification; the differences between appraised and actual cost by component are not explained; and it is not stated whether external audits of the project were qualified or not.

a. Quality of ICR Rating: Unsatisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions