Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Emergency Energy Assistance


  
1. Project Data:   
ICR Review Date Posted:
08/05/2013   
Country:
Kyrgyz Republic
PROJ ID:
P101392
Appraisal
Actual
Project Name:
Emergency Energy Assistance
Project Costs(US $M)
 15.0  14.96
L/C Number:
C4524, CH429
Loan/Credit (US $M)
 11.0  14.96
Sector Board:
Energy and Mining
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  12/09/2008
 
 
Closing Date
04/30/2010 03/31/2012
Sector(s):
Energy efficiency in power sector (50%), Power (50%)
Theme(s):
Infrastructure services for private sector development (100% - P)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Richard L. Berney
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
According to the Emergency Project Paper (EPP, page 1) "The development objective of the project is urgently to increase the volume and reliability of the national energy supply part (sic) in winters, especially, through investment in thermal power generation and support to the implementation of the Government of Kyrgyz Republic’s Energy Emergency Mitigation Action Plan (EEMAP)."

According to the Financing Agreement (FA Page 5), "The objective of the Project is to assist the Recipient, in accordance with the Recipient’s Energy Emergency Mitigation Action Plan (EEMAP), to increase the Recipient’s volume of energy supply, especially, the supply of thermal power in the winter season, within the Kyrgyz Republic, and to improve the sustainability of such energy supply.

This review assesses achievement of the objectives of the Financial Agreement. Reliability, mentioned in the EPP statement, will also be assessed.

Additional Financing of US$4 million equivalent was approved by the Board on November 5, 2009. The project objectives did not change.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
1: Rehabilitation of the Electricity and Heating Systems (Appraisal US$14.84 million; Actual US$14.90 million)
This component included carrying out the rehabilitation of the electricity and heating systems of the Recipient's Power Plants Company, including the rehabilitation of electrostatic precipitators and the provision of low sulfur fuel oil to the Bishkek and Osh combined heat and power plants as well as the provision of brass pipes and coal handling equipment, such as bulldozers, in support of such rehabilitation.

2: Capacity-Building (Appraisal US$0.16 ; Actual US$0.06 ) This component supported project implementation by strengthening the institutional capacity of the recipient's Power Plants Company to carry out project management, monitoring and evaluation, including the capacity to carry out activities pertaining to environmental management and safety, procurement, disbursement and financial management.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Cost: Final project cost was US$14.96 million, compared to an appraisal estimate of US$15 million.

Financing: The original IDA financing was for US$11 million (including a US$5 million Project Preparation Fund), half as Grant and half as Credit. US$4.0 million of Additional Financing was approved by the Board on November 5, 2009 with the objective of supporting the whole project cost, including the repair of the Bishkek combined heat and power plant boilers and attached assets such as eight exhausters at four boilers, and supply of coal-handling equipment such as lifts and conveyer belts for six conveyers. These items could not be supported at the time of approval of the original Grants and Credits because of insufficiency of IDA funds. The Additional Financing was also half Grant and half Credit. All the Grants and Credits were nominated in Special Drawing Rights (SDRs), which appreciated in value relative to the US dollar over the course of the project. Of the total approved IDA financing of US$15 million, an undisbursed amount of SDR 37,195 (US$ 57,652 equivalent) was cancelled on July 31, 2012.

Borrower Contribution: The Borrower contribution comprised the staff of the Project Implementation Unit and all the workers for the repair and upgrading of the boilers. Its contribution is not quantified.

Dates: The project was extended twice, for a total of almost two years, first from April 30, 2010 to May 31, 2011 in conjunction with the Additional Fiancing, and then to March 31, 2012, to allow all the Grant and Credit funds to be used after the effectiveness of the Additional Financing had been delayed by political instability.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Substantial.

The objective was relevant to the 2007 Joint Country Support Strategy (JCSS) for the Kyrgyz Republic, and addressed one of the strategic objectives of the JSCC, which underlined that “there is an urgent need to stem the deterioration in key infrastructure and social services that will reduce non-income dimensions of poverty” and a “need to support the government’s efforts to ensure and enhance the provision of, and access to, essential public services”.

The Kyrgyz Republic’s energy supply relied on hydroelectric power generation for 90 percent of its power output in 2008/2009. Ninety percent of this hydroelectric power came from one river whose flow had reached critically low levels during an exceptionally dry 2007-2008 period. As a result, at the time of project appraisal, the Kyrgyz Republic's energy supply was at an emergency low level. The country was also facing a gas and fuel oil supply crisis as a result of rapidly rising import prices and limited foreign funds to purchase these commodities. This created a substantial risk of freezing and subsequent bursting of central heating and hot water supply pipes, which, if it had occurred, would have put an excessive burden on the electricity supply, since it would have been the only other heating source for the major urban centers. Without hot water or electricity, the population of Bishkek would have faced a real danger of freezing. The project objective of improving the output of the Bishkek and the Osh combined heat and power plants represented the only solution to this problem.

The 2011 Interim Strategy Note for the Kyrgyz Republic supported Additional Financing for the ongoing Bishkek and Osh Urban Infrastructure project because it contributed to social stability in urban areas, notably in the south, by (i) providing adequate essential basic infrastructure services, (ii) ensuring sustainability of the physical improvement, and (iii) strengthening the management of service delivery, thorough better accountability and governance of the municipalities and local utilities. It was envisaged to expand the project funding to small towns. (Page 26)

b. Relevance of Design:
Substantial.
The statement of development objectives was clear and straight forward. The project results chain included all the inputs and activities necessary to achieve the project’s expected outputs and outcomes. The output of the Bishkek and Osh combined heat and power plants needed to be increased to provide the essential heat supplies for the Kyrgyz Republic's two main urban centers. Improving their performance was critical for ameliorating the urban winter heating supply problems. The project's initial costing provided for substantial improvements in the performance of these power plants.


4. Achievement of Objectives (Efficacy) :

1. To urgently increase the volume of the national energy supply in winter. High

Outputs:
Project implementation of the original project proceeded mostly on schedule, The urgently needed materials and goods – low sulfur fuel oil (20,000 tons), brass pipes, bulldozers, and spare parts for electrostatic precipitators, new conveyor belts – were delivered and installed on time, before the arrival of cold weather, and all activities under the original project were completed by the original closing date of April 30, 2010.

Institutional improvements include establishing a program of daily monitoring of coal and fuel oil stock balances at the Bishkek and Osh CHPs, which are critical data for determining the amount of energy that these plants are capable of supplying, and a daily monitoring of the water inflow and outflow at the Toktogul hydroelectric power plant, which enables the authorities to evaluate the medium term electric generation capabilities of this hydroelectric plant. Both of these data gathering and dissemination activities were initiated as part of the project's M&E program.

Outcome:
The key indicators were: (i) additional heat output during the winter seasons; (ii) greater plant operating time; (iii) additional thermal electrical power output during the winter seasons. As can be seen from the table below, the outcomes were substantially greater than the targets for the first critical winter of the original project (2008/09). Output of the target indicators decreased in the following winter of 2010/11, due to political instability in the Kyrgyz Republic and the consequent delay in the effectiveness and implementation of the Additional Financing. However, at the completion of the project, plant production had increased significantly, with winter heat output 54% higher than the base year, power output 48% higher than the base year, and plant operating factor (number of hours operated at installed capacity) 75% higher than in the base year.


2007/0 2008/9 2009/10 2010/11
Base Year Target Actual Target Actual Actual

Winter Heat Output (GCal) 1292 1382 1564 1412 1592 1991
Improvement over base year 90 272 120 300 699

Power Output (GWh) 492 537 684 572 595 730
Improvement over base year 45 192 80 103 238

Plant Factor (hours) 739 >739 1027 >739 894 1293
Improvement over base year 288 155 554

2. Increasing the reliability and sustainability of the energy supply: Substantial

Given the emergency nature of the operation, sustainability will be taken to refer only to physical sustainability and not to financial sustainability.

Outputs:

The outputs under the first objective are also relevant to this one.

The ICR notes that the operating staff have been trained in operational performance monitoring and appropriate preventive measures.

The gathering and dissemination of critical operating information and the status of hydrological conditions relevant to the output of the hydroelectric power generating system has been institutionalized and is now regularly used internally within the Power Plants Company. Under the project, Power Plant staff gained additional training and experience in preventative maintenance as part of their operational performance monitoring activities.

Outcomes:
Plant reliability improved greatly over the course of the project, as seen by the increase in the number of hours that the Combined Heat and Power (CHP) plants were able to operate, from 739 hours in 2007/08 to 1,293 hours in 2010/11. (See Plant Factor in the above table).

According to the ICR (page 10), the new materials and equipment supplied under the project, while helping to address the particular emergency situation, have a long operational life (up to 15 years) and would therefore be durable and sustainable. The repaired boilers are expected to operate for about five years, and the new electrostatic precipitators will contribute to a longer operating life for the boiler pipes. The coal handling equipment (the conveyor belts and bulldozers) supplied under the Project replaces equipment that had been in use for almost 40 years, and was worn out.

Because the project was designed to provide emergency assistance to the sector, it focused on short term energy production problems, and did not address the sector’s financial issues. Inadequate cost-recovery tariff levels are exacerbated by high technical and commercial losses, and low collections rates. These issues are only just beginning to be addressed.

5. Efficiency:

The ICR does not analyze the project’s ex-post financial or economic rates of return, nor does it make any other quantitative assessment of the efficiency with which project resources were used. It reports the Government's estimate that the economy could seriously slow down during the crisis, due to curtailment of electricity supplies and concomitant reduction in economic activity. And it claims that the economic value of the electricity and heat generation under the project is indisputable, since the economic cost, including loss of jobs due to disruption of energy supply to industries, small and medium enterprises and commercial customers, would have been considerable, and in the absence of the emergency response, the entire city of Bishkek could have frozen, with catastrophic consequences for human welfare loss. The ICR (page 13) also states that the project's benefits were realized in a cost-effective manner, and that the costs were considered reasonable compared to the achieved benefits and recognized norms.

There were some administrative and operational inefficiencies. Procurement performance (primarily shopping on a prior review basis) was mixed. Most contracts were awarded according to the agreed procurement schedule but a few packages suffered delays due to low capacity of both the implementing agency and the bidders. The delay in effectiveness of the Additional Financing led to cancellation and re-tendering of a number of procurement activities that had been initiated shortly after Board approval, but in advance of effectiveness. This caused an implementation delay of one winter season, and there was a delay of nearly two years in completing implementation.

Efficiency is rated modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project's objective remain substantially relevant to the needs of the country and to the Bank's assistance strategy for the Kyrgyz Republic. Relevance of design was also substantial. The efficacy of the first objective -- increasing the volume of energy supply, especially that of thermal power in the winter season was high. Higher than targeted increases in the output of heat generation during winter seasons of the Bishkek and Osh Combined Heat and Power Plants greatly eased the energy emergency that the Kyrgyz Republic faced starting in 2007/08. The efficacy of the second objective is rated substantial. There is evidence of increased reliability and durability provided standard maintenance standards are complied with. Efficiency, however, was modest since there is no ex-post quantitative analysis of the efficiency with which project resources were used and there was a delay of nearly two years in completing implementation. Overall outcome is rated moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

The equipment supplied under the project has a long operational life period and should therefore be durable and sustainable. The repaired boilers have an expected operating life of fifteen years without new major repairs. The Power Plant staff is fully trained in the operational performance monitoring needed to identify the need for preventive measures. An emergency plan is in place and is updated annually. However, the combined heat and thermal power capacity needs to be expanded further to reduce the country's over-dependence on hydroelectricity in the winter.

While there have been improvements in the functioning of the energy sector, several important policy reforms still need to be implemented. Financially, the energy sector remains weak, as a cost-recovery tariff level has not yet been achieved. This problem is exacerbated by high technical and commercial losses and low collection rates. Until these issues are properly addressed, the long term sustainability of the sector, and therefore of the assets that were renewed under this project, will remain at risk.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:
The Bank responded quickly to the Government's request to help manage the energy crisis and provide a donor coordination role. An energy team arrived within days of the Government’s request and assisted it in preparing a structured emergency response. It extended a Project Preparation Facility of US$5 million, primarily to provide essential low sulfur fuel oil to keep the heating plants operating, but also to initiate procurement of coal handling equipment. The Bank re-prioritized its IDA program for the Kyrgyz Republic, allocating US$11 million to finance the most urgent priorities in the Government's Energy Action Plan (EEMAP). Project design was simple and focused on the priorities of rehabilitating the critical CHP equipment. Procurement documents were prepared and reviewed by the Bank before the project became effective. The ICR reports that the Bank closely coordinated its efforts with other external partners and recognized the comparative advantage of each. However, no details are provided.

Quality-at-Entry Rating: Highly Satisfactory

b. Quality of supervision:
Project activities were implemented satisfactorily under both the main project and the Additional Financing. The Bank team closely monitored the procurement progress .The task team leader was located in Bishkek and the procurement and financial management staff were located in the region, ensuring timely responses to procurement issues as they arose. Bank supervision was flexible and prompt in responding to Government requests under changing political conditions, including that for the Additional Financing. Fiduciary compliance was ensured by the establishment of an Action Plan which was followed during project implementation. The Bank’s financial management specialist regularly conducted supervision to ensure compliance with the Bank requirements.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Government demonstrated its strong commitment to the project and its objectives. When it recognized the emergency nature of the situation, it acted swiftly to formulate an Emergency Energy Mitigation Action Plan, which it followed. This strong commitment was a key factor that not only helped formulate the project concept/design, but also facilitated its implementation. An extensive participatory process was followed to ensure constant interaction and information sharing. This approach helped in successful project implementation even during the time of significant changes in the Government. However, in the wake of the political instability that began in April 2010, the Government’s focus was distracted from project implementation and resulted in the delayed effectiveness of the Additional Financing. The new Government continued to provide strong support for the project.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:
The project was implemented by the Project Implementation Unit of the Power Plants Company (PPC), which worked efficiently to avert the potential crisis of inadequate heat for the 2008/09 winter season. A part time procurement specialist and a financial specialist who were also working on other Bank projects were hired to ensure smooth running of the procurement process when the staff of the PPC needed assistance. Procurement activities were carried out in accordance with the guidelines. The audit reports were submitted in a timely manner and were acceptable to the Bank. The ICR also reports that Bank safeguards policies were complied with (see Section 11 below).

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
M&E design clearly reflected the project’s development objectives. The indicator for intermediate output performance was the utilization factors of the Bishkek and OSH CHP plants. The two output/outcome indicators were the annual increase in heat production and power production during the winter seasons. The baseline was provided by the CHP plant management and collected by the PIU. As designed and implemented, these key intermediate and final performance indicators were fully able to assess the achievement of the objectives and to test the links in the results chain. Baseline data on the current outputs of the two plants were collected. However, there was no attempt to collect data related to estimate what would have been the output of the plants if funds had not become available for plant and equipment rehabilitation.

b. M&E Implementation:
The Power Plants Company and the Bishkek CHP plant collected indicator data regularly. Baseline indicators were established at effectiveness and target outputs and outcomes were clearly defined and were updated during the preparation of the Additional Financing project to reflect the up-scaling of output expected from this additional financing. Actual figures were compared to target values and were made available to IDA quarterly. The Implementation of the M&E system proved fully sufficient to assess the ongoing achievements of the project's objectives.

a. M&E Utilization:
This M&E data were used to measure the achievement of project targets throughout project implementation. The M&E program also included the collection of data on a daily basis by the Power Plants Company on water inflow/outflow at the hydropower plant and by the Bishkek and Osh Plants on the levels of the coal and fuel oil stock. The dissemination of this information was used to inform the direction of the project. It enabled both the Government and the Bank team to follow the progress of improvements the power sector's ability to meet its energy needs and to make projections of the country’s medium term energy balances and to take preventive measures, such as increasing fuel oil supplies, when necessary. These M&E arrangements are expected to be sustained beyond the project implementation period as the daily monitoring practices have been institutionalized and are regularly used internally within the Power Plants Company. The data gathered by the M&E system, as designed, were used effectively to assess the achievement of the project's objectives and to test the links in the results chain.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:
At appraisal, the project was classified under the environmental category “B” for environmental assessment purposes, as it was expected to have a limited and manageable impact on the environment. According to the Emergency Project Paper, the only safeguards policy to be triggered would be Environmental Assessment (O.P. 4.01). The works involved routine civil, mechanical and electrical rehabilitation, concerning existing structures only. No additional structures were built and no land was acquired. The ICR reports that the Environmental Management Plan was prepared to address the main environmental issues of the project: diligence in transport and handling of the fuel, safe storage at the power station site and the availability of permits for its combustion and related emissions. These requirements were part of the work contracts. According to the ICR, the project fully complied with the environmental management plan and with Bank procedures.

b. Fiduciary Compliance:
The financial management capacity assessment, conducted at project appraisal, concluded that the existing financial management arrangements of the implementing entity would not fully meet Bank requirements. The ICR reports that an Action Plan was developed and followed during project implementation, reportedly resulting in full compliance with requirements during project implementation. A qualified and experienced financial specialist was recruited to handle accounting, reporting and disbursement functions. Financial reports were prepared regularly and in a timely manner. External audits of the project financial accounts were carried out by auditors acceptable to the Bank and with timely submission of all the reports. The audit reports were found to be satisfactory (that is, unqualified) and acceptable to the Bank.

c. Unintended Impacts (positive or negative):
In addition to ensuring that the energy supply system did not collapse, the rehabilitation of electrostatic precipitators in the coal fired Bishkek combined heat and power plant substantially decreased particle emissions. Ash emission was reduced by about 1,000 tons during the winter period, which and had a direct positive impact on air quality in the areas near the plants. Improved availability of central heat and power also had a positive indirect environmental impact by avoiding emissions from improvised individual household multi-fuel stoves and the cutting of trees to supply fuel.

By increasing the availability of heat and electrical power, the project has indirectly contributed to the power company's ability to increase the amount of water that it has been able to store behind its hydroelectric dam in years of low hydrological flow, thereby increasing its availability for agricultural uses in the summer in years of low hydrological flow.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Moderately Satisfactory
The ICR contains no quantitative analysis of the efficiency with which project resources were used, thereby leading to a modest assessment of efficiency. 
Risk to Development Outcome:
Significant
Significant
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Moderately Satisfactory
Government performance is rated moderately satisfactory since, in the wake of the political instability that began in April 2010, the Government’s focus was distracted from project implementation and resulted in the delayed effectiveness of the Additional Financing.  
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
These lessons are taken from the ICR with some adaptation of the language:
  • Projects that are prepared in a short time frame in response to an emergency, have to be designed as simply as possible, in order to deliver quick results and to respond rapidly to evolving needs on the ground.
  • Although emergency operations are not designed to address long-term policy issues, they do serve as good entry points for a longer term engagement. As such, it would be prudent for emergency operations to be followed by a longer-term engagement, as was the case in this project.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR covered all the essentials. However, an important shortcoming was the absence of any quantitative analysis of the efficiency with which project resources were used. A discussion of the results of the donor coordination efforts might have been helpful for future emergency projects. A discussion about why heat and electricity output declined in the winter of 2010/11 would also have been useful.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions