Search in:
Go
Advanced Search

Implementation Completion Report (ICR) Review - Social Infrastructure Retrofitting Project


  
1. Project Data:   
ICR Review Date Posted:
04/19/2012   
Country:
Belarus
PROJ ID:
P044748
Appraisal
Actual
Project Name:
Social Infrastructure Retrofitting Project
Project Costs(US $M)
 59.36  52.39
L/C Number:
L4876, L7056
Loan/Credit (US $M)
 37.6  37.6
Sector Board:
Energy and Mining
Cofinancing (US $M)
   
Cofinanciers:
Board Approval Date
  06/05/2001
 
 
Closing Date
12/31/2006 12/31/2010
Sector(s):
District heating and energy efficiency services (94%), Central government administration (6%)
Theme(s):
Access to urban services and housing (67% - P) Pollution management and environmental health (33% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Fernando Manibog
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:

According to the Project Appraisal Document dated May 10, 2001 (PAD, page 2), the Project Development Objective (PDO) was "to improve the functional and health environments of social sector facilities, with particular emphasis on reducing energy consumption." The Additional Financing Project Paper dated November 5, 2007 indicated the same objective. Schedule 2 of the Loan Agreement dated September 26, 2001, however, was worded differently, as follows: "The objective of the project is to improve the functional and health environments of social sector facilities, such as orphanages and community homes for the elderly and disabled." Thus, the Loan Agreement gave examples of the types of facilities covered by the project, but did not refer to the project's "emphasis on reducing energy consumption." This ICR Review is based on the PAD statement of the PDO, which is more monitorable.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

The project included two components:

A. Energy Retrofitting (US$56.36 million appraisal and additional financing estimate; US$46.31 million actual)
This component consisted of energy retrofitting in schools and medical facilities, and associated supply-side improvements. The specific activities included: boiler conversions to gas-firing instead of oil, coal or electricity; boiler replacements and heating system improvements; rehabilitation of selected portions of the District Heating Networks to capture savings at the heat production source; piloting of renewable fuels such as wood chips; and energy efficiency measures. Up to 500 public buildings were to be retrofitted.

B. Technical assistance (US$2.73 million appraisal and additional financing estimate; US$4.82 million actual)
This component financed technical and financial audits, public awareness campaigns, training of the Project Management Unit staff, energy policy and environmental studies, and associated workshops. The social surveys and information dissemination campaigns aimed to increase beneficiary participation and ownership of energy-saving measures. Tariff and subsidy studies were intended to pave the way for Bank-Government dialogue on policy issues of high priority for the country's energy sector.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Cost:
Including the IBRD front-end fee and Project Preparation Fund, the project's appraisal and additional financing estimate was US$59.36 million, while the actual cost was US$52.39 million.

Financing and Borrower Contribution:
The original IBRD loan was US$22.6 million. An Additional Loan of US$15 million was approved by the Board on December 4, 2007 to finance the cost overrun and the scaling-up of energy efficiency activities in about 140 additional buildings. The cost overrun of about US$6 million resulted from bid prices that were higher than estimated due to the strong euro and increased wages in Belarus. The total IBRD loan was US$37.6 million. The Borrower's actual contribution was US$14.79 million, or 68% of the US$21.76 million estimate at appraisal and with the additional financing. No cofinancing was involved.

Dates:
The project was approved on June 5, 2001 with an original closing date of December 31, 2006. A mid-term review was conducted in December 2004. The project closed on December 31, 2010, after three extensions totalling four years beyond the original closing date. The three extensions were made to allow reallocation of funds, completion of a final contract under the original project, and completion of the activities financed by the Additional Loan. The dates for full disbursement and closing of the original loan and the additional loan, were March 31, 2008 and December 31, 2010, respectively.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Objectives: High
The relevance of the project's objectives was, and remains, high in relation to the needs of the country's energy sector and the Bank's Country Assistance Strategy (CAS). Since the mid-1990s, with the structural shifts of the economy away from Soviet-era infrastructure, Belarus has actively sought to improve energy efficiency in its industrial and public sectors. In 1996, the Government created the Committee for Energy Efficiency, now the Energy Efficiency Department (EED). Between 1996 and 2000, it invested US$370 million in energy-saving measures such as installing metering devices, rehabilitating and replacing old heat boilers, and designing more energy-efficient materials and technologies. These initial efforts were extended up to 2005 under an Energy Savings Program. At the same time, the Bank's CAS -- after lending was resumed in FY99 when certain macroeconomic triggers were met -- identified the energy and social sectors as potential areas of cooperation and renewed dialogue, given the commitment shown by the Government. In particular, measures to reduce energy consumption, with a focus on improving functional and ambient health conditions, were seen as low-case scenario projects that have robust economic and social benefits even if the macroeconomic and structural environment were to become uncertain. The project's objectives, therefore, were highly relevant given the Bank and country context at that period. These objectives are even more relevant today given Belarus' drive to use energy savings as a major instrument to reduce its dependence on energy imports; moreover, the Bank's current CAS (FY08-11) continues to support the country's energy efficiency program under the pillar related to Global Goods.

b. Relevance of Design:

Design: Substantial
The project's straightforward design, which focused on energy retrofitting backed up by technical assistance, was substantially relevant to the energy intensity issues that Belarus was trying to address. The project focused on poor lighting and heating conditions in schools and hospitals, which were below even basic standards and resulted in poor delivery of educational services, high absenteeism among students and teachers, frequent illnesses, repeated hospital visits, longer healing times, etc. The design was also tailored to respond directly to the Government's request for the Bank's assistance in incorporating the social sector into the country's high-visibility energy efficiency program. More specifically, the Government requested the Bank for a project design that took into account the Bank's experience in addressing the energy efficiency problems of Soviet-design structures, as well as the lessons learned from the Bank's financing of energy efficiency projects in the Russian Federation, Lithuania, Estonia, and Ukraine. However, the outcome indicator targets should have been quantified at the appraisal stage rather than only later during the first supervision mission.


4. Achievement of Objectives (Efficacy) :

The degree of achievement of the PDO -- to improve the functional and health environments of social sector facilities, with particular emphasis on reducing energy consumption -- was substantial.

The project has resulted in reducing energy consumption by about 243,000 MWh per year, while improving heating services for public buildings in the social sectors (education and health), improving user satisfaction, and reducing greenhouse gas emissions. Annex 2 of the ICR (Outputs by Component) compares (a) the original targets that were quantified in the Implementation Status and Results Report (ISR) from the first supervision mission with (b) the expanded targets that were set during the Additional Financing stage. The table shows that nearly all of the targets were met or exceeded. Some highlights of targets that were exceeded include the following:

  • 745 buildings were retrofitted, compared to the previous targets of 717 buildings in the Additional Financing Project Paper, 643 buildings in the first ISR, and 480 to 500 buildings in the PAD
  • 380 buildings were equipped with lighting improvements, compared to 301 buildings in the Additional Financing Project Paper, and 202 buildings in the PAD
  • 120 buildings were equipped with more energy-efficient window replacements, compared to 38 and 28 buildings targeted in the Additional Financing Project Paper and the PAD, respectively
  • 9 audits were carried out, compared to the target of 1 audit per year

Quantitative targets related to numbers of boiler houses and heat substations renovated, tariff/subsidy and environmental studies, social surveys, and public awareness campaigns were all met. In schools where lighting improvements were made, luminosity almost doubled from 200 to 300 lux without the project to more than 500 lux after the completion of project improvements, while at the same time saving about 15 percent in electricity consumption. It is estimated that the project also resulted in a reduction of about 52,000 tons of CO2 emissions. A monitoring and evaluation contract was also carried out.

Qualitatively, surveys have shown a high degree of user satisfaction in having warmer, more comfortable, better lit and less polluted environments for learning, working and getting well. Funds saved through energy efficiency improvements were used for other priority needs in the school and hospital facilities. Eight demonstration sites have also provided examples to the visiting public on the range of possible energy-saving measures.

5. Efficiency:


Efficiency is rated substantial. The recalculated economic rate of return (ERR) and financial rate of return (FRR) were approximately 25 % and 20 %, respectively. The economic price of natural gas that was being avoided through the project's energy savings had increased significantly at project closing compared to the level during project appraisal, i.e., US$287 and US$65 per thousand cubic meters (TCM), respectively. While this would have resulted correspondingly in significant increases in the EIRR, there were countervailing increases in investment costs due to the strong euro, as well as general wage and price increases in Belarus. For the FIRR recalculation at closing, the actual natural gas price was US$151/TCM, compared to US$30/TCM at appraisal.

The project closed on December 31, 2010, after three extensions totalling four years beyond the original closing date. The three extensions were made to allow reallocation of funds, completion of a final contract under the original project, and completion of the activities financed by the Additional Loan. The dates for full disbursement and closing of the original loan and the additional loan, were March 31, 2008 and December 31, 2010, respectively.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
24.4%
85%
ICR estimate:
Yes
24.9%
85%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


The relevance of the project's objectives was high and design relevance was substantial. The objectives remain highly relevant today given the continuing high priority for Belarus of reducing energy imports by saving energy, which are being pursued in both the Government's sector strategy and the Bank's most recent CAS. The project's efficacy is substantial -- nearly all of the targets were met or exceeded. At 25 % ERR and 20 % FRR, the project's efficiency is also rated substantial. The overall project outcome rating is satisfactory.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:

Government strategy in the sector may be expected to continue supporting energy efficiency programs. Public awareness of the benefits of saving energy is strong, as is the ownership of necessary energy efficiency measures. Based on the life cycle of new equipment installed under the project, the energy efficiency improvements may be expected to last 15 to 20 years, but newer and better technologies that achieve higher energy efficiencies may also be expected during that period. Beneficiaries have put maintenance arrangements in place, and suppliers have provided adequate training in the use of the improved equipment.

a. Risk to Development Outcome Rating: Negligible to Low

8. Assessment of Bank Performance:

a. Quality at entry:

In preparation for this first project in Belarus after many years of non-lending, the Bank carried out an energy sector review in April 1995, which was updated in 1998 and in 2000. The Bank also provided a Project Preparation Facility (PPF) advance of US$1 million in February 1998 to finance pilot energy efficiency investment activities in Minsk, public awareness campaigns by a local NGO, and a social survey to verify project benefits as perceived by the beneficiaries and to help devise the project's monitoring indicators. A PHRD Grant was also approved in November 1998 to pilot energy efficiency activities in schools and hospitals, launch an energy conservation media campaign, review utility billing and collections systems, and prepare tender documents. The PPF and PHRD (both approved in 1998) helped establish and train the Project Management Unit sufficiently in advance of the project appraisal in February 2001, which significantly facilitated the subsequent implementation of the project. However, the outcome indicator targets should have been quantified at the appraisal stage rather than only later during the first supervision mission.

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:

Shortly after project effectiveness in December 2001, the Bank conducted a project launch workshop in May 2002. The project was supervised regularly, with adequate budget and staff resources. The project team alerted the Government to implementation issues as they arose and facilitated timely remedies, according to the ICR. The mid-term review mission in December 2004 helped to resolve project issues (e.g., related to procurement procedures, limited number of bidders, inter-agency coordination, selection of sites) and also carried out a second rapid social assessment (which, for example, indicated that operation & maintenance and rapid personnel turnover were problems in some sites).

The project team responded expeditiously to the Government's request in 2007 for additional financing to cover cost overruns and scale up the project. A social scientist and environmental specialist were also added to the project supervision team, in response to recommendations of the the Quality Assessment of the Lending Portfolio that was carried out for the project in 2008 by the (then) Quality Assurance Group. The Bank team was highly decentralized, which facilitated close supervision on the ground, while specialized technical support was provided from headquarters.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Government -- through its priorities, policies, and actions prior to and during the project -- demonstrated consistent commitment to the project's objectives. By 2009-2010, the Government was spending about 1.2 percent of GDP on energy efficiency projects, and the project was an essential component of the Government's much larger energy efficiency program that started about 15 years earlier. The Government has put in place an adequate legal, regulatory and institutional framework to support energy efficiency. These include an energy efficiency law, regional and national programs with monitorable targets, and the creation of the Energy Efficiency Department (EED) with overall responsibility for promoting energy efficiency at the national and local levels.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:


    EED had the overall responsibility for implementing the project, while day-to-day operational implementation tasks rested with the Project Management Unit (PMU). EED coordinated inputs from the various ministries, especially the Ministries of Health and Education. The PMU was responsible for procurement and contract supervision, financial management, compliance with the project's environmental management plan, reporting to the Bank and EED, and monitoring & evaluation. According to the ICR, the PMU worked closely with staff of local governments and sector ministry representatives to ensure the project's timely implementation. For example, the PMU hired a local engineer to cover each oblast and to monitor, report and supervise project progress and performance of contractors.

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The project's indicators were appropriately keyed to measuring megawatt hours/year of energy savings, number of buildings rehabilitated, number of boiler houses and substations rehabilitated, number of buildings with lighting improvements, number of buildings with window replacements, and number of demonstration sites. One shortcoming of the results framework, however, was that the baselines and end-of-project targets "were developed and revised over time in the various ISRs during implementation" according to the ICR, instead of being quantified in the PAD. Social indicators were also not quantified, hence difficult to monitor and attribute solely to the Project.

b. M&E Implementation:

The PMU collected data regularly on the project's performance vis-a-vis the target values developed during implementation. The PMU prepared semi-annual Project Progress Reports that were made available to the Bank and the Government. The calculation of savings after completion of the energy efficiency measures were based on energy audits that were done in each of the facilities before rehabilitation. The calculations were made by the beneficiary institutions.

a. M&E Utilization:

Local and regional authorities, as well as the EED, made use of the M&E findings for decision-making when planning energy efficiency investments, according to the ICR.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:

The project was rated a category B for the purposes of the Bank’s OP 4.01 on Environmental Assessment. According to the ICR, there were no environmental issues during implementation, and PMU followed the Environmental Management Plan (EMP), including the removal of any hazardous materials, mitigation of noise and pollution levels during construction, and safety. Compliance with the EMP was confirmed by the Bank's environmental specialist during site visits.

b. Fiduciary Compliance:

The PMU maintained a satisfactory financial management system, according to the ICR. The PMU submitted quarterly financial monitoring reports and annual audit reports to the Bank. Except for a mall delay in receipt of audit reports in 2008 and 2009, audit reports were received in a timely manner and "were acceptable to the Bank."

According to the ICR, procurement under the project was conducted in compliance with the applicable guidelines, provisions of the Loan Agreement and agreed procurement plan. All contracts concluded under the project were subject to the Bank’s prior review. Procurement supervision was performed by the Bank procurement specialist during regular project supervision missions, including physical inspection of contracts, observed availability and high quality of goods, works and materials, procured by the Borrower under the project.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Satisfactory
 
Risk to Development Outcome:
Negligible to Low
Negligible to Low
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

The following key lessons that can be learned from the project's preparation and implementation are taken from the ICR with adaptation:

  • When initiating or resuming lending to client countries where there has been no prior Bank lending or where operations have been suspended temporarily, it is important to focus on the sector(s) for which the Government has consistently shown adequate commitment, and select high priority projects with large and visible economic and social benefits, especially if the macroeconomic and institutional environment continues to be uncertain.
  • Baseline data at entry and end-of-project outcome indicators need to be quantified prior to implementation in order to ensure that project performance is measurable during supervision and attributable to the project at completion.
  • To facilitate least-cost procurement in cases where contracts may be too small and too geographically dispersed to attract international bidders, the procurement approach needs to include: (i) the provision of workshops and information to local bidders; (ii) revision of the conditions and terms of the bidding documents to elicit a sufficient number of bids; and (iii) meeting(s) between the Bank team, the PMU, and potential local bidders to inform them about the Bank’s procurement procedures and determine the reasons for their lack of interest or failure to bid.
  • To avoid costly change orders and contract amendments during implementation, it is important to ensure the accuracy of basic data for technical specifications, by following these steps: (i) require the beneficiary institution to sign off on the data; (ii) have the design and supervision consultant and the project implementing agency visit the sites before finalization of the technical specifications to verify the basic data; and (iii) request the contractors to visit the sites before the start of works.
  • To minimize investment and operation and maintenance costs, if is important to review and revise the design practices in the country with a view to reducing the capacity requirements and the potential over-sizing of boiler specifications.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:


The ICR provided the essential details regarding the project's implementation experience and results, and is consistent with the preparation guidelines. It provided sufficient sector context that underlined the importance of energy efficiency programs to Belarus. It would have been useful if some description of the two follow-on projects had been provided, given the long-term strategic vision of the Government in the energy efficiency area.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)