Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Power Transmission Project

1. Project Data:   
ICR Review Date Posted:
Project Name:
Power Transmission Project
Project Costs(US $M)
 55.4M  54.1
L/C Number:
Loan/Credit (US $M)
 48.00M  46.3M
Sector Board:
Energy and Mining
Cofinancing (US $M)
 0  0
Board Approval Date
Closing Date
12/31/2010 12/31/2011
Power (100%)
Infrastructure services for private sector development (100% - P)
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Richard L. Berney
Robert Mark Lacey Soniya Carvalho IEGPS1

2. Project Objectives and Components:

a. Objectives:
The project development objectives are stated in the Project Appraisal Document (PAD, page 3) as follows: "The primary objective of the project is to improve the efficiency of the power transmission operation in Azerbaijan through technical and institutional strengthening of the generation/transmission utility. The project has as a secondary objective to contribute to strengthening the financial position of Azerenerji [ the publicly-owned power generation and transmission utility and the Borrower]."

According to the Loan Agreement (page 16), “the objectives of the Project are to: (a) improve the efficiency of the power transmission operation through building the Borrower’s technical and institutional capacity; and (b) strengthen its financial condition.”

This Review uses the Loan Agreement's statement of objectives as it is more monitorable.

b. Were the project objectives/key associated outcome targets revised during implementation?

c. Components:
1. Power System Management Equipment (Appraisal US$45.8 million; Actual US$50.4 million)
Subcomponent 1.1: Power System Management
(i). Installing a Supervisory Control and Data Acquisition/Energy Management System (SCADA/EMS), to enable real time acquisition and analysis of operational information from generation stations and high voltage transmission substations, and control of the generating plants and high voltage transmission systems;
(ii). Upgrading the telecommunications network, including broadband communications facilities between major offices and facilities; and
(iii). Installation of generating plant and substation control and metering equipment, including fuel metering on major thermal generating units.
Subcomponent 1.2: Transmission Network Rehabilitation; Rehabilitation of selected High Voltage transmission lines and High Voltage transmission stations.
2. Management Assistance (Appraisal US$3.6 million, Actual US$2.2 million): Improving Azerenerji's management systems and preparing the company for a future restructuring of the energy sector.
3. Project Implementation Unit (Appraisal US$ 0.7 million, Actual US$1.5 million). Financed the Unit's incremental operating costs.
The components were not changed.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: The cost of the physical investments (the SCADA system and transmission rehabilitation) increased from US$45.8 million at appraisal to US$50.4 million at completion, an increase of US$4.6 million (10%), which is slightly less than the US$5.2 million contingency allowance in the Appraisal cost estimate.
Financing: US$1.7 million of the Bank loan remained undisbursed at closure, and was cancelled. There were no other external financing sources.
Borrower Contribution: The Borrower was initially expected to contribute US$7.4 million, but actually contributed US$8.0 million. The additional amount was for further training on the use of the SCADA system.
Dates: There was a one year extension of the closing date to December 31, 2011, to enable the completion of the SCADA/EMS.

3. Relevance of Objectives & Design:

a. Relevance of Objectives:
The CPS for FY 2011-2014 states that "Recent extensive government investments in the energy generation, transmission and distribution capacities (supported by the World Bank) have resulted in notable improvements in the quality of utility services." (Para 39). And that "The Government sees the continuation of priority infrastructure projects as critical for development of businesses," and "The need to ensure uninterrupted power supply and gasification is a priority" (para 26). In addition, the project’s objectives complemented the sector policy reforms that had been supported by the Bank's adjustment operations.
The project’s objectives were also relevant to the FY 2003-2005 Azerbaijan Country Partnership Strategy (CPS), one of the goals of which was to reverse the decline in infrastructure services. The power sector, in particular, was performing poorly. There were frequent rolling blackouts, with many areas of the country receiving only a few hours of electricity per day. System reliability was poor with frequent localized outages and occasional widespread system failures. The delivery of affordable and reliable electricity was, and continues to be, an essential requirement to enhance living standards, and supporting new income generating activities.

b. Relevance of Design:
There was a clear and logical causal chain between all of the activities that the project was designed to carry out and the expected attainment of its objectives. Power system management tools, including the installation of a SCADA/EMS were to enable real time acquisition and analysis of operational information, while upgrading generating plant and substation control and metering equipment was needed to feed operating information automatically into the SCADA system. These are all critical elements in enabling the operators to minimize generation costs, and effectively stabilize system output, thereby improving the efficiency of the power transmission system. Rehabilitation of selected High Voltage transmission lines and High Voltage transmission stations is also critical for minimizing transmission losses and power outages. The investment activities were effectively complemented by technical and managerial assistance to enable Azerenerji to operate the enhanced infrastructure efficiently.

4. Achievement of Objectives (Efficacy) :

(a) Improve the efficiency of the power transmission operation through building the Borrower’s technical and institutional capacity. High.
  • The high voltage transmission system rehabilitation was completed successfully.
  • The Power System Management program, including the installation of a SCADA/EMS and controls and metering equipment in generating plants and substations, and the upgrading of the telecommunications network, were completed by project closure. Elements of the SCADA system are undergoing final testing and fine tuning during the second half of 2012. Staff are receiving additional training to enhance their capabilities to use all the system elements.
  • A transmission network study was completed.
  • Transmission losses, which were expected at Appraisal to decline from 5.0% to less than 4.5%, actually fell to 3.6 percent in 2011.
  • Forced outages in rehabilitated transmission lines declined significantly as did overall network outages. Total hours of outages went from 1,425 in 2003 to 158 in 2011, while total power lost went from 81 GWh down to 13 GWh. The dispatch system improvements have enabled Azerenerji to (i) monitor and control the load flows on transmission lines in real time; (ii) identify flexible alternatives for correcting localized faults in the transmission system; (iii) plan outages of substations and lines as necessary; (iv) archive for analysis of unanticipated “events” in the power system; (v) forecast load requirements for all substations and transmission lines; (vi) estimate the dynamic and static stability of the system as a whole.
  • These achievements have enabled Azerenerji to reduced fuel consumption by 11 percent between 2007 and 2011, from 353 grams of fuel per kWh generated (g/kWh) to 314 g/per kWh, seven times the 1.5 percent improvement objective established at Appraisal. It has also enabled the company to maintain grid voltage to within +5%, -10% stipulated in the nominal bulk supply contracts, and to reduce frequency variation to within 0.1% of 50 Hz, both of which were established at appraisal as energy quality improvement objectives.
  • It must be noted, however, that these improvements would have been unlikely without the concurrent rehabilitation and upgrading of the system’s generating plants which provided the surplus in capacity needed to maintain grid voltage and frequency. The generation plant investments were financed by IFC and other international financial institutions (EBRD, KfW, Islamic Development bank and JBIC) outside the framework of the project.

(b) Strengthen the Azerenerji's financial condition. Substantial
Azerenerji’s financial condition improved over the life of the project and remained stronger at project closure than at appraisal, despite declining operating margins in 2009 and 2010.
  • Technical assistance for improved company and system management was provided in the following areas: project management; project finance management; procurement; assets evaluation; project and company audits; assistance in Azerenerji’s transition to International Financial Reporting Standards; development of an integrated management information system; and dispatch system training.
  • The EMS information management system has enabled Azerenerji to significantly improve its corporate and financial management. It has enable the utility to better control its billing system. Together with the technical and managerial assistance provided under the project, these improvements in management controls have made it possible for Azerenerji to implement a substantial internal restructuring along functional and financial lines.
  • Better data collection has enabled Azerenerji to bring its accounts in line with International Financial Reporting System requirements. Audits of these accounts have been completed since 2006 and in 2008 Azerenerji received an unqualified (clean) audit. The improvements in its corporate and financial management have enabled Azerenerji to transform itself, in 2010, from a Soviet-style State Enterprise, to a more transparent Corporation. Azerenerji has separated billing from collections, created a strategy department and launched, in 2011, a commercial restructuring and financing plan, which “is expected to positively affect Azerenerji’s finances starting 2015-2017” (ICR, page 10).
As a result of the above-mentioned reforms, Azerenerji’s financial situation is considerably stronger than prior to the project. Specifically: After suffering operating losses in most years prior to the project (losses were US$84 million equivalent in 2003), there has been a persistent operating surplus in every year since 2007. This is due to the following actions:
  • Tariffs were adjusted so as to reach cost recovery. In January 2007, the weighted average retail tariff was tripled to AZN0.06 (about 7.5 US cents) per kWh. Although regulatory reform does not appear to be on the agenda, the Borrower, in its comments on the ICR, stated that tariffs are legally set to at least cover production and transmission (though not collection) costs, and in fact have been on a “cost plus” basis since 2007 (ICR, page 51). According to the project team, “the tariff level remains adequate to cover operating cost at full collections.”
  • Collections have improved for all classes of customer from a low baseline of 36% in 2003. Collections from households went up from 37% of the amount billed in 2006 to 85% in the first nine months of 2011, due mainly to the installation of 300,000 “smart” meters and the adoption of a pre-payment regime among metered clients. At the wholesale level, collections reached 80% in 2010, up from 50% in 2004.
  • There have been improvements in operating efficiency through (a) the gradual removal of high-cost fuel oil from the generation fuel mix (fuel oil was 70% of the mix in 2000 and by 2010 had been replaced entirely by less costly natural gas); (b) improved fuel efficiency in generating units as a result of the rehabilitation of older plants (between 1998 and 2011, fuel consumption fell by 25% from 400 grams per kWh generated to just over 300 grams); and (c) lower transmission costs – technical losses fell from 4.05% in 2008 to 3.6% in 2011, and can be expected to fall further once the SCADA system is fully operational.
  • Direct Government subsidies to Aerenerji, which had reached US$400 million in 2004, have been eliminated. Although the utility is indirectly subsidized through purchasing natural gas at about half the international market price, the price it pays is still in excess of the gas production price, so that the indirect subsidy does not constitute, for the present, a financial burden to the Government (which owns the gas production and distribution facilities). The Government’s position is that this indirect subsidy is an appropriate alternative to raising tariffs.

Reflecting the utility’s strengthened financial situation and its careful attention to auditor’s recommendations, Azerenerji received, in May 2012, a Long-term foreign currency Issuer Default Rating (IDR) of BBB and a Short-term foreign currency IDR of “F3, with a positive outlook on the long term IDR,” from Fitch Ratings.

Although Azerenerji’s financial condition has been strengthened, a number of challenges remain to be overcome before full financial sustainability would be attained:
  • There is no mechanism in place to permit regular and consistent adjustment of tariffs. Such adjustments are infrequent. Prior to 2007, there had been no adjustment since the 1990s, and tariffs are not expected to be increased again until 2015. With annual inflation averaging 8 percent, Azerenerji is once more losing ground.
  • Azerenerji has not yet been able to resolve issues surrounding full payment for electricity provided to large wholesale customers, including Government-owned industries, and to the Baku regional distribution company. According to the 2010 Auditor’s Letter, the largest wholesale customer systematically underpays about 18% of its monthly bills. Azerenerji classifies most of this debt as impaired losses (special non-recurring charges taken to write down an asset with an overstated book value), which are periodically written off. In 2010, impaired losses were over 10% of revenue, though they fell back to 2.3% in 2011.
  • After strongly improving in 2007 and 2008, Azerenerji’s operating margin (operating income divided by net sales) declined from 45% in 2008 to 23% in 2009 and to only 2% in 2010, before recovering to 11% in 2011. Although this was partly due to lower capacity utilization in the face of sluggish domestic demand, the main cause was large impairment losses.
  • Other causes for concern are Azerenerji’s rising debt ratio and lack of financial autonomy. The former is due to the fact that Azerenerji is in the process of embarking on a large investment program for grid modernization with an estimated cost of €3.1 billion (about US$4 billion equivalent). Its internal cash generation is far from being sufficient to meet the investment needs. Embarking on this investment program has meant that Azerenerji was unable to comply with financial covenants under the project with regard to debt service ratios (ICR, page 33). Regarding lack of financial autonomy, the Government sets gas prices and electricity tariffs, and decides on state employees’ compensation. This means that some 97% of Azerenerji’s revenues and 30% of its costs are directly controlled by the state (ICR, page 34).

5. Efficiency:

The physical investments were completed at slightly below the appraisal estimates. Delays during implementation were relatively minor, considering that this was a complex undertaking with a new Borrower, and were addressed in an expeditious manner. Delays in the starting of the SCADA procurement required that the project completion date be extended once (by one year), to complete the installation of the system. Although physically installed, the system still required further testing and fine tuning SCADA is expected to become fully operational in 2012. The ICR estimated the ex-post economic rate of return (ERR) for the SCADA system to be 22.1% and for the transmission rehabilitation 41%, compared to 36% and 48% respectively at Appraisal. These high returns for the transmission system are consistent with expected economic value or "willingness to pay" for avoiding blackouts, which is substantially higher than the average willingness to pay for electricity. Efficiency is rated Substantial.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:

Rate Available?
Point Value
ICR estimate:

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project fully achieved its first objective of improving the efficiency of the power transmission operation through investments in transmission rehabilitation and the SCADA/EMS, while at the same time building the Borrower’s technical and institutional capacity. Efficacy of this objective is rated high. Efficacy of the second objective -- strengthened financial condition -- is rated substantial. Improved financial management and results were obtained during the life of the project, and Azerenerji's financial condition was stronger at project closure as a result, although important challenges remain to be addressed before full financial viability would be attained. Relevance of objectives is rated high, and that of design substantial. With efficiency rated substantial, overall outcome is assessed as satisfactory.

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:

Although the implementation of the SCADA/EMS investment components was technically complex, these components were successfully implemented, are currently in operation, and are providing significant operational benefits. The technical risks are assessed as low and manageable. The technology and operation of the upgraded transmission lines and transformer stations is well-established and fully understood by Azerenerji.
Primary training for the SCADA system, which is a complex, integrated system for information retrieval and analysis, has been completed. However, when the system was tested, Azerenerji’s management realized that optimal utilization of its many analytical capabilities required additional training. This additional training program is currently being implemented at Azerenerji’s expense, which increases the probability that the system will be well utilized. However, because the outcome of the training is not yet known, the technical risk with regard to the SCADA system is assessed as moderate.

Although Azerenerji's financial position has improved since project appraisal, the risks to financial stability remain high. Azerenerji has not yet been able to resolve issues surrounding full payment for electricity provided to large Government owned industries and to the Baku regional distribution company. In addition, there is no mechanism for regular tariff reviews that would minimize the effects of inflation before the next tariff increase scheduled for 2015. The utility's operating margin, after improving markedly in 2007 and 2008, began to decline again in 2009 and fell to 2% in 2010 as impaired losses rose to 10% of revenues. There was some improvement in 2011, when lower impaired losses allowed the operating margin to rise to 11%, but this is insufficient to demonstrate definitively that the deteriorating trend that began in 2009 is at an end. Azerenerji is, moreover, embarking on a major investment program which is far beyond its internal cash generation capacity; its debt service obligations will, therefore, likely increase.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:
The project's objectives and its individual components were well defined and relevant to the country’s and the Borrower’s needs. Economic analysis was thorough. Technical design was well advanced. The implementation program was well designed, including monitoring and evaluation arrangements, which were clearly quantified, and mechanisms for collecting data put in place. However, the staffing plan for the Project Implementation Unit (PIU) was not fully developed, which slowed procurement and created problems with contract management. Technical risks were appropriately assessed. Revenue collection issues were meant to be covered by an agreement to separate and privatize Azerenerji’s distribution operations, but the financial risks associated with the introduction of foreign companies to take over distribution responsibilities were not fully identified or mitigated. An agreement was reached on eliminating direct subsidies and on increasing tariffs sufficiently to bring the Borrower into compliance with financial covenants. However, agreement was not reached on the establishment of an institutional mechanism to maintain tariffs at an adequate level over time, or on how the Government was going to address the issue of accounts receivable. Moreover, the impact of the large grid modernization investment program on debt ratios was not taken sufficiently into account. An environmental assessment was carried out in compliance with safeguards policies.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:
Because this was a new Borrower, Bank supervision missions were undertaken, on average, three times per year. There were early issues of slow disbursements, non-compliance with financial reporting requirements, and inadequate staffing of the Implementation Unit. The Bank supervision team responded proactively to these issues by bringing forward the mid-term review by three months to April 2008, during which agreement was reached with the Borrower on a 19-point action plan to resolve outstanding problems. Most of these actions were undertaken within six months, and the implementation rating in supervision reports became satisfactory. The pressing issue of improved SCADA operator training was identified in time to reach agreement with the supplier on an extended training program. However, some important issues, including the establishment of a system for regular tariff adjustments and the under-payment of billings by major wholesale customers remained unresolved at project closure.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Government demonstrated ownership of, and commitment to, the project, and was fully supportive during project implementation. It provided direct support to Azerenerji as agreed during project preparation and it took steps to improve the sector's financial viability by its one time 50% tariff increase. It has not, however, established a mechanism for regular tariff adjustments. When it was established that the foreign concession distribution companies were not respecting their contractual obligations for new investment and payments for energy received, the Government acted proactively by cancelling these agreements and returning the concessions to Azerenerji. The Government has required Azerenerji to launch a large rehabilitation and construction investment plan for new power plants and transmission and distribution facilities, which meant that Azerenerji was unable to comply with financial covenants regarding debt service ratios.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:
Azerenerji was fully committed to achieving the project’s Development Objectives. Although there was some delay in establishing the appropriate staffing, when these staffing shortcomings were resolved, Azerenerji’s Project Implementation Unit provided quality technical management for project implementation, performed its procurement and financial management functions in a satisfactory manner. and worked well with Bank staff in resolving issues as they arose. However, Azerenerji has not complied with all the project’s financial covenants. While Azerenerji has made adequate arrangements for the regular operation of the project’s physical investments, the ICR (page 15) notes that there is no evidence that it intends to continue using the established project indicators to monitor system performance

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
There were clear links established in the PAD between objectives and component output and outcome indicators. The M&E system was focused on issues related to physical implementation of the project and on the project's measurable outputs related to efficiency, including the fuel consumed per kWh produced, the number of outages per months and their duration, and the quality of electricity measured by frequency and voltage level variations. They are all quantitatively measurable and relatively easy to collect. Baseline values for all indicators were available. Azerenerji was to be responsible for managing and operating the M&E system.

b. M&E Implementation:
The PAD enumerated indicators were, and continue to be, collected by Azerenerji as part of its normal operations. The data is collected initially through computer inputs, which increases reliability.However, the baseline data is not always comparable with recent information because of the re-absorption of previously independent distribution entities into Azerenerji. Data was effectively and efficiently transferred from Azerenerji to the Project Implementation Unit and then provided to the Bank. These indicators have provided direct evidence for evaluating project results. The data will in the future be collected through the SCADA system.

a. M&E Utilization:
The SCADA system as a whole is designed to be a M&E device, constantly evaluating and adjusting system performance, so it seems likely that Azerenerji management will continue to use it.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:
The Project was classified as category “B” for Environmental Assessment purposes since the physical investments are confined to existing facilities and existing rights of way. According to the PAD (page 16), the only safeguards policy to be triggered was Environmental Assessment (O.P. 4.01). The ICR (page 9) reports that an Environmental Management Plan (EMP) was prepared as required under O.P. 4.01. There were no polychlorinated biphenyls (PBCs) in the transformer oil used in Azerbaijan. There were no subprojects in disputed areas or conflict zones. The ICR reports (page 9) that satisfactory consultations about, and public disclosure of, the EMP took place. Although the ICR does not state specifically that OP 4.01 was complied with, it reports (page 9) that implementation of the EMP was closely followed by Azerenerji, and that the Bank supervision team received regular reports on issues such as orderly removal of old equipment, transformer oils and scrap metal, and the status of measurements of electric and magnetic fields.

b. Fiduciary Compliance:
Financial Management
Financial audits have been implemented since 2006. The first of these audits was submitted with a significant delay as a result of poor delivery performance by the Auditor. In 2008, Azerenerji received an unqualified (clean) audit, though those of 2009 and 2010 were qualified. According to the ICR, the auditors’ recommendations have been addressed. Significant progress was observed over the life of the project regarding Azerenerji’s financial management . Azerenerji transitioned to International Financial Reporting Standards (IFRS) in 2010.
A financial management reporting process was established at the start of implementation, and a project accounting system was also put in place. Quarterly reports were consistently submitted in a timely manner and were acceptable to the Bank. According to the ICR (page 9), financial management arrangements were fully compliant with Bank standards.
The ICR reports (page 9) that all procurement was carried out in a manner consistent with Bank guidelines. Although the technical complexity of the SCADA/ EMS component led to delays in preparation of the technical specifications, and the procurement process therefore took longer than originally anticipated, by project closure all contracts for equipment had been completed satisfactorily. There were no reported cases of misprocurement.

c. Unintended Impacts (positive or negative):

d. Other:

12. Ratings:

IEG Review
Reason for Disagreement/Comments
Risk to Development Outcome:
Azerenerji’s future financial stability is compromised by its inability to achieve full payment due from major wholesale customers. In addition, there is no regular mechanism in place for tariff adjustments, and no adjustments are foreseen until 2015. There is absence of definitive evidence that the deteriorating trend in operating margins since 2009 is at an end. The utility's major ongoing investment program, combined with limited internal cash generation, will likely lead to rising debt service obligations. 
Bank Performance:
Moderately Satisfactory
There were moderate shortcomings in Quality at Entry. The staffing plan for the Project Implementation Unit (PIU) was not fully developed, which slowed procurement and created problems with contract management. The financial risks associated with the introduction of foreign companies to take over distribution responsibilities were not fully identified or mitigated. Agreement was not reached on the establishment of an institutional mechanisms to maintain tariffs at an adequate level over time, or on how the Government was going to address the issue of accounts receivable.  
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
Quality of ICR:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following lessons are taken from the ICR with some adaptation of language:
  • Government commitment to institutional and reform is essential for the long term financial viability of the power sector. Most importantly, there needs to be a strategy and an institutional and legal framework for adjusting tariffs on a regular basis as costs increase. Without this commitment, energy sector institutions will continue to have financial difficulties and will be unable to restructure effectively. (i.e. separate generation from distribution and begin the privatization process.) In this case, the lack of a continuing policy of tariff adjustment threatens to undermine financial performance enhancements supported by the project.
  • It is important to make arrangement for an adequately staffed project Implementation Unit before Board Approval, and to ensure that the Unit is fully staffed before project Effectiveness. This is particularly important in situations where there is a first time implementing agency, since they are unlikely to be fully cognoscente of the magnitude of effort involved in meeting Bank procurement and reporting requirements.

14. Assessment Recommended?

To verify the ratings and draw lessons, including on sustaining the financial health of power companies.

15. Comments on Quality of ICR:

The ICR focused on outcomes, covered all the issues in a manner consistent with the guidelines, provided appropriate evidence for judging project performance, and was internally consistent. It was written in a concise and readable manner. It reviewed and updated all the project’s economic assumptions and implemented rigorous economic analysis. However, the Lessons learned section was diluted by the inclusion of several general observations that apply to all projects, such as the need for strong Government and implementing agency commitment and close Bank supervision. The Disbursement Profile is missing from the Data Sheet.

a. Quality of ICR Rating: Satisfactory

© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions