Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Lb-emergency Power Sector Reform Capacity Reinforcement


  
1. Project Data:   
ICR Review Date Posted:
06/27/2014   
Country:
Lebanon
PROJ ID:
P104774
Appraisal
Actual
Project Name:
Lb-emergency Power Sector Reform Capacity Reinforcement
Project Costs(US $M)
 8.00  6.12
L/C Number:
Loan/Credit (US $M)
 5.00  5.00
Sector Board:
Energy and Mining
Cofinancing (US $M)
 0.97  0.97
Cofinanciers:
French Development Agency (AFD)
Board Approval Date
  03/19/2007
 
 
Closing Date
06/30/2009 12/31/2012
Sector(s):
Central government administration (50%), Power (50%)
Theme(s):
Other public sector governance (50% - P) Regulation and competition policy (25% - S) Infrastructure services for private sector development (25% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Richard L. Berney
Ridley Nelson Christopher David Nelson IEGPS1

2. Project Objectives and Components:

a. Objectives:
The development objectives of the project in the Grant Agreement were “to accelerate the implementation of [energy sector] reforms and [the] restructuring of Electricite du Leban (EdL) - by enhancing the capacity o f MOEW, EdL and the multi-ministry higher level committee ” (Page 6 ) The appraisal document used identical wording. (Document T7692, page 6).

This Review considers the enhancement of the capacity of Ministry of Energy and Water (MOEW), EdL, and the Multi-Ministry Higher level Committee (MMHLC) as the ways through which the objectives are to be implemented treating them as output targets.

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 12/31/2010

c. Components:
1. Provision of support to strengthen the capacity of the MOEW to implement power sector policy reform:(Actual U$1.2 Million (at appraisal the component costs were not differentiated)) The technical assistance under this component included policy advice on key sector strategic issues, on the completion of on-going projects, and on the establishment of a regulatory agency. During the restructuring of December 2010 a new study, “The LNG solution for Lebanon”, was added.
2. Provision of support to the Government of Lebanon for the restructuring of EdL. (Actual U$1.85 million) The technical assistance under this component, envisaged advising the Government on the restructuring strategy of EdL, and proposing appropriate capitalization levels for each of the separate, independent entities that the restructuring would establish .
3. Provision of support to EdL to improve its operational and financial performance and to support the implementation of its restructuring. (Actual U$2.19 million) The service contract under this component was intended to address ways to improve generation efficiency including system reliability and efficiency, as well as commercial, financial and accounting issues.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost, Financing and Borrower Contribution: The estimated project cost at appraisal was US$8.0 million, financed with US$5.0 million from a Trust Fund Grant and US$3.0 million from the Government of Lebanon. The final project cost was US$6.12 million, which included US$5.0 million of the Trust Fund Grant, US$0.97 million grant from the French Development Agency, ADF, and US$0.15 million from the Government.

Dates: In June 2009 the project closing date was extended by 6 months to December 31 and then again for an additional year to end December, 2010, The rationale for these extensions was to provide additional time to ensure the adequate completion of the studies. The project was restructured in December 2010 and extended for another two year (to Dec 31, 2012).


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
Rating High :The project’s objectives were and continue to be highly relevant to Lebanon’s development strategy. The Government’s reform program was presented in the 2007 Paris III Conference. It put considerable emphasis on power sector reform as part of the country’s sustainable reconstruction and economic recovery program. The Bank’s 2011-2014 Country Partnership Strategy includes electricity as an explicit strategic infrastructure priority.

b. Relevance of Design:
The initial project component design provided comprehensive technical assistance to MOEW in the implementation of sector reforms (including the restructuring EdL), and to EdL to assist it to improve its operational and technical performance. Both of these project elements were highly relevant to the project’s stated objectives. The initial development targets were logically linked to these targets. However, the emergency project instrument to tackle issues unresolved for many years was unsuitable and the ambition excessive. The Relevance of Design rating for the initial project is Modest
The Dec 2010 restructuring added a new project component: a study of a proposal for the importation of LNG. This was an important study for the power sector as a whole, as it was expected to be a critical element in improving the sector’s efficiency by reducing the cost of electricity generation. The rating for the Relevance of Design of the restructured project at a more modest scale is Substantial.


4. Achievement of Objectives (Efficacy) :

The original pre-restructured project

Objective 1. Accelerating the implementation of [energy sector] reforms: Rating Modest

Output: The technical assistance program produced a wide ranging report for MOEW, with recommendations for strengthening its capacity to implement power sector reforms. These included (i) a proposed power sector strategy; (ii) a fuel sourcing strategy; (iii) a proposal for establishing an energy sector regulator, (iv) tools and procedures for a sector regulator; and (v) a series of proposals designed to encourage Independent Power Producers. These studies provided background material for the Government’s Policy Paper for the Energy Sector Policy Paper (ESPP) which emphasized the energy sector reforms that were needed for the country to develop a cost-effective electric power system. The ESPP was approved by the cabinet in June 2010.

Outcomes:
Target 1: The preparation and beginning of implementation of a National Energy Strategy integrating a power sector least cost plan and a tariff study, Only Partially Achieved - The Government’s Energy Sector Policy Paper substituted for a national energy strategy. There is no least cost sector investment plan. A tariff study could not be completed in the absence of a sector restructuring plan.
Target 2: Development of a fuel sourcing strategy. Achieved.: A fuel source strategy has been established and used to determine fuel mix of next generation of projects.
Target 3: Evaluate Proposed IPPs and, if viable, contracting them as a basic element in a power sector strategy: Not Achieved – No IPPs have been proposed.

Development Objective 2: Accelerating the implementation of the restructuring of EdL: The original pre-restructured project: Rating Modest

Output: Support for restructuring EdL through corporatization was to be implemented in two phases. The first phase was to provide policy guidelines for a process of corporatizing EdL. The second phase was to provide assistance for implementing these recommendations. Only the first phase was completed. Recommendations were made for a electricity sector market structure, including the roles of the public and private sector in the context of attracting private investment into generation and distribution, the critical elements of which were (i) the unbundling of EdL and corporatizing each sector; (ii) establishing a competitive structure for potential independent power producers; and (iii) establishing a power purchasing entity independent from EdL generation. The contract for the second phase was cancelled, due to divergent views on the phase one report’s recommendations.

Outcome:
Target 1: That the Government would agree on a process for restructuring EdL and the Cabinet would approve the decrees necessary to implement the process. Not Achieved
Target 2: Asset register completed and level of capitalization and shareholding structure proposed for restructured companies. Expected to be at least Partially Achieved -


Objective 3: Enhancing the capacity of Electricity de Lebanon (EdL) Rating Modest

Output: Technical assistance was provided in a wide range of operational and management areas with the objective of improving EdL’s operational and financial performance.

Outcomes: The original outcome targets for this objective were mostly not achieved. They were:
Target 1: EdL is better managed with a more skilled staff base: Not Achieved - EdL has been unable to fill key financial, procurement, technical and commercial positions.
Target 2: EdL’s productivity gains make it more efficient; Not Achieved – there have been no reported productivity gains.
Target 3: The reliability of electricity supply is improved: Not Achieved – no new investment has been made to improve reliability of supply.
Target 4: Cost reduction projects are implemented. Expected to be Achieved - Efficiency improvement plans have been developed for EdL’s power plants. These improvement plans are being implemented for two plants by their O&M contractors, and are planned for EdL’s other two other plants as part of rehabilitation work currently being tendered.
Target 5: The financial gap in distribution is reduced:. Likely to be Achieved – Private sector management contracts for distribution service providers' that are currently being implemented are expected to reduce the financial gap in distribution. However, it is too early to have quantitative evidence supporting this.

The Restructured Project:

The Government cancelled the contract for the second phase of the EdL study in early 2011. The US$1.8 million that had been allocated for this study was reallocated to support: (i) a revised roadmap for EdL corporatization; (ii) a study on the viability of using imported liquefied natural gas for power generation. The Development Objectives were not changed, but the key performance target indicators were.

All three of the outcome indicator targets were dropped and new targets for energy sector reforms were approved during the restructuring.

In parallel with the objectives prior to restructuring, the following two objectives were the remaining focus of the reduced post-restructuring project (objectives numbered as for the pre-restructuring period.)

Objective 1: Accelerating the implementation of energy sector reforms: Rating Modest

Output: A study implemented as part of the restructured project evaluates the technical details of a LNG importation project.

Outcome:
Target 1: Establishment of a fuel source strategy: Achieved - A decision has been taken to proceed with a new LNG importing facility, which is to use rented offshore floating terminal facilities. This project is in the process of being implemented, and procurement has started.
Target 2: Bidding documents to be prepared for new IPPs. Not Achieved- This target replaced the original target of evaluating specific IPPs. Not Achieved: MOEW is waiting for a firm commitment from Kuwaiti fund to finance consultants to complete a master plan and a location study for IPPs, and no potential private sector power producers have been identified.

None of the other original targets related to the implementation of energy sector reforms were achieved in the restructured project.

Objective 2; Accelerating the implementation of the restructuring of EdL: Rating Modest

Output: A second study of the restructuring process initiated in 2010. It produced a roadmap for EdL’s corporatization and restructuring that differed in timing and sequencing from the recommendations of the first study.

Outcome:
Target 1 :Not Achieved. A roadmap for Ed restructuring would be finalized and agreed with the Government.. This replaced the original indicator target that EdL would be restructured and better managed, with a more skilled staff, and the intermediate target that the Cabinet would approve the decrees necessary to implement the restructuring process was also dropped. However, the new, less demanding target was also not achieved. The relevant Government Agencies still have substantial disagreements on how the process should be implemented, some of them supporting the proposal in the first study and some the proposals in the second study. As a result, the Government has not yet approved a way forward. Some progress has, however been made, in that there are two feasible alternative from which to choose, whereas before the project there were none.

The intermediate output target that EdL's assets be registered- Partially Achieved - This process is an essential element of the corporatizing process and is expected to be achieved for the assets involved in electricity distribution, but not for those involved in generation or transmission. Following the approval of the 2010 PPES, electricity distribution activities (distribution to the final consumers) has been allocated to private distribution companies. These companies have been tasked with developing their own asset registers.


5. Efficiency:

This was a technical assistance project, so no explicit economic or financial analysis was undertaken at the time of appraisal. Potential long term economic impacts from the financed studies included, inter alia: reduction of sector investment needs; lower long term operating costs; and reduced need for individual small scale, inefficient diesel power generators. Benefits that could potentially be associated, at least indirectly, with the outcomes of studies implemented under the project, include savings from rehabilitating two power plants evaluated under the project, estimated by the ICR to be about US$30 million per year, and savings from establishing a LNG import facility to enable the switch from fuel oil to imported LNG, which the ICR estimates has a NPV of US$200 million. However, neither of these two outcomes has been achieved under the project.


The project financed two studies for developing a proposal for restructuring EdL. While there may have been some overlap, the studies approached the restructuring challenge from different perspectives, and recommended different avenues for EdL’s transition towards corporatization. The choices provided by the two studies are important elements in the subsequent discussion of the optimum restructuring path. However, none of the recommendations from either study have been chosen for implementation


This Emergency Project was designed to be implemented in two years, but took four and a half years to be completed. While some of the objectives were met, important and fundamental issues evaluated in project financed studies, such as the restructuring of EdL have still not been resolved. Efficiency is therefore judged to be Modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project design involved a series of studies over a two year period, studies that were expected to lead the government making important decision on energy policies that had been politically controversial for many years. This type of project design would have been more consistent with a standard technical assistance project than with an emergency technical assistance project. Its relevance for an emergency assistance project was negligible although other design aspects leave Relevance of Design, on balance, rated Modest.

The project was restructured after three years, during which, with approximately two thirds of the project's fund spent, the Government had been unable to implement any specific actions in follow-up on the recommendations of the completed studies. The rationale for the restructuring was the desire to utilize funds that had been allocated for the second stage of a study on corporatization of EdL. The Government had been unable to move forward on this second stage study because of disagreements on its terms of reference. The freed up funds were to be used for a new study on corporatizing EdL and for an evaluation of an imported LNG project proposal.

The achievement of the targets related to the project's objectives were modest for both the original project and for the restructured project, in the latter case partly because the objectives themselves were unaltered. The project made some contribution to enhancing the capacity of the Ministry of Energy to implement a comprehensive energy sector plan but its contribution to the objective of improving the performance of EdL was very limited, both in terms of hiring new senior staff to improve its internal efficiency and corporatization to improve the efficiency of the sector as a whole. The efficacy of the restructured project, which accounted for only $1.4 million of the $5.0 million Credit, was modest; the study on importing LNG has confirmed that this option should be pursued, but the other study on the corporatization of EdL failed to lead to a Government agreement on a corporatization plan. Efficiency is also rated as modest. Weighting before and after restructuring, when key indicators were changed, results in no significant rating differences so outcome is rated as unsatisfactory.

a. Outcome Rating: Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

A political consensus on the proposed sector reforms and roadmap for restructuring of EdL will be hard to reach. The likelihood that the Council of Ministers may not be able to reach decisions on these issues any time soon is significant. Sectarian interests and political instability could also be a challenge to the implementation of any EdL restructuring plan. Without an early consensus, the tools provided by this project for EdL’s restructuring would quickly become outdated. The lack of the necessary sector reforms (e.g. tariff restructuring) would have a significant negative impact on the sector, as it would strongly undermine the sustainability of the planned investments, of the introduction of IPPs for generation, and of the overall sector efficiency. Given the inherently political nature of these factors, and the subsequent uncertainties created by the fighting in Syria, the risk to development outcome is deemed to be high.

a. Risk to Development Outcome Rating: High

8. Assessment of Bank Performance:

a. Quality at entry:
The project rationale was clear, its goals were in line with the Government’s priorities as defined in the framework of the Paris III conference, indicating that there was a significant commitment to the project’s Development Objectives from the relevant Government branches. This technical assistance was aimed at providing support for the reforms of a Development Policy Loan, which was also under appraisal and was approved a few months later. However the project design had several problems.

  • The choice of instrument was unsuited. There was insufficient logic in classifying a technical assistance project as an emergency operation when it involved studies of issues that had been unresolved for many years.
  • The appraisal overestimated the real level of political consensus around the 2002 Electricity Law and the restructuring EdL, and the likelihood that the parallel reform implementation envisaged in the Development Policy Loan would achieve that consensus.
  • The choice of project development indicators suggests that the project’s potential outcomes (impacts) were significantly oversold, including the speed at which at which IPPs could be evaluated and contracted, as well as the speed that EdL could be restructured, could become more efficient, and could implement new cost reduction projects.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:
Bank supervision was competently handled during the early implementation with the contracting of the initial technical assistance project. However the follow-up supervision was weak. The phase one consultant report on EdL restructuring was delivered in 2008, but phase two was blocked by disagreements within the Government on the appropriateness of its recommendations on how to proceed with the commercialization process. The supervision team left unresolved the issue of how to move forward for two years, until shortly before the project was to close. If the project had been closed, the unused $1.4 million allocated for the second phase study would have been cancelled. Agreement on the restructuring of the project was achieved shortly before the closing date. The Government agreed to cancel the controversial phase two study and a new study, with new terms of reference, was approved. As explained in Section 6, above, the project’s objectives were left unchanged but most of the development targets were dropped or modified, solely on the basis that the original targets were unattainable. Project implementation proceeded smoothly after task team leadership was relocated to Lebanon in 2010 to facilitate implementation of the remaining studies, but even with the Bank’s considerable efforts the problem of opposing concepts of how to restructure of EdL remained unresolved.

Quality of Supervision Rating: Moderately Unsatisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:
The Council for Development and Reconstruction (CDR) is identified in the Grant Agreement as the implementing agency. The other agencies, MOEW, HCP, and EdL are considered in the ICR to be part of the Government, even though they were responsible for designing and implementing the various studies. There were difficulties in establishing consensus among these agencies, and in the case of EdL, in establishing internal agreement within EdL on various aspects of EdL related studies. Most importantly, the MMHLC, which was supposed to coordinate the project activities and reach agreement on how to implement the results of the various studies, never met. The ICR also indicates that the Government did not provide their planned US$5 million contribution.

Government Performance Rating: Unsatisfactory

b. Implementing Agency Performance:
The management of CDR fulfilled its fiduciary functions as specified in the Grant Agreement. In addition, when different branches of the Government had diverging opinions on how to proceed with the EdL restructuring study, it played a catalyzing role in unblocking overdue payments to consultants and in inducing consensus for the revised study.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Unsatisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:
The M&E design was overly ambitious. The PAD had 17 intermediate project development Indicators covering reinforcing the MOEW, restructuring of EdL and strengthening EdL. While some were related to other ongoing projects, rather than this project, and some could not be expected to be implemented within the two year horizon of the project, others were well designed as measures of project success. This included expectations for the establishment of a national energy strategy, and the creation of a Regulatory Agency,

b. M&E Implementation:
The indicators were revised during the restructuring in late 2010 to reflect new expectations for project outcomes. Many of the original indicators were dropped when it was decided that they were unattainable, and others more likely to be achieved were added.

a. M&E Utilization:
Rather than being used to support implementation of the project, the indicators were adjusted to follow likely outcomes.

M&E Quality Rating: Negligible

11. Other Issues:

a. Safeguards:
This was a category C technical assistance. No safeguard policies were triggered.

b. Fiduciary Compliance:
The project complied with the Bank’s financial management procedures and requirements. It submitted interim un-audited financial reports on time, and audit reports were submitted with unqualified auditor’s opinion acceptable to the Bank. All implementation and status reports (ISR) have consistently rated FM as satisfactory.

c. Unintended Impacts (positive or negative):
None

d. Other:
None



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Unsatisfactory
The outcome is rated unsatisfactory mainly because an agreed restructuring plan for EdL was not achieved. Relevance of Design pre-restructuring, Efficacy and Efficiency were rated as modest, Weighting before and after restructuring did not change the overall rating. 
Risk to Development Outcome:
Significant
High
There is still no political consensus on the proposed sector reforms and roadmap for restructuring of EdL. Nor is there any agreement on increasing tariffs to an economic level. 
Bank Performance:
Moderately Satisfactory
Moderately Unsatisfactory
Quality at entry is moderately unsatisfactory because the classification as an emergency project was inappropriate for studies on issues that had been constraints for many years. Supervision was rated as moderately unsatisfactory because it left the issue of the phase two EdL restructuring activity unresolved for two years.  
Borrower Performance:
Moderately Satisfactory
Moderately Unsatisfactory
The Government's performance is rated as Unsatisfactory because its Multi-Ministry Higher Level Committee did not fulfill its agreed upon role. It was supposed to coordinate the project activities and reach agreement on how to implement the results of the various studies, but it never met.  
Quality of ICR:
 
Unsatisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:
The following lessons have been taken with some adjustment in language from the ICR:
  • A pragmatic and focused project development objective that is consistent with the level of resources involved is essential to provide a realistic assessment of a project’s likely impact. In this case, the expectation that this small financial grant would be adequate to bring about significant restructuring of EdL was unrealistic and diluted the effectiveness of resources spent.
  • Transparency in the objectives of the energy sector reform process, and an ex-ante buy-in from major stakeholders is essential for the success of an energy sector restructuring. In this project, there was a significant divergence between EdL’s and MOEW’s vision of how to proceed. The first EdL restructuring study was led almost exclusively by the HCP, and the second EdL restructuring study was led almost exclusively by the MOEW. Neither agency appear to have had established a significant buy-in from other important agencies and players.
  • It is essential to understand who the winners and losers are in any significant power sector reform, so as to create a pro-reform coalition that can help foster the process. Otherwise, narrow interests are likely to overshadow national level interests.

  • 14. Assessment Recommended?

    No

    15. Comments on Quality of ICR:

    The ICR was satisfactory in explaining the problems that the project faced and in detailing the changes made during restructuring. However, while it discusses project outputs, it provides little analysis of development outcomes, which are substantially less impressive when measured against the project’s unchanged Development Objectives. The ICR argues that problems with project design and formulation of the intermediate development objectives were a natural result of the time pressure of an emergency recovery intervention. However, it fails to follow this up and explore the logic behind classifying a technical assistance project as an emergency operation when it involved studies of politically sensitive issues that have been unresolved for many years. It fails to discuss the fact that the only rationale for dropping many of the outcome targets was that they could not be attained. An explanation of why the Government failed to provide the US$5 million indicated in the PAD would also have been useful.

    a. Quality of ICR Rating: Unsatisfactory

    (ICRR-Rev6INV-Jun-2011)
    © 2012 The World Bank Group, All Rights Reserved. Terms and Conditions