Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Rw-compet & Enterprise Dev (fy01)


  
1. Project Data:   
ICR Review Date Posted:
01/14/2013   
Country:
Rwanda
PROJ ID:
P057295
Appraisal
Actual
Project Name:
Rw-compet & Enterprise Dev (fy01)
Project Costs(US $M)
 41.14  46.25
L/C Number:
C3499, CH363
Loan/Credit (US $M)
 41.14  46.25
Sector Board:
Cofinancing (US $M)
 0.32  1.0
Cofinanciers:
Board Approval Date
  04/19/2001
 
 
Closing Date
07/31/2007 07/31/2011
Sector(s):
General finance sector (39%), General public administration sector (25%), Power (22%), General industry and trade sector (9%), General information and communications sector (5%)
Theme(s):
Other financial and private sector development (29% - P) Regulation and competition policy (29% - P) State-owned enterprise restructuring and privatization (14% - S) Micro Small and Medium Enterprise support (14% - S) Legal institutions for a market economy (14% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Xiaolun Sun
Clay Wescott Navin Girishankar IEGPS2

2. Project Objectives and Components:

a. Objectives:


    The project development objective was to establish an enabling environment for growth and development of the private sector that would help reduce poverty in Rwanda. (Project Appraisal Document, p.2 and Financing Agreement, Schedule 2).

    While both the Project Appraisal Document (PAD) and the Financing Agreement (FA) are consistent in their presentation of the overall objective, the PAD is more specific about the sub-objectives. It notes that objective of "promoting a competitive climate" would be achieved by " (i) streamlining the business environment; (ii) reducing the costs and increasing the efficiency of telecommunications, water and electricity utilities, and the tea industry; and (iii) improving access to financial services and provide support services to local entrepreneurs."

    For the purposes of this review, IEG will use the PAD's definition of objectives and sub-objectives. They are more precise about the linkages between outputs and the intended outcomes.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

According to the PAD, the project's four components are as follows:

    1. Improving the business and legal environment (US$21.79 million at appraisal, US$27.16 at completion, including additional funding from restructuring): Aimed at streamlining the business environment to facilitate entry or growth of private businesses, this component had four sub-components that focused on (i) justice system and legal framework (US$2.89 million) by supporting the establishment of a Commercial Court, building the capacity of the Arbitration Center, improving the commercial legislative and regulatory framework, and support to a Documentation Center to the Ministry of Justice (MINJUST); (ii) investment promotion and trade facilitation (US$3.71 million) by strengthening the capacity of the newly established Investment Promotion Center; (iii) support to emerging businesses (US$8.89 million) by providing TA to the newly established Centre d'Appui aux PME du Rwanda, establishing an enterprise start-up fund, and providing TA to competitiveness improvement; and (iv) financial services (US$6.89 million) by addressing certain issues in the commercial banking sector and by strengthening the micro-finance sector.

    2. Reducing costs and improving efficiency (US$13.57 million at appraisal, US$11.07 at completion): This component would provide TA support to instill efficiency and cost effectiveness in the provision of water, electricity and telecommunication services. The three sub-components included (i) privatization of RWANDATEL, capacity building for the Ministry of Public Works, Transport and Communications, and technical assistance to strengthen corporate management in postal services (US$2.52 million); (ii) financial support for restructuring ELECTROGAZ (US$9.03 million); and (iii) support for establishing a multi-sector regulatory agency and capacity building for the Privatization Secretariat (US$2.70 million).

    3, Promoting a market-based tea industry (US$2.25 million at appraisal, US$0.82 million at completion): This component aimed to build a market-based tea industry through the introduction of private management and investment in tea estates and factories and participation of stakeholders in policy making and marketing through an independent Tea Board.

    4. Support to Private Sector Federation (US$1.48 million at appraisal, US$7.2 at completion): This component planned to assist the newly established Private Sector Federation with the HIV/AIDS information campaigns and support the Project Coordination Unit (PCU).

    DCA Schedule 2 presented almost identical project components, but listed Support to Private Sector Federation as a sub-component of Component #2, and envisioned capacity building of its member associations. Support to PCU was the only activity under Component #4.

    Financing Agreement of 07/24/2008, Schedule 1, stated that when the project was restructured for additional financing, five sub-components were added to the financial services sub-component of Component #1, including improving access to credit and financial services, improving accounting and auditing standards, enhancing non-bank financial institution regulation and supervision, strengthening the national pension fund, and modernizing payment systems. This strengthened the project's support for financial services, but did not change the PDO.

    The ICR (para. 19) stated that support to Rwanda Bureau of Standard, Kigali International Arbitration Centre, Masaka Business Incubation Centre and the leasing program were added to Component #1, while the HIV/AIDS campaign was dropped from Component #4 because it was financed by other donors. In its place, a host of other institutions such as Rwanda Housing Bank, National Tender Board, Kigali Institute of Science, Technology and Management, Ministry of Youth, Sports and Culture (MIJESPOC), Rwanda Office of Tourism and National Parks and National Insurance Commission, received support under this component, mainly in the form of equipment provision and staff capacity building. Support to establish a Commercial Court and to improve commercial legislative and regulatory framework, however, was not mentioned in the ICR.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Dates: The project closing date was extended twice: in 2007 for one year to allow for implementation of project activities not yet implemented; and in 2008 for three years due to additional financing to support the financial sector. (ICR, para. 20.)

Financing: At appraisal, the project cost was estimated to be US$41.14 million, of which US$40.83 million was financed by the Bank. The actual project cost were US$41.23 million, of which Borrower contributed US$1.00 million. In 2008, US$6.00 million additional financing was approved for a US$5.75 million restructuring of Component #1. US$5.02 million was the actual spent. Actual spending on Component #4 was substantially higher than initial plan ($7.2 vs $1.48 million) due to additional activities eventually included in this component. (ICR, p.35.)


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Relevance of objectives, which were unchanged over the project life, is rated high. The Rwanda CAS FY09-12 has two main objectives (i) promoting economic transformation and growth, and (ii) reducing social vulnerability. The CASPR of FY11 confirmed that the focus of the rest of the CAS period would be primarily on the first objective. (CASPR, p.4) Although the project was initiated at a time when the focus was to support Rwanda's reconstruction after the war, it remains relevant to the latest CAS and strongly supports CAS outcome 1.3: Improved environment for private sector development.

b. Relevance of Design:

Relevance of design is rated high. The four project components were all relevant to achieving the stated project development objectives, and each had its own intermediate outputs for its sub-set of activities. Lessons from similar projects were incorporated in the project design, which followed a reasonable causal chain linking inputs and outputs supported by the project to the objectives. In response to the changing country context and WBG's strategy in Rwanda, part of the project was restructured with additional financing to provide more support to the financial sector. However, there was no discussion of the specific exogenous factors or unintended effects that would affect the achievement of the objectives.


4. Achievement of Objectives (Efficacy) :

The overall objective was to be achieved through four sub-objectives (and associated project outputs). These are assessed below:

    1. Improving the business and legal environment.

    (i) Justice system and legal framework. Rwanda has done much to improve the legal environment for private sector development, and is recognized as a top reformer by the Doing Business Reports. The ICR presented clear indication that the TA provided through the project helped Rwanda achieve these successes. In addition, the ICR indicated that the project funded the Doing Business Reform Unit, which paved the way for the more focused and more comprehensive IFC Rwanda Investment Climate Reform Project that was launched in 2008.

    (ii) Investment promotion and trade facilitation. The project aimed to strengthen the capacity of the newly established Investment Promotion Center. The ICR, however, indicated that this did not happen. Instead, project funding was transferred to five Rwanda Embassies and Rwanda Investment and Export Promotion Agency Nodal Office in selected countries to promote investment. The Nodal Offices were not effective and the project stopped its support; there is no information on how effective the Rwanda Embassies were in promoting investment, or how much of their achievement could be attributed to this project.

    (iii) Support to emerging businesses. The project deviated from the original plan of financing two funds (Rwanda Small Loan Fund and Risk Capital Fund) destined to help SMEs start and grow, and contributed instead to the Rwanda Enterprise Investment Company (REIC) which was designed to operate like an equity fund for promising SMEs. The REIC was not a success and was restructured in 2011. The ICR indicated that the project also financed a Business Plan Competition to stimulate entrepreneurship development, and provided capacity building to a variety of institutions and small businesses, although the impact of these activities were difficult to assess.

    (iv) Financial services. Under the initial financing, the ICR reported that the project supported a number of institutions such as a Chart of Accountants for the banking sector, a Credit Reference for all sectors, the School of Finance and Banking, as well as funding for recapitalizing of the largest financial institution in Rwanda and the creation of a leasing program for SME access to finance. The scope of activities was expanded in 2008 with additional funding. The ICR provided evidence that most outcomes were achieved by the end of the project cycle. Rwanda's financial sector was stronger at closing of project than 10 years before when the project was launched, and this project directly contributed to this outcome.

    2. Reducing costs and improving efficiency. The ICR provided evidence that the project achieved the outcomes of facilitating the establishment of a fully functional multi-sector regulatory agency and stimulating competition and private sector investment in the telecommunications sector. The project strengthened the Privatization Secretariat and Rwanda's ability to implement an effective privation program. Although the privatization of Rwandatel, and the restructuring of Electrogaz, encountered multiple setbacks, the efficiency of utility provision has improved through various other measures. There is no information on the nature of Bank support to postal services, and it is difficult to tell the relative contribution of this project and other energy sector projects implemented at the same time.

    3, Promoting a market-based tea industry. The project provided specialized TA to the creation of the Tea Board and the privatization of seven out of nine tea estates and factories. These achievements should have contributed to the establishment of a more market-based tea industry, although the ICR did not comment on this. Their contribution to tea price increases and Rwanda tea exports, evidence provided by the ICR on the achievement of this component, is not clear.

    4. Support to Private Sector Federation. According to the ICR, the project funded training and study tours of Federation staff, business association leaders, entrepreneurs and trainers. The impact of such activities is difficult to assess and the ICR did not provide further evidence on the impact of such support. Other activities under this component were associated with those supporting emerging businesses of Component #1 above. There is no information on project support to the Project Coordination Unit.

    Although not monitored under the project and not possible to be attributed specifically to any project outputs, it is plausible that the improved business environment has contributed to the reduction of poverty in Rwanda by 12 percent between 2005/06 and 2010/11. Taking all of this evidence into account, IEG rates the efficacy of achievement of objective during this period as Substantial.

5. Efficiency:


Efficiency is rated Modest. NPV and ERR were calculated at appraisal and closure, but using different assumptions and methodologies as the original methodology was not recorded. The ICR did not present a cost-effectiveness analysis, evidence of implementation at least cost, or other indicators of "value for money". The nature of the economic return under this project - macro benefits to the economy and key sectors also makes attribution extremely difficult. Project disbursement closely followed the formally revised disbursement plan; the actual cost was 98 percent of appraisal, as pointed out by the ICR.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
Yes
24%
100%
ICR estimate:
Yes
31%
100%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The objective and design were highly relevant to Rwanda's country context and the WBG's assistance strategy throughout the project implementation period. Efficiency is rated modest, and efficacy as substantial. The four outputs are linked by a reasonable causal change leading to achievement of the objective. For the most part, the intended outputs were achieved even though many adjustments had to be made to project plan in response to changing country, donor assistance and other circumstances. The aggregate result is that Rwanda has made progress in improving its business environment for private sector development and is poised to continue to grow.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

Business environment reforms depend critically on political will and institutional capacity. Rwanda has displayed very strong political will and leadership in carrying out reforms and established a solid track record. The project devoted significant attention to building institutions, thus mitigating the capacity risks. Nevertheless, neither risk is negligible given the reforms that still lie ahead and the limited resources and capacity available.

a. Risk to Development Outcome Rating: Moderate

8. Assessment of Bank Performance:

a. Quality at entry:

The good fit between the project objectives and country conditions and CAS strategies, the relevance of the project design to project objectives, the stake holder analysis, and the analysis of risks and mitigation measures all indicate that the Bank did well in identifying, preparing and appraising this project. As the ICR pointed out, however, the project lacked an appropriate results framework and outcome indicators to measure results.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:

The project activities was monitored closely, with regular supervision missions and reports. A field-based TTL in the latter half of the project ensured good collaboration with the clients and timely resolution of issues. Project disbursement was on schedule. Adjustments to project activities, as well as closing dates, were made to account for new circumstances.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Government of Rwanda has had a solid track record in implementing reforms. Their achievement in improving business environment is reflected in the Doing Business ranking, by which Rwanda continued to be a top reformer in many areas. The Government's completion report provides a critical assessment of the project and very thoughtful discussion of lessons learned. This is another indication of its commitment to and close involvement in the project

Government Performance Rating: Highly Satisfactory

b. Implementing Agency Performance:

Initially, implementation performance was impeded by insufficient staffing, high turnover of staff and lack of transfer of knowledge to new recruits. For example, when the project started, the PIU had no staff with specialized procurement skills, because there were none available in the country due to the genocide (which killed mostly the skilled people). The Project Coordinator had procurement experience and was later trained to be able to carry out the functions of a procurement specialist. At that time, the Country Office was also small and had procurement advise and clearances provided through the Burundi Country Office and Bank headquarters.

As stated in the ICR, after the midterm review, country circumstances changed and it was possible to recruit skilled people. The project was later charged with the responsibility to support and strengthen the National Procurement Office and Accounting Professionals because of lack of such skills. There are two additional indications, inter alia, demonstrating effective implementation capacity. First, concerning the extension of the project closing date, the Borrower estimated the timing correctly and was able to implement project activities as planned. Secondly, concerning disbursement of funds, the PIU was able to disburse all funds from the original project and 94 percent of the additional financing.

Implementing Agency Performance Rating: Satisfactory

Overall Borrower Performance Rating: Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The project's M&E framework was unclear. It contained a large number of indicators, but a results matrix that did not show a clear results chain. The Key Performance Indicators defined on the first page of the PAD did not correspond to anything in the matrix in Annex 1. The results matrix, called project design summary, provided details of output indicators, some of which were, however, outcome indicators, and project reports, which would serve as an M&E tool. At the design stage, several indicators lacked baseline data.

b. M&E Implementation:

As indicated in the ICR, a survey in 2006 (five years after project launch) tried to retrofit the baselines for the indicators. This proved very difficult for some indicators. As project activities changed, the results framework was not updated. The ISRs showed that a subset of the indicators from the long list in PAD Annex 1 was tracked to monitor project progress, but there was no indication whether appropriate collection methods were used to ensure data quality.

a. M&E Utilization:

The project was modified several times on the basis of issues identified via various sources, including implementation agencies' reports, external auditor reports, and supervision mission reports.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

An environmental pre-audit was conducted, and approved by the Bank and the Borrower before the project was approved. (PAD and ICR)

b. Fiduciary Compliance:

The ICR noted some issues and a technical problem encountered by the PCU in following financial management guidelines. Nevertheless, there was no indication of breach of financial management policies.

c. Unintended Impacts (positive or negative):
NA

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Moderate
Moderate
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Satisfactory
Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

This Review concurs with the ICR on the following lessons:
  • Political commitment at the highest level is needed to guide implementation of far-reaching reforms.
  • A good M&E framework with clear causal chains linking activities to outcomes provides a critical management tool to make timely adjustment.
  • PCU capacity needs to be sufficiently factored into project design to ensure smooth implementation.
  • Decentralization of project supervision enhances dialogue, timely identification, and resolution of issues.
  • In a multi-donor context, coordination among partners is critical to ensure efficiency and impact.
  • As a top reformer of the business environment, Rwanda’s experience would be of interest to many countries and should be shared widely through South-South knowledge exchange program.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

    The ICR is comprehensive and candid, providing evidence on both the project's achievement and its weaknesses. The lessons are consistent with the analysis. One shortcoming is the argument, presented in the ICR, that efficacy would have been rated satisfactory had the indicators had been better formulated (p. 27). However, given the availability of standard investment climate measures for Rwanda over the project period, it should be possible to use these standard ratings to assess achievements. Also, the ICR states that the outcome rating is due to the fact that not all achievements could be attributed solely to the project (p. 28). In IEG's view, what matters is a reasonable causal chain linking project outputs to achievements, and the absence of any convincing evidence that other non-project factors contributed to the achievements.

    a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions