Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Administration Capacity Building Project


  
1. Project Data:   
ICR Review Date Posted:
12/22/2012   
Country:
Burkina Faso
PROJ ID:
P078596
Appraisal
Actual
Project Name:
Administration Capacity Building Project
Project Costs(US $M)
 7.125  not available
L/C Number:
CH151
Loan/Credit (US $M)
 7  6.59
Sector Board:
Public Sector Governance
Cofinancing (US $M)
 0.125  not available
Cofinanciers:
Board Approval Date
  03/22/2005
 
 
Closing Date
08/31/2010 08/31/2011
Sector(s):
Central government administration (80%), Sub-national government administration (20%)
Theme(s):
Administrative and civil service reform (40% - P) Health system performance (20% - S) Public expenditure financial management and procurement (20% - S) Decentralization (20% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Stefano Migliorisi
Michael R. Lav Navin Girishankar IEGPS2

2. Project Objectives and Components:

a. Objectives:


    The original project development objective (PDO) was “to improve administration structures and processes in light of the decentralization policy.” (Project Document, p. 3). These improved capacities were expected to “yield, in the medium-term, measurable impact on provision of services to the citizens of Burkina.” (PD, p. ii).

    The Development Grant Agreement (DGA, p. 16) defines the original objective differently: “to assist the Recipient in improving the rationality, transparency, accountability and performance of its administration, by: (i) implementing decentralization and deconcentration of responsibilities and services, and (ii) building up institutional and legal infrastructure.”

    For the purposes of this review, IEG uses the definition in the Program Document because it explains how intermediate outcomes such as administration structures are related to higher level objectives of service delivery. The definition in the DGA are focused on intermediate objectives.

    The revised PDO was " (i) to strengthen the Recipient’s central government resource management capacity, and (ii) to enhance the Recipient’s capacity in the planning, organization and monitoring of the implementation of the decentralization process.." (Amendment to the DGA, p. 4). See Section 2b.

b. Were the project objectives/key associated outcome targets revised during implementation?
Yes

If yes, did the Board approve the revised objectives/key associated outcome targets? Yes

Date of Board Approval: 06/29/2010

c. Components:

The project had six original components (actual costs refer to end of 2009, before restructuring):

    1. Component 1: Support to the Ministry of Civil Service and State Reform (original cost: US$ 0.7 million, actual cost: US$ 0.58 million) was designed to strengthen the ministries role of guiding the de-concentration process, implement Jumen Resource Management (HRM) deconcentration (e.g., update and improve the SIGAPSE system and de-concentrate it to line ministries and possibly to regions, and increase Civil Service performance by implementing training programs in evaluation policy).
    2. Component 2: Support to Ministry of Finances and Budget (original cost: US$ 2.3 million, actual cost: US$ 1.23 million) financed the Ministry’s implementation of restructuring efforts to improve internal efficiency. This included completion of public account audits, strengthening the links between budget programming and the PRSP process, strengthening PEM de-concentration through expansion of CID across additional regions, strengthening internal audit capacity, and strengthening revenue collection.
    3. Component 3: Support to the Ministry of Economic Development (original cost: US$ 0.68 million, actual cost: US$ 0.45 million) through establishment of procedural manuals, establishment of a single methodology for elaborating, monitoring and evaluating sectoral and local policies, and preparation of procedural manuals and training of staff in their use.
    4. Component 4: Support to the Ministry of Territorial Administration and Decentralization (original cost: US$ 0.75 million, actual cost: US$ 0.81 million) included adoption and implementation of a new organizational chart, strengthening of the leadership role of the Ministry in implementing basic guidelines for the decentralization policy (disseminating the Decentralization code implementing decree), and strengthening of local governments‟ capacity to fulfill their role by preparing, adopting and implementing a capacity building strategy for local governments.
    5. Component 5: Support to the Ministry of Health (original cost: US$ 1 million, actual cost: US$ 0.24 million) financed activities to implement the decentralization strategy and strengthen the Ministries‟ role as regulator and planner through activities such as providing support to contracting with the private sector, and strengthening the management capacity of the Ministry.
    6. Component 6: Support to the Prime Minister’s Office (original cost: US$ 0.68 million, actual cost: US$ 1.4 million) focused on improving internal efficiency and inter-ministerial coordination through implementation of the recommendations of an organizational audit.
Revised Components. The June 2010 restructuring reduced the number of components from six to three. Amount indicated in parenthesis refer to the cost of the components based on resources still available at project restructuring, when the Bank did not increase its grant of US$7 million equivalent (of which US$2.11 were still undisbursed and US$0.44 had become available due to the appreciation of the SDR against the US dollar), but the Borrower increased its contribution from US$125,000 to US$4 million. The revised components were:
    1. Component 1: Strengthen the Recipient’s central government efficiency and controls (US$ 2.5 million equivalent, of which IDA US$ 2 million equivalent) by (a) strengthening the capacity of selected Recipient’s ministries to undertake longer term planning and evaluation of policies; (b) improving de-concentration of human resource management, including through the development of information technology tools; and (c) improving the pay management system through the delegation of the processing of pay changes to the Regions.
    2. Component 2: Strengthening policy formulation coordination and monitoring in key recipient’s Ministries responsible for decentralization through a decentralization road-map and a medium-term (three to five year) decentralization implementation plan (US$ 560,000 of which IDA US$140,000) by (a) supporting the definition of policies and procedures to implement decentralization; and (b) supporting planning, allocation of responsibilities and costing of activities in terms of both human and financial resources.
    3. Component 3: Supporting project management including inter alia, the operation of the PCU (US$ 1 million equivalent, of which IDA US$ 0.41 million equivalent) to support project implementation through the financing of the technical staff of the PCU and the completion of some studies.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Cost: The project costs increased after restructuring from US$7.125 million to US$11 million. No data are available on actual project costs at closing as there is no information available on the actual amounts disbursed by the Borrower.

Financing: The project was financed by an IDA grant equivalent to US$7 million and a Borrower contribution of US$ 0.125 million. At the time of restructuring, the Borrower increased its contribution to US$4 million, while IDA had disbursed by then US$4.89 million and allocated the remaining balance of the grant plus the additional resources in dollar terms created by the appreciation of the SDR against the dollar to the three revised components for a total of US$2.55 million. Of this amount, only US$ 1.68 million was actually disbursed before project closing.

Borrower Contribution: The Borrower agreed to increase its contribution from US$125,000 to US$4 million during restructuring but the ICR does not provide any information on what share of the revised contribution was actually disbursed.

Dates: There were some minor deviations from planned dates during the course of the project. Approved on March 22, 2005, the project became effective on August 25, 2005, just ten days later than planned and a mid-term review was carried out in May 2008 only two months later than scheduled.The closing date was extended twice from August 31, 2010 to February 28, 2011 in June 2010 and to August 31, 2011 in January 2011. The extension was due to delays in the recruitment of a consultant to prepare the decentralization roadmap and the selection of contractors to extend the computerized systems for processing pay data in the regions. The project closed as per revised schedule.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Relevance of Original Objectives

The original objective was to improve administration structures and processes in light of the decentralization policy. These improved capacities were expected to “yield, in the medium-term, measurable impact on provision of services to the citizens of Burkina.” (PD, p. ii). IEG assesses the relevance of this PDO as follows:

  • This original PDO is consistent with the GoBF’s Strategy for Accelerated Growth and Sustainable Development (SCADD) presented in 2010 and covering the period 2011-2015. Its third pillar is focused on strengthening good governance in the country, including macroeconomic management, public financial management, enhanced accountability, and more extensive decentralization. It is also consistent with the vision of “Burkina 2025” developed in 2005.
  • The objectives are aligned with the latest Bank’s Country Assistance Strategy covering the period FY10-FY12, in particular the CAS's second strategic theme (promoting shared growth through improved social service delivery). This pillar envisaged an increased emphasis on public participation and local development.
  • While the original objectives broadly link efforts to strengthen administrative capacity to service delivery, they does not specify the type of capacity that the project aims to build. As noted in the 2005 World Bank Africa Capacity Building Taskforce Report, experience from other countries indicates that a key success factor for capacity building initiatives is to specify the functions to be strengthened (for example, resource management, service delivery, regulation, or enforcement). In addition, it is not clear what the PDO intends to do "in light of decentralization policy" (for instance, create local structures or central ones, further decentralization or manage the associated risks).

Given the lack of specificity in the PDO, relevance of the original objective assessed as modest.

Relevance of Revised Objectives

The revised objectives continued to be aligned with the PRSP pillars and the CAS strategic themes discussed above. They also improve on the original objectives by specifying the types of capacity that the project aims to build -- that is, central capacity to undertake resource management and to support decentralization processes. Relevance of the revised objectives is assessed as substantial.

b. Relevance of Design:

Relevance of Original Design

The original project contained too many components dispersed across a number of ministries. Given the low levels of capacity in the country, this particular design lacked coherence and was unwieldy. While it mentions service delivery as a high level objectives, it did not present a convincing causal chain between inputs (for example, technical assistance and training), outputs (for example, types of administrative capacity at the central and local levels, increased resource flows, controls, and strategies), and outcomes (basic service delivery in key sectors). The Project Document also did not explain how a series of relatively small interventions in six ministries could lead to across-the-board improvements in service delivery.

In its comments on the draft ICR Review, the Bank's team informed IEG that the original design was "based on the idea of allowing ministries to compete for additional resources for institutional reform implementation." However, there is no evidence (for example, an operational manual) of a rules-based system of competition between ministries. If the original components resulted from such a competition, the project did not provide for re-allocation funds if ministries' performance fell short of expectations. The experience in other countries suggests that such operations could use challenge grants or similar funds that support sub-projects based transparent rules of access.

The relevance of design was therefore modest.

Relevance of Revised Design

The 2010 restructuring consolidated the number of components, and focused them around cross-cutting central government controls and decentralization policies. While the revised design demonstrated closer links between capacity building outputs and project objectives, the revised results framework did not clarify causal links between outputs and intended service delivery outcomes. The revised outcomes were in actuality outputs (for instance, improved resource management capacity of central government measured by reduction of time needed to process HR decisions and modify payroll data; and improved planning, organization and monitoring of the decentralization process measured by design of a decentralization implementation roadmap). Measurable outcomes -- for instance, plausible improvements in service delivery -- were not defined.

The relevance of design therefore remained substantial.


4. Achievement of Objectives (Efficacy) :

Original PDO

The original PDO was to improve administration structures and processes in light of the decentralization policy. The actions supported by the project produced some important outputs. The efficacy of the original PDO is rated as substantial.

Outputs

The project achieved several of its original outputs. All Ministries and 13 regional offices are now connected to SIGASPE, the Integrated System for Human Resource Management, and thanks to it, the average time needed to modify payroll data has been reduced from 90 days to 38 days. The Integrated Financial Management System (IFMIS) was also extended to line ministries in the capital city and to all regions, the latter thanks to funding from the project. The project also helped draft legal and regulatory framework for decentralization and devolution of human resources and financial resources to local units and municipalities. The project finally assisted in training supervisors to apply the staff performance evaluation system and initiate systematic professional development and training for the core civil service.

Outcomes

The PDO of improving administration structures and processes in light of the decentralization policy was measured by two outcome indicators: program budgets that are better aligned with PRSP objectives and the national MTEF, as well as a greater share of national budgets transferred to and executed by local governments. These indicators referred to small part of the PDO, reflecting only a few elements of the changes needed to achieve it. Both outcome indicators were met according to the ICR. Ten Ministries had satisfactory program budgets, while the share of national budgets transferred to and executed by local governments grew from 1% in 2007 to 3% in 2011.

Revised PDOs

The revised PDO was to strengthen the Recipient’s central government resource management capacity; and to enhance the Recipient’s capacity in the planning, organization and monitoring of the implementation of the decentralization process.(Restructuring Paper, p. 3). Resource management capacity improved and the enabling legislation for decentralization and devolution was drafted. The efficacy of the revised PDOs is therefore rated as substantial for resource management capacity and modest on decentralization.

Outputs

The project achieved several of its revised outputs of the first PDO. SIGASPE was fully implemented, The pay/salary adjustments were processed in twelve regions. As noted above, the IFMIS was extended to line ministries in the capital city and the regions. In addition, the Bank team informed IEG that the project helped support drafting the legal and regulatory frameworks for decentralization and devolution of human and financial resources to local units and municipalities.

Outcomes

The PDO of improving resource management capacity was achieved. The average time to modify payroll data in SIGASPE was reduced from 90 days to 64 days in 2010 and 38 days in 2011. The times required for several other activities (i.e., secondment, leave without pay, transfer to a different department, creating an internship) were also substantially reduced by percentages between 20 and 70. Even though the enabling legal framework for decentralization was drafted, the ICR provided no evidence of actual improvements in the planning, organization and monitoring of the implementation of the decentralization process.


5. Efficiency:


The ICR does not discuss efficiency at all as the project appraisal did not include a formal cost benefit analysis. However, it should have been possible to provide information on unit rate norms for technical assistance that IEG cannot calculate in the absence of detailed data on days of technical assistance and their cost.

Several aspects of design and implementation reduced the project’s efficiency, as it covered a very fragmented set of activities (about 130 of an average size of US$50,000) carried out by six ministries. This increased transaction costs without a real justification, while relying on weak implementing capacity at the PCU.

The efficiency was therefore modest both before and after restructuring.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


Original PDO

The efficacy of the original PDO was substantial. There were, however, shortcomings in the relevance of objectives (modest), relevance of design (modest), and efficiency (modest). In accordance with the IEG/OPCS Harmonized Evaluation Criteria, this combination of ratings leads IEG to an outcome rating of moderately unsatisfactory.

Revised PDO

Even though 76 percent of disbursements took place before the restructuring, the relevance of revised objectives and relevance of design were substantial as well as the efficacy of resource management objectives. The efficacy of decentralization objectives was modest as was project efficiency. In accordance with the IEG/OPCS Harmonized Evaluation Criteria, IEG assesses project outcome as moderately satisfactory.

Overall Outcome Rating

Despite the improvements following the restructuring, IEG calculated a weighted average outcome rating of 2.24, which is moderately unsatisfactory.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

IEG concurs with the ICR’s assessment of significant risk to development outcome. The outcomes that could be traced to the project were improvements in SIGASPE and improved processing of HR actions, the IFMIS roll-out, and the drafting of decentralization legislation. While the HR changes are unlikely to be reversed, continued efforts will be needed to institutionalize IFMIS and related processes. The risk to the establishment of a decentralized framework is also significant since it requires sustained political commitment to ensure implementation and institutionalization of the framework. It will also require an extended program of support to build local institutions and change management effort to reorient line ministries. The ICR does not provide insights into the political economy of decentralization and devolution in Burkina Faso.

a. Risk to Development Outcome Rating: Significant

8. Assessment of Bank Performance:

a. Quality at entry:

The project addressed covered too many components and ministries at the same time in a low capacity environment.There were 130 activities over the life of the project that had to be completed by six different ministries. As noted in the ICR (p. 31), some sub-components or activities could have been projects in themselves. For instance, the project aimed at training all the hundreds of elected local officials and local civil servants in all 358 local councils, an immense task, out of eleven tasks assigned to the Ministry of Territorial Administration. The Monitoring and Evaluation Framework was initially rather weak and was improved only after restructuring, one year before project closing. Almost all of the above issues were addressed during restructuring, but this happened late in project implementation.

Quality-at-Entry Rating: Moderately Unsatisfactory

b. Quality of supervision:

There were eleven supervision missions over the life of the project. However, reporting was frequently delayed and some ISRs were overly positive on project performance. The Bank was nevertheless pro-active in addressing the project’s poor performance, although the restructuring took very long and the project was restructured only one year before closing. There were no compliance issues, with unqualified audit reports being delivered on time for the entire life of the project.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Government showed sufficient commitment to the project’s objectives and is still pursuing its decentralization objectives with Bank support. However, such commitment was not constant over the life of the project with frequent delays in procurement, disbursement and implementation, and substantial staff turnover in the Ministries involved, negatively affecting project performance. As noted in the ICR (p. 21), the demands from the various Ministries were not properly filtered and this led to a substantially fragmented capacity building program.

Government Performance Rating: Moderately Satisfactory

b. Implementing Agency Performance:

Notwithstanding the complexity of project design, the PCU was relatively successful in coordinating project activities of several different Ministries. The PCU was proactive in assisting Ministries in project implementation. When the various implementing agencies faced difficulties in hiring qualified accountants, the PCU often completed the accounting, control, and reporting of implementing agencies (ICR, p. 21). The ICR concluded that “the project was well managed by the PCU Administrator who appeared to be the ‘oil’ that kept the project running smoothly.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The original project design included an integrated management system with the PCU in charge of data collection and analysis. However, the results framework included too many activities and indicators from too many ministries, making data collection particularly challenging.

b. M&E Implementation:

The project suffered in particular from the absence of a dedicated M&E officer. A part time M&E officer was hired after the restructuring.

a. M&E Utilization:

Project data were not collected systematically and could therefore not be used in analyzing performance.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

There were no safeguards issues.

b. Fiduciary Compliance:

There were no fiduciary issues during the life of the project, only a few procurement issues caused by misunderstanding on World Bank procedures. Such issues were corrected in the course of project implementation.

c. Unintended Impacts (positive or negative):

None

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Unsatisfactory
Despite improvements in the relevance of objectives and design, the restructuring took place after 76 percent loan proceeds were disbursed. The weighted average of ratings for outcomes lead to a moderately unsatisfactory rating. 
Risk to Development Outcome:
Moderate
Significant
Risks to decentralization relate to the political commitment to pass draft legislation and to provide a program of support to ensure its implementation. Similarly, IFMIS will require sustained capacity building support. 
Bank Performance:
Moderately Satisfactory
Moderately Unsatisfactory
Both quality at entry and quality of supervision were moderately unsatisfactory. 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Unsatisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

IEG identified the following lessons from the ICR and the overall experience of this project:
    • Resistance to reform should never be underestimated and project design should be kept simple, particularly in low capacity environments. The lack of progress on decentralization was determined, at least in part, by resistance to reform, and lack of prioritization of project activities.
    • Projects focusing on public sector reform should be ready to fund more than training, studies and technical assistance.
    • While demand driven interventions can be useful, some activities needs to be fully define at the design stage and requests need to be filtered, not simply supported on a first-come-first served basis
    • Behavioral changes in the civil service require a focus on processes, resources and incentives in addition to the provision of training and IT hardware
    • M&E systems need to be rigorous and realistic, and attention must be paid to M&E staffing issues already at the design stage.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

While ICR was generally candid, it did not provide critical information to assess the efficacy of decentralization objectives. Furthermore, analysis of risks to development outcome did not include an assessment of the political economy of decentralization and devolution in Burkina Faso. In the end, IEG's assessment of efficacy of the revised objectives required additional information from the Bank's country team, following the preparation of the draft ICR Review.

a. Quality of ICR Rating: Unsatisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions