Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Public Service Capacity Building Project (PSCAP)


  
1. Project Data:   
ICR Review Date Posted:
02/27/2006   
PROJ ID:
P050400
Appraisal
Actual
Project Name:
Public Service Capacity Building Project (PSCAP)
Project Costs(US $M)
 28.0  28.42
Country:
Zambia
Loan/Credit (US $M)
 28.0  28.0
Sector, Major Sect.:
Central government administration, Law and justice,
Law and justice and public administration; Law and justice and public administration
Cofinancing (US $M)
   
L/C Number:
C3329      
   
Board Approval (FY)
  00
Partners involved
DFID 
Closing Date
12/31/2003 06/30/2005
         
Evaluator: Panel Reviewer: Division Manager: Division:  
Jeffrey Balkind
Chad Leechor Kyle Peters IEGCR

2. Project Objectives and Components:

a. Objectives
Primary development objective:to make public service delivery processes more effective and efficient in order to facilitate economic growth and reduce poverty. The DO supported two strategic areas of the 1999 CAS: (1) improved governance through improving transparency and accountability, strengthening public sector efficiency and capacity, fostering local government and decentralization; and (2) better access to basic services and direct poverty reduction interventions. The PSCAP also supported the PRSP -- which was focussed on accountability, expenditure management and good economic governance.

b. Components (or Key Conditions in the case of Adjustment Loans):
The project was designed as an APL. The Phase 1 DO was: to design and implement critical system-wide reforms that would support improved service delivery. These reforms included rationalizing staffing, particularly in the core civil service, reforming pay, introducing more effective management of payroll and establishment, replacing the cash budget with a Medium Term Expenditure Framework (MTEF), improving financial management systems, and accelerating privatization and private sector participation in the economy.

Phase 2 was to have rolled out the systems developed and piloted under Phase 1 throughout the public service and result in service provision being at least partially decentralized with greater community involvement. Phase 3 was to have been a consolidation phase during which the benefits of the previous phases would have been monitored and evaluated, and appropriate adjustments made to make them more effective.

c. Comments on Project Cost, Financing, Borrower Contribution, and Dates
The credit became effective six months after Board approval. Originally, Phase 1 of the APL was meant to be a three year program with a closing date of December 31, 2003. Due to delays in implementation, and an unrealistic initial design, the closing date was extended to June 30, 2005. The APL is continuing as the lending instrument for the SWAps, which comprise Phase Two of the APL. IDA’s funding for continued public service capacity building is therefore being made through a harmonized approach of pooling of funds with other donors (two SWAPs in partnership with 11 other donors plus a stand-alone Bank funded retrenchment activity).


3. Relevance of Objectives & Design:

The project was meant to address key public administration issues and its objectives were aligned with both the CAS and the PRSP. The relevance of the project, however, was significantly diminished by unrealistic design and over-ambitious goals. Phase 1 of the program did not adequately address public service delivery processes. Phase 2 was meant to get to these important aspects of service provision and direct poverty reduction, but for the reasons stated above it never proceeded. The wide ranging reforms required a sufficiently long enough time frame for implementation -- longer than was allowed under the project design.This was especially critical because the political and bureaucratic environment was undergoing much change, and the complexity of the reforms required a good body of time for implementation.

4. Achievement of Objectives (Efficacy) :

The project had six components, each with a large number of subcomponents. Achievements were:

1. Rightsizing and Pay Reform Partially achieved. Public service restructuring, payroll reforms, introduction of performance contracts, strengthening public service social safety net programs, and divestment of non-core functions. Staffing in the public service was reduced by 17.5 percent from the target baseline (2 percent short of the target). 14 of the 24 allowances recommended for abolition in the pay reform were not abolished or included in base salaries.

2. Policy and Public Service Management Partially achieved.Linkages between ministerial strategic plans and the MTEF planning cycle remain weak in some cases, although some improvements are noted in the ICR, especially in integrating policy, program and budget planning.

3. Financial management, Accountability and Transparency Partially achieved. There was provision of more effective human resources management. A Payroll Management and Establishment Control (PMEC) system, funded and supervised by DFID was introduced. However, although Permanent Secretaries were placed on contract and an Annual Performance Appraisal System (APAS) designed, performance contracting for senior officials was not introduced. The APAS was not made functional.

4. Judicial and Legal Reform Partially Achieved. There was improvement of resource allocation and financial management systems of the public service. The MTEF was introduced in association with Activity Based Budgeting (ABB). Capacity building, capital works and resource provision in the Office of the Auditor General (OAG) resulted in the auditing of government financial statements meeting legislative requirements and in the strengthening of OAG provincial office operations. However, the MTEF remains a work in progress.

5. Decentralization Not achieved. The objective was to restructure, decentralize, and improve accountability and transparency in the public service. However, "agentization", privatization and private sector economic participation were not significantly enhanced. The project’s decentralization activities were partly implemented with the development of a decentralization policy and draft Decentralization Implementation Plan. Revisions of the Public Finance Act and the Audit Act, which would have complemented other accountability reforms, were not undertaken. Interventions designed to provide greater public participation in the policy process and to survey citizens on service delivery quality were also not initiated.

6. Establishment of a Project Implementation and Coordinating Unit (PICU) to manage the project. Achieved. The PCIU operated competently.

5. Efficiency:

The project design and implementation was effective within the constraints of a weak civil service. Annex 4 of the PAD has cost benefit analysis computing the NPV of the planned restructuring reforms. The PAD stated that the retrenchments undertaken since 1997 had been negative, and it estimated positive net benefits of $6.1 million in FY02 from the new PSCAP. However, the ICR makes no attempt to measure the actual net benefits -- Annex 3 is left blank as 'not applicable.' This is an illustration of analysis done for Board approval purposes, with no attempt to follow-up the analysis.
6. M&E Design, Implementation, & Utilization:

The project made good use of many indicators through the LOGFRAME categorized according to the six components. A rating of each indicator was presented in the ICR using a six-point scale. Annex 1 of the ICR mentions that the mean score for all indicators is 3.47, which would place it between moderately satisfactory and moderately unsatisfactory. The Table in Annex 1 computes a mean score of 3.81 at the end.
7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative):

The Internal Finance Management Information systems (IFMIS) subcomponents of the project was not introduced, although the design study for IFMIS was completed. Instead the Government upgraded its Financial Management System (FMS) and included a commitment control capability in it, to enhance budget execution and control in all ministries down to the provincial level. The FMS has resulted in more accurate and on-time production of monthly expenditure returns, leading to greater expenditure predictability and reduced government borrowings on the domestic public debt market. The implementation of the enhanced FMS resulted in the Bank and the IMF waiving the IFMIS conditionality for Zambia to reach the HIPC completion point in 2005.

8. Ratings:
ICR
ICR Review
Reason for Disagreement/Comments
Outcome: 
SatisfactoryModerately SatisfactoryThe outcome indicators (mean score) place the project as 'Moderately Satisfactory.'
Institutional Dev.: 
ModestModest
Sustainability: 
LikelyLikelyReforms can be sustainable through use of SWAPs. Consequently, Phase 2 of the APL and Phase 3 will not be processed.
Bank Perf.: 
UnsatisfactoryUnsatisfactoryProject was over-ambitious in design. Project supervision was satisfactory.
Borrower Perf.: 
SatisfactorySatisfactoryDespite the weaknesses in design, the Government agencies did a satisfactory job in implementing a credible portion of the program.
Quality of ICR: 
Satisfactory

NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness.

9. Lessons:

1. The Mid-term review was a lost opportunity The MTR should have made a harder assessment of progress, problems, capacity and willingness to implement the original project design. It did not take into account changes in government policy, and priorities, so that the relevance of continuing with the original design could be assessed. Ultimately, Phase 2 and Phase 3 of the APL were dropped and reforms were pursued through two SWAPs. No information is presented on how those SWAPs are faring.


2. Avoid disconnects between various project ratings through greater realism The PSRs and ISRs were not sufficiently candid.They maintained satisfactory ratings throughout and did not give an accurate picture of the mixed progress. Actual disbursements lagged estimated figures by 50 percent for much of the project period, hence the team had an incentive to maintain satisfactory ratings in order to keep up the disbursements. The problem goes back to the original three-year time frame versus the five years it actually took -- just for Phase 1.

3. Correct use of instruments is important. Data is not available on the performance of the two SWAps as Bank disbursement to them has not commenced and they were only approved by the Board on January 5, 2006. However the harmonized approach with other donors made possible under a SWAp has clear benefits in ensuring that agreed reforms get implemented.

4. Broader partner involvement The PAD mentioned that DFID, Irish AID and UNDP would be involved in the program. Ultimately it was only DFID. DFID commented that certain payrolls continue to be excluded from PMEC (uniformed defense payrolls, some parts of Office of the President among others), but it agrees with the ICR finding that implementation of PMEC was a significant achievement.


10. Assessment Recommended?  No

          Why?  

11. Comments on Quality of ICR:

The ICR is comprehensive and well presented, drawing on the stated indicators and their results. The ICR team did not quantify the net benefits of the program, since almost all the assumptions used in Annex 4 of the PAD proved to be inaccurate and could not provide a reliable baseline. Calculation of the economic net benefits of complex public sector reform projects with large numbers of interventions across multiple agencies can be difficult, especially if the baseline data are not agreed upon. Greater attention should be devoted to this aspect in the newly approved SWAp.

(ES-Rev4B-Dec/05)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions