Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Development Of The National Statistical System


  
1. Project Data:   
ICR Review Date Posted:
03/12/2012   
Country:
Burkina Faso
PROJ ID:
P085230
Appraisal
Actual
Project Name:
Development Of The National Statistical System
Project Costs(US $M)
 13  13
L/C Number:
C3878
Loan/Credit (US $M)
 10  10
Sector Board:
Public Sector Governance
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  03/25/2004
 
 
Closing Date
07/31/2009 07/31/2010
Sector(s):
Central government administration (85%), Information technology (15%)
Theme(s):
Economic statistics modeling and forecasting (100% - P)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Fareed M. A. Hassan
Kris Hallberg Ismail Arslan IEGPS2

2. Project Objectives and Components:

a. Objectives:


    The objectives were to build a sustainable statistical system which would efficiently and effectively collect, process and disseminate accurate, timely, coherent and reliable statistical data concerning the economy and social conditions of the Borrower, through: (i) the improvement of the statistical policy and the regulatory and institutional framework; (ii) the establishment of a system for human resource development for the statistical system; (iii) the development of statistical data and dissemination; and (iv) the development of statistical infrastructure (DCA, p. 14). There is no difference in the wording of the objective in the PAD and the ICR. The DCA (Schedule 6), the PAD Results Framework ( Annex 3), and ICR (Section 1.2) identified four outcome indicators: (i) rate of user’s satisfaction increase in terms of accessibility to data, timeliness and periodicity for national accounts and social statistics; (ii) targeted statistical products are easily accessible; (iii) reduction in time lag between data collection and dissemination for targeted statistical products; and (iv) targeted statistical products are released within the time limits and with the frequency meeting IMF General Data Dissemination System (GDDS) requirements. Eleven intermediate outcome indictors covering the project components were are also identified.

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

The project has five components:

Component organizational Development (appraisal $0.43 million, actual $0.67 million)
This component would help create the legal and institutional framework for a sustainable central organization to collect and disseminate statistics. Specifically, it would include activities to enforce the existing law for statistical systems and operations, revive the Conseil National de Coordination Statistique (CNCS) as a design and implementation organization for statistical policies, and initiate dialogues between producers and users of data.
Component 2-Human resource development (appraisal $2.31 million, actual $2.38 million)
This component would train producers and users of statistics. The primary target group for the component would be made up essentially of Government statistical officers (through scholarships), but also policy planners, journalists, businesses, NGOs, and trade unions.
Component 3-Data development (appraisal $3.78 million, actual $3.73 million)
This Component was designed to generate new statistics, or improve the timeliness, accuracy, and presentation/dissemination of existing statistics, on national accounts, poverty and living conditions, demographics, health, education, agriculture, labor, and transportation.
Component 4-Physical infrastructure and equipment (appraisal $2.76 million, actual $2.54 million)
This component was designed to help establish information technology and physical infrastructure for statistical operations, including vehicles, computers, networking equipment, and a headquarters building for the Institut National de la Statistique et de la Démographie (INSD).

In addition there was a small component (to support the unit implementing the project (appraisal $0.73 million, actual $0.87 million). There were some weakness in the Project Management Unit (PMU). The PMU was initially not familiar with Bank's procedures (e.g. procurement, financial management, and M&E) and faced shortage of staff and insufficient number of vehicles for conducting surveys.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

The credit was approved on March 25, 2004 for US$10 million equivalent. The total project cost was estimated at US$13 million, with a US$3 million government counterpart contribution. The original closing date of July 31, 2009 was extended by one year to address delays experienced by the population and housing census for which counterpart funding was not secured in time.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

Project objectives were consistent with the strategic objectives of the government’s reflected in its 2000 PRSP and Statistical Master Plan ((Schéma Directeur de la Statistique – (SDS), and supported four of the five strategic objectives of SDS: (i) produce statistics that meet changing needs; (ii) share statistics effectively; (iii) reform institutions, to improve production and use of statistics; (iv) strengthen human resources; and (v) design and implement a sustainable financing strategy for production of statistics. The project objectives were also consistent with the Bank’ CAS that supported the government efforts on implementing the PRSP and SDS. These strategic documents recognized the importance of evidence-based policy-making that required timely, comprehensive, reliable and accessible statistical information. Progress in these areas contributes to informed resource allocation, and improved governance, a major PRSP pillar. The project was also consistent with the STATCAP program, which is the first explicit effort to provide IDA lending for statistical capacity development. The relevance of objectives is rated substantial.

b. Relevance of Design:

The components in the project were consistent with the overall objectives. The project design took a comprehensive approach by combining the methodological and IT improvements (components 4 and 3) with support for organizational and management improvements (components 1 and 2). Consultation and coordination with other partners such as the IMF began at project inception. Special attention was given to involve stakeholders, like sector ministries and primary data users, in the design of the project. A solid analytical base came from the diagnostic work underpinning the 2003 Statistical Master Plan (SDS). Support to the development of the SDS was provided by a Trust Fund for Statistical Capacity Building that also funded a workshop to define statistical sector strategies. The project was properly designed to address the key issues raised and to carry-out the actions proposed in the SDS. However, the project design was prejudiced by not having an adequate Results Framework with baselines or well specified indicators. The incomplete framework was not addressed until the 2007 Mid-Term Review, indicating that supervision was being conducted without fully specifying progress against target. A 2006 Quality of Supervision Assessment (QSA7) noted that supervision reports (ISRs) were not updated, but reissued with the same data and baseline data or targets were often missing. The ICR (p.17) acknowledges the weaknesses of the M&E system and Section 10 of this review provides details. On balance, the relevance of design is rated substantial.


4. Achievement of Objectives (Efficacy) :


The objectives of building a sustainable statistical system that would efficiently collect and disseminate accurate, timely , coherent and reliable national accounts and social/sector statistics were substantially achieved with the exception of timeliness:
    • First, there was an increase in the rate of user satisfaction in terms of accessibility, timeliness and periodicity for national accounts and social statistics and the statistics user survey indicated that for social statistics, the average rate of user satisfaction exceeded the target, whereas for national accounts it fell below the target.
    • Second, easy accessibility of targeted statistical products (e.g. National Accounts, Statistical yearbooks, survey data/reports) was achieved for all statistical data, except for vital registration statistics as reflected in the statistics user survey.
    • Third, reduction in time lag between data collection and dissemination for targeted statistical products was partly achieved as there were delays due to local printing capacity and the indicator relating to website release set a target of 60 percent and was not met and on average 36.5 percent of reduction of the time lag was reached by 2009 for only 4 out of 8 products.
    • Fourth, of the targeted eight statistical products, only three have been distributed in accordance with the GDDS standards.
    • Fifth, in terms of sustainability, the government has continued to fund the recurrent activities of INSD and the sectors and routine data production and reporting has continued, maintaining the changes supported by the project. Several ministries (e.g. health and labor and social security) have recognized the importance of statistics and have created a department of statistics at the level of directorate or directorate general.

Substantial progress on intermediate outcomes/outputs was also achieved:
      • First, in terms of organizational and human resources development, the project led to development of statistical law and related decrees which were published in June 2007, and CNCS Permanent Secretary was operational. However, mechanisms for effective dialogue between the data users and producers were partially developed. The project contributed to long term training of 303 staff of which 57 were new recruits: 39 demographer degree; 34 Level-One statistical engineer degrees; 45 Level-Two statistical engineer degrees, 4 diplomas in Statistics for Development, 84 Statistical technician degrees and 101 statistical technical agent degrees . The hiring of newly trained statisticians has increased the ratio in the statistical agencies from 11 percent to 47 percent at INSD and from 42 percent to 73 percent overall, exceeding envisaged targets. In addition, several short-term training sessions were conducted to the benefit of staff in charge of statistical data production in the various ministries and agencies. In total, 109 staff received short-term training in Burkina or abroad (77) or attended study tours (32) on the statistics of poverty and household conditions, national accounts, environmental statistics, population census, demographic analysis, statistics coordination, project planning and project M&E. Finally, a total of 34 agencies were identified as statistical data users and training was provided to 49 members of these agencies.
      • Second, in terms of data production, the project helped strengthen the production of timely, reliable data, following internationally accepted methodologies in the following sectors:
        (i) National accounts. Prior to the project, national accounts had not been produced since 1998, and the move to the System of National Accounts, revision 1993 (SCN93) from the 1968 version represents a significant increase in the coverage of national accounts through the inclusion of the informal sector. National accounts for 1999-2001, 2002-2006, 2007 were completed and published on the web site. Those for 2008 are completed but not yet published (available on demand). Those for 2009 are being finalized. The preparation of 2010 NA will start soon.
        (ii) Poverty and living conditions. The 2006, 2007 and 2008 Tableau de Bord Social (TBS) on the data synthesis of socio-economic indicators were produced and disseminated. These data are expected to help monitor and understand poverty and social conditions.
        (iii) Demographics. The final results of the 2006 General Population and Habitat Census (Recensement Général de la Population et de l’Habitation - RGPH) were published.
        (iv) Education, Labor and Social Security, Health, Transport, Agriculture and Employment. The statistical yearbooks for these sectors are produced on a regular basis and are expected to help inform the formulation of sector strategies.
      • Finally, the project helped build a new headquarters of the Institut national de la Statistique et de la Démographie (INSD) and improve its working environment. Computer and office equipments were acquired for INSD, the PMU and the other data producing agencies. The main equipment acquired was 413 typewriters, 176 computers and 60 printers, 12 cars and 4 motorcycles.

5. Efficiency:


The PAD (p. 104) noted that the project was not amenable to a detailed cost-benefit analysis. The ICR noted that improvements in statistical operations would produce higher quality data with broader coverage. These data would reduce the information cost of all economic activity, and increase efficiency throughout the country. The ICR also noted that the unit cost of project activities were comparable to other Bank projects with similar type of expenses. Efficiency is rated substantial.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
%
%
ICR estimate:
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:

The project objectives and design were relevant to the government’s and the Bank’s strategy. Substantial progress toward the project objectives was made. A legal and institutional basis for a sustainable central statistical organization was set up and facilitated data development, dissemination, and utilization. Starting from a weak base (e.g. no national accounts since 1998), the project-supported changes were important, including timely production of data on national accounts, social statistics, and sector data on education, health, labor, transport. Data supported by the project were integrated into critical strategy documents and management tools such as PRSP. The INSD was able to produce most indicators for monitoring the various dimensions of poverty. The independent evaluation of the PRSP in June 2010 confirms that INSD has played an important role in the regular production of data on living conditions of households, whose quality is widely recognized. Sector level data such as education, health are expected to help these sectors anchor their strategies and action on reliable and timely data. Mostly user’s satisfaction has increased but there have been moderate shortcomings. First, timing between production and dissemination of targeted statistics has improved by 36 percent but still under the target (of 60 percent) due to some unexpected problems. Second, dissemination based on GDDS standards has been achieved for only 3 out of the targeted eight statistical products. This review rates the outcome as moderately satisfactory.

a. Outcome Rating: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating:

In approving the Master Statistical Plan (SDS), the Government committed to allocate the budget to support the implementation of the SDS over a period that would go beyond the lifetime of this project. Government’s actions show that many project changes are sustained. Prior to closing, government had used its own resources to fund the operating costs for INSD regional offices, taking over from the project. The recurrent activities of INSD and the sectors are funded and routine data production and reporting has continued, sustaining the changes brought about by the project. Government has also continued the training of statisticians. A new statistical development strategy was prepared for 2011-5, focusing on the (a) consolidation of the legal, institutional and organizational framework; (b) improvement of the statistical data production and analyses; (c) improvement of data storage, diffusion and use; and (d) the strengthening of human, financial and material capacity. The Capacity building supported by the project has provided the country with a body of trained statisticians. Improved capacity in statistics and managerial skills through the project has also helped strengthened the system in the long run. The project proposed the integration of PMU staff within INSD so as to ensure greater sustainability. This review concurs with the ICR assessment that the risk to development outcome is low to negligible.

a. Risk to Development Outcome Rating: Negligible to Low

8. Assessment of Bank Performance:

a. Quality at entry:

The project objectives were consistent with the government’s own priorities as well as with the Bank’s strategy. The project was properly designed to address the key issues raised and to carry-out the actions proposed in the government’s Master Statistical Plan (SDS). The project components and activities were in line with the overall project development objectives and followed a sector-wide approach combining the methodological and IT improvements with support for organizational and management improvements. Attention was given to involve all partners and stakeholders in the design of the project.. A shortcoming was the weak design of the project M&E system.

Quality-at-Entry Rating: Moderately Satisfactory

b. Quality of supervision:

In terms of supervision, the Bank team maintained a regular dialogue with policy makers and technical staff on progress. The intense and sustained supervision (an average of three missions per year) has helped improve project performance. However, as noted earlier the 2006 Quality of Supervision Assessment (QSA7) found that the supervision mission reports (ISRs) were poor in quality, did not provide a good review of project progress and problems.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

    The operation was developed within the objectives of the government’s Statistical Master Plan ((Schéma Directeur de la Statistique ( SDS) that has a clear vision for reform in the statistical system. The government requested Bank support for SDS implementation and made a commitment to contribute effectively over 23% of its financing (supplementing training and construction works, taking over operational costs of regional offices). The government also remained committed during implementation. It approved the proposed reforms for the National Statistical System, regulated and organized the country’s statistical activities, and implemented several decrees. The government has also prepared a new national statistical strategy for 2011-2015 and approached the Bank for its support through a second phase project. All three ministers that were successively in charge of finance during project implementation were supportive of the project.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:

There were three implementing agencies. First, the Institut national de la Statistique et de la Démographie (INSD) has been responsible for the implementation of the project, as the central institution responsible for the national statistical system. The INSD has improved its capacity to manage and coordinate the national statistical program and adopted sound financial and administrative regulations that strengthen its role as the central statistical coordinator. Second, a Project Management Unit (PMU) was established within the INSD to carry out the day-to-day implementation of the project. The PMU performed relatively well, except during the first year, when the project faced some difficulties. Third, with few exceptions, line ministries statistical departments implemented project components in their sectors, under the guidance and supervision of the PMU to ensure compliance with the procurement, disbursement and financial management policies and procedures.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The design of the M&E system was weak. The ICR noted (p.17) the linkages between some of the objectives, impacts and indicators were sometimes unclear. Some of the outcomes identified did not translate into measurable indicators and could not be monitored during project implementation. The ICR also noted that the project was one of the first sector-wide approaches for statistics, and thus there was little experience on which to base the M&E system. As indicated in the PAD, data for results monitoring was expected to be obtained from routine administrative data that INSD and agencies produce and external specific evaluations as well as those carried by other beneficiaries, including line ministries and donors. Some baseline monitoring data was obtained from the SDS results framework, the IMF Report on the Observance of Standards and Codes (ROCS) and the GDDS process that assess the quality of statistics production.

b. M&E Implementation:

The M&E Framework proved to be too complex to be implemented, despite the fact that a survey to produce the missing baseline indicators was conducted. The M&E rating dropped to highly unsatisfactory by the time of the 2007 Mid-term Review (MTR). Following the MTR, lessons learned on the M&E system were applied to improve the system during the last two years of the project. Responsibility for M&E was moved from the PMU to a directorate in INSD, whose staff undertook the M&E activities, leading to an increase in the amount of information on project activities and outputs. At the time of project completion, the M&E system improved and progress reports on results were produced as planned.

a. M&E Utilization:

The delays experienced in producing the indicators and setting up an M&E team at INSD resulted in the lack of data use for monitoring project results in the first years of the project as pointed out by the 2006 QSA7 review. During the last project years, INSD focused more on outcome indicators and a Road Map for project completion was set up based on the indicators, to ensure that targets would be met as planned. The major effect of the project on the INSD M&E system was to establish regular reporting mechanisms, raise awareness of its usefulness and foster a results management focus at INSD well beyond the implementation of the project. The M&E findings were largely disseminated to various stakeholders.

M&E Quality Rating: Modest

11. Other Issues:

a. Safeguards:

The construction of INSD headquarters led to the production of dust, disposal of large amount of waste and noise. Actions recommended to mitigate the impact of the INSD construction on the environment were executed. No other safeguard policies are triggered by this project.

b. Fiduciary Compliance:

No procurement/fiduciary problems to report.

c. Unintended Impacts (positive or negative):

None

d. Other:

None



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Satisfactory
Moderately Satisfactory
 
Risk to Development Outcome:
Negligible to Low
Negligible to Low
 
Bank Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
 
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

Lessons derived from this project include (i) supporting a statistical system has proven to be more efficient if it is intergated into the framework of an overall national statistical strategy (i.e. Statistical Master Plan ((Schéma Directeur de la Statistique ( SDS)); and (ii) creating a critical mass of statisticians is a pre-requisite for the sustainability of a national statistical system. In addition, a number of lessons learned from past projects that were used in the design of the project under review includes the following. The general lesson learned in developing statistical capacity is that fragmented, uncoordinated programs of support do not result in sustainable systems. Furthermore, investments in information technology infrastructure, or improved methodologies, require accompanying organizational and management change in order to be effective. These lessons are reflected in the project design by approaching the development of the statistical system through the Statistical Master Plan. This ensures that appropriate assistance and investments are made in all areas of the statistical system, and that a wide range of both users and producers are involved and consulted.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The overall quality of the ICR is satisfactory. The ICR is comprehensive, containing all necessary information to evaluate the project. It is frank in its assessments and conclusions.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions