Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - India: Second National Tuberculosis Control Project


  
1. Project Data:   
ICR Review Date Posted:
10/02/2013   
Country:
India
PROJ ID:
P078539
Appraisal
Actual
Project Name:
India: Second National Tuberculosis Control Project
Project Costs(US $M)
 342.0  n/a
L/C Number:
C4228
Loan/Credit (US $M)
 170.0  178.85
Sector Board:
Health, Nutrition and Population
Cofinancing (US $M)
 129.5  n/a
Cofinanciers:
GFATM, DFID, USAID, WHO, GDF
Board Approval Date
  08/22/2006
 
 
Closing Date
03/30/2012 09/30/2012
Sector(s):
Health (72%), Sub-national government administration (20%), Central government administration (8%)
Theme(s):
Tuberculosis (33% - P) Health system performance (33% - P) HIV/AIDS (17% - S) Decentralization (17% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Judith Hahn Gaubatz
Judyth L. Twigg Christopher D. Gerrard IEGPS2

2. Project Objectives and Components:

a. Objectives:


    According to the Development Credit Agreement (DCA, page 19), the objective of the project was:
      • To support the Borrower's revised National Tuberculosis (TB) Control Program phase II, aimed at treating and reducing the incidence of tuberculosis throughout its territory.

    This was consistent with the higher-level program objective articulated in the Project Appraisal Document (PAD, page 3), which was "to help reduce mortality and morbidity due to TB and interrupt transmission of infection through consolidating program performance and expanding program coverage." More specific outcomes (and corresponding key indicators) were indicated in the PAD (page 4) as follows and are also used as the basis for this review:
      • To achieve the global targets of 70% case detection and 85% cure rate in all districts of the country (key project indicator: the number of districts that have achieved a detection rate of at least 70% and a cure rate of at least 85%); and
      • To reduce the incidence of smear-positive tuberculosis in the zones where Directly Observed Treatment Short-course (DOTS) has been under implementation for five or more years (key project indicator: decline in incidence of smear-positive tuberculosis).

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:
The project was part of a broader national program for TB control, with coordinated support from international donors. Given this sector-wide nature of the program, specific costs per component (appraised and actual) were not available.

1. Consolidation of the Revised National Tuberculosis Control Program (RNTCP) Services: This component aimed to sustain the quality of public TB services nationwide, under the framework of a second phase of the national TB program (RNTCP II). Activities were to include: improvement of supervision and quality assurance of laboratory services; strengthening of monitoring systems at all levels (including a computerized management information system); conducting of TB infection risk surveys; operations research to improve effectiveness of TB control activities; conducting of information and education campaigns (IEC) to create awareness of TB symptoms and demand for free DOTS treatment; and improvement of institutional capacity through training and technical assistance.

2. Expansion of RNTCP Outreach Services to Targeted Special Groups: This component aimed to maximize inclusion of TB patients under the DOTS program. Activities were to include: implementation of strategies to ensure services for tribal (through Tribal Action Plans) and poor (districts with high proportion of poor) populations, private sector providers, HIV/AIDS patients, pediatric patients, and multi-drug resistant TB (MDR-TB) cases. Special incentives were to be provided for health personnel working in difficult areas, and additional resources were to be provided for low-performing areas.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

Project cost:

    • The estimated project cost reported in the PAD was US$342.0 million. This included Bank financing, government contribution, and donor contributions to the national TB program. The actual project cost reported in the ICR included only the Bank and government contributions, at US$232.2 million. The national TB program cost including donor contributions was not reported in the ICR.

Financing:
    • The Bank provided US$178.85 million as a Credit, slightly higher than the estimated amount of US$170.0 million.
    • The PAD reported that donor contributions were estimated at US$129.5 million; the actual amount provided by each donor was not reported in the ICR.

Borrower contribution:
    • The government provided $55.2 million in counterpart funds, higher than the estimated amount of US$42.5 million.

Dates:
    • The project closing date was extended from March 2012 to September 2012, in order to allow additional time for the preparation of a request for Additional Financing. The government subsequently withdrew this request due to an expected large increase in domestic funding for the health sector.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:
High. Although India has achieved global targets in tuberculosis (TB) case detection and treatment success rates on a national level, it continues to have the largest number of new TB cases annually, with an estimated 2.2 million new cases occurring each year. Moreover, there remain significant disparities among different districts at the sub-national level. The project objectives are therefore highly relevant to the country situation. The objectives are also highly relevant to the Bank's corporate priorities, as reflected in the Millennium Development Goals, and to the Country Partnership Strategy for FY2013-2017, which identifies strengthened public and private health delivery systems as a key outcome.

b. Relevance of Design:

Substantial. The project design drew directly from the country's Revised National TB Control Plan (RNTCP), for which project interventions were considered technically sound, based on internationally recommended practices for detecting and treating TB. The project focused on reaching targeted sub-populations to ensure more complete coverage of TB services, such as tribal and poor populations, TB patients in the private sector, pediatric patients, HIV patients and Multi-Drug Resistant TB (MDR-TB) patients. The project addressed numerous institutional capacity issues as well, such as the development of information systems, human resources, and decentralization of administrative and service delivery roles. The SWAp was a relevant approach as it aimed to improve coordination among donors (Global Fund, USAID, DfID, WHO and the Global Drug Facility (GDF)) and drew on a common results framework and financing plan.


4. Achievement of Objectives (Efficacy) :


During the period covered by the project, the Bank provided approximately 43% of the funding for the national TB program, with the government providing 14% and the Global Fund 40% (largely in the form of drug procurement). Observed outcomes are therefore plausibly attributable, in large part, to Bank-financed interventions.

To increase coverage of TB treatment/ To achieve the global targets of 70% case detection and 85% cure rate in all districts of the country is rated Substantial, due to achievements in increasing, or maintaining, case detection and treatment success rates, particularly among vulnerable/underserved populations. The project identified specific sub-populations that would be targeted in order to expand treatment coverage: tribal populations, poor populations, HIV patients, pediatric patients, MDR-TB patients, and private sector patients. There was evidence of substantially improved coverage among most of these targeted groups; however, there were modest improvements in reaching private sector providers.

Outputs
    • Provision of DOTS treatment to over 8.6 million TB patients. Of these, almost 600,000 patients were pediatric patients.
    • Establishment of 49 MDR-TB management sites. 16,826 patients with MDR-TB were started on DOTS-Plus treatment.
    • Training of 108 DOTS-Plus treatment officers.
    • Referring of almost 1.8 million suspected cases of TB from HIV/AIDS service providers. Of these, almost 190,000 suspected cases were confirmed.
    • Participation of 2,325 NGOs in the RNTCP.
    • Participation of 13,397 private providers in the RNTCP. The ICR (page 20) suggests that these numbers may still be considered inadequate when compared to the size of the private sector in the country and (page 10) that administrative inefficiencies led to delayed payments and limited the scale of many public-private schemes. Possible factors included inadequate financial incentives for private providers to engage with RNTCP, delayed payments under the public-private scheme, and lack of capacity at the local levels to fully engage with large numbers of private providers.
    • Implementation of Tribal Action Plans, which included doubling the per capita number of TB Units and microscopy centers, providing travel allowances to patients and attendants, and providing salary incentives and travel allowances for health staff.
    • Implementation of an intensified TB/HIV package, as part of the joint TB/HIV framework in 14 states with high HIV prevalence rates.

Outcomes
At the national level, reflecting project support to the national TB program:
    • The number of zones (the country is divided into five - originally four - geographic "zones" for the purpose of incidence surveys) that reached the treatment success rate target was 5, achieving the target of 4. However, only one zone reached the global target for case detection rate of 70% (the other four zones had rates of 65%, 65%, 59%, and 68%).
    • According to additional data provided by the project team, comparable countries have reached similar "plateaus" once high case detection and treatment success rates have been reached. In China, the CDR fluctuated between 87% and 90% from 2008-2010; TSR between 94% and 95% from 2005-2009. In Nepal, CDR fluctuated between 70% and 75%; the TSR from 88% to 90%. In Brazil, the CDR fluctuated between 84% and 88%; the TSR from 72% and 75%.

Among target vulnerable populations, reflecting the project's particular focus on underserved groups:
    • The 85 targeted tribal districts maintained their case detection rate at 79% in 2006 and 80% in 2011. According to additional information provided by the project team, lower-performing tribal districts in particular improved their case detection rates, as 25 of the 40 tribal districts that were below the 70% target at the start of the project period saw increases by the project close.
    • The 145 targeted poor districts increased their case detection rate from 55% to 67%, falling slightly short of the target of 70%.
    • The 85 targeted tribal districts (districts with 50% Scheduled Tribe populations) maintained treatment success rates at 86% in 2006 and 88% in 2011. According to additional information provided by the project team, lower-performing tribal districts in particular improved their treatment success rates, as 23 of the 25 tribal districts that were below the 85% target at the start of the project period saw increases by the project close.
    • The 145 targeted poor districts slightly increased their treatment success rate from 85% in 2006 to 89% in 2011.
    • 49% of MDR-TB patients successfully completed DOTS-Plus treatment, falling short of the target of 70%. The ICR (page xii) suggests that patients in the initial phases of the scale-up were likely to be suffering from advanced states of the disease, while the 24-month course of treatment and adverse drug reactions caused defaulting.


To reduce the incidence of TB / To reduce the incidence of smear-positive TB in the zones where DOTS has been under implementation for five or more year is rated Substantial, due to evidence of improved quality of TB services, which likely contributed to reduced incidence. WHO-estimated outcomes over the project period also suggest reduced incidence due to the national TB program, for which the Bank provided major support.

Outputs
    • Training of 35 IEC officers.
    • Conducting of IEC activities, including baseline survey and media campaigns.
    • Establishment of network of National and Intermediate Reference Laboratories (including one national, 31 state, and 3 private). Over 13,000 designated microscopy centers were also established (including one per 100,000 population and one per 50,000 population in tribal and hilly areas).
    • Training of 334 TB Officers.
    • Development of accreditation system to monitor the quality of diagnosis and staff performance. However, the ICR (page 23) reports that poor quality of lab infrastructure and high turnover rate among lab technicians hinder overall progress in improving laboratory services.
    • Systematization of monitoring and evaluation system (including introduction of web-based reporting system at the state level).
    • Staffing of 90% of endorsed District TB Officer positions, 82% of DOTS Plus and HIV/TB Supervisor positions, 94% of Senior TB Lab Supervisor positions, and 100% of data entry positions.

Outcomes
    • The Annual Risk of TB Infection (ARTI) in zones were DOTS has been implemented for more than five years decreased from 1.5% in 2005 to 1.1% in 2012, surpassing the target of 1.34%.
    • The proportion of health care providers who were aware of DOTS treatment increased from 52% in 2007 to 97% in 2010. Community awareness of DOTS (i.e. DOTS is the correct treatment for TB; DOTS is available for free at government health facilities) also increased during the project period.
    • There were no drug stock-outs during the project period.

The following WHO estimates also reflect achievements in reducing the incidence of TB:
    • The annual TB incidence rate decreased from 209 per 100,000 people in 2005 to 181 per 100,000 in 2011.
    • The annual TB prevalence rate decreased from 358 per 100,000 people in 2005 to 249 per 100,000 in 2011.
    • The annual TB mortality rate decreased from 36 per 100,000 people in 2005 to 24 per 100,000 in 2011.

5. Efficiency:

Substantial. The project design was based on a widely accepted cost-effective intervention (DOTS). Specifically for India, cost-effectiveness studies for the RNTCP program estimated that the cost per Disability-adjusted Life Year (DALY) gained ranged from US$19 - $63. The ICR reports that this range is considered to be at the low end of costs when compared to other public health interventions. Other studies estimating the cost-benefit ratios for the RNTCP reported economic benefit of US$115 to US$191 per US$1 spent.

Evidence of efficient use of project resources other than direct costs of DOTS treatment (i.e. IEC activities, lab quality interventions) is not reported in the ICR The use of a targeted approach to increase coverage of unreached populations may have contributed to increased efficiency, although more resources were spent for these targeted interventions compared to reaching the general population.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


Relevance of the project objectives is rated High, while relevance of the project design is rated Substantial. Achievement of the objective to increase coverage of TB treatment/ to reach global targets in case detection and treatment success rates is rated Substantial, due to evidence of improved coverage among targeted sub-populations (including tribal and poor districts). Achievement of the objective to reduce TB incidence is rated Substantial due to evidence of decreased risk of TB. Efficiency is rated Substantial due to evidence of cost-effectiveness of the primary project intervention (DOTS).

a. Outcome Rating: Satisfactory

7. Rationale for Risk to Development Outcome Rating:


Strong political commitment to addressing TB is likely to be sustained, in part reflected by the government's development of the RNTCP National Strategic Plan for 2012-2017. The ongoing integration of TB services within the broader National Rural Health Mission program, along with planned increases in domestic funding for the health sector, will also contribute to the sustainability of the project activities. While there are still some capacity constraints at the decentralized level, central management capacity remains strong, supported by strengthened fiduciary and monitoring systems.

a. Risk to Development Outcome Rating: Negligible to Low

8. Assessment of Bank Performance:

a. Quality at entry:

The project design for this next phase of the national TB program drew heavily from a 2006 joint donor mission, which recommended the following: more attention was needed on quality of services; systems for procurement, human resource development, and monitoring needed strengthening; state-level capacity needed to be developed; and improvements were also needed in community awareness of TB services, coordination with HIV/AIDS services. management of MDR-TB, and private sector participation. These recommendations were clearly reflected in the project design. Support was closely coordinated with international partners to ensure technical quality and to enable scale-up. The risk assessment, which rated the project risk as overall Significant, was comprehensive. Effective mitigation measures were identified for the more significant risks, which included inability to monitor achievements among targeted groups, deficiencies in the procurement system, and lack of interest among private providers in using DOTS. The risk of inadequate capacity at the state level materialized (including shortage of staff, delays in hiring, and high turnover) and had a negative impact on project implementation, as the rapid expansion of the RNTCP II program could not be matched by human resources. M&E arrangements relied on existing national systems, which were already well-established and well-harmonized among the various donors, and on dedicated monitoring staff in the various national, state, and district level TB units. However, there were some shortcomings in the choice of indicators, given the significant disparities in TB outcomes among the districts (see Section 10).

Quality-at-Entry Rating: Satisfactory

b. Quality of supervision:

Annual joint missions were coordinated by the Bank team, which provided an effective coordinating mechanism for the government and donors. The ICR (page 29) notes the very positive feedback from counterparts and partners regarding strong Bank supervision. Fiduciary management was overall satisfactory, with particular attention given to addressing procurement challenges and financial management difficulties arising from the merger of the TB program with the National Rural Health Mission program. Although local procurement capacity was not developed to the extent planned, there were no major procurement problems related to the external contracted agency . Shortcomings in M&E (choice of indicators) were not sufficiently corrected.

Quality of Supervision Rating: Satisfactory

Overall Bank Performance Rating: Satisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The national government has continued to demonstrate strong commitment to the project objectives, with a high level of ownership over the RNTCP program. The government ensured supportive policy and institutional arrangements. Counterpart funds were provided at a higher level than estimated.

Government Performance Rating: Satisfactory

b. Implementing Agency Performance:

The Central TB Division (CTD) within the Ministry of Health and Family Welfare demonstrated effective implementation capacity, which had been established through the prior phase of RNTCP implementation. The CTD conducted effective and regular monitoring of project activities, including using the information to identify low performing districts in need of more attention. Fiduciary performance was moderately satisfactory, including adherence to Bank recommendations on procurement. However, there were ongoing human resource shortcomings (staff shortages and delays in hiring) and delayed disbursement of funds in some states, due to the merging of the project management responsibilities with the National Rural Health Mission program.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The ICR (page 10) notes several deficiencies in the choice of indicators: (i) the denominator for the PDO indicators relating to district-level rates were flawed in that incidence estimates were based on surveys that divided the country into large geographic areas ("zones") rather than existing districts; (ii) out of 22 intermediate outcome indicators, 12 did not have specific target values and 8 did not have a baseline figure at project approval; (iii) the use of international targets for case detection and cure rates may not have been appropriate as these rates were already high, and increasing them even by a small margin would likely take more time than the project period. Also, the ICR (page 7) notes that reaching the stated global targets in all districts may have been ambitious given the wide disparities among the districts, with particular challenges faced by those with significant poor or hard-to-reach populations. Monitoring arrangements made use of the existing national system which was already well-established at the start of the project period, as well as dedicated monitoring staff at the various national, state, and district level TB units.

b. M&E Implementation:

M&E data was compiled at the state level using a computer-based system. Field supervision and reviews on implementation progress and achievement of outcomes were carried out on a regular basis, both by the CTD and the states. Joint donor missions were conducted on a regular basis.

a. M&E Utilization:

According to the ICR (page 10-11), the CTD and states regularly analyzed the monitoring data and provided feedback to districts for further analysis and corrective action. The feedback included validating program data, sharing implementation experiences by states and districts, and strengthening administrative and political support. M&E data were compiled into an annual report and made publicly available on the internet.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:

The project was classified as a Category "B" project and triggered the Environmental Assessment safeguard policy (OP/BP 4.01) due to occupational health and safety issues and medical waste management. Actions identified in an Infection Management and Environmental Plan were largely carried out, including training of health staff, guidelines on occupational practices, and a reporting system on medical waste.

The project also triggered the safeguard policy on Indigenous Peoples (OP/BP 4.10) due to outreach efforts to tribal populations. A Tribal Development Plan was prepared, based on a Social Assessment of TB patients and community members in tribal populations, migrant workers, slum inhabitants, and industrial workers. Actions included developing special measures to overcome information, access, and provider gaps to these populations.

Safeguards compliance on both measures was satisfactory with no major problems reported.

b. Fiduciary Compliance:

Procurement: A Detailed Implementation Review (DIR) had been carried out by the Bank for all health sector projects in the portfolio, during the early part of the project period. Although the timing of the Review contributed to a lengthy time period between appraisal and Board approval, the ICR (page 9) suggests that the Review findings led to strengthening procurement arrangements which had a positive impact on overall project quality and outcomes. External procurement agencies were contracted, initially as an interim measure while Ministry capacity was developed. Donor efforts to increase local procurement capacity had limited impact, although a Central Procurement Agency was eventually established. Misprocurement was declared on several contracts at the state and district level, due to ineligibility for Bank financing. Actions were taken by the Bank and CTD to prevent further incidents of misprocurement.

Of particular note, drug procurement was managed effectively, with technical assistance from WHO. Procurement processes included pre-dispatch and post-delivery quality testing of all procured pharmaceuticals, and sourced first-line TB drugs from manufacturers with WHO prequalified products. There were no drug stock-outs throughout project implementation.

Financial management: Financial management arrangements were based on the existing systems from the previous Bank TB project. The government took measures to integrate these separate TB systems into the common system for all health programs under the National Rural Health Mission umbrella. Although this reform effort is intended to have longer-term benefits, it led to some implementation delays (slow disbursement of funds and administrative inefficiencies) as well as delays in submission of audit reports. The ICR (page 13) reports findings of qualified external audit reports which included weaknesses in compliance with the internal control framework (especially with respect to the management of advances), preparation of bank reconciliation statements, and uneven quality of accounting records. These issues were eventually resolved through proactive efforts of the Bank team and the CTD.

c. Unintended Impacts (positive or negative):
None reported.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Satisfactory
Satisfactory
 
Risk to Development Outcome:
Negligible to Low
Negligible to Low
 
Bank Performance:
Satisfactory
Satisfactory
 
Borrower Performance:
Satisfactory
Moderately Satisfactory
There were ongoing human resource shortcomings (staff shortages and delays in hiring) and delayed disbursement of funds in some states.  
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

Lessons from the ICR, with adaptation from IEG:
    • Strong government commitment is essential for consolidating program successes and achieving more complete national coverage. In the case of this project, the government provided support to policy frameworks - MDR-TB treatment and joint TB/HIV coordination - which enabled outreach to vulnerable populations outside the reach of national programs.
    • Focused efforts are needed to increase the engagement of the private sector and NGOs in providing DOTS. In the case of India, a large number of TB patients in the private sector are not being reached by the national program and may create an obstacle to more complete national coverage.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:

The ICR is notable for its clarity and its clear focus on outcomes. The quality of the evidence is overall satisfactory, although additional evidence on outcomes among the targeted sub-populations would have strengthened the analysis. The project team subsequently provided clarification on these outcomes.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions