Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - Rural Women's Development


  
1. Project Data:   
ICR Review Date Posted:
05/30/2006   
PROJ ID:
P044449
Appraisal
Actual
Project Name:
Rural Women's Development
Project Costs(US $M)
 53.8  na
Country:
India
Loan/Credit (US $M)
 19.5  10.43
Sector, Major Sect.:
General agriculture fishing and forestry sector, Central government administration, Health, Other social services, General water sanitation and flood protection sector,
Agriculture fishing and forestry; Law and justice and public administration; Health and other social services; Health and other social services; Water sanitation and flood protection
Cofinancing (US $M)
 19.2  8.21
L/C Number:
C2942      
   
Board Approval (FY)
  97
Partners involved
IFAD 
Closing Date
12/31/2003 06/30/2005
         
Evaluator: Panel Reviewer: Division Manager: Division:  
Nalini B. Kumar
Ridley Nelson Alain A. Barbu IEGSG

2. Project Objectives and Components:

a. Objectives
The overall objective of the project was to strengthen processes that promote economic development of women and create an environment for social change. The specific objectives were to (a) establish women's self-help groups (SHGs) which build self-reliance and self-confidence and provide them greater access to and control over resources; (b) sensitize and strengthen the institutional capacity of support agencies (government, NGOs and banks) to proactively address women's needs; (c) increase the incomes of poor women through their involvement in income generating activities, thereby contributing to poverty alleviation; (d) develop linkages between SHGs and lending institutions to ensure women's access to credit financing; and (e) improve access to better health care, education and drudgery reduction facilities.

Project objectives were not revised.The project was designed to be implemented in 6 states, Haryana, Bihar, Gujarat, Karnataka, Madhya Pradesh and Uttar Pradesh but was actually implemented in 9 states as 3 states, Uttar Pradesh, Bihar and Madhya Pradesh, were bifurcated during the implementation period of the project.

b. Components (or Key Conditions in the case of Adjustment Loans):
The project had four components:
1. Institutional capacity building for Women's Development (appraisal estimate for total cost US$14 million , actual total cost US$7.45 million ). To support the establishment of self-reliant women's groups based on saving and credit, building capacity of new and existing SHGs through training and awareness raising and assistance to SHGs to establish a track record of sound credit management to make them attractive clients to lending institutions.
2. Establish support mechanisms for women managed income generating activities (appraisal estimate for total cost US$ 20.3 million, actual total US$ 5.5 million). To provide support for SHGs to access group loans from lending institutions for investments in income generating activities, both on-farm and non-farm, following demonstration of capacity for credit management based on the group's own savings.
3. Establish mechanisms to access social programs and leverage funds for community assets' creation (appraisal estimate for total cost US$ 7.8 million, actual total cost US$ 2.32 million). To assist in improving women's access to social services such as health and child care and creating or improving community assets such as drinking water and multipurpose meeting halls by leveraging contributions from on-going government programs or the community.
4. Provide effective project management system (appraisal estimate for total cost US$9.0 million, actual total cost US$6.41 million) for effective implementation of project activities with the support of a comprehensive monitoring and evaluation system.

c. Comments on Project Cost, Financing, Borrower Contribution, and Dates
At appraisal, total project cost was estimated to be US$53.8 million, of which US$19.5 million was to come from IDA, US $19.2 million from IFAD, US$3.7 million from Government of India/State Governments, US$8.5 million from institutional finance and US$2.9 million from beneficiaries. The ICR does not provide complete information on actual total project cost. There is no information in the completion report on whether the institutional finance amount (U$8.5 million) or the beneficiary contribution (US$2.9 million) materialized.There is also a discrepancy in the total IDA $ amount at appraisal as noted in the appraisal document and the completion report--the appraisal document notes the amount as US$19.5 million (same as the Controller's database) whereas the completion report notes it as US$18.41 million. The Controller's database reports that US$7.03 was canceled, whereas the completion report notes that US$7.33 million was 'surrendered' however there is no explanation in the completion report as to why that happened. It is also not explained why the IFAD loan was significantly less than the original amount.

Though the project was approved in March 1997 it did not become effective until April 1999 owning to lack of Cabinet Clearance from Government of India and noncompliance with several conditions of effectiveness. The project closing date was extended by a year and a half because of the late start of the project.


3. Relevance of Objectives & Design:

The project objectives were substantially relevant to Government of India's goals of making women equal partners and participants in the development process and was consistent with the Department of Women and Child Development's women-in-development strategy emphasizing women's increased access to basic amenities and services, addressing issues of gender bias and social constraints in access and creating an effective demand for services and linking economic and social services to enhance effectiveness. The project was also consistent with the Bank's Country Assistance Strategy for FY 96-98 emphasizing the need to support the government's development policies aimed at reducing poverty, facilitating greater private sector participation and accelerating human resource development with special attention to gender issues.

The project strategy emphasized a combined focus on promoting women's empowerment and development activities to have a broader impact on the lives of poor women.While the project components were mostly in line with the objectives and spanned a range of relevant activities from mobilization and self help group formation to institutional capacity building, the project design was complex and as noted in the QAE, was weak in taking into account borrower ownership and capacity of the executing agency to implement the project.

4. Achievement of Objectives (Efficacy) :

Specific objectives
(a) establish women's self-help groups (SHGs) which build self-reliance and self-confidence and provide them greater access to and control over resources.Substantially Achieved. A total of 17,647 self-help groups were established against an original target of 7,400. However, it is difficult to tell on the basis of evidence provided how far the project helped build self-reliance and self-confidence among women. The ICR does note that 89% of groups reported ability to travel outside the village for reasons of training, visits, meetings etc. as compared with the baseline of 25% but it is not clear how far this can be attributed to the project. The ICR reports that several improvements in terms of women's access to increased incomes, improved school enrollment etc. occurred amongst both project and non-project villages. The ICR also notes that an assessment of the extent of awareness, perceptions and attitudinal changes among the members of self-help groups was not possible because of lack of data.

(b) sensitize and strengthen the institutional capacity of support agencies (government, NGOs and banks) to proactively address women's needs.Modestly Achieved. The ICR claims that the impact of institutional strengthening is reflected in the sustainability of the self-help group activities. However, five issues raised in different parts of the completion report raise concerns about whether the institutional capacity of support agencies was actually enhanced and whether they were proactively able to support women's groups: first, the ICR notes that more than 80 percent of the Business Counselling Centers were established during 2003/04 which was too late for the centers to prove their viability before project closure; second, the ICR notes that "Group members' satisfaction with the support and nurturing received from NGOs and banks as well as the guidance received for income-generating activities was rated as average by 52% and good by 37%. 11% groups claimed not to have received any support. The support from banks was rated as good by 30%, as average by 53% and 17% informed that no support had been received" (page 6) (It is not clear what average meant in the survey but it is assumed it meant not good and not bad i.e.about half way); third, the ICR notes that while 1213 clusters as compared to a target of 1244 were formed, only 5% of these have been able to provide forward and backward linkages to the members; fourth, the ICR also notes that "while the coverage of skills-based training is fairly good, this component seems to have suffered from the lack of linkage between skills training and support and guidance in how to prepare, plan and implement income generating activities. Linkages to market and marketing surveys have also been missing. The overwhelming risk is to train the self-help group members in skills, which they are not able to use or that they initiate income-generating activities without proper assessment of the economic soundness of an enterprise. " (page 9); fifth, the completion report notes several problems with project management such as recruitment and frequent turn-over of staff, lack of coordination and poor understanding of its role in the Lead Training Agency.

(c) increase the incomes of poor women through their involvement in income generating activities, thereby contributing to poverty alleviation. Modestly Achieved. While the self help group members include a higher proportion of women from disadvantaged groups, the ICR notes, "it is difficult to assess the impact of project initiated activities due to the absence of comparable data." (page 7) Non-project villages have also progressed economically and a large number of changes in the economic status of women have come about because of general economic improvement. The ICR itself notes that a reflection of the lack of linkage between skills training and support and guidance is the 'comparative low increase of income generated from these activities.'

(d) develop linkages between SHGs and lending institutions to ensure women's access to credit financing.Substantially Achieved. The number of self-help groups availing loans directly from financial institutions rose to 97% by project end.

(e) improve access to better health care, education and drudgery reduction facilities. Modestly Achieved. A total of 14,743 self-help groups are reported to have channeled benefits from various literacy, sanitation, health and forestry programs. However, the ICR notes it is difficult to assess the actual impact of the project on a range of the reported improvements, since it covers areas of indirect rather than direct project intervention, where a host of other factors are at play. Non project villages have demonstrated similar improvements to project villages and where there are differences the ICR does not report on whether they are significant.

Overall objective
to strengthen processes that promote economic development of women and create an environment for social change. Difficult to tell how far this objective was achieved on the basis of information presented in the completion report.

5. Efficiency:

At appraisal an ERR of 25% was estimated. However no recalculations were done at the completion stage. There is also very little information in the completion report to be able to assess whether the assumptions under which the ERR calculations were made at the appraisal stage hold. For example, there is no information on interest rate on lending from SHGs and interest rate on borrowing or other evidence to be able to assess the financial health of the SHGs.The ICR review has concerns that the benefit stream at appraisal may not hold. The calculations of the benefit stream at appraisal took into account economic models for 23 typical income generating activities. While the ICR provides no information on the kinds of activities supported it does acknowledge that there has been comparatively low increases of income generated from the activities supported (ICR page 9). Also, the delay in getting the support services going suggests an actual benefit stream relatively more lagged than was used in the appraisal document. The sensitivity analysis undertaken at the appraisal stage had noted that a decrease in project benefits by even 10% would bring the ERR down to 12%. Thus the ERR was highly sensitive. There are also questions about efficiency of resource use because of weaknesses in financial management. The ICR notes "Financial management performance remained weak throughout the project with weak financial reporting, internal control systems and inadequate corrective actions taken." ICR page 14.
6. M&E Design, Implementation, & Utilization:

The project design provided for a comprehensive M&E system, however the ICR does not provide information on the M&E system put in place. In addition, it was also visualized that a baseline survey would be undertaken and an impact evaluation carried out to assess poverty impact. While an impact evaluation was undertaken there is no information on its quality or methodology.
7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative):

While no safeguard issues are reported, the ICR does point to weaknesses in financial management,accounting and internal control functions which could have implications for fiduciary compliance.

8. Ratings:
ICR
ICR Review
Reason for Disagreement/Comments
Outcome: 
SatisfactoryModerately Satisfactory[the ICR's current four-point scale does not allow a "moderately sat." rating]. The main stated objectives were the process objectives and therefore this review gives more weight to these, including the institutional strengthening that supported the process. Less weight is given to the immediate SHG formation itself. There are some questions about how well the project enhanced the processes. In addition, with a very sensitive ERR at appraisal and given the doubts about income increases there are concerns about efficiency of resource use. For these two reasons the rating of moderately satisfactory best fits the evidence.
Institutional Dev.: 
SubstantialSubstantial
Sustainability: 
LikelyLikely
Bank Perf.: 
SatisfactorySatisfactoryBut only moderately so. There were weaknesses in quality at entry and a better assessment of borrower ownership and institutional capacity to implement the intervention would have provided for smoother implementation.
Borrower Perf.: 
SatisfactoryUnsatisfactoryFor several reasons: (i) The delay in Cabinet Clearance from Government of India and noncompliance with conditions of effectiveness caused considerable loss of momentum in commitment and pace of implementation leading to slow start-up and disbursement lags; (ii) unsatisfactory project management during most of the project and problems of delays in filling posts and frequent turn-over of staff that negatively affected project performance; (iii) weak financial management and internal control; (iv) insufficient delegation of authority to state level implementing agencies and unclear division of roles and responsibilities among different project partners that slowed project implementation in the beginning; (v) hurdles in release funds from Government of India to the states and from states to NGOs negatively affected implementation.
Quality of ICR: 
Unsatisfactory

NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness.

9. Lessons:

Based on the ICR several key lessons are identified.

(i) In a multi-state project, clearly defined functions and responsibilities of the Center and the individual states are necessary for successful implementation.

(ii)A robust monitoring and evaluation system that provides critical information on the process and outputs and outcomes is essential to ensure not only that the project is on track but also to assess how far the various components contribute to achievement of project objectives.

(iii) Mapping and assessing the expertise and capacities of partner organizations for implementing/supporting different project objectives is critical.

(iv) The project experience points to the need to tie women's training to market demand to ensure sustainability. This means not simply assessing market demand but also assessing demand at a price and estimating whether the to-be-trained women can deliver adequately at that price.


10. Assessment Recommended?  Yes

          Why?  It is difficult to tell how far the project achieved its overall objective effectively and efficiently on the basis of evidence contained in the ICR. A project assessment needs to be undertaken to: (i) verify the ratings, particularly the efficiency; (ii) draw lessons of experience; (iii) assess fiduciary compliance.

11. Comments on Quality of ICR:

The ICR is rated unsatisfactory for the following reasons:
  • As seen under section 4 it provides insufficient information to assess whether the project achieved the overall objective of strengthening processes that promote economic development of women and create an environment for social change;
  • Cost data is incomplete and there is no information on why there was so much variation in actual costs of components from those at appraisal. The ICR also fails to explain how the objectives could have been achieved when nearly half the IDA amount was actually disbursed. Appraisal cost estimates for components in the text do not match those in the table in Annex 2.
  • It draws on some key impact evaluation reports as evidence that project objectives were achieved but provides no information on the methodology that was used to collect data for these reports;
  • Borrower comments are not included. The ICR guidelines require that at least a summary of the Borrower's inputs be included in the ICR;
  • Contradictory evidence is presented in different parts of the report. For example, para 4.2.1 page 9 reports that a total of 1224 clusters were formed and about 255 (more than 20%) are providing forward and backward linkages to members. However, on page 7 the ICR notes that 1213 clusters as compared to a target of 1244 were formed and only 5 % of these have been able to provide forward and backward linkages to the members.
  • The ICR notes that the calculation of economic and financial rates of return are not applicable to the project and fails to provide any evidence on efficiency of resource use. The report also does not mention that an attempt at calculating ERR was made at the appraisal stage. The case for not doing an ERR ex post is weak since it was done ex ante which is harder. Ex post the types of investments made are known.
  • The ICR does not report fully on the key performance indicators noted in the appraisal document. In some cases, the unit of the outcome indicators reported in the completion report is different from that noted in the appraisal document. For example, the appraisal document emphasized reporting on various indicators at the SHG level such as volume of saving. However at the ICR stage information is available on total volume of saving generated under the project. Reporting at the ICR stage is also incomplete on the output indicators established at appraisal.

(ES-Rev4B-Dec/05)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions