|1. Project Data:
ICR Review Date Posted:
|Sergipe State Integrated Project: Rural Poverty
Project Costs(US $M)
Loan/Credit (US $M)
|Agriculture and Rural Development
Cofinancing (US $M)
Board Approval Date
|General agriculture fishing and forestry sector (55%), Water supply (15%), Power (15%), Sub-national government administration (10%), Other social services (5%)|
|Rural services and infrastructure (29% - P)
Participation and civic engagement (29% - P)
Rural non-farm income generation (28% - P)
Rural policies and institutions (14% - S)|
||ICR Review Coordinator:
|John R. Eriksson
|2. Project Objectives and Components:|
a. Objectives:The PAD (pp.14. 49) begins with a statement that the Project Objective is "an integral part of the Government of Sergipe’s rural development with inclusion strategy." This phrase is a presumed characteristic of the Project Objective, but is not the objective itself. Following this phrase, the PAD sets our three specific Project Objectives.
"The project will ....
(i) improve the rural poor’s access to basic socioeconomic infrastructure,
(ii) raise incomes and capital assets through investments in productive activities and increased linkages between small producers and markets, and
(iii) strengthen the cross-sectoral integration of investments in rural areas through improved participatory planning and monitoring at the local, municipal and State levels."
The Loan Agreement, dated May 11, 2009, gives the following formulation of the project objective (p.6, Schedule 1):
"The objective of the Project is to increase social and economic opportunities for Municipalities’ rural poor by improving their access to basic social and economic infrastructure and thus contributing to the Borrower’s objective of improving its Human Development Index."
The first part of this definition is an elaboration of the first objective in the PAD formulation ((i) above), but the Loan Agreement formulation lacks the second objective in the PAD, to "raise incomes and capital assets through investments in productive activities and increased linkages between small producers and markets." This objective is considered by the ICR to be the "core of the project" (p.14). However, the Loan Agreement formulation adds a specific objective not found in the PAD. Namely, to contribute to the “Borrower’s objective of improving its Human Development Index.” As will be explained in sections 3 and 4 below the Human Development Index was never envisioned as an objective with baselines and targets in the project, but rather as a criterion for selecting municipalities to participate in the Project. Accordingly, IEG adopts the specific objectives as formulated in the PAD.
The PAD states a third specific objective, to "strengthen the cross-sectoral integration of investments in rural areas through improved participatory planning and monitoring at the local, municipal and State levels." The ICR treats this as a project objective and assesses it. The PAD and ICR also refer to this objective as a project component, “Component 3” (“Horizontal Integration,” briefly described in section 2-c below). The participatory aspect could be viewed as an objective but the processes involved are means to achieving the first and second project objectives. IEG will assess the third objective but recognize that it is also a means.
b. Were the project objectives/key associated outcome targets revised during implementation?
c. Components:The Project was the second operation of a planned Bank-supported two-phase Rural Poverty Reduction Program supporting the State of Sergipe. The Federal Government divided the planned operation of US$41.6 million into two loans of US$20.8 million each, with performance conditions for moving to a second phase. Due to fiscal non-compliance issues between the Federal Government and the State of Sergipe, the State could not proceed with an Additional Financing and Phase I closed in June 2006. Consequently, there was a gap of 3.5 years between the Concept Note Review (March 2005) and Board Approval (September 2008) of Phase II and another 10 months to Project Effectiveness in July 2009. The Project implementation period, including a six-month extension, was 3 years.
The Phase II Project had four components:
1. Community Subprojects
2. Institutional Development
3. Horizontal Integration
4. Project Administration, Supervision, Monitoring and Evaluation
Component 1: Community Subprojects (appraisal: US$23.46 million; actual: $US21.86 million)
To support, through matching grants to community associations, implementation of about 1,000 subprojects for (i) small-scale socioeconomic infrastructure and (ii) productive and market-oriented investments with technical assistance (small-scale agricultural and nonagricultural related). This Component was intended to improve the livelihoods and wellbeing of an estimated 100,000 people in about 670 communities. As under the first phase project, communities, through their associations and participatory Municipal Councils, were to identify, prioritize and execute investments. Some 70% of subproject investment resources were to be focused on the 41 poorest municipalities. The remaining 30 municipalities were to receive an estimated 30% of funds for subproject investments.
Component 2: Institutional Development (appraisal: US$1.38 million; actual: $US1.05 million)
To be executed by the implementing agency, the Sustainable Development Company of the State of Sergipe, and to include: (a) technical assistance and training to support mobilization and strengthening of community associations to identify, prepare, operate and maintain subproject investments; (b) capacity-building for municipal councils to manage responsibilities defined in the Project Operational Manual, including assessment and supervision of Community associations, participatory planning, financial management of community subprojects, and environmental management of small community subprojects; and (c) workshops and seminars for municipal councils and community associations to exchange implementation experiences and facilitate integration with other federal and state poverty reduction programs.
Component 3: Horizontal Integration (appraisal: US$0.82 million; actual: --- [no expenditure])
To provide technical assistance, consultancies, software acquisition, and systems installation to the State Secretariat of Planning to strengthen its results-based management capacity to improve planning, integration, monitoring and evaluation of public policies and investments for poverty reduction; to improve the alignment of public expenditures with the State’s development priorities, and to support modernization of state public administration.
Component 4: Project Administration, Supervision, Monitoring and Evaluation (appraisal: US$0.54 million; actual: US$1.68 million)
To finance operational costs of project administration, coordination, supervision, monitoring, and impact evaluation.
d. Comments on Project Cost, Financing, Borrower Contribution, and DatesProject Cost
The total actual project cost was US$24.59 million, before an IBRD Front-end Fee of US$0.52 million, which brought the total to US$25.10 million. This compares with an estimated total cost at appraisal of US$27.1 million, comprised of the sum of the costs of the four components, US$0.80 million for contingencies and US$0.05 million for a Front-end Fee (PAD, p.64).
The project was financed by an IBRD loan of US$18.49 million or 73.7% of the total cost (including front-end fee). The balance of US$6.62 million was contributed by the Borrower and Local Communities.
The Borrower contributed US$4.43 million or 17.6% of the total cost and Local Communities, US$2.19 million or 8.7%. The Borrower's share of total cost increased from 14.1% at appraisal to 17.6% at closing (Local Communities’ share declined slightly from 8.9% at appraisal to 8.7% at closing). The increase in the Borrower's share resulted from an absolute increase in its contribution from US$3.83 at appraisal to a contribution of US$4.43 million at closing, as well as from a decline in the IBRD loan from US$20.80 million at appraisal to the final amount of US$18.49 million. The decline reflects a cancellation of US$2.31 million, owing to a refusal by the Bank to extend the closing date beyond six months (ICR, p.12). The reason for the Bank position is explained in section 5 below, on Efficiency.
The project was approved 09/23/2008 and closed 06/30/2012 after a six-month extension.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:The relevance of objectives is rated substantial.
The assessment of relevance and achievement of objectives in the next two sections focus primarily on the first two specific objectives of the project (see section 2-a above): "(i) improve the rural poor’s access to basic socioeconomic infrastructure; (ii) raise incomes and capital assets through investments in productive activities and increased linkages between small producers and markets." The relevance of these objectives is substantial. At appraisal, the State of Sergipe – the smallest State in Brazil – ranked 23rd nationally in its Human Development Index (HDI) with 54% of its population still classified as poor. Rural poverty was especially severe with 50% of rural households depending on less than one minimum salary per month (about US$280 at the time) and 61% on less than two. While aspects of socioeconomic infrastructure had improved, with government programs taking over the provision of electricity, almost half of households still lacked piped water, and jobs and income generation had, with the demands of participatory community institutions, become an urgent focus of public policy.
The Bank's Brazil Country Partnership Strategy 2012-2015 noted relevant Government priorities for addressing climate change and other environmental concerns, ameliorating slum conditions and associated poverty, and investing in sustained water resource use (pp. 9, 24, 30, 32, 126). Of the 24 Bank lending operations identified in the Indicative Lending Program for FY 2012-2013, half are focused on States in the Northeast of Brazil and most deal with poverty, rural development or environment issues (pp.44-45). Of Economic and Sector Work (ESW) underway or planned, about a third is focused on the above subjects in the Northeast (pp.47-48).
The third specific objective, “strengthen the cross-sectoral integration of investments in rural areas through improved participatory planning and monitoring at the local, municipal and State levels," is modestly relevant since it is a means to accomplish the first and second specific objectives. It could be considered substantially relevant if “cross-sectoral integration” and “participatory planning and monitoring” are considered objectives but the relationship between these constructs as means and as objectives is not made clear in the ICR.
As noted above, a specific objective in the Loan Agreement is “contributing to the Borrower’s objective of improving its Human Development Index.” The components of a Human Development Index, such as infant mortality, are related to the first specific project development objective (socioeconomic infrastructure) contained in the PAD. But no attempt was made in the ICR to establish a linkage or measure it. Moreover, the Team points out that the components of the Human Development Index change slowly over time and would therefore not be relevant for the Project, even if change could be measured.
b. Relevance of Design:The relevance of design is rated modest. The first two Project Components --Community Subprojects and Institutional Development-- are relevant to the first and second specific project objectives.
The relevance of the third Component --Horizontal Integration-- is not clear. The PAD indicates that the Component is “to provide technical assistance, consultancies, software acquisition, and systems installation to the State Secretariat of Planning to strengthen its results-based management capacity to improve planning, integration, monitoring and evaluation of public policies and investments for poverty reduction; to improve the alignment of public expenditures with the State’s development priorities; and to support modernization of state public administration.” These activities could presumably contribute to the first and second specific project objectives, as would Component 4 (“Project Administration, Supervision, Monitoring and Evaluation”), which considerably overlaps Component 3.
The Team indicates that the purpose behind the Horizontal Integration concept was to build capacity in State Secretariats of Planning in States of the Northeast to manage integration of policies and programs across sectors for greater development efficiency and impact. Sergipe State declined to borrow for such activities but the Project Team indicates that one element of Component 3, the training of middle managers in the Secretariat of Planning, was conducted under the horizontal integration umbrella although not funded by the Project. In any event, the link is not apparent between the Horizontal Integration Component 3 and the third specific project objective, to “strengthen the cross-sectoral integration of investments in rural areas through improved participatory planning and monitoring at the local, municipal and State levels.” Moreover, the project funds projected in the PAD for Component 3 (US$0.82 million) were not spent. The ICR explains that the State Government (Secretariat of Planning) was “not committed” to the Component (pp.6, 21).
The Loan Agreement refers to a Human Development Index, and the PAD and ICR refer to such an index for Sergipe State as a whole and to a Municipal Human Development Index covering 71 municipalities in the State. This information was used to describe characteristics of Municipalities of the State and establish their eligibility to participate in the Project. But nowhere are these indices framed as objectives.
The linkage of the Results Framework Analysis to the project objectives is mixed. There are 21 so-called "Intermediate Outcome Indicators." It is difficult to see how some of them link to the 7 “Project Development Objective Indicators” (“PDO Indicators”) or to the three specific Project Development Objectives. Virtually all the so-called intermediate outcome indicators are output indicators, with the exception of Indicators 1 and 4, which could be viewed as intermediate outcomes and are stated in terms of families and communities benefited, respectively. However, no definition of "benefited" is provided. Two of the seven PDO indicators (2 and 3) are clearly outcome indicators and have implications for the project development objectives. The other five PDO indicators are at the intermediate outcome or output level. Their contribution to project objectives is conceivable, but in three cases assessment of achievement is not possible owing to measurement problems.
|4. Achievement of Objectives (Efficacy) :|
Efficacy is assessed by each of the three project objectives.
Objective 1: Improve the access of the rural poor to basic socio-economic infrastructure.
While efficacy at the output level had minor to moderate shortcomings, the lack of adequate data to assess outcomes results in a modest rating for Objective 1.
- 74% of project financing for socio-economic infrastructure subprojects went to the 41 poorest rural municipalities (target:70% against a baseline of zero)
- 22,006 families benefited with socio-economic infrastructure investments (target: 20,000 families against a baseline that over 27,000 families benefited in Phase I project)
- 547 community socio-economic infrastructure subprojects financed (target: 500 community subprojects implemented [revised from 1,000] against a baseline of 1,030 subprojects financed in Phase 1) Comment: "implemented" and "financed" are not necessarily the same.
- Female-led associations with financed subprojects comprised 30-35% of all subprojects (target: 35% against a baseline of 31% in Phase I)
- 40% of eligible quilombola (Afro-Brazilian minority) communities benefited from socioeconomic infrastructure investments (target: 50% against a baseline of 74% in Phase I project)
- All eligible quilombola communities represented in Municipal Councils (target: 75% against zero baseline)
- Household incomes of project beneficiaries changed by an unknown amount (target: increase of 30%, including incomes of "special" (minority) populations against an indeterminate baseline) Comment: it was not possible to calculate a percentage increase, owing to the lack of achievement and baseline data. This was one of two unambiguous outcome indicators among the 28 indicators in the ICR Results Framework Analysis.
- Value of household assets of project beneficiaries changed by an unknown amount (target: increase of 20% against an indeterminate baseline) Comment: this was also an outcome indicator, but it was not possible to calculate a percentage increase, owing to the lack of achievement and baseline data.
- Municipal Human Development Index. No information about change in this index during the project period.
Objective 2: Raise incomes and capital assets through investments in productive activities and increased linkages between small producers and markets. The ICR states that:
"this sub-objective was essentially the Project’s "core." Its success depended on the financing and management of sustainable productive activities which could be marketed in an organized manner ... under contractual arrangements." (pp.14-15) The performance of this objective at the output level was less satisfactory than for Objective 1. Taken together with the lack of information at the outcome level, efficacy of achievement of Objective 2 is rated negligible
- 30.1% or 74 of productive subprojects were led my women (target: 35% against a baseline of zero)
- 69% of beneficiary associations surveyed for the Final Evaluation accessed credit from other sources after their subproject was financed (target: 30% against a baseline of zero)
- 30% of households with productive investment subprojects had contracts with commercial buyers (target of 70% against a baseline of zero) The ICR explains that the implementing agency (Sustainable Development Company of Sergipe) "lacked technical capacity to lead an organized strategy and target was unrealistic" (p.iv)
- About 23% of productive beneficiaries received some form of regular TA (technical assistance) focused on small-scale agriculture (target: 80% of productive investments with regular TA against zero baseline)
- No results at the outcome level.
Objective 3: Strengthen the cross-sectoral integration of investments in rural areas through improved participatory planning and monitoring at the local, municipal and State levels. The ICR cites the following achievements for the third objective, but it is not clear how any of these results measure “cross-sectoral integration of investments through improved local participatory planning and monitoring.” Efficacy of achievement of this objective is rated modest.
- The project is claimed to have leveraged US$6.00 of additional resources for every US$1.00 of project investment funds from other programs and projects (target: US$5.00 against a baseline of zero)
- 86% of Municipal Development Councils "were involved in leveraging activities" from Federal and State programs (target: 50% against a baseline of zero) Comment: the ICR is internally inconsistent when it claims in a comment that "all 71 Councils" or "200%" leveraged resources from other programs (p.vii).
- An improvement of 30% in a “social capital index” was to occur by the end-of-project, but achievement was not measured since the data collected did not align with the index variables.
- The third indicator above (“social capital index”) might conceivably have been an outcome. But its lack of definition makes it impossible to determine its achievement. As noted above, it was not measured.
There are several other achievements cited by the ICR Framework that cut across the specific project objectives. In some cases, it is not clear whether the achievement is an outcome or an output.
Four training outputs are presented. The achievements met or exceeded targets in each case (although the training of State Secretariat of Planning personnel was not financed by the Project).
- Municipal Development Council Members
- State Secretariat of Planning middle managers and technical personnel
- Sustainable Development Company of Sergipe (the implementing agency) staff
- Beneficiary Associations entering the Project
Evaluation and Related Studies The Results Framework Analysis includes information on four evaluation-related indicators. These are discussed in section 10 on M&E.
Human Development Index
The ICR provides no way of measuring change in this index, cited by the Loan Agreement as the “Borrower’s Objective.”
Other The Framework includes several other indicators dealing primarily with internal project management, namely, issuing an Operational Manual, including Safeguards and anti-corruption policies, staff of the implementing agency visits to subprojects, and establishing and upgrading a functioning MIS (Management Information System, which was a Project subcomponent but funded entirely by the Government). All these targets were achieved.
Due to multiple factors adversely affecting project implementation, efficiency is rated modest. The cost-benefit analysis of efficiency by the ICR is limited to a random sample of 37 subprojects financed under Project Component 1 (the total universe was 794 subprojects financed by the Project, which accounted for almost 90% of total project costs). A weighted average is obtained to yield an estimated Economic Rate of Return (ERR) of 22%. This average hides the fact that the rates of return on individual subprojects are skewed. From 32%-to-46% of the analyzed subprojects yielded returns below the 10% cutoff employed by the Bank (implying a negative Net Present Value). The ex ante (before-project) ERR estimated at Project appraisal was 25%. This was based on economic analyses of previous projects in twelve categories that corresponded to those planned for the current project
However, the Project was beset with a number of implementation difficulties that adversely affected efficiency (pp.6-9). Some of these were apparent at entry and will be discussed further in section 8-a. Others relate to the Borrower and will be discussed in section 9. The ICR sums up the implementation of the project as “dysfunctional” (p.7). The following are the most salient manifestations of problematic implementation:
- Weak management on the part of the implementing agency, lacking a consistent, efficient approach.
- In comparison with the Phase I Project, the Phase II project experienced a marked increase in bureaucracy in Government layers, affecting all stages of subproject approval, resulting from a "host" of new legal, fiscal, environmental and documentation requirements.
- Government-required re-registration of community organizations and restructuring of Municipal Councils.
- A State Budget Law prohibiting funds transfer to non-governmental organizations deemed of “public utility," subsequently replaced by a 2-year waiting period before a new association could participate in the Project.
- A supply-driven approach promoted by the State Government that led to widespread acquisition of tractors, raising concerns about land management and environmental impact. Tractor cost almost doubled and the justification for using grants for this purpose was questioned.
- Institutional restructuring after the Midterm Review that "virtually paralyzed" project execution through to closing.
- Reduced priority over time given by the State Government to the Project in favor of Federal cash transfer programs.
- Disbursement was extremely slow until the final 6 months when almost half of planned disbursements occurred, which raised concerns of the Bank team about the quality of subprojects..
- Actual cost of Component 4 (Administration) exceeded appraisal cost by 311% ($1.68 million vs. $0.54 million).
The Project was extended a relatively brief time, from 12/31/2011 to 06/30/2012, although the Borrower had originally requested an extension of 18 months to permit full disbursement of remaining loan funds and apparently expected a further extension after the first six months (ICR, p.8). But the Bank refused to consider any extension beyond six months and $2.31 million of the loan was canceled. Even with the six-month extension, the effective implementation period of the project was only three years.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated