|1. Project Data:
ICR Review Date Posted:
|Tocantins Sustainable Regional Development Project
Project Costs(US $M)
Loan/Credit (US $M)
Cofinancing (US $M)
Board Approval Date
|Sub-national government administration (50%), Roads and highways (50%)|
|Rural services and infrastructure (25% - P)
Municipal governance and institution building (25% - P)
Environmental policies and institutions (24% - P)
Participation and civic engagement (13% - S)
Decentralization (13% - S)|
||ICR Review Coordinator:
||Robert Mark Lacey
|2. Project Objectives and Components:|
The project development objectives in the Project Appraisal Document (p. 4) were:”to contribute to raising living standards, reducing inequalities and regional disparities by improving access to markets, job opportunities, infrastructure and social services for rural communities in Tocantins' poorest regions, while ensuring sustainable use of natural resources and the protection of fragile ecosystems.”
The objectives according to the Loan Agreement (Schedule 2, p. 24) were: “to assist the Borrower in raising living standards of its rural poor population and in reducing inequalities and regional disparities among its inhabitants, by improving access to markets, job opportunities and infrastructure and social services for rural communities in the Borrower’s poorest regions, while ensuring sustainable use of natural resources and the protection of fragile ecosystems.”
The objectives in the Loan Agreement will be used as the benchmark for this evaluation.
b. Were the project objectives/key associated outcome targets revised during implementation?
The cost table by components in Annex 1 of the ICR is incomplete, the figures just refer to the Bank Loan amount. The project included following three components:
1. Participatory planning and management of regional and municipal development (appraisal US$6.5 million, actual US$4.1 million) to: (a) establish and strengthen mechanisms for local civil society participation; (b) set-up, in each region of the project, regional units of the Planning Secretariat (SEPLAN) to support the local and regional planning and management capacity; and (c) strengthen the state's capacity to undertake multi-sectoral state-level planning, administer public programs and monitor/evaluate the performance of such programs.
2. Environmental management (appraisal US$10.2 million, actual US$19.30 million) to: (a) strengthen the state's land management capacity; (b) consolidate the state's environmental protection system; and (c) support the regulation and promotion of sustainable land use.
3. Rural transport improvement (appraisal US$42.7 million, actual US$36 million) to: (a) strengthen the state's road management capacity; (b) define in a participatory fashion and improve, to all-weather condition, about 6,000 km of priority road sections of the main municipal road networks and improve about 120 km of high-priority state feeder roads; (c) promote cost-effective rural transport services; and (d) provide support to the project coordination unit.
d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost: Annex 1(a) of the ICR refers only to the Bank Loan amount. According to the project team the actual project cost was US$112 million.
Financing: The loan of US$ 60.0 million was fully disbursed. There were no other external sources of financing.
Borrower Contribution: The project team confirmed that the Borrower's actual contribution was US$52.0 million and not US$ 23.0 million as reported in Annex 1 (b) of the ICR. This was higher than the appraisal commitment of US$ 40 million.
Dates: On August 13, 2009, the project closing date was extended by one year from December 31, 2009, to December 31, 2010, to allow minor adjustments to the loan amount distribution and to use the unallocated category which constituted 11.5% of the loan amount. On December 16, 2010, the closing date was further extended by 10 months, until October 31, 2011 to allow the project implementation to take advantage of the dry period for completing physical works.
|3. Relevance of Objectives & Design:|
a. Relevance of Objectives:Rated high.
The objectives are relevant to the Bank’s Country Assistance Strategy and to Brazil's priorities. Both the 2008-2011 and 2012-2015 Country Partnership Strategies (CPSs) aim for increased efforts towards a more equitable and sustainable economy. The project directly addresses two of the four objectives of the later CPS, namely to enhance equity and focus on environmental sustainability. The objectives are also relevant to the State of Tocantins’ Pluri-Annual Plan focused on building core infrastructure that supports the lagging regions and aims to promote social inclusion.
b. Relevance of Design:Rated modest.
By improving the rural roads and promoting cost-effective rural transport services, it is plausible that the project could contribute to improving the access to markets, job opportunities, and social services for rural communities. However, the linkage to raising living standards, reducing inequalities and regional disparities is not spelt out in the results framework.
The project's environmental management component included activities to consolidate the state's environmental protection system and support the regulation and promotion of sustainable land use. These activities are directly linked to the project objective of ensuring sustainable use of natural resources and the protection of fragile ecosystems.
|4. Achievement of Objectives (Efficacy) :|
The project's objectives - (i) raising living standards of Tocantins' rural poor population, and (ii) reducing inequalities and regional disparities among its inhabitants are assessed below. Outputs are common to both objectives.
- The project improved 144 km of roads compared to the target of 120 km. However, the proposed output based rehabilitation and maintenance contracts were not implemented.
- The project supported the construction of bridges and culverts.
- The project financed the State Water Master Plan, a Hydro-geological Water Resources Master Plan and river water flow studies, and then operationalized recommendations through five river basin action plans and through creation of basin management committees.
- With project support, the Borrower carried out four zonings, including 3 thematic and 1 geographic, which establish measures for sustainable development with a view to ensuring that economic development activities take account of environment and natural resources protection and of biodiversity conservation.
- The project funded a large number of activities such as municipal development agendas and regional development agendas. 67 municipalities had finalized their municipal development agendas by project closure. During the initial stage, there were delays in the process of establishing regional development agendas. However, all four regions targeted by the project had prepared regional development agendas by project closure. The new four year State Development Agenda is based on a series of 9 regional development agendas.
- None of the six protected areas which were to be established under the project was set up. Preparatory activities for 18 areas were finalized. The ICR reports (p. 31) that the establishment of the protected areas was delayed by lengthy processes of consultation and community mobilization, political decisions, and handling of social issues to manage stakeholders expectations.
- The project planned to create four inter-municipal consortia. None was established.
- Numerous studies were carried out of the State fauna and flora, and program of biodiversity conservation were developed; preparatory work was carried out for legislation for the protection of fauna and the formulation of a State policy on fishing.
(i) Raising living standards of Tocantins rural poor population: rated substantial.
The following evidence on intermediate outcomes is obtained for the most part from two Impact Evaluation surveys carried out by the Borrower, the first in 2003-2005 (before the start of the civil works), and the second in July, 2011, four months before project closure. The results of the surveys are reported in the ICR.
- In the southeast, 92% of the households surveyed stated having all-weather access to a health facility, against 68% before the project.
- Travel time to health facilities was reduced. For example, in the Southeast region, the travel time to the nearest hospital was reduced from 2 hours to 1 hour and 30 minutes. In the north, the travel time to hospital was reduced from 1 hour 52 minutes to 1 hour 25 minutes.
- In the Southeast region, 94% of the households surveyed stated having improved access to education facilities, versus 79% before the project. The travel time to a preschool decreased marginally from 29 to 27 minutes. The time to reach a secondary school was reduced from 45 to 32 minutes. In the North region, 79% of the households surveyed stated having improved access to education facilities, versus 66% before the project. The travel time to pre-school was reduced from 21 to 13 minutes, and from 44 to 14 minutes for secondary school. In both regions, the travel time to the closest elementary school was not reduced.
- The surveys indicate that the project improved access to markets. In the Southeast region 18% of the production (the production does not only refer to crops but to any type of product) was destined to other municipalities after the project against 7% before the project. In the North region, 100% of the traded production was destined to other municipalities after the Project against 26% before the project. This is calculated based on the share of cash crops in total agricultural production. The ICR reports (p. 28) that data were not available to estimate the impact of road improvements on commodity prices.
- A household survey showed that in the Southeast, the average distance to the municipal center decreased from 38.5 to 22.5 km, while in the North the same average distance decreased from 18.5 to 10 km.
- A survey carried out by the Road Agency on 27 private firms contracted for civil works estimated that about 2600 temporary jobs were created, of which 84% were local and regional (55% beneficiary municipalities and 29% in neighboring municipalities) and 16% were jobs generated in the firm's headquarters. The duration of these job assignments varied between 4 to 8 months in the field and between 6 to 10 months in headquarters.
- The project attempted the formation of social capital through Rural Producers Associations. In southeast, the 11 Rural Producers Associations increased their average membership from 35 to 43 per Association. However, in the North, the number of Associations of Rural Producers declined from 20 to 11, and average membership decreased from 90 to 72.
- The project supported participatory mechanisms which relied on extensive consultations in each municipality on their municipal development agendas. This helped the communities in prioritizing investments on municipal roads. The beneficiaries voted on roads that the Government would improve that best responded to their needs. However, “the project inadvertently created unfulfilled expectations that could have been avoided with clearer communication and outreach” (ICR, page 23).
The ICR provides no evidence of improved living standards of the rural poor in Tocantins. It is assumed that the road improvements and their intermediate outcomes listed above would have raised living standards but the link is not made explicit and no indications are provided that living standards did in fact increase.
However, the project team subsequently provided IEG with the results of an Impact Evaluation which was not available in final form at the completion of the ICR. The project team reports that the methodology used in the Evaluation underwent a thorough peer review process in Fiscal Year 2013/2014, and that the Evaluation is expected to be published under the auspices of DEC. According to this Evaluation, the project's rural road interventions contributed to the creation of about 220 jobs in the interviewed rural communities. The project also had a likely positive impact on agriculture employment in two of the four regions (+17%) and in public service delivery in the other two. The Evaluation estimates that the monthly household income increased by R$100 or about US$50, and that while household income has tended to increase regardless of the project, the increase was larger where the project was implemented.
The project team further reported that the trends of a human development index (the IFDM or Indice FIRJAN de desenvolvimento humano, a composite municipal index based on health, education and income indicators) across the municipalities of Tocantins show an 9% growth in the municipalities of Eastern Tocantins (where the project was implemented) between 2005 and 2010, compared to a 6% growth in Western Tocantins.
According to the project team, while the health impacts are inconclusive, the educational impact of improved roads was positive in two of the four Project target regions (Bico do Papagaio and Southeast) corresponding to the regions where the surveys had the most relevant samples of control group. The number of girls attending school has increased by 0.03 girls per household (respectively about 0.05-0.06 girls per household) in these regions, a figure considered to be statistically reliable.
(ii) Reducing inequalities and regional disparities among the inhabitants of Tocantins: rated modest.
The ICR did not include any evidence on either intermediate or final outcomes related to this objective. It reports that the project generated employment opportunities, infrastructure and social services for rural communities in Tocantins' poorest regions that might have led to reducing inequalities. A bottom-up approach was adopted to foster rural communities empowerment in local development through participatory planning, and promoting better social inclusion and enhanced appropriation of economic opportunities through accompanying measures to local development initiatives. However, no evidence is provided of reductions in either income inequalities or regional disparities in income and wealth.
According to the project team, the improved rural road connectivity changed people’s transport modal choice. More people came to use public buses and individual cars after the rural road improvement. Numerically, households that used public buses to go to school have increased by 3% after the project compared to the control group. Use of individual cars to access nearest populated place, municipal center, nearest hospital and school has increased by 4-8% (values varied by each destination). Individual car ownership has also increased by 4%. Similarly ownership of bicycles increased by 8%.
The ex-post Economic Rate of Return (ERR) was 20.2% compared to the appraisal estimate of 18.5%. The ex-post economic analysis, which used the same methodology as at appraisal, focused on project roads (which accounted for 60% of total project cost). Investment costs were taken from the actual civil works contracts. No maintenance costs were included on the grounds that “structures put in place are expected to remain useful for decades, with basically no resources being needed for maintenance, an important factor considering the general lack of capacity of the municipalities to undertake such maintenance” (ICR, page 19). The benefits consisted of the estimated increase in producer surplus resulting from the project investments. The improvements to all-weather condition of the municipal roads were expected to increase this surplus through: (i) reduction of overall production costs, notably, reductions in transport costs, decrease in stocks, and reduction of losses related to transport and stocking; (ii) productivity gains, which varied according to the product; (iii) increased size of cultivated areas; and (iv) faster changes towards more productive agriculture in areas where available land is scarce (ICR p. 34).
Principal among the issues associated with this methodology are, first, it is unclear how the area of influence of the roads was estimated; second, it appears to attribute the whole increase in producer surplus in the areas of influence to the project-financed road improvements; third, the sources for the municipal-level agricultural production, price and cost estimates prior to and after the project are not clear, and there is no discussion of their reliability; and fourth, estimates purport to show a considerable increase in agricultural production, and hence of generated traffic on the rural roads. Despite this, no estimate of maintenance costs is included.
The project experienced a delay of one year and ten months due for the most part to operational and administrative inefficiencies. Given these, and the methodological issues with the cost-benefit analysis, efficiency is rated modest.
a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:
* Refers to percent of total project cost for which ERR/FRR was calculated