Independent Evaluation - Home > Search

Implementation Completion Report (ICR) Review - State Judicial Modernization Project


  
1. Project Data:   
ICR Review Date Posted:
11/09/2012   
Country:
Mexico
PROJ ID:
P074755
Appraisal
Actual
Project Name:
State Judicial Modernization Project
Project Costs(US $M)
 37.5  16.5
L/C Number:
L7250
Loan/Credit (US $M)
 30.0  13.5
Sector Board:
Public Sector Governance
Cofinancing (US $M)
 0  0
Cofinanciers:
Board Approval Date
  07/01/2004
 
 
Closing Date
09/30/2010 12/31/2011
Sector(s):
Law and justice (100%)
Theme(s):
Judicial and other dispute resolution mechanisms (33% - P) Other public sector governance (33% - P) Law reform (17% - S) Access to law and justice (17% - S)
         
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Jiro Tominaga
Michael R. Lav Navin Girishankar IEGPS2

2. Project Objectives and Components:

a. Objectives:


    The development objective described in the Project Appraisal Document (PAD) is to support the improvement of institutional performance of judiciaries in a few states through National Bank of Works and Public Services' (Banco Nacional de Obras y Servicios Públicos: BANOBRAS's) credit program for state judicial modernization by learning while doing. In these states, the Project would: (a) strengthen institutional capabilities, organizational culture and knowledge; (b) strengthen efficiency and effectiveness of judicial services; (c) improve judicial transparency; (d) increase access to justice; and (e) support Project coordination, monitoring and evaluation, and learning, including consultation with Project stakeholders (p.2, PAD).

    The Loan Agreement presents the project development objective slightly differently. It states that the objective of the Project is to assist the Borrower with the implementation of its judicial modernization credit program by supporting the efforts of the Participating States to: (a) strengthen justice sector institutions and promote cultural change and knowledge sharing among such institutions and with civil society; (b) develop efficient and effective judicial services; (c) increase judicial transparency; (d) enhance access to justice for the poor and disadvantaged populations of the Participating States; and (e) disseminate good practices and learning-by-doing strategies within and among Participating States; and promote efficient monitoring and evaluation procedures, including consultation with Project stakeholders. (Schedule 2, Loan Agreement)

    The two formulations are not materially different, but the objective presented in the Loan Agreement is more specific. Therefore, this review assesses the performance of the Project against the objectives specified in the Loan Agreement. This is consistent with the approach taken in the Implementation Completion and Results Report (ICR).

b. Were the project objectives/key associated outcome targets revised during implementation?
No

c. Components:

There were five components at the time of project appraisal. The Project envisaged to fund subprojects or separate projects in participating states which include the activities in the following components.

1. Institutional improvement, organizational culture and knowledge diffusion (cost at appraisal $9.0 million, actual cost $3.4 million): The activities under this component emphasize strengthening institutional capabilities, development of a culture of service and increasing knowledge about justice institutions and their operations. They also help strengthening jurisdictional processes and raising awareness of the judicial function. This component encompasses: (a) improvement of the capabilities of judges and staff in such areas as strategic planning and budget programming, judicial organization, technology use, dissemination of institutional innovations, sharing modernization plans with stakeholders, encouragement of civil society to participate in the evaluation and feedback on institutional transformations and impacts; (b) awareness raising of judicial operators to act consistent with a culture of service and continuous improvement; and (c) sharing knowledge with local, national and international judiciaries and other institutions involved in judicial modernization.

2. Improving efficiency and effectiveness of judicial services (cost at appraisal $10.2 million, actual cost $5.9 million): This component addresses improving the efficiency of judicial branches and the effectiveness of judicial decisions to raise productivity and quality while reducing costs and delays. It encompasses: (a) better design, follow-up, control and evaluation of judicial policies with respect to jurisdictional and administrative management; (b) improvement of the judicial branch management model, procedures and organization; (c) design and implementation of a judicial career system; (d) systematization and automation of procedures focused on case management and documentation for the Superior Tribunal of Justice, first instance courts and other courts, and finance and human resources systems for the Branch’s administrative structure and/or Judicial Councils; (e) development of training, research, information and document centers for judges and operators; (f) development of information for decision making and interpretation; (g) provision of equipment and infrastructure for designing and implementing of information and communication technology (ICT) plan; (h) design and implementation of plan for construction, rehabilitation and remodeling of courthouses and judicial offices; and assistance for pre-investment studies on further enhancement of the efficiency and efficacy of the Judicial Branch.

3. Increasing judicial transparency (cost at appraisal $6.0 million, actual cost $2.0 million): This component encompasses: (a) creation of new or the strengthening of existing organizational units responsible for the dissemination of information about jurisdictional and administrative procedures, and for the management of judicial documentation to support judges and judicial operators; the carrying out of outreach campaigns to provide information about the functioning and structure of the judicial system; and the development of annual performance reports by the Judiciaries including relevant auxiliary institutions; (b) development of discipline and accountability mechanisms for judges and judicial operators; (c) organizing events disseminating the benefits of an enhanced transparency of judicial processes; and (d) carrying out studies and research on improving transparency about discipline systems, access to information, accountability systems and judicial ethics.

4. Strengthening access to justice for all users (cost at appraisal $7.0 million, actual cost $2.3 million): To increase accessibility to justice administration by reducing barriers to services, this component encompasses: (a) development of special programs for women, minors, indigenous peoples and other poor and socially disadvantaged users of the judicial services and small entrepreneurs; school orientation and education programs for children and adolescents to gain a greater understanding of justice administration; and community outreach seminars on gender-based violence, and access to justice for indigenous populations; (b) creation of new, or the strengthening of existing alternative dispute resolution institutional mechanisms; (c) development of innovative programs aimed at diversifying the range of judicial services; (d) development of training programs for public defenders, legal aid providers and staff of public prosecutor’s offices; (e) development of programs to improve the interactions between the judiciary and legal professionals; and (f) carrying out of studies for the strengthening of access to justice for the public at large, with special emphasis on the poor and disadvantaged populations.

5. Project Coordination, Monitoring and Evaluation and Learning (including pre-investment assistance).(cost at appraisal $5.0 million -- component dropped): This component encompasses: (a) support for the design and operation of the State project coordination units (PCUs), promotion of good practices and knowledge sharing; and (b) support for strengthening BANOBRAS's capability to supervise the project, conduct semi-annual M&E reviews, promote learning and prepare future projects.

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates

The project cost at appraisal was $37.5 million financed by an IBRD loan of $30 million with the Borrower financing the remaining amount. The project was appraised in April 2004, approved by the Board on July 1, 2004, and became effective on January 25, 2006. It was restructured in August 2008 which resulted in dropping the project coordination, M&E and learning component (Component 5 above). The project closed on December 31, 2011, 15 months after the original closing date, September 30, 2010.

The Loan Agreement was amended four times: (a) on January 19, 2007 to amend the currency conversion letter; (b) on September 4, 2008 to: (i) increase the percentage of financing under the Loan through the adoption of Mexico’s Country Financing Parameters; (ii) cancel Component 5; (iii) reallocate the proceeds of the Loan; (iv) extend the Closing Date to December 31, 2011; and (v) increase the amount of the Authorized Allocation of the Special Account; (c) on July 2, 2009, to accept BANOBRAS’ request for cancellation of the amount of US$16.5 million from the unwithdrawn Loan balance effective as of May 4, 2009; and (d) on January 19, 2010 to include the provision of works’ supervision as eligible works under subprojects. The Project was extended once from the original closing date of September 30, 2010 to December 31, 2011.

Component 5 was dropped in the restructuring because BANOBRAS did not have specific unit in charge of M&E of judicial reforms. Also, BANOBRAS internal financial policies did not allow external debt to be used to support routine credit management activities such as M&E. Instead, it was planned that each State Judiciary would carry out M&E through the line units in charge of implementing the Judicial Modernization Plans pursuant to the provisions of the relevant agreement to be entered into between BANOBRAS and each sub-national. The general M&E unit of BANOBRAS would conduct the supervision of such activities in accordance with the standard practice of a Bank loan. The Bank task team would coordinate its own M&E activities with BANOBRAS and with the participating States. The proceeds of the loan allocated to this component were transferred to Component 2 (Improving Efficiency and Effectiveness of Judicial Services).

In the original design, the eligible Subprojects would have had two sources of funding: the BANOBRAS credit line and the cash/in-kind contributions of the participating Sub-nationals. In the particular case of the only subproject financed by the Project of the Federal District Government (GDF)/TSJDF, the Federal Government rules on the use of public debt limited the Bank/BANOBRAS financing to goods and works. As a consequence, as part of the subproject arrangements entered into between BANOBRAS and the GDF/TSJDF, it was agreed that the GDF/TSJDF would finance with its own resources the technical assistance needed to carry out the non-work and non-good portion of the TSJDF Judicial Modernization Plan, in particular the advisory services required to make the integrated ICT solution operational.


3. Relevance of Objectives & Design:

a. Relevance of Objectives:

The 2004 Country Partnership Strategy (CPS) highlighted the importance of assistance to the judicial system, which provided inefficient and inadequate services. A series of government's development plans note the priority assigned to improvement in the justice sector. The 2008 CPS encompasses judicial reform as one of the goals under the Strengthening Institutions pillar. It notes that strengthening institutions remains one of the key country development goals that the Bank will continue to support through this Project. The relevance of objectives is substantial.

b. Relevance of Design:

A salient feature of the project is that it left flexibility in terms of the number of participating states and states' ability to prioritize among the building blocks for judicial modernization which are Components 1-4 described in Section 2.c. of this report. The learning-while-doing approach in developing states' own Judiciary Modernization Plans is intended to make strategic choices in these plans iteratively while ensuring broad participation by relevant staff. These approaches were intended to address diverse challenges in different states in a demand-driven manner. The effort by the task team to carry out the highly participatory processes that it went through to prepare this project. However, the wide participation by states did not materialize even though 16 Judicial Modernization Plans were prepared between 2003 and 2006. There were also four states which signed the Letter of Intent. In the end, only the Federal District accessed the BANOBRAS credit program.

    The main reasons for reluctance by the states to participate, according to the ICR (paragraph 24) were: (i) lack of coordination with the executive branches of the state governments which are legally responsible for states' financial decisions; (ii) inability to reach BANOBAS creditworthy status for some initial candidates and availability of more favorable alternative financing sources for others; and (iii) limited expertise and capacity within BANOBRAS in promoting judicial reform projects. There was also uncertainty related to on-lending terms and disbursement mechanisms. The Bank later started work to revise the description of the project to reflect the fact that only the Superior Justice Tribunal of the Federal District (Tribunal Superior de Justicia del Distrito Federal: TSJDF) participated in the Project. However, this restructuring was not finalized. Given that the original design had to go through multiple changes, this review considers that the relevance of design was modest.


4. Achievement of Objectives (Efficacy) :


The overarching objective of the project was to assist the Borrower (BANOBRAS) with the implementation of its judicial modernization credit program by supporting the efforts of the Participating States to: (a) strengthen justice sector institutions and promote cultural change and knowledge sharing among such institutions and with civil society; (b) develop efficient and effective judicial services; (c) increase judicial transparency; (d) enhance access to justice for the poor and disadvantaged populations of the Participating States; and (e) disseminate good practices and learning-by-doing strategies within and among Participating States; and promote efficient monitoring and evaluation procedures, including consultation with Project stakeholders (Schedule 2, Loan Agreement). Given that only the TSJDF subproject was financed, the project did not achieve the full PDO envisaged at the time of appraisal.

As allowed for in the project design, the GDF/TSJDF selected a few of the activities eligible for financing under the Project. The TSJDF subproject focused financing on infrastructure remodeling (no construction envisaged) and information and communication technology (ICT) equipment. Due to constraints imposed by the internal regulations of SHCP for GDF public debt, the TSJDF Subproject could not finance technical assistance or training out of project funding but the TSJDF conducted the support activities with its own budget resources.

The outcome/impact indicators set in the PAD were: stakeholder satisfaction on the efficacy of modernization activities; and improved capacity building and communication practices. No target or baseline was set in the PAD for either item. To gauge stakeholder satisfaction on the efficacy of modernization activities, the TSJDF subproject tracked the number of user service applications uploaded to the institutional ICT network. It aimed to have 10 user service applications uploaded that serves as the backbone of the management model concerning increased user confidence and stakeholder satisfaction in TSJDF services. The subproject resulted in 5 applications being uploaded. Twenty two public kiosks have access to the status of judicial decisions and location of cases in target courts. A survey on TSJDF user’s satisfaction suggests that more than half of respondents (55.25 %) were satisfied with TSJDF services. The level of satisfaction with the image, attention and functioning of the target criminal courts (Reclusorios Norte, Oriente, Sur and Santa Marta Acatitla) was 63%. Users’ satisfaction reported by this survey is higher than the trust in the justice system as reported for the whole country by Americas Barometer, LAPOP, in 2010, in which Mexico scored 35.5%.

As for improved capacity building and communication practices, the TSJDF subproject followed the new ICT network that provides timely information on demand and supply of justice services in the courts targeted by the subproject; training and knowledge sharing for implementation of the new management model; and the inter-institutional arrangements with other sector agencies. The achievements in these areas seem to be assessed based on some of the outputs produced as results of the subproject. These outputs are associated with the activities to achieve the subobjectives (a)-(e) indicated in the PDO. The key results for each sub-objective are as described below.

Subobjective (a) strengthen justice sector institutions and promote cultural change and knowledge sharing among such institutions and with civil society: The TSJDF subproject aimed to strengthen institutional capability for continuous service delivery in four (4) critical court locations through the integrated ICT network. It aimed to: (i) make 69 target trial courts capable of processing statistics online; (ii) make 81 trial, appeal and judgement execution courts able to share case tracking information; and (iii) enable 80 percent of trial courts to have access to e-mail, intranet and extranet; and (iv) have the Institutional ICT network cover 80 percent of target TSJDF staff (1,600 staff). The key modernization activity financed was an integrated ICT solution to support the TSJDF's new information model for decision-making which is now operational in 69 courts in the four locations selected for the subproject (Reclusorio Norte, Reclusorio Sur, Reclusorio Oriente, and Santa Marta Acatitla) in low income areas. This new system provides: (i) information on financial material and human resources optimization; (ii) data to be used to prepare and implement a results-based budget linked with previously defined indicators; and (iii) tools to present and support the budget before the Secretariat of Finance and the Legislative Assembly of the DF. Statistics can be processed online in 69 target criminal trial courts, 40 criminal peace courts, 2 sentence execution courts, 15 juvenile courts and 26 appellate courts. ICT network covers 100 percent of target TSJDF staff who also have access to e-mail, intranet and external websites. In parallel, the TSJDF, with its own resources developed and updated the court services that would be operating through this ICT network. No cultural change or knowledge sharing activities were included in the subproject other than the program sponsored by the Bank and Paris 21.

Subobjective (b) develop efficient and effective judicial services: The subproject aimed to: (i) reduce the average processing times in target trial courts from 2.5 months to 2.0 months; (ii) improve target courts clearance rate from 77.8 percent (2008) to 100 percent; (iii) increase the total workload from 18,180 cases a year in the target courts to 20,000 cases a year; and (iv) make 35 target courts capable of carrying out virtual trials and videotaped oral-based trials. The subproject reduced the average processing times in target trial courts to 2.3 months; the target courts' clearance rate is 108.6 percent; total criminal courts workload was 20,589 in the end of 2011; however no target courts are yet to be capable of carrying out virtual trials which require legislative changes. The subproject did not include the organizational and management model for the administrative structure of the TSJDF, a coordination and communication model, human resources systems, or developing support units for training, research and information. Relevant activities were covered by TSJDF's own resources.

Subobjective (c) increase judicial transparency: The subproject had pursued two indicators: increased availability of public information; and increased opportunities for community engagement. As for the increased availability of public information, the subproject targeted: (i) 6 court databases to operate through the integrated ICT network developed under the subproject and accessible to court staff and general public; (ii) 4 judicial statistics reports compiled quarterly for decision-making purposes and published annually for public dissemination; and (iii) up to 19,000 judicial decisions a year disseminated through the integrated ICT network. Five court databases are now operating through the integrated ICT network developed under the project and are accessible to court staff; monthly judicial statistics reports are made available for decision making purposes, since 2010; annual statistics reports have been published and disseminated; the statistics portal in the TSJDF webpage has 53 data banks available to public; and there is an online bulletin making available the status of judicial decisions and location of cases in courts. In eight TSJDF buildings, 22 kiosks are available for citizens for case status consultation. The TSJDF website has a transparency portal that permits public public access and consultation of the financial asset declaration of TSJDF judges and staff and the TSJDF transparency indicators. In addition, 620,896 judicial decisions (including final judgments) have been filed and are available for a fee to the TSJDF staff. Access to these decisions by the public was expected to be available by June 2012.

As for increased opportunities for community engagement, the subproject did not have specific community engagement activities.

Subobjective (d) enhance access to justice for the poor and disadvantaged populations of the Participating States: The subproject had no specific vulnerable groups access activities other than adapting infrastructure to users' needs, including informational and orientation centers at the four targeted courts located in the vicinity of vulnerable group areas. Out of the 302,220 annual users of the targeted buildings, 58.8 percent are persons whose family income is less than US$450 per month, and 30.8 percent are users whose family incomes is between US$450 and US$1,000 per month. 39.8 percent of all users are women.

Subobjective (e) disseminate good practices and learning-by-doing strategies within and among Participating States: and promote efficient monitoring and evaluation procedures, including consultation with Project stakeholders: The component that includes the relevant activities was dropped.

Overall, although the TSJDF subproject generated some positive results, the areas it covered were not as broad as originally planned in the PAD. Several subobjectives were not included in the TSJDF subproject as oftentimes these areas were addressed by TSJDF own resources. The PAD envisaged that the Project would support "a few" states, but only one subproject materialized. Given these points, the efficacy of the project is rated modest.

5. Efficiency:


Net Present Value/Economic Rate of Return, Financial Rate of Return, and Cost Effective Analysis criteria were not applied to this Project. The Summary of Project Analysis section of the PAD argues that the returns of this project to be realized through improved delivery of judicial services, efficiency, and transparency of the superior courts. The examples of such returns include the reduced costs of litigation and improved collection of outstanding business liabilities. Indirect benefits include the stable and uniform application of the rule of law, strong business confidence and a stable investment climate. It also notes the expected benefits from lowering the levels of corruption, equitably applying the rule of law, and predictably and uniformly interpreting commercial laws. However, these impacts were not monitored and measured, so it is not possible to measure project efficiency based on these assumptions. Based on the assessment that the project achieved the development objectives partially in coordination with the activities financed by TSJDF's own financing, this review rates project efficiency to be modest.

a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:


Rate Available?
Point Value
Coverage/Scope*
Appraisal:
No
%
%
ICR estimate:
No
%
%

* Refers to percent of total project cost for which ERR/FRR was calculated

6. Outcome:


Given the assessment of the relevance of objectives (substantial), relevance of design (modest), efficacy (modest), and efficiency (modest), this outcome is rated moderately unsatisfactory. The TSJDF subproject generated some important results through investments on its new ICT system. However, it was not able to attract states to participate, resulting in partial achievements of the PDOs. The ICR assesses that part of the reasons for non-participation by states is associated with the nature and terms of BANOBRAS credits, pointing to a weakness in the relevance of design.

a. Outcome Rating: Moderately Unsatisfactory

7. Rationale for Risk to Development Outcome Rating:

The TSJDF subproject received strong support from the Chief Justice who was recently re-elected for an additional four years. The TSJDF management and staff have shown strong commitment to the project including allocation of own resources to implement relevant activities.

a. Risk to Development Outcome Rating: Negligible to Low

8. Assessment of Bank Performance:

a. Quality at entry:

The project was prepared through considerable efforts for consultation with state judiciaries. However, the project suffered from less-than-expected participation of states due partly to the nature of BANABRAS credits. The area of BANOBRAS' expertise was in large infrastructure projects rather than in promoting judicial reforms. The ICR also points to the inadequate consultation with the state ministries of finance, which might have been able to identify the potential challenges associated with the line of credit conditions, as the executive branch makes financial decisions on state affairs. Based on these, this review rates the quality at entry to be unsatisfactory.

Quality-at-Entry Rating: Unsatisfactory

b. Quality of supervision:

After the August 2007 mission, the Project’s performance was rated unsatisfactory by the Bank and the possibility of cancellation was considered. The project had no disbursements for three years. Instead, the Bank initiated pro-active measures which include: (a) modifications of some key aspects of Project design (such as financial covenants, cancellation of component 5, and the extension of the closing date); (b) establishing an additional technical assistance program for the TSJDF on criminal justice to strengthen institutional capacities to implement a criminal reform package (comprising study tours, workshops and visits of international experts) partly intended to offset the high financial cost of borrowing from BANOBRAS; (c) facilitating the TSJDF engagement in a complementary program sponsored by the Bank and OECD (Paris 21) for strengthening capacity to generate and analyze relevant statistical information; and (d) cancelling US16.5 million from the Loan. In addition, a restructuring package was prepared in advance but the Bank decided not to pursue the restructuring because it was considered that there was not enough time before the Closing Date given the amount of coordination needed. It was also difficult to expect that a restructuring at that late stage would make a substantial difference in the assessment of Project PDO. Overall, the Bank showed proactivity and commitment in helping project implementation, but it could not finalize the full-scale restructuring. This review rates the quality of supervision as moderately satisfactory.

Quality of Supervision Rating: Moderately Satisfactory

Overall Bank Performance Rating: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance:

The Federal Government, which guaranteed this Loan, showed strong commitment with state justice systems modernization at a critical time of project implementation, when cancellation was being seriously considered. During the October 2007 meeting of the National Association of Judicial Operators (AMIJ), President Calderon reiterated the Federal Government’s commitment to State judicial modernization, and publicly requested BANOBRAS to establish the appropriate mechanisms to activate the Bank-financed Project to strengthen the States’ Judicial Branches. The Secretariat of Finance and Public Credit (Secretaría de Hacienda y Crédito Público: SHCP) took the lead on the Federal Government side to accomplish this objective in conjunction with BANOBRAS. A joint effort between BANOBRAS, SHCP, GDF, TSJDF, and the Bank was made to find realistic solutions to address the high financial costs that the GDF/TSJDF faced in participating in the project. However, it appears from the ICR and relevant materials that the shortcomings in the original design that had substantial impact on project outcomes could have been thought through during project preparation. Given these points, this review rates government performance as moderately unsatisfactory.

Government Performance Rating: Moderately Unsatisfactory

b. Implementing Agency Performance:

During project preparation, considerable efforts were made by BANOBRAS to involve as many stakeholders as possible to discuss policy options available and in the overall decision making process. This effort led to 16 State Judicial Modernization Plans--a record in terms of the number of participatory exercises attended by hundreds of local stakeholders, not only for the Judiciaries but from bar associations, academic institutions and civil society organizations. During implementation, a senior management team in the Technical Assistance and Multilateral Projects Unit of BANOBRAS took responsibility for project implementation, in close coordination with other functional and territorial units of BANOBRAS that cooperated closely with the Bank team. A close working relationship between BANOBRAS and the TSJDF helped improve performance in the last two years of Project implementation. The delays of the first years of Project implementation were partially offset by the rapid implementation of the “turn-key” contract that packaged all goods, works and services between January and December 2010 with the assistance of the Bank’s procurement specialist.

BANOBRAS, GDF and the TSJDF worked closely with the Bank’s team in implementing the TSJDF subproject. Appropriate levels of review and approval were in place; financial accountability was observed; expenditures were duly authorized; and documentation was properly maintained for periodic review. The Project did not suffer from any counterpart funding problems. The Loan Covenants were fully complied with (including the financial covenants related to audits), with no delays. Audit reports were satisfactory, and financial and procurement ex-post reviews were also acceptable.

The TSJDF provided overall policy guidance through the Chief Justice working in conjunction with members of the Judicial Council. The Chief Justice entrusted his Oficial Mayor (General Manager) the overall responsibility for Project implementation. The technical units of TSJDF performed well. The responsible Directorates: (i) responded to the demands for assistance from the TSJDF field units in carrying out subproject components in four different locations; (ii) ensured an effective liaison with the decision-making levels of the TSJDF and promoted the adequate utilization of TSJDF’s installed institutional capacity; and (iii) ensured compliance with the Bank’s requirements in the areas of procurement, disbursement and financial management.

Based on the close coordination between relevant agencies and appropriate handling of day-to-day project implementation achieved by the implementing agencies (BANOBRAS implementation responsibility and TSJDF), this review rates the implementing agency performance as moderately satisfactory.

Implementing Agency Performance Rating: Moderately Satisfactory

Overall Borrower Performance Rating: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design:

The baseline and target for the indicators to be monitored in the Project were not specified in the PAD. This is likely due to the project design that is based on demand from states which may or may not participate in the Project. The indicators were also difficult to measure and were not particularly connected with potential Project outputs or activities. The TSJDF identified the indicators more specifically, which included some outcome indicators (e.g. average processing time) and those defined based on outputs (e.g. the number of user service applications uploaded).

b. M&E Implementation:

The TSJDF subproject developed its own robust set of input, output and impact indicators. These indicators were tracked during project implementation. A specific coordination and implementation unit was established in the TSJDF. It had full-time staff in charge of monitoring and evaluation responsibilities, and coordination with the various units of the TSJDF engaged in the implementation of the subproject.

a. M&E Utilization:

The TSJDF subproject indicators were tracked to help effective implementation of project activities. The single ICT network that allowed the integration of statistical data generated by all the TSJDF courts, and the Paris 21 program that helped improve the statistical analysis and reporting capacity of the Central Statistics Unit of the TSJDF.

M&E Quality Rating: Substantial

11. Other Issues:

a. Safeguards:

There was no reported safeguards related issues during project preparation and implementation. The project was rated as Category C and did not trigger any safeguards policies.

b. Fiduciary Compliance:

There was no reported fiduciary compliance issues.

c. Unintended Impacts (positive or negative):

The ICT platform introduced through the TSJDF subproject had effects broader than those envisaged in the project. Using the ICT platform, statistical information is currently collected from all of the 278 TSJDF courts, 17 justice support offices and 13 administrative offices--well beyond the target 69 courts. Socio-economic information is collected by TSJDF for the National Institute of Statistics and Geography (INEGI). The ICT platform has also made the implementation of a pre-trial detention system, which enhanced monitoring of the defendants rights and to better controlling compliance with due process standards.

d. Other:



12. Ratings:

ICR
IEG Review
Reason for Disagreement/Comments
Outcome:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Risk to Development Outcome:
Negligible to Low
Negligible to Low
 
Bank Performance:
Moderately Unsatisfactory
Moderately Unsatisfactory
 
Borrower Performance:
Moderately Satisfactory
Moderately Satisfactory
In cases such as this where either government performance or implementing agency performance is in the satisfactory range while the other is in the unsatisfactory range, the overall Borrower performance rating, according to ICR guidelines, normally depends on the Outcome rating. According to this guidance, the Borrower Performance for this project should be rated moderately unsatisfactory. However, given that the government was the guarantor for this project and that TSJDF's contributions to the overall project results was significant, this review puts a higher weight to the implementing agency performance and therefore rates the Borrower performance to be moderately satisfactory.  
Quality of ICR:
 
Satisfactory
 
NOTES:
- When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
- The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons:

This review agrees with the lessons identified in the ICR, particularly the one that calls for a realistic assessment of justice sector issues at the early stages of the preparation process. Although 16 State Modernization Plans were prepared in close consultation with justice operators and users, these plans ended up not being financed by the project. Failure to support the judicial modernization plans in multiple states was the key reason for not being able to achieve the PDO. The initial analysis to assess the feasibility of project concept turned out to be the critical element for overall project outcome.

    This review also notes the unique challenges associated with projects that support the judicial branch at sub-national level. They require engagement with various levels of stakeholders in addition to the main counterparts in the state judicial branch, namely the executive branch at the state and federal levels. BANOBRAS being the borrower in this project also added to the number of stakeholders involved. While ensuring coordination among all is possible, the challenge of engaging the key players is certainly greater.

14. Assessment Recommended?

No

15. Comments on Quality of ICR:


The ICR presents a candid assessment of project outcomes. It provides a substantial amount of information on the events before and during project implementation, although in several sections, the descriptions are repetitive.

a. Quality of ICR Rating: Satisfactory

(ICRR-Rev6INV-Jun-2011)
© 2012 The World Bank Group, All Rights Reserved. Terms and Conditions