Search in:
Advanced Search

Implementation Completion Report (ICR) Review - Innovation For Competitiveness 1st Phase Apl

1. Project Data:   
ICR Review Date Posted:
Project Name:
Innovation For Competitiveness 1st Phase Apl
Project Costs(US $M)
 466.60  330.98
L/C Number:
Loan/Credit (US $M)
 250.00  250.00
Sector Board:
Financial and Private Sector Development
Cofinancing (US $M)
Board Approval Date
Closing Date
12/31/2009 12/31/2010
Tertiary education (70%), General industry and trade sector (21%), Central government administration (5%), General information and communications sector (4%)
Education for the knowledge economy (25% - P) Technology diffusion (25% - P) Export development and competitiveness (24% - P) Other financial and private sector development (13% - S) Small and medium enterprise support (13% - S)
Prepared by: Reviewed by: ICR Review Coordinator: Group:
Robert J. Anderson
Kris Hallberg Ismail Arslan IEGPS2

2. Project Objectives and Components:

a. Objectives:

    The statements of objectives contained in the PAD (Project Appraisal Document) and the Loan Agreement differ slightly.
        The Program seeks to support the Government's efforts to improve the competitiveness of the Mexican economy by strengthening the innovative capacity of the private sector, accelerating advanced human capital formation, and increasing the international integration of the innovation system. (PAD, page 6)
        The objective of the Project is to support the Borrower’s efforts to improve the competitiveness of its economy through: (a) strengthening National Council for Science and Technology (CONACYT)’s institutional capacity; and (b) supporting CONACYT’s existing grant and scholarship programs designed to (i) increase the innovative capacity of the Borrower’s private sector, (ii) accelerate advanced human capital formation, and (iii) increase the international integration of the Borrower’s innovation system. (Loan Agreement, page 21)
    Since the Loan Agreement makes strengthening CONACYT's institutional capacity an additional explicit objective of the project, the Loan Agreement statement of objectives is used here as the basis for IEG's review of the project ICR. The project ICR also uses the Loan Agreement objectives statement as the basis for assessing project efficacy.

b. Were the project objectives/key associated outcome targets revised during implementation?

c. Components:

Part A: Business Innovation Promotion (appraisal $57.79 million, actual $54.49 million).
Supporting the following CONACYT funding programs for business innovation
and promotion, including, linkages with the private sector, through:

      1. provision of grants to public research centers (consortia and research networks) to promote linkages with groups of enterprises with existing research and development absorption capacity;
      2. provision of grants to support the NAFIN Venture Capital Fund;
      3. provision of grants to private sector companies and public research centers to help transform scientific and technological developments into viable high value products and services;
      4. provision of grants to private sector companies to support innovation and technological development projects that would strengthen the competitiveness of said companies; and
      5. provision of grants to eligible graduates of master and doctoral program to promote their insertion in the Borrower’s labor market.
    Part B: Scholarships (appraisal $155.25 million, actual $185.77 million). Provision of scholarship grants to support the CONACYT Scholarship Program.

    Part C: Institutional Strengthening (appraisal $10.71 million, actual $6.46 million). Provision of technical assistance, training and equipment (as necessary) to
    strengthen CONACYT’s capacity to, inter alia:
        1. support policy development in the areas of science, technology and innovation;
        2. establish a comprehensive information, monitoring and evaluation system;
        3. design and implement a public information campaign to disseminate CONACYT’s programs; and
        4. strengthen CONACYT and the Project Coordination Unit (PCU) to assist in the carrying out of the Project.

    d. Comments on Project Cost, Financing, Borrower Contribution, and Dates
    The closing date of the project was extended from December 31 2009 to December 31 2010 to allow for the completion of project activities. Funding was reallocated between different disbursement categories in April 2006, November 2007, and August 2009, resulting significant changes included increasing allocations for scholarships, business innovation finance and hardware, and decreasing allocations for consultants.

    3. Relevance of Objectives & Design:

    a. Relevance of Objectives:

    Competitiveness and growth objectives and the development agenda to address them are central elements of the Government's development strategies and the Bank's supporting assistance strategies for the project period. The objectives of the ICP are aligned with those of the government’s 2007-2012 National Development Plan and the Bank’s supporting Country Partnership Strategy (CPS) FY2008-2013. Enhancing the national innovation system is a critical element of the sustainable growth pillar of the CPS FY2008-2013, reaffirmed in the CPS progress report issued in FY2010. The relevance of objectives is thus rated as "high".

    b. Relevance of Design:

    The Bank's support programs for the period of implementation of ICP envisage a multi-faceted approach to increasing productivity and competitiveness that addresses (inter alia) policy constraints in factor markets (e.g., in labor markets), education, and infrastructure, as well as in S&T and innovation - the focus of the ICP project.
    The specific design of the ICP flows from a Bank analytical work and other studies analyzing trends in competitiveness indicators and comparing Mexico's performance with other LAC and OECD countries in the S&T area. The Bank's analysis traces Mexico's relatively poor performance to the following factors: (1) Inadequate total investment in R&D; (2) Insufficient private investment in R&D; (3) Inadequate linkages between public R&D and the productive sector; (4) Few Science and Technology (S&T) based start-ups; (5) Low S&T skill base; and (6) Concentration of resources and capacity in a few public R&D entities. The ICP is designed to address these constraints.

    The knowledge base underpinning the ICP is thus substantial and the Bank's transformation of it into a project results framework is sensible. However, the linkages between S&T development interventions such as scholarship programs and consortia formation on the one hand and competitiveness outcomes on the other are long in gestation (as the ICR emphasizes) and less precisely empirically known than (say) the linkages between factor market reform and productivity. There is thus less certainty about the timing and extent to which some of the interventions supported by the ICP project will contribute to increases in innovation, productivity and competitiveness. The results framework of the project, taken in the context of the APL program of which it is a part, is thus plausible on a priori grounds. Design relevance is therefore rated as "substantial".

    4. Achievement of Objectives (Efficacy) :

    The ICR does not directly address the issue of the contribution of the ICP project to the improvement of competitiveness of the economy. This is reasonable for this project given the long gestation period between most of the activities supported by the project, the specific sub-objectives (e.g., increase the innovative capacity of the private sector) linked to this overall objective, and competitiveness of the economy. As time passes however, it will be important to track progress with regard to the broader productivity/competitiveness objective.

    Recent World Economic Forum Global Competitiveness Index data suggest that Mexico is making progress in these dimensions, as reflected in the Innovation component of the indicator. One indicator is provided the Global Competitiveness Report's classification of economies by stages of development corresponding to main drivers of growth (i.e., Stage One - Factor Driven, Stage Two - Efficiency Driven, and Stage Three - Innovation Driven). At project inception, Mexico was classified as a Stage Two economy; at present, it is classified as in transition from Stage Two to Stage Three, indicating increasing importance of innovation activity. On the Innovation Factors sub-component of the GCI, Mexico's score increased from 2.95 in 2008-09 data to 3.19 in the 2011-12 data, and it was ranked on this factor at 90 th out of 134 countries in 2008-09, and 63 rd out of 142 countries in 2011-12.

    Outcomes with regard to the specific sub-objectives of the project were mixed but, in general, substantial progress was made toward the achievement of all sub-objectives.

      Objective (a): Strengthening CONACYT's institutional capacity.
      CONACYT’s business innovation programs were strengthened substantially, as were the National Program for Quality Graduate Programs, M&E activities, and CONACYT's IT platform. Its business innovation programs were in the start-up phase at project inception; at the conclusion, project-supported programs had been restructured to take advantage of early experience and the results of program evaluations and scaled up, with increased funding from the Mexican budget. The National Program for Quality Graduate Programs, which developed and implemented a program to strengthen graduate education resulted in substantially increased numbers of accredited graduate programs (from 680 in 2006 to 1304 in 2010). Improvements to IT systems have enabled more up-to-date monitoring and efficient management of CONACYT’s programs, as well as supporting the data systems needed to conduct more rigorous evaluations of CONACYT's programs. The ICR notes that changes in the structure and management of the Policy, Monitoring and Evaluation Directorate have placed a much higher importance on evaluations, which has been complemented by project financing for several evaluations. Project financing contributed to all of these aspects of CONACYT's institutional development. Efficacy of the ICP with regard to achievement of this objective is rated as "high".
      Objective (b.1) Increase the innovative capacity of the Borrower’s private sector
      Achievements with respect to this sub-objective of the project are mixed. The indicators utilized by the project as presumptive correlates of the unobservable "innovation capacity" are "private R&D as a percentage of GDP" and "proportion of private R&D as a share of total R&D". Private R&D increased on average by 6.2 percent in real terms from 2004 through 2008 (later figures were not available at the time of preparation of the ICR) - from 0.15 percent of GDP to 0.17 percent of GDP. The ICR speculates (reasonably) that private R&D investment probably was strongly negatively affected by the global financial crisis during 2009 (overall private investment in Mexico fell by 17 percent in real terms in that year). Private R&D as percentage of total R&D increased from 2004 through 2006, but declined in 2008 due to a boost in public sector R&D as STI programs gained a higher profile in the government’s policy agenda. The ratio is likely to have declined again in 2009 due both to a contraction in private sector investment and the aforementioned increased public funding for STI. Project targets were for these two PDO indicators were thus not met.
      As the ICR notes, however, intermediate outcome indicators provide a better indication of the effects of project interventions at this relatively early stage of the planned 10 year business innovation APL program. These generally show considerable progress in promoting research institution business linkages and business innovation activities. After an early learning period, the program to strengthen linkages between S&T institutions and enterprises by creation of consortia took off. Forty-eight (48) public-private research consortia were created with project funds, bringing together 173 companies (versus a target of 8 consortia and 24 companies). In 2009, CONACYT scaled up the consortia program by launching a new initiative, PROINNOVA. The ICP financed consortia-formation activities (Mode A of CONACYT's program to support consortia) and counterpart funds under PROINNOVA financed corsortia operations. A total of 259 consortia were established between 2006 and 2010, involving 287 companies.

      Thirty-four (34) new business lines were supported through the venture capital (VC) fund, Emprendedores, (34) nearly attaining the project’s target (40). Private sector investment leveraged was US$36 million, close to the target of US$39 million. Overall, the fund leveraged 2.5 dollars of private investment for each dollar of public investment.

      The number of projects that reached commercial stage with support from the Precompetitive Fund (at least 20) exceeded the target (15). The output figure is an estimate based on the 2008 evaluation, and the actual number is likely to be substantially higher.

      Total private sector investment for innovation leveraged through the Sectoral Innovation Fund with the Secretaría de Economía was double the target. Sixty nine patents were reported in the final project survey, but only two thirds of beneficiaries participated in the survey and only half provided information on this indicator (see also Section 10 of this review on M&E). This number is a low boundary and the actual number of patents registered is very likely to be substantially higher.

      The intervention that substantially under-performed expectations was the effort to promote hiring of doctoral and masters graduates as researchers in industry. In hindsight showed, the demand for employees with advanced degrees of Mexican firms was far more limited than had been expected.

      Taken together, the progress registered in these dimensions is sufficient to conclude that the ICP achieved most of what could reasonably have been expected of this first-phase of the program. Efficacy with regard to increasing the innovation capacity of the private sector is therefore rated "substantial".

      Objective (b.2) Accelerate advanced human capital formation.

      The results framework takes as a indication of achievement of effective advanced human capital formation an increase in advanced human capital factor intensity in the economy, as measured by the number of researchers in relation to the total employment (number of researchers per thousand employment). This ratio increased by 25 percent between 2004 and 2009 i.e., from 8.3 per thousand to 10.4 per thousand (against a target of 12.3). The target was overly ambitious given the lag between start of graduate training and entry into the labor force, and the high likelihood that some fraction of graduates do not enter the work force as researchers.

      Targets for achievement of intermediate outcomes (mostly project outputs in the form of an increased number of scholarships in highly ranked graduate programs in demand-oriented areas) were generally achieved. CONACYT awarded 72,888 scholarships between 2006 and 2010, of which roughly two-thirds were in science and technology. The Bank project financed roughly half of those scholarships (35,282), 83 percent of which were in science and technology. About 67 percent of scholarships were awarded for master programs and 29 percent for PhD programs, and the number of scholarships awarded annually grew steadily from 10,027 in 2006 to 21,019 in 2010. In addition, the percentage of scholarships allocated to demand-oriented (as determined by designation by states, sectoral and private sector representatives) programs increased substantially reaching 66 percent of the total, slightly short of the target of 70 percent. All national scholarships were allocated to highly-ranked programs under the National Program for Quality Graduate Programs (Programa Nacional de Posgrados de Calidad, PNPC), surpassing the project target of 90 percent. The performance of the PNPC has also improved substantially in recent years, in particular the process for evaluating and classifying graduate programs.

      Efficacy with regard to achievement of the accelerated formation of advanced human capital is rated as "substantial" on the strength of both progress with regard to increases in the factor intensity of advanced human capital (though not tightly linked due to the gestation period to project interventions) and the significant scaling up of scholarship quantities and quality supported by the ICP.

      Objective (b.3) Increase the international integration of the Borrower’s innovation system.

      The ICP tracked the number of internationally-published scientific articles by Mexican scientists as a correlate or indicator of the degree of international integration of Mexico's innovation system. The indicator target (9229 articles in 2009) was exceeded by about 2 percent. Data supplied subsequently by the Region indicated that the proportion of internationally-published scientific articles published by Mexican researchers also increased - from 0.82 percent in 2006 to 0.86 percent in 2009. Achievements with regard to targeted intermediate outcomes were mixed. The percentage of international scholarships allocated increased three-fold from 4 percent to 12 percent, a substantial increase, although short of the 20 percent target. The number of students studying abroad under collaborative agreements between national and foreign programs, including becas mixtas (mixed scholarships-a program under which Mexican graduate students spend a period of study abroad), reached 720, well-above the target of 320. The ICR notes that there has also been a significant internationalization of the National Program for Quality Graduate Programs (PNPC), inter alia through the incorporation of international experts in the evaluation process for PNPC accreditation, increases in scholarships for students and professors to undertake a portion of their work internationally, enhanced linkages with international doctoral programs, and the inclusion of students from other LAC countries in PNPC graduate programs - all of which should contribute to international integration. The achievement of this objective is rated as "substantial".

    5. Efficiency:

    The ICR states that no formal efficiency analysis was presented in the PAD and is not possible at this time given the nature of the project interventions. It notes however that project administration costs were low (0.56 percent of the total loan amount), that project interventions addressed specific constraints identified in the PAD and that project interventions are in line with the literature and international best practices. The project was completed with little delay. Project efficacy (see Section 4 above) was substantial or better with regard to all objectives. An additional indication that project funds were well used is provided by the substantial amount of money the Mexican Government committed from its own budget to modifying and scaling up project interventions and the strong participation of the private sector.

      Taken together, these factors suggest that an efficiency rating of "substantial" would be warranted.

      a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return at appraisal and the re-estimated value at evaluation:

    Rate Available?
    Point Value
    ICR estimate:

    * Refers to percent of total project cost for which ERR/FRR was calculated

    6. Outcome:

    The objectives of the ICP are highly relevant and project design relevance is substantial. Project efficacy was high with regard to strengthening CONACYT's capacity to promote innovation, and substantial with regard to increasing the capacity of the business sector to innovate, increasing the formation of a base of human capital to support innovation, and increasing internationalization of Mexico's S&T. Project efficiency was also substantial.
      The outcome of the ICP is therefore rated "satisfactory".

    a. Outcome Rating: Satisfactory

    7. Rationale for Risk to Development Outcome Rating:

      ICP outputs, as was intended, increased both the supply of and demand for factors that should logically lead eventually to increased innovation, faster total factor productivity growth, and enhanced competitiveness. As noted above, however, the linkages between STI programs and these development outcomes are not as well known as those between other interventions and productivity (i.e., there are "technical" risks associated with uncertainties in the state of knowledge linking S&T interventions and productivity/competitiveness outcomes), and there are a number of important intervening exogenous factors, (e.g., the security situation, the S&T efforts of other countries) - most of which are recognized in Mexico's comprehensive development strategy but which are very difficult to predict or control. Sustainability of outcomes is also crucially dependent upon sustained government and private sector commitment, backed by sustained commitment of resources.
      The probabilities of unfavorable technical or intervening exogenous factor disturbances are impossible to estimate, but they are probably not negligible, and the negative impact on outcomes of unfavorable outcomes of these factors are likely to be moderate to significant. The risks to government and private sector commitment appear to be low, as reflected in Mexico's long-standing commitment to and budget support of S&T development and substantial and growing private sector participation. However, both remain vulnerable to cyclical shocks, and would also be vulnerable to negative shocks described in the preceding paragraph that would tend to cast doubt on the payoffs from continued support of S&T interventions.

      For these reasons, risk to development outcome is rated as "significant".

      a. Risk to Development Outcome Rating: Significant

    8. Assessment of Bank Performance:

    a. Quality at entry:

    The ICP’s objectives and design were based on a substantial body of prior analytical work and project experience, well-thought out and embedded in a comprehensive strategy (e.g., embracing education, factor market reform, and other factors related to productivity) to strengthen Mexico’s competitiveness. It addressed institutional and policy constraints to development of Mexico’s STI programs related to competitiveness and procurement and financial mangement arrangements – as reflected in generally smooth implementation of the project – were satisfactory. M&E design, while deficient in recognition of the gestation period required for project activities to affect outcomes and in the specification of some indicators, was satisfactory. Risks were well-identified, appropriately appraised, and mitigated effectively. The Bank provided appropriate skills and maintained good task team continuity throughout the implementation period.

      Bank performance with regard to ensuring quality at entry, notwithstanding the deficiencies in M&E design, is therefore rated as "satisfactory".

    Quality-at-Entry Rating: Satisfactory

    b. Quality of supervision:

    Bank supervision, during the early part of the project period, focused primarily on implementation issues. Supervision was effective in identifying implementation issues and flexible in identify workable solutions for their resolution. Later in the implementation period, increasing attention was paid to monitoring outcomes against those expected to flow from the ICP and subsequent projects. Bank supervision of procurement and financial management was satisfactory, with thorough independent reviews conducted. Supervision reporting was generally complete, candid, and reflected close Bank engagement in identification and resolution of problems. Effective arrangements have been made for continuation of project supported programs - in many instances scaled up - with government budget support, and for continuing the Bank's planned APL program of support to Mexico's STI programs.

      Quality of supervision is therefore rated "satisfactory".

    Quality of Supervision Rating: Satisfactory

    Overall Bank Performance Rating: Satisfactory

    9. Assessment of Borrower Performance:

    a. Government Performance:

      Government performance was strong in virtually all aspects of project design and implementation. The competitiveness agenda and supporting STI programs received strong government backing, as reflected in support for policy and institutional development measures cutting across the development agenda, and budget backing - through relatively difficult times - for STI programs. The programs supported by the ICP were primarily government-conceived programs. The Government provided effective counterparts to the Bank team and worked diligently in all aspects of project/program design and implementation, including strengthening of M&E of STI programs. Government performance is therefore rated as "satisfactory".

    Government Performance Rating: Satisfactory

    b. Implementing Agency Performance:

    All indications are that CONACYT and other agencies participating in the implementation of the ICP were and are committed to achieving project development objectives. Implementation “craft” (i.e., procurement and financial management)were generally competent, and where problems arose, these were addressed pro-actively and constructively. Responsibilities for M&E implementation were met, and actions were taken in response to M&E findings and recommendations. Project-funded activities have continued – sometimes in modified form as a result of project experience and the aforementioned M&E efforts – with government budget support, and implementing agencies are now working with the Bank in the development of the second phase of planned three-phase APL program. Implementing agency performance is therefore rated as "satisfactory".

    Implementing Agency Performance Rating: Satisfactory

    Overall Borrower Performance Rating: Satisfactory

    10. M&E Design, Implementation, & Utilization:

    a. M&E Design:

    The PDO indicators selected are reasonable in the context of the planned ten-year APL program, although as the ICR notes, the gestation period between project activities and possible effects on these indicators is relatively long. The evolution of PDO indicators over the ICP implementation period is thus primarily reflective of other factors - including earlier CONACYT activities that were precursors to ICP activities. The analytical work underpinning the ICP design suggests a number of other indicators that might also be considered (e.g., net exports of technology, total factor productivity) in tracking competitiveness. The World Economic Forum competitiveness indicators (particularly the Innovation sub-indicator and its constituent components) might also be considered in this regard.

      The intermediate outcome indicators are also reasonable choices, although some tweaking of indicators for outcomes carried forward in subsequent operations may be in order (e.g., articles published by Mexican authors in international journals might better be expressed in relation to the total number of such articles; citation data - both references by Mexican authors to the international literature as well as references to articles by Mexican authors; advanced degree completions). It would also be appropriate-given the fact that enough time will now have passed to expect to begin to see some effects of project activities - to shift the focus of some indicators from outputs (e.g., consortia formed) to outcomes (e.g., patent activity, sales, profitability, company valuation).

    b. M&E Implementation:

    Indicators were tracked and reported regularly. When new information became available, indicator baselines and targets were revised. Subsequent information provided by the task team indicates that indicators are continuing to be tracked after project closing as an input into formulation of the second APL in the planned program. M&E implementation would have been more effective, as the ICR notes, had the survey methodology used to track certain indicators been designed to elicit a higher response rate - perhaps through face-to-face interviews.

    The disappointing response rate in M&E surveys also raises the possibility that the "hurdle" for private sector participation in the program may have been set too low, or that the participants' evaluations of the value of the program (as might be inferred from unwillingness to take the time to respond) may not be as great as might be surmised based solely on the other participation statistics tracked as intermediate outcome indicators.

    The project also contemplated, supported and produced a number of evaluations of CONACYT's programs. Although the number of evaluations completed fell short of the intermediate outcome indicator target for evaluations conducted, the evaluations undertaken appear to have been of a broader scope (i.e., "program" level as opposed to "project" level) and depth than envisaged when the quantitative target was set. All indications are that the evaluations completed were generally of high quality.

    a. M&E Utilization:

    M&E outputs, including the aforementioned program evaluations carried out by CONACYT, were and are being used by CONACYT in the reformulation of its STI programs, and by CONACYT and the Bank in formulation of the next phase of the Bank's support program for Mexico's STI activities.

    M&E Quality Rating: Substantial

    11. Other Issues:

    a. Safeguards:

    The ICP was categorized "C" - for purposes of application of the Bank's operational policies governing environmental assessment. In this regard, the PAD (page 17) states: "The project is expected to have a neutral or positive environmental impact. Support would only be provided to sub-projects that are environmentally sound, and therefore are in line with Federal environmental laws and regulations. In order to assure compliance, the PCU is tasked with the responsibilities of: (i) raising awareness of relevant environmental regulations among potential beneficiaries; (ii) introducing environmental aspects in project‘s operational manuals; and (iii) monitoring their implementation." The ICR, however, provides no information concerning Bank or government activities during implementation with regard to fulfillment of these obligations.

    b. Fiduciary Compliance:

    Notwithstanding some areas for improvement identified by an Independent Procurement Review carried out under the project, procurement performance under the project as generally found to be satisfactory. Financial management reports were submitted as required, reviewed by financial management specialists and found to be satisfactory.

    c. Unintended Impacts (positive or negative):

    The ICR cites learning concerning how Mexico could strengthen its innovation system by Mexican STI officials in the context of a delegation visit to US counterparts as an unintended positive impact. Although this visit was not specifically envisaged in the original project design, it is clearly linked to international integration and, therefore, "intended" in the sense that it is linked to project development objectives.

    d. Other:

    No other relevant aspects of which IEG is aware.

    12. Ratings:

    IEG Review
    Reason for Disagreement/Comments
    Risk to Development Outcome:
    Negligible to Low
    There are a large number of "random factors" in the results framework linking between outputs of S&T interventions and improved productivity and competitiveness. The likelihood of negative outcomes with respect to at least some of these is non-negligible and the effect of such negative outcomes could be significant.  
    Bank Performance:
    Moderately Satisfactory
    The ICR's rating of Bank performance does not conform to ICR Guidelines. In the ICR, one sub-rating is MS (Quality At Entry) and one is S (Supervision). In that case, the overall Bank Performance rating must per ICR guidelines be MS (i.e., it is rounded to the lower rating). This review assigns a rating of S for Quality At Entry because it does not discount this rating as heavily for the deficiencies in Monitoring and Evaluation. 
    Borrower Performance:
    Quality of ICR:
    - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006.
    - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

    13. Lessons:

    The lessons of general applicability (clear communication of program changes to participants, monitoring and evaluation grounded in a framework that is not only conceptually reasonable but also realistic with regard to causal links and temporal dimensions, the need for an interview element in project participant surveys, and the high value of knowledge exchange on international experience to many of the Bank's middle income clients) accord well with those drawn across the Bank's portfolio. The lesson on M&E is particularly important and very well-stated. It is also particularly relevant to projects and programs like the ICP and the APL program of which it is a part, where there are long gestation periods and a number of exogenous and/or random factors impinging on development outcomes.

    The lessons of applicability specifically to programs designed to support innovation (flexibility to attract private sector participation, need for multi-year funding, need for improved understanding of demand factors and incentives in private decisions to undertake research and hire researchers, need for an adequate gestation period for formation of public private partnerships, need for professional management in venture capital funds, and need to consolidate programs) are well-grounded in experience with the implementation of ICP. IEG would note, however, that some degree of judgement is warranted in applying the "flexibility" lesson. Risks that funds may be applied in a way inconsistent with the spirit of Bank policies and standards need to be appraised and, if needed, mitigated in a way that minimizes disincentives to private participation while meeting Bank fiduciary responsibilities.

    14. Assessment Recommended?


      As noted, the linkages between S&T development interventions and competitiveness/productivity are not as thoroughly explored as those between (say) factor market reforms and productivity/competitiveness. A re-examination of this project at the time program completion, in the context of the full program and - perhaps - in the context of similar programs in other countries, could add substantially to knowledge of these linkages.

    15. Comments on Quality of ICR:

    The ICR is complete in its coverage of the ICR rubric, candid in its assessments, and well-presented. Its analyses of project implementation experience and project efficacy - including attribution - are strong and well-supported by data. The lessons it draws are well-grounded and presented in a way that is likely to make them useful in future operational work. There are two minor shortcomings. The first is an inconsistency in project cost data between the ICR text and data presented in Annex 1. Table (a) on p. 26 shows total project costs as $248.75 million at appraisal and $246.73 million at completion (as of March 2011). These amounts are in fact loan amounts - exclusive of the up front fee. Table (a) should have reported total project costs, funded by the Bank and the Government. The project was in fact fully disbursed - no funds were cancelled. The second shortcoming is the miscalculation of the overall Bank Performance rating (see Section 12). Overall, without these minor problems, ICR quality could have warranted a rating of "exemplary".

    a. Quality of ICR Rating: Satisfactory