• How much progress has been made toward each strategic objective? (You may want to check on the evolution of outcomes and outputs indicators)
• You may want to provide a summary of the activities that were conducted, outlining:
  • Which activities were completed
  • Which activities are still ongoing
  • Which activities are pending
  • Which activities have been retired

In this area, you will want to examine whether strategies and actions have been effective, relevant, and sustainable as well as whether they made an impact.

Effectiveness

Effectiveness refers to whether the objectives were met, including any differential results across groups. While progress towards objectives is determined by effectiveness, the actual objectives are identified by relevance. Effectiveness and impact are complementary criteria focusing on different levels of results. Effectiveness considers the achievement of results, while impact focuses on higher-level results.

• Elements for analyzing effectiveness include:
  • Achievement of the objectives: whether or not strategies and actions achieved intended results
  • Weighing the relative importance of what was achieved: this may draw upon the analysis of relevance, which should have addressed the potential differences between the priorities and needs of the various stakeholders
  • Differential results: considering inclusiveness and equity of results
  • Influencing factors: examination of the factors that influence results, such as management, human resources, financial aspects, regulatory aspects, implementation modifications, or

• Some possible questions may be:

  • Did the strategies and related actions successfully contribute to reaching the objectives?

  • What are the main reasons for achievement of targets?

  • What were the conditions for success?

  • If any strategies and related actions were not successful, what were the reasons and what needs to be done differently?

  • Depending on your context, addressing inequality in service provision (regarding access to pre-primary education and/or use of pre-primary education services) may have been part of the pre-primary section of the ESP.

    • You may want to disaggregate data by: gender, place of residence, geographical location, disability, poverty status, displacement

      • The evaluation should address these issues and come up with answers on the extent to which the ESP has reduced inequalities

      • What were major successes in the pre-primary subsector with regard to the equitable delivery of quality pre-primary services to various stakeholders?

Relevance

Relevance refers to whether the ESP focused on the right aspects. The evaluation of relevance provides a foundation to understand if needs are met as a part of effectiveness and impact. Many of the elements of relevance are also critical factors in sustainability, as the relevant strategies and actions are likely to have greater support among stakeholders, which can influence the degree of ownership of resulting benefits, and thus their sustainability.

• Elements for analyzing relevance include:
  • Responsiveness to needs, policies, and priorities:
  • Sensitivity and responsiveness to the context: how strategies and actions took into consideration economic, environmental, equitable, social, cultural, political economy and capacity considerations
  • Quality of design: how strategies and actions addressed relevant priorities and needs and whether goals were clearly specified
  • Adapting over time: ongoing adaptation to external contexts and internal changes
• Some possible questions may be:
  • To what extent were the ESP strategic and specific objectives valid and appropriate? (For example, did they respond to teachers’ and learners’ needs and did they continue to do so under any changed circumstances?
  • What changes might need to be considered in the future?

Sustainability

Sustainability refers to whether any beneficial changes will last. Sustainability is linked to relevance, with the level of relevance to key stakeholders being a key factor affecting their ownership and buy-in to eventual benefits, which in turn drives sustainability. The evaluation of the continuation of results also relies on results being achieved (effectiveness) and that higher-level effects were demonstrated (impact).

• Elements for analyzing sustainability include:
  • Building an enabling environment for sustainable development: how systems, institutions, or capacities were strengthened to support future development
  • Continuation of positive effects: looking at actual sustainability, or the continuation of benefits that are already evidence, as well as looking at prospective sustainability, or benefits for key stakeholders that are likely to continue into the future
  • Risks and potential trade-offs: considering actors that may enhance the sustainability of benefits over time as well as factors that may inhibit sustainability
• A possible question may be:
  • Will the benefits last?

Impact

Impact refers to whether any differences were made, whether any significant positive or negative, intended or unintended, higher-level effects were generated. Impact addresses the ultimate significance and potentially transformative effects of the ESP’s strategies and actions that are longer term or broader in scope than those already captured under the effectiveness criterion.

• Elements for analyzing impact include:
  • Significance: the “so what” question - how much the ESP’s strategies and actions mattered to those involved
  • Differential impact: Positive impacts overall can hide significant inequity, so it is important to consider that impact for different groups can be monitored and evaluated
  • Unintended effects: these can be either positive or negative. Where positive, evaluators should consider their overall significance and whether there is scope for innovation, scaling, or replication of the positive impact. Where negative, attention should be paid particularly on unintended impacts on vulnerable groups
  • Transformational change: this can be thought of as addressing root causes and refers to “holistic and enduring changes in systems or norms”
• A possible question may be:
  • What difference was made in the pre-primary subsector as a result of the ESP’s strategies and actions?

What constraining factors, barriers, or bottlenecks have hindered the system’s ability to achieve pre-primary goals and targets?

• What policies/strategies need:
  • To be pursued (those which were effective and valid)?
  • To be dropped (those which were ineffective or not relevant)?
  • To be redesigned
  • To be added to meet emerging and changing contexts?
• What are key issues that the Ministry of Education (or the ministry in charge of pre–primary education) should focus on in order to positively shape the pre-primary subsector’s development agenda?
• What are the set of key priorities the Ministry of Education (or the ministry in charge of pre-primary education) should pursue over the duration of the next ESP?
• What resources and changes in capacities, processes, and practices will be required in order to enhance the system’s ability to deliver quality pre-primary services in an equitable, inclusive, cost-efficient, and sustainable manner?