1. Complete the evaluation overview
This section of the toolkit is structured to mirror the Evaluation overview template to give additional guidance, section by section.
An evaluation overview is completed during program design (see Stages for planning and commissioning an evaluation) and is part of the Cabinet submission template. It is mandatory for all new programs seeking additional funding of $1 million or more in a year to complete an evaluation overview (refer to the Cabinet handbook). It should be a concise summary of the key outcomes the program is trying to achieve, and how and when success will be measured. The overview should be completed in consultation with the agency’s evaluation manager and or the Program Evaluation Unit at the Department of Treasury and Finance (DTF).
When seeking Cabinet approval for new programs, agencies must:
- consider previous evaluations as part of the new policy/program design to ensure continuous learning and improvement
- identify the outcomes the program aims to influence
- identify the key performance indicators against which the success of the program will be measured
- identify data sources to monitor program effectiveness, including baseline data
- include sunset clauses
- include provision for evaluations as part of the initial funding request.
Evaluation overviews will be reviewed by the Department of the Chief Minister and Cabinet and DTF as part of normal Cabinet submission processes. If a program is approved to proceed, a full evaluation work plan (see section 2: Complete the evaluation work plan) is required and must be provided to DTF within six months of the approval.
Explain why the program is needed and what it will aim to achieve. Consider how the program will impact future demand on government services if successful, and the time frame for this impact. This will need to be described in more detail in the evaluation work plan if the program is approved to proceed. Further guidance is at section 2.2: Program overview.
Show how the program’s inputs logically flow through activities to outputs and outcomes. Ideally, this is done using a program logic (see section 2.5.1: Program logic). If a program logic has not yet been developed for the program, the table below can be used as a starting point. A full program logic will be required as part of the evaluation work plan if the program is approved.
Table 4. How will the program achieve this?
The resources (funds, expertise, time) required for the delivery of activities.
The day-to-day tasks an organisation must undertake in order to provide a product or service.
The products or services generated by program activities; deliverables. The provision of outputs is typically under the control of the program and is related to the quantity and quality of program implementation.
The intended results of a program – the change it seeks to create. Outcomes are typically outside the control of the organisation.
Explain how external factors could influence the program’s outcomes, including economic, geographic, legislative, competitive and technological factors, and proposed risk management strategies.
List relevant programs with similar aims run by other agencies or governments, or non-government organisations to show how other programs may influence the outcomes of this program. See section 2.5.1: Program logic for further guidance.
Show how the program’s success will be measured for the activities, outputs and outcomes from Table 4. Indicators should be based on the SMART principles (specific, measurable, attainable, relevant and time-bound). Further guidance on selecting indicators can be found at the BetterEvaluation website.
Table 5 should build on Table 4. A new row should be added to the table for each activity, output and outcome to identify an indicator, baseline, target and data source. Clearly identify and explain any gaps.
Table 5. How will the program's success be measured?
While there are many different types of evaluation, the Program evaluation framework is based on three main types of evaluation, linked to the program lifecycle:
- Process evaluation: was the program implemented as expected? (Approximate timing ≤18 months.)
- Outcomes evaluation: are the desired outcomes being achieved or on track to be achieved within the agreed timeframe? (Approximate timing >2 years.)
- Impact evaluation: did the program influence the achievement of the desired outcomes, and was it value for money? (Approximate timing >3<5 years.)
Table 6 should list the evaluations required for the first five years of the program. All programs require a process evaluation within 18 months of commencement. The timing and need for an outcome and/or impact evaluation will depend the program’s cost, strategic significance and degree of risk. As a general guide, only programs that are a high strategic priority would require all three evaluation types. Further guidance on evaluation types is at section 2.5.3: Types of evaluation, and section 2.6: Budget and resourcing has guidance on estimating the budget for evaluations.
Table 6. What evaluations will be required in the first 5 years of the program?
Briefly address the following considerations:
- Is internal program data and/or analysis required?
- If data is required, is this data already being reported in the format required? If not, what resources are required to get the data?
- If analysis is required, what unit will undertake this work, have they been consulted in the preparation of the submission and do they have the skills and resources required? Section 2.6: Budget and resourcing has further guidance.
Further information on data collection is at section 4. Manage the implementation of the evaluation.
Last updated: 14 December 2020
Share this page:URL copied!