Integrating evaluation into program design and budget development

Evaluation as part of the program cycle

For the purposes of the program evaluation framework, a program is broadly defined as: “A set of activities managed together over a sustained period of time that aim to deliver an outcome for a client or client group”.[1] Essentially, programs deliver government functions.[2] A program can also include related government spending on a single intended outcome.[2] A program can be as broad as all government expenditure to reduce cost of living pressures, or as specific as a single social concession.[2] Education, health and policing services are deemed functions, not programs. Further, as a general rule, the Program evaluation framework will not apply to infrastructure and information and communications technology projects (which are covered by separate review processes) or externally funded programs.

The term ‘program’ is sometimes used interchangeably with project, service, initiative, strategy or policy. In practice, programs vary in size, duration and structure, and may span multiple agencies. Whole of government programs can be large and significant strategies, action plans or frameworks that encompass multiple agencies and locations, and comprise many agency-level programs, sub-programs and projects (Figure 1). Regardless of program size, when designed and conducted well, evaluation can yield useful evidence about the effectiveness of programs.

A strategic approach to evaluation (see Evaluating strategically) includes evaluations at several levels. For example, evaluating at the whole of government level to identify how different components of a strategy work together to achieve outcomes, and evaluating at the project level to examine specific aspects of a program.

Figure 1: Program hierarchy

Inverted pyramid image

Integrating evaluation into the program lifecycle ensures cost-effective evaluation is delivered in time to support key decision making points (Figure 2). Planning for evaluation should start at the program design stage so all stakeholders understand the key performance indicators the program will be assessed against and how and when evaluation will occur. Early planning also ensures data requirements are identified prior to commencement and lessons learned from previous evaluations can be used effectively.

Figure 2: Integrating evaluation into the program lifecycle

Program cycle diagram

Further guidance on good practice policy development is available on the APS Policy Hub.

Evaluation as part of budget development

Integrating evaluation into the budget process allows governments to make better use of resources.[3] The program evaluation framework integrates evaluation into the Territory Government’s budget process through an evaluation overview as part of the Cabinet submission template,[4] an evaluation work plan for approved programs, sunset clauses and a rolling schedule of evaluations.

Evaluation overview

An evaluation overview is required as part of the Cabinet submission process for programs requesting funding of $1 million or more in a year. The overview should be a concise summary of the key outcomes the program is trying to achieve and how success will be measured (further info at section 1 Complete the evaluation overview).

A full evaluation work plan will be required within six months if the program is approved to proceed.

Evaluation work plan

The detailed evaluation work plan[5] outlines future evaluation activity for a particular program over the next five years. The template requires agencies to consider:

  • the program’s theory of change (the program logic model)
  • key evaluation questions, indicators, and data sources (the question bank and data matrix)
  • appropriate types and timing of future evaluations (combined with the logic model and data matrix to form the program’s evaluation work plan).

See section 2. Complete the evaluation work plan for further guidance on completing the evaluation work plan.

Sunset clauses

Programs subject to sunset clauses are funded for a finite period, with the decision for further funding (either wholly or in part) informed by an evaluation.

Currently, agencies’ annual recurrent budgets represent an accumulation of funding decisions by a variety of governments over time. Without program evaluation, the Budget Review Subcommittee and Cabinet have little visibility of:

  • the effectiveness of existing programs
  • the applicability of existing programs to the current policy context
  • whether delivery models for the existing program provide the best value for money.[2]

A sunset clause is a built-in decision point for government. Unless otherwise directed by Cabinet, funding for new programs (or extensions of existing programs) that impact the Territory Government’s operating balance by $1 million or more in a year will be subject to an initial five-year sunset clause. This ensures that ongoing funding for programs is informed by evaluation.

Sunset clauses have been included in the Cabinet template and handbook and will be supported by a:

  • Sunset clause guide for agencies (in development)
  • Sunset clause guide for Treasury analysts (in development).

[1] NSW Government Evaluation Framework August 2013.

[2] Program Evaluation: Sunset Clauses, Agency Guide, Government of Western Australia, 2014.

[3] Building Better Policies, Chapter 6 Monitoring and Evaluation Systems and the Budget, World Bank, 2012.

[4] In the absence of exceptional circumstances, all submissions seeking additional funding must be considered as part of the Budget development process under the Northern Territory Government’s Charter of Budget Discipline.

[5] The Evaluation work plan template has been adapted from the Evaluation strategy template with permission from the Evaluation Office at the Commonwealth Department of Industry, Science, Energy and Resources. Publicly available at the APS Policy Hub.

Last updated: 19 January 2021

Give feedback about this page.

Share this page:

URL copied!