The NSW Government has recently released an updated version of its program evaluation framework, with the aim of improving government programs and providing more rigorous evidence of their outcomes. It continues to recommend the ‘program logic’ approach to evaluation, which sets out the relationship between program outputs and program outcomes. Below is a simple example of the logic for a learn to swim program.
This approach has a number of benefits:
- It makes a connection between the program’s activities and intended outcomes. For example, satisfaction with the learn to swim program may result in more kids completing the program.
- It clearly demonstrates how the program contributes to the overall goals of the organisation. If the evaluation indicates that the program isn’t achieving the organisation’s goals, then the appropriateness of the program can be challenged.
- It helps agencies focus their data collection efforts and avoid the two sins of evaluation: either collecting data on outcomes (such as drowning rates) that are too broad and likely to be influenced by other factors (such as pool fencing regulations); or collecting data on outcomes that are too narrow and do not provide an insight into what the program has achieved (such as number of children attending the program).
While the learn to swim example is quite simple, articulating the program logic of many government programs is often a complex and challenging process. Many agencies find it difficult to measure outcomes that are within the scope of their program. The goal is to be clear and concise, without being overly simplistic. It’s also important that the program evaluation is neither too broad or narrow, but achieves a ‘sweet spot’ where the agency can see if it’s made a real difference without drawing causal connections to broader outcomes in the community that are beyond its direct control.
Nexus has substantial experience developing program logic for a range of government and non-government organisations. Greg Masters worked in the Program Evaluation Unit many years ago, which pioneered the use of program logic in the evaluation of programs and services across NSW government agencies. We work with program staff to chart the logic of your programs and understand how they connect to common higher-level outcomes at an organisational level.
Greg Masters also designs and runs training programs on developing program logic and program evaluation. His recent clients include the Office of Environment and Heritage, NSW Kids and Families, Centre for Aboriginal Health and the Ministry of Health. Greg has also designed a training course for the Institute of Public Administration Australia NSW, which uses real-life case studies that give participants practical, hands-on experience in applying program logic in the context of the NSW Government’s evaluation framework.
