Named the runner up in the not-for-profit category of the Impact Measurement Awards, The Smith Family provides long-term support to disadvantaged children and young people to keep them engaged in education. It has an integrated, whole-of-organization approach to measurement based on the Results (Outcomes) Based Accountability (RBA) framework which is embedded in its policies, processes and systems. This organization-wide approach to measurement includes its five-year strategic plan, program logics, outcomes framework and staff performance management processes. At the core of this work is the drive to maximize the impact of its work with children and young people.
The Smith Family’s strategic plan identifies its mission, which is linked to its strategic priority of providing effective support through innovative quality programs, and its five long term measures of effectiveness. There is an annual corporate operating plan identifying key performance indicators which monitor progress on achieving these long term outcomes. These measures of effectiveness are reported on publicly in the annual report.
The key performance indicators are approved by the board with accountability for delivering on them embedded in the performance management processes which apply to all staff. These KPIs are reported on monthly, with a quarterly review undertaken by the executive team and board to assess progress being made and to identify any refinements that are required.
There is a strong organizational focus on developing a culture of impact measurement and using such measurement to improve programs and processes. A range of simple visuals which link mission, measures of effectiveness, strategic priorities and short and long term program outcomes have been developed and are used extensively with all staff, including through induction processes. The CEO’s monthly email to all staff has a strong focus on impact measurement, as do state conferences and team meetings, and staff have a range of opportunities to reflect on the implications of the outcomes which are being achieved.
Program logics and theories of change, which are research- and practice‐informed, have been developed for all core programs. These underpin program implementation and impact measurement. An outcomes framework has been developed for all core programs using the RBA approach. The outcomes framework identifies the expected short- and long-term outcomes, at the individual, school and community level and the associated output, quality and effectiveness measures and indicators for each program.
The Smith Family has invested in a custom-designed database and in-house research team to enable detailed data analysis that informs both program delivery and external advocacy. Responsibility for data entry is shared by frontline staff and volunteer operations centers, ensuring there is ownership for outcomes measurement across the organization. Each young person for whom data is entered has a unique identifier, enabling impact at both aggregate and individual levels to be measured.
In particular, The Smith Family is tracking three long-term outcomes – school attendance, advancement to Year 12, and post- school participation in employment and education – of the 34,000 young people it is supporting through its Learning for Life program. Learning for Life provides disadvantaged young people and their families with long‐term financial support, support from a Smith Family staff member and access to programs from the early years to tertiary level, with the goal of keeping them engaged in education.
Outcomes measurement has informed program and process improvement for The Smith Family in a number of ways. Initial analysis of the attendance rate data in 2011 highlighted significant issues with data quality and quantity with only 67 percent of required data available. In 2012, a concerted multi‐pronged strategy was implemented including communication with staff and volunteers across Australia to emphasize the importance of quality data and a range of new data collection processes. As a result, 97 percent of attendance rate data was available in 2012. Analysis of both the advancement and engagement rates also highlighted particular challenges staying engaged in education among Learning for Life Year 11 students, prompting the organization to pilot and evaluate a range of new program models with the aim of improving program effectiveness for this group.
