Evaluation
Evaluation of parent Autism Spectrum Disorder education programmes
Client:
Ministry of Health & Ministry of Education
Project Background:
Over the past 10 years the Ministries of Health and Education have purchased a number of parent education programmes for parents and whānau of children with Autism Spectrum Disorder (ASD): EarlyBird and ASD Plus for parents of children from 0 to school starting age. The aim of the programmes is to increase knowledge and understanding of ASD and the strategies and services available for parents and whanāu.
Our Involvement:
In 2010 Litmus was commissioned to concurrently evaluate EarlyBird and ASD Plus. The evaluations drew on a range of data sources:
- Review and analysis of programme data and documentation (provider reports to the Ministry, administration and monitoring data, and other relevant documentation)
- Expert interviews
- Quantitative surveys of participating families
- 360° Immersion Visits including in-depth qualitative interviews with programme providers and parents/ whānau.
Key Outcomes:
Ministries were provided with a detailed assessment of the programmes, and recommendations on improvements which could increase their effectiveness for the targeted audience. Further reading 
Evaluation of Community Action on Youth and Drugs (CAYAD)
Client:
Ministry of Health
Project Background:
Community Action on Youth and Drugs (CAYAD) is a community action programme aligned with the New Zealand National Drug Policy (NDP). It aims to limit the use of drugs and reduce harm from drug use amongst young people. The programme was first piloted in 1997 and is focused on 30 communities across New Zealand identified as high-need. Many CAYAD sites, in response to community need, have expanded their focus to also include reducing demand and controlling supply of alcohol to young people in their communities.
Our Involvement:
Litmus was engaged to evaluate the process and impact of the CAYAD programme. The evaluation involved a mixed-methodological approach consisting of:
- A review of literature and documentation relating to the CAYAD programme
- Interviews with key national-level stakeholders
- Case studies of eight CAYAD sites, involving interviews with managers, co-ordinators (kaimahi), reference group members, parents and young people
- An online survey of the remaining 17 CAYAD providers.
Key Outcomes:
Litmus provided the Ministry with an assessment of successes and challenges arising from the implementation of CAYAD, and actionable recommendations on how the CAYAD programme could be strengthened throughout the country to bring about the desired outcomes. Further reading
Review of the implementation of the New Zealand Disability Strategy
Client:
Office for Disability Issues (ODI)
Project Background:
The New Zealand Disability Strategy (the Disability Strategy) was instigated in 2001, as a means of enabling a vision of society that highly values the lives and continually enhances full participation of disabled people. It provides an enduring framework to ensure that government departments and agencies consider disabled people before making decisions. It was developed in consultation with disabled people and the wider disability sector, and reflects many individuals’ experiences of disability.
Our Involvement:
Litmus was commissioned to review progress made by central government agencies implementing the Disability Strategy over the period April 2001 to June 2007. The sector-wide review involved:
- A review of documentation related to the Disability Strategy
- In-depth interviews with disabled people, disabled persons’ membership organisations, parents of disabled children, disability support providers, central government agencies, local authorities, District Health Boards, tertiary education institutions and lead implementation agencies
- The development of an Outcomes Framework to model how the implementation of the Disability Strategy contributes to changes in central government agencies and other agencies’ policies and services, which ultimately result in positive changes in the life outcomes and value of disabled people
- The development of a Monitoring and Reporting Framework for the 10 year evaluation, and associated outcomes indicators.
Key Outcomes:
The review provided ODI with a detailed assessment of government’s progress towards achieving Disability Strategy Objectives and Action Points, and a clear framework for monitoring its on-going implementation. Further reading
Review of the New Zealand Coaching Strategy
Client:
Sport and Recreation New Zealand (SPARC)
Project Background:
The New Zealand Coaching Strategy was launched in 2004 in response to the 2001 Ministerial Taskforce Report ‘Getting Set for an Active Nation’. The Strategy was developed by coaches for coaches, and is aligned to athletes’ needs.
Our Involvement:
Litmus conducted the four-year review of the New Zealand Coaching Strategy’s implementation, which included:
- In-depth interviews and focus groups with national sports organisations, regional sports trusts, clubs, coaches and SPARC
- An online survey of grassroots and elite coaches
- A review of documentation on the development and implementation of the Strategy.
Key Outcomes:
The report provided SPARC with evidence on how the Strategy is being implemented, progress made towards the Strategy’s objectives and vision, and facilitators and barriers to uptake.
Reviewing Facilitation of Local and Central Government on Community Outcomes Process
Client:
Department of Internal Affairs
Project Background:
The Community Outcomes Process was introduced by the Local Government Act 2002 (the Act). Part 6 of the Act requires local government to facilitate a process with their communities, at least every six years, to identify community outcomes. This process should help communities (including central government agencies and businesses) identify and promote community outcomes. The Department of Internal Affairs is required to facilitate the central and local government interface in the Community Outcomes Process.
Our Involvement:
Litmus evaluated the Department’s facilitation role from 1 January 2005 to 30 June 2006. The evaluation comprised four key components:
- Internal stakeholder discussions, which resulted in the development and sign-off of an evaluation plan and draft intervention logic
- Analysis of documentation provided by the Department
- In-depth interviews with central government agencies and national bodies of local government
- Local government case studies involving in-depth interviews across local authorities, and interviews with regionally based participants from central government agencies.
Key Outcomes:
The Department of Internal Affairs received local government case studies demonstrating central and local government interaction and the Department’s facilitative role in this relationship. A cross-case report highlighted the successes of the Department’s facilitation and the barriers and enablers to local and central government interaction. Litmus also facilitated a future-focused workshop to enact the evaluative learnings into workforce development. Further reading
Review of the Youth Access to Alcohol Community Action Project
Client:
Alcohol Advisory Council of New Zealand (ALAC)
Project Background:
Following implementation of the pioneering Youth Access to Alcohol (YATA) Project, which empowers local communities to redress youth alcohol issues using a 'community action' model, ALAC wished to evaluate the Project's success in the 30+ communities nationwide that had adopted it.
Our Involvement:
Litmus evaluated the YATA Project by undertaking eight community case studies, providing an in-depth, '360°' view of how YATA works in different communities. We:
- Carried out case studies and associated tasks, including a documentation review; on-the-ground observation; and interviews and workshops involving YATA group coordinators, community partners and target groups
- Conducted a quantitative survey and literature review.
Key Outcomes:
Identification of key success and risk factors for the YATA project helped shape strategic direction at a community and national level.
|
