Public Sector

We also designed, tested and refined a developmental work research (DWR) methodology that enables policy and regulatory agencies to engage with the people, organisations or industries of interest to them in a way that enables the two parties to co-design the policy or regulatory artefact or action. This research-based innovative instrument is termed the Co-design Laboratory.

We also undertook evaluations of policy and policy or operational programmes. We evaluated the effectiveness of agency engagement and activity.

Key Focus Areas

• Research for policy and operations
• Evaluation of policy and programmes
• Connect policy and practice
• Case-study based methods for policy and operations

Research into the design and effectiveness of public funding schemes

Performance Based Research Fund – Tertiary Education Commisison (TEC)

Policy development and research & evaluation for the public sector

WEB Research uses case study-based evaluations to help government agencies bridge policy and practice. The approach is a form of developmental evaluation that allows the agency to understand the systemic links between:

• Its policies and processes (for example, labour market policy); and
• The activities and practices of those who are affected by these policies and processes (for example, the behaviour of firms, managers and employees).

WEB and our research associates developed and trialled the case study-based approach with the Department of Labour (DoL) in a series of projects carried out between 1996-2002. The initial impetus for DoL to undertake this form of evaluation stemmed from their need to develop ‘evidence-aware’ policy in understanding firms, workplaces and the impact of departmental policy and regulation. DoL wanted to augment its standard research practices with a more bottom-up, practice-oriented approach. Case study-based evaluations allowed the department to investigate what was actually happening in workplaces and allowed for that information to flow back to policy in a meaningful way. Our evaluations are developmental for both policy groups and firms and workplaces.

Track record

In the past 13 years, WEB Research has undertaken a substantial number of policy evaluation projects in New Zealand to inform or evaluate public policy and its operational effectiveness. This work includes the Phase One Process Evaluation of the PBRF for the Tertiary Education Commission and the Evaluation of the Marsden Fund (in partnership with Technopolis) for the Ministry of Research, Science and Technology. Other clients include the Ministry of Education, the Department of Labour, the Ministry of Social Development, the New Zealand Qualifications Authority and Inland Revenue.

NGO: Developmental evaluation

What is it?

1. Developmental evaluation is a facilitated, high involvement process to help organisations and groups get robust evidence of the effectiveness and value of their policy, programme, or funding initiative.
2. Research tools are used to document and share lessons learned, using a user-friendly approach that has high ‘face validity’ (Davidson, 2007).
3. Our approach is designed to build an ongoing evaluation capability within those organisations that use our services.

Where did it come from?

• WEB Research developed the method by drawing on:
  • Insights and tools from our research experience on how adults learn and change their practices at work; The work of international colleagues, particularly at the University of Helsinki in Finland (Engestrom, 1996);
  • Our case study-based approach to evaluation developed jointly with the Department of Labour between 1996-2002 (see Sankar and Hill, 2002);
  • Our work as researchers with Charitable Trusts and NGOs in New Zealand; and
  • The expertise of evaluation experts such as New Zealander Jane Davidson (2007).
• Integral to the method is a close partnership between the organisation commissioning the evaluation, the research agency (WEB), and those whose projects and activities are being evaluated.

Who uses it?

NZ Charitable Trusts and NGOs in funding relationships

• Developmental Evaluation enables a Charitable Trust or NGO to develop a partnership relationship with funded service providers in a short, medium or long-term relationship.  The method allows the funder to:
  • Evaluate the process and outcomes of the service provided;
  • Reflect upon and re-design (with the active participation of the service provider) monitoring and reporting tools, and processes for engagement between the funder and provider; and
  • Change the behaviour or services of each other to better secure the agreed outcomes.
• Traditionally funders let contracts to organisations that carry out research or intervention programmes for people in a particular community sector. In these cases, the connection between the funder and the people it seeks to assist is mediated by the contracting organisation.
• This arrangement no longer suits some funders.  They want to create the possibility of all parties – the commissioning and commissioned organisations and the people they intend to benefit – collectively monitoring, reviewing and re-designing interventions on a rolling basis.
• This is a sophisticated approach and requires a medium to long-term high trust relationship between the parties. A pure contracting model is not possible; the precise form, tools and behaviours of the new relationship have to be created by the parties as they undertake their respective roles and tasks.

Evaluation research with charitable trusts and NGOs

A five-year evaluation project for the Wayne Francis Charitable Trust (2007-2012). Carried out in collaboration with Dr Jane Higgins. The project evaluates how well the Trust’s process of funding projects works, and how funded organisations view and experience the Trust. The work also includes outcome evaluations of funded organisations.

An Evaluation of the Salvation Army Employment Plus (2008). This was an evaluation of that part of the Salvation Army that operates as a Private Training Establishment.