The measurement of training opportunities course outcomes: an effective policy tool?

• Author: Stolte, Ottilie

Abstract

Training Opportunities is an active labour market policy initiative, and part of a response to the entrenched problems of unemployment in Aotearoa New Zealand. The funding and implementation of Training Opportunities are determined in part by a particular system for measuring course outcomes. This paper argues that this measuring system should not be used for policy development, due to measurement errors and problems assigning causality to the intervention. Consequently, various disincentives arise that contradict the objectives of Training Opportunities. While accountability is important, the over-reliance on the narrowly defined Training Opportunities outcomes undermines the ability of providers to assist the unemployed, and thereby contribute to the policy goals of reducing unemployment and labour market disadvantage in New Zealand.

INTRODUCTION

Despite varying levels of economic growth, unemployment and underemployment (2) are now entrenched features of Aotearoa New Zealand society. One active labour market policy that offers direct assistance to specified groups of unemployed adults is Training Opportunities. (3) This training policy aims to address unemployment by assisting individuals to overcome their impediments to full labour market participation. Like most other areas of public spending, Training Opportunities is under constant scrutiny and must operate in accordance with business accountability principles.

The prime focus of Training Opportunities is to assist learners to acquire a critical bundle of foundation skills that will enable them to move effectively into sustainable employment and/or higher levels of tertiary education.

This paper focuses on two aspects of research conducted on Training Opportunities (Stolte 2001). First, the paper considers the emergence of Training Opportunities as a particular response to unemployment in New Zealand. A brief historical context is followed by an outline of the operational context for the policy initiative. Second, this paper argues that the particular design of policy mechanisms for Training Opportunities is counterproductive to broader policy goals. Specifically, the paper asks whether the current outcome measurement system is an appropriate tool for policy development to ensure that these courses are a prudent and constructive response to unemployment. This is an important question not only because of the need for accountability, but also because of the persistence of unemployment and labour market disadvantage.

BACKGROUND: A STUDY OF TRAINING OPPORTUNITIES

The research informing this paper grew out of a concern about unemployment in New Zealand and the failure of economic growth as a "solution" to unemployment. Bertram (1988) and Higgins (1997) assert that there is a need for scholarly investigation of the policy frameworks and mechanisms for employment assistance and training programmes in New Zealand. In particular, they stress the need for work at a practical level that draws on participatory research approaches.

When the research began in 2000, Training Opportunities was seen as a major form of employment assistance available to unemployed adults. The pilot phase of the research indicated that course outcomes were a controversial issue for both the providers and funders of Training Opportunities. A qualitative methodology appeared most useful to investigate why there were problems with the system for measuring outcomes. The research combined case studies, discourse analysis and theoretical engagement. (4) The fieldwork included open-ended key informant interviewing with funders and providers. Informal encounters and participant observation occurred in various training course settings. These approaches were less viable within the public sector organisations, so the research contained a somewhat greater focus on the providers' experiences. Secondary data were derived from policy documents, media releases and government publications.

Due to time and resource constraints, most of the fieldwork was located in Hamilton, although I did visit Wellington on two occasions to discuss the research with staff in central government agencies.

CONTEXT: THE EMERGENCE OF TRAINING OPPORTUNITIES

This section provides the context for the emergence of Training Opportunities as a historically specific response to the problem of unemployment in Aotearoa New Zealand (Wallace 1998). Since the 1970s many different employment assistance initiatives and training schemes have been developed in response to the persistence of unemployment.

The Entrenchment of Unemployment in Aotearoa New Zealand

The political and economic reforms that began after the 1984 elections were based on the assurance that principles of free-market economics would lead to growth and prosperity, and would eventually reduce unemployment. Yet between 1984 and 1998 economic growth remained virtually static and the unemployment rate doubled (Chatterjee 1999:65-67).

Many academics and commentators have detailed the entrenchment of unemployment in New Zealand since the 1970s (Easton 1989, 1997, Green 1994, Kelsey 1993, 1995, Morrison 1991, Waldegrave and Coventry 1987). In contrast, more optimistic reports claim that unemployment rates are declining overall and that the labour market is expanding into new areas (Brash 2000, Cocrombe et al. 1991, Rose 1990). Another group of researchers, however, highlight the increasing prevalence of underemployment, labour market disadvantage and income inequality, arguing that this prevalence often escapes the variables currently used in labour market research (Briar 2000, Clogg 1979, Callister 2001, Easton 1996, 1997, Martin 2000, Peace 1999, Waldegrave 1998). In effect, they say, the possibility of stable and reasonable employment conditions is now an elusive goal for large groups of the population.

The increase of casual and impermanent work means that many people officially recorded as being "in the labour force" are faced with frequent occurrences of poverty and disadvantage (Brown and Scase 1991). In Aotearoa New Zealand being unemployed or underemployed usually means being poor (Waldegrave and Coventry 1987). The logical progression is that assisting people to move towards more stable forms of employment becomes a crucial factor in addressing poverty and creating a more inclusive society. However, radical social theorists such as Beck (2000) and Gorz (1999) highlight the irony of basing social inclusion on paid employment in a society where full-time paid work is a dwindling prospect. Can the Training Opportunities providers be expected to produce employment outcomes when there are too few jobs?

TRAINING OPPORTUNITIES

In 1993 the Training Opportunities Programme (TOP) was introduced to replace ACCESS, which involved employment schemes for low-skilled workers. (5) The decreased availability of low-skilled work undermined the rationale of ACCESS. Although such work schemes were of some value to participants, they were questioned overall because what was the point in training people for jobs that did not exist (O'Connor 1983)? In response to the impasse, employment assistance policies were realigned towards improving the individual performance of participants in the labour market. The TOP courses were introduced as an integral part of the National government's education strategy, which was aimed at:

... raising achievement levels; increasing the participation of underrepresented groups and individuals in education and training; increasing opportunities in the post-school sector; and ensuring that the system is more responsive to changing needs. (ETSA 1992:8) The TOP courses involved a greater emphasis on learning job-seeker strategies, (6) improving general work attitude (7) and providing entry-level skills targeted at areas of high labour demand. For some trainees, Training Opportunities served as a steppingstone to further training and education (ACNielsen 1999). (8)

In their current form, the Training Opportunities courses are designed to assist people who are regarded to be at risk of "long-term unemployment" (DWI/WINZ 2000). These training courses are delivered by private training establishments (referred to as providers in this paper)--organisations associated with churches, iwi groups, community trusts; some private businesses; educational institutions such as polytechnics; and employers. The providers must be approved and registered with the New Zealand Qualifications Authority before they can tender for a contract to deliver a course (ETSA 1992). The Training Opportunities providers offer an adult learning environment for skills acquisition and to build self-esteem and confidence. In 2001, there were 413 training providers; as at 31 July 2001 there were 9,043 trainees, with 21,600 participating over the calendar year (Skill New Zealand 2001). The total cost of Training Opportunities for the year was $94 million (Statistics New Zealand 2001).

The Training Opportunities courses are "targeted", with criteria intended to prevent disadvantaged individuals from being "crowded out" by less disadvantaged people (Ministry of Education 2002). The eligibility criteria are:

* Aged 18-19 years with low qualifications, left school in the last 26 weeks and registered with Work and Income; (9) or

* Registered with Work and Income as an unemployed job seeker for at least 26 weeks, with low qualifications; or

* Registered with Work and Income for fewer than 26 weeks, with low qualifications and assessed by Work and Income as being at risk of long-term unemployment; or

* Registered with Work and Income as an unemployed job seeker for at least 26 weeks, with more than two School Certificate passes or more than 40 credits and assessed by Work and Income as lacking foundation skills; or

* Has Refugee status, with higher qualifications and registered with Work and Income; or

* Participated in Youth Training in the last three months and granted approval by the...