Increasing social science research capacity: some supply-side considerations.

• Author: Lunt, Neil

Abstract

This paper discusses the development of policy-related research capacity, with particular reference to the place of tertiary education. With regard to the recent demands for more policy-focused research, it argues that deficiencies on the supply side may prove a systematic inhibitor to a more satisfactory "utilisation equilibrium". These are issues that the public sector in the broadest sense need to act upon. Otherwise there is the danger that a capacity gap will result in growing dissatisfaction with the search for answers to the evidence-based questions "What works?" and "Where is the proof?"

INTRODUCTION

An important shift in policy thinking, both locally and internationally, has been the call for better evidence on which to base policy, programme and professional intervention. At all levels of policy development key questions being articulated are "What works?" and "where is the proof?" The question of what qualifies as adequate proof has always been a complex one, combining as it does political influences, organisational memory, tacit knowledge, and the persuasiveness of various proponents (Polanyi 1957, Kuhn 1962, Lindblom 1979, Smith 1999). However, and perhaps even because of a recognition of that complexity, within the range of sources privileged as acceptable as evidence, research has always laid claim to a prized place.

A number of factors are exerting pressure for evidence-based, or at least evidence-influenced, activity. The debate is a wide one, encompassing a plethora of policy spheres, disciplines and methodologies. A key driver here is the continuing emphasis within the public sector to secure value for money, close gaps, ensure accountability, and improve service delivery. It has been suggested that high-quality evidence is increasingly being demanded as part of the policy process (Davies et al. 2000, Ministry of Social Policy 2001). There has been recognition of the limited part played by more focused evaluation, and a debate has re-emerged about the relationship of research, evaluation and policy (State Services Commission 1999, Smithies and Bidrose 2000, Ministry of Social Policy 2001). We say re-emerged because the debate about the role of research and its links to government policy is a longstanding one, with discussions around the use of social science within government beginning in the 1930s (cf. Robb 1987) with the Social Science Research Bureau; growth within the government sector and the universities in the post-war period (Marsh 1952, McCreary 1971, Mackay 1975) and disillusionment from the 1970s and the corresponding search for an appropriate organisational model and configuration of social science (e.g., Gibson 1970, Shields 1975, Fougere and Orbell 1975, Keir 1982).

The purpose of this paper is not to rehearse debates about the value of research in the policy-making process, but to offer some ideas about preconditions for usage. The paper then identifies some of the barriers currently present in the tertiary and public sector that will--in all probability--militate against that maximisation, and it ends with some suggestions about what could be done to overcome those barriers. Essentially, our discussion is driven by what the Ministry of Social Policy (2001) discussion document refers to as a "rightward shifting demand curve by central agencies", and what we identify as a particular bottleneck to supply emanating from the tertiary sector.

Internal and External Capacity

High-quality, policy-relevant work may be sourced from a range of suppliers. Here we identify three main types:

* an agency's own internal research and evaluation capacity;

* a contractor-commissioner relation (with the contract going to the tertiary sector, market research sector, private contractors or a mixture); and

* a hybrid relationship, which may involve internal research capacity combined with external modifiers--typically consultants bringing specific skills.

Genuine contractor-commissioner relationship took hold from the 1970s, with grants-in-aid being replaced by tighter contractor specifications. The period since has seen a refinement of the contractor relationship and "Rothschild principles" (2) (Kogan and Henkel 1983), with research commissioning and management an identifiable public service role (Hawkins 2000).

A range of steps have been identified as crucial to ensuring good-quality research, and these focus on:

* effective commissioning;

* effective research in the field; and

* effective utilisation.

Clearly, these stages are in practice interrelated. Determinants of the final quality and impact of commissioned research include (but are not restricted to):

* adequate conceptualisation of research and framing of questions--projects that ask interesting, but not the most policy-relevant, questions, or adopt inappropriate methods are foils to evidence-based activity;

* research that has clear and achievable objectives, and promises outcomes that are timely;

* effective research management and liaison to ensure early identification of problems (commonly, scope creep or timeframe compression);

* rigorous analysis that addresses research questions and has policy relevance, but also recognises the limitations as well as the strengths of the research; and

* highly trained and prepared researchers, with adequate supervision, working within optimal-sized teams (Weiss 1977, Weiss 1998).

Many of these considerations were identified and discussed by the State Services Commission in its timely review of evaluation and policy advice (1999), and are echoed in the social policy capacity review and Ministerial Statements (Ministry of Social Policy 2001, Minister for Research Science and Technology 2001, Social Science Reference Group 2001). The State Services Commission's report focused on a range of issues around the utilisation of the research results. Discussion of research utilisation and relationships between research commissioners, policy advisers and Ministers are to be welcomed. Our discussion here is far more modest in scope, and driven by the fifth point we have outlined above (researcher capability). Arguably, it is one that must be addressed for evidence-based thinking to gain a stronger foothold in particular policy areas.

Instead of focusing on the demand side of social science research (what research gets commissioned, and what happens to the products of research), we discuss supply-side considerations and, in particular, research training and preparation. The piece is informed by historical and contemporary reflections on the development of social science and social research within New Zealand; teaching and writing in the area of social research methods; and our experience of contract research work and relationships with government departments.

Broadly, we argue that better research preparation will contribute to narrowing the gap between supply and demand of policy-relevant research. We suggest a number of possible directions, including rapprochement between academic teachers/researchers and those based within central departments. We write from disciplinary traditions of sociology, political science and psychology, drawing on backgrounds that lean towards social--as opposed to more economic--research. Many comments we make, however, will also speak to those more heavily economic constituencies.

REVISITING THE SUPPLY SIDE: THE ORGANISATION OF APPLIED SOCIAL RESEARCH

It is worthwhile briefly reviewing the roles of those involved in the contracting/ commissioning relationships, and how these roles have altered in recent years as the result of particular pressures.

Government Departments

Government departments have had a relationship with social (and economic) research that stemmed from a desire to collect routine statistics and information about their activities to inform reporting and service delivery. Some of the earliest examples of specialised policy advice being utilised in making government policy were in the form of economists giving evidence to Royal Commissions and establishing networks with senior public servants and Ministers (Endres 1987, Endres and Fleming 1994). Governmental research capacity was largely established during the 1950s and 1960s. The 1970s was a period of continued development and discussion about the appropriate place of social science research--and growing pressure (albeit unsuccessful) to establish an independent social research agency away from immediate short-term pressures (Shields 1975, Fougere and Orbell 1975). The 1980s and early 1990s can be seen as times when less emphasis was placed on research activity, and major changes were introduced without (and sometimes in the face of contradictory) research evidence. There was a running down of research capacity, and social science was seen to have a more limited contribution to make to society and policy than previously anticipated. (3) There was a focus on a priori economic models as a basis for action, rather than an investigation of the interplay of theory, action and evidence (cf. Easton 1997).

Since the early 1990s there has been increased emphasis on applied research, with a particular focus on evaluation of effectiveness. Central to the latter has been the attempt to address the question, "Does it work and how can we improve it?". There has been an attempt to go beyond costs and inputs, to prioritise outcomes and impacts. The language of evaluation (formative, summative), a desire for outcomes, and stakeholder consultation within the research process, are increasingly emphasised as part of central agency activities. A range...