Chapter 17: Has it worked globally? – External Quality Audit

17

Has it worked globally?

Chenicheri Sid Nair

Abstract:

This chapter attempts to draw on the themes that have been brought forward by the international contributors. In so doing, it looks at the evidence presented to answer the question often discussed and debated in scholarly publications and within the walls of higher education institutions – has external audit worked?

Key words

evidence

student voice

quality culture

Introduction

Quality, quality, quality, words that resound within higher education institutions worldwide! The quality factor is much discussed within these institutions, governments and funding bodies. This emphasis on quality harks back to the growth of the education industry. The exponential growth of higher education providers, including offshore providers, has raised concerns about the quality and integrity of programs across the tertiary sector. This growth, which has occurred on an international level, is without doubt due to the push by governments to increase the number of citizens with basic and higher-level post-school qualifications in order to support economies and increase competiveness (Wheelahan et al., 2012). Wheelahan et al. (2012) also suggest that the agenda on social inclusion in higher education has also been a driving factor in the growth of the industry.

This book is the first attempt to draw together international contributions to the discussions and debates that have taken place over many years on the effectiveness of quality systems in terms of the audits that have been carried out. The arguments that have been put forward by critics, scholars and researchers are wide and varying, from the need for accountability to superficiality of processes, lack of impact on teaching and learning, ineffectiveness to support change, and the intrusion of audits on academic freedom. Although Shah and Stanford point out succinctly in their chapter that most of the research that has been carried out in this area is based on opinions and predictions as to the likely impact of quality audits, the chapters by Paewai and by Puteh, Habil and Azli reinforce many of these opinions and predictions on the basis of actual perceptions of the audit process gathered from teaching staff.

What this book brings together are the thoughts, perceptions and experiences of an international group of contributors with regard to the impact that quality audits have on them, their institutions and the higher education community in general.

Have audits been successful?

The crux of the question about audits is whether all the work over the years has made any discernible impact on the sector. Have the time and effort been in vain? Has the audit exercise been just that, an exercise that institutions and agencies go through year after year without any concrete change or improvement or commitment?

In order to get a feel for the state of play and to draw a final conclusion, one would have to look at the specifics.

One of areas that the book touches upon is the student voice. The evidence here suggests that this area has developed significantly since quality audits were instituted (see chapters by Jennings and Cameron; Kubuabola; Lange and Singh; McTaggart; Palmer). On the Australian and Fijian scenes effective monitoring systems and processes have been developed to allow students to provide feedback on all areas of teaching and learning. The New Zealand experience highlights the introduction of surveys on the quality of student engagement as a result of the quality agenda. Though the consensus of the contributors is that such systems were built to address a shortcoming, there were also weaknesses where the audits had no impact. For example, in the Australian context teaching evaluations are considered by many institutions to be private and confidential to the academic, with little that can be done to effect change. However, Shah and Nair (2012a) point out that there has been some movement where institutions have sought to address this issue by linking the outcome of the survey to the annual performance review process. Although it is acknowledged that there are weaknesses in the systems and processes, there is also clear acknowledgement that external quality audits have resulted in a notable increase in the discussion by academics and management of issues relating to teaching and learning.

A further example cited by the contributors is the increased engagement of students, resulting in management’s recognition of differing levels of achievement in teaching and learning at institutions. However, McTaggart’s chapter clearly enunciates that although there was movement in this area, there is something of a dilemma in terms of investment in teaching and learning from the perspective of support given to academics. The argument seems to stem from the fact that universities are investing the bare minimum to ensure meeting their key performance targets, with no real investment in improving teaching at the individual level.

One area that seems to have emerged internationally as a result of audits is recognition of benchmarking as an essential process in continuous improvement. Although this important development has been recognized, little has been said about the effectiveness of such benchmarking exercises; nor has benchmarking been used for improvements or comparison tied to internal key performance indicators. There is, however, a clear indication that key performance indicators are a driving force in the quality agenda of a number of institutions.

Management of quality was another key theme in many of the chapters. The evidence presented by a number of contributors points to a management structure in which quality has been recognized as being part and parcel of the operations of the institution (see chapters by Kubuabola; Padró; Puteh et al.; Lange). The experience of contributors is that one result of quality audits has been a number of initiatives on the management front. These include the setting up of specific committees to address areas of concern, senior management staff including quality assurance in their portfolio, definition of policies to address quality matters, definition of processes to address quality activities and the development of reporting strategies.

Though the major discussions in the book revolve around audit within the public university sector, two chapters address a sector that has been growing rapidly in the last few years: that of private provision (see chapters by Gupta and by Shah and Stanford). The reach of private higher education has grown in many parts of the world, with more than 70 per cent of students attending private higher education institutions in India, Malaysia, Japan, South Korea, Indonesia and the Philippines, more than 30 per cent in mainland China and more than 15 per cent in Thailand and Vietnam (Gupta, 2008).

The private higher education sector has not been well researched, as compared to the publicly funded universities. This having been said, private higher education has played a revolutionary role in a number of countries, where it has provided an alternative tertiary education choice for many students who otherwise might not have had direct access to university education. The sector has provided a different kind of access and opportunity for diverse student groups, with location, ease of entry, specialized areas of teaching and small class sizes seen as key factors influencing student choice (Shah and Brown, 2009).

Such rapid growth has raised concerns about the quality and standard of education on offer (Shah and Nair, 2012b). Although Shah and Nair (2012b) highlight a number of issues relating to private providers, the author of the present chapter systematically shows how an audit process can be used to add value, resulting in enhancements and positive outcomes. The author of the present chapter highlights this positive outcome by citing evidence of high satisfaction in a post-audit feedback at the private institution, thus lending support to the contention that external quality audits raise internal capabilities for continuous quality assurance (Scott and Hawke, 2003). Gupta, in her chapter, alerts us to the complexity and sheer size of higher education in India and the effect of private providers in the delivery of educational outcomes. She clearly shows that there have been positive changes as a result of government measures relating to the quality of private providers’ delivery.

The view of many of the chapters is that external audits are useful and have matured with time, thus helping institutions to deliver better courses and programs, better teaching and learning and support to students, and resulting in improved management structures (see chapters by Ala-Vähälä and Saarinen; Jennings and Cameron; Kubuabola; Liu; McTaggart; Palmer; Shah and Stanford; Stensaker; Lemaitre et al.). It is safe to say that the general consensus in these chapters is that quality audits have been successful in embedding quality culture in the various stakeholders engaged in the education sector (see chapters by Gupta; Kubuabola; Lange; Palmer; and Shah and Nair, 2011). An important observation is that many developing countries have experienced positive outcomes as a result of quality audits and accreditation. This is important, as these developing economies are striving to achieve the best they can in a competitive global environment. Whether developed or developing, the international perspective is that external audit has a positive effect on change dynamics in higher education. With regard to improvements as a result of external audits, views are mixed, ranging from opinion that changes have been effective to skeptical. Most contributors are of the view that there is little evidence of external audits being successful in totality, but audits have resulted in improvements in specific areas, as discussed earlier in this chapter.

Where to from here?

An interesting observation in this book is that all moves towards quality assurance have been initiated by government in the respective countries. There is no evidence to suggest that quality systems were already in place in order to ensure that institutions maintained appropriate services or effected changes as and when necessary. The evidence of the contributions suggests that the shift to a quality culture has been greater when this was forced upon institutions. As pointed out earlier, however, it is safe to say that the general consensus is that external quality audits have been the means to successfully embedding a quality culture among the various stakeholders involved in the education sector.

The question that I am left with at the end of this book is: where to from here? Should we continue with the same approach, live with the shortcomings and hope that things will change? Or do we, as a community, improve on the shortcomings? Palmer, in his chapter, enunciates the principles of evolutionary change, recognizing strengths and weaknesses and building on these to create better systems. This is the quality cycle at work and puts forward the notion that the system we have is in fact effective. While I agree that this is the most logical approach to change dynamics, I am left pondering on the quality agenda in Australia, which has made a seismic move from the external audit regime to a regulatory framework.

There is now a regulatory body in Australia for quality, the Tertiary Education Quality and Standards Agency (TESQA). The aim of this body is to strengthen the quality and standard of all providers of higher education in Australia. The legislation relating to the functions of this body is yet to be fully tested, but the initial signs are that TESQA has the necessary powers to step in at any time to address any ‘risk’ issues relating to quality, including the ‘reputation’ of higher education in Australia. This leaves the question of what the roles of universities are, and feeds into arguments about the restriction of academic freedom. But the greater question is this: what is the role of higher education providers in providing innovative education programs? If an external body has the power to step in, does it have the power to resource changes that it deems necessary?

I ask this question as one particular issue has sent shivers down my spine: an Australian university was asked to explain why an honorary doctorate was awarded to a foreign dignitary who was not supported by a number of citizens of the dignitary’s homeland. Here I question what the quality issue is and what the risk factors are that will impact on the institution in its delivery of high-quality teaching, learning and research.

Is this the quality route for higher education for the future? This is a road untraveled, and the whole world is watching to see if this model will provide a better route to a better and higher quality of higher education. Is this approach indicative of the way in which higher education institutions will function in the future, with all decisions made by a central body?

Whatever the future holds, the evidence in this book suggests that external quality audits have been an effective vehicle for engagement with quality issues. Maybe the best is yet to come!

References

Gupta, A. International trends and private higher education in India. International Journal of Education Management. 2008; 22(6):565–594.

Scott, G., Hawke, I. Using an external quality audit as lever for institutional change. Assessment & Evaluation in Higher Education. 2003; 28(3):323–332.

Shah, M., Brown, G., The rise of private higher education in Australia: Maintaining quality outcomes and future challenges. Proceedings of the Australian Universities Quality Forum (AUQF). Alice Springs, 2009:143–150.

Shah, M., Nair, C.S. The influence of strategy and external quality audit on university performance: An Australian perspective. Tertiary Education and Management. 2011; 17:139–150.

Shah, M., Nair, C.S. The changing nature of teaching and unit evaluations in Australian universities. Quality Assurance in Education. 2012; 20(3):274–288.

Shah, M., Nair, C.S. Private for-profit higher education in Australia: Widening access and participation and opportunities for public–private collaboration. Higher Education Research and Development Society (HERDSA); 2012. [forthcoming].

Wheelahan, L., Arkoudis, S., Moodie, G., Fredman, N., Bexley, E. Shaken not stirred? The development of one tertiary education sector in Australia. Adelaide Australia: National Centre for Vocational Education Research; 2012. [Research report].