This was my research-based dissertation for my Master’s degree. Click here to download a pdf of the full research study.

Abstract: 

Australian organisations have experienced unprecedented changes over the past two decades but there has been little research into whether these changes have achieved their objectives. This paper sets out to redress this lack of empirical evidence.

This paper uses a meta-analysis of 50 case studies covering 6079 Australian organisations to discover whether organisational change and development in Australia actually works; whether some change methods work better than others in an Australian context; and whether some methods work better in some industry sectors than others.

The paper discusses the background to Australia’s love affair with organisational change, relating it primarily to industrial and economic reforms of the 1980s.  It examines the underlying theoretical foundations of organisational change techniques and proposes a model for identifying the orientation of each technique on two continuums – one showing the alignment to scientific versus behavioural management, and the other showing the technique’s position in relation to contingency theory versus ‘one best way’.  Three research questions are identified, and various statistical techniques are used to test them. 

Research findings indicate that approximately half the organisational change initiatives in Australian organisations are successful, and half are not.  Those that are aligned to principles of behavioral management and that follow a process of contingency theory in their application appear to be significantly more effective than change initiatives that hold to scientific management principles and follow a prescriptive or ‘one best way’ application.

The findings are disquieting organisational change in Australian organisations has, at best a chequered track record.  There are strong indications that some methods are more effective than others in bringing about lasting organisational development in an Australian context.

The findings from this research inform significantly the current debate on change and development within Australian organisations.  They establish a clear agenda for future research which, it is hoped, will contribute to a greater level of sustainability and international competitiveness in Australian organisations.

TABLE OF CONTENTS

CHAPTER ONE : BACKGROUND………………………………………………………. 5

CHAPTER TWO : INTRODUCTION…………………………………………………… 9

What is organisational change and development?…………………………………… 9

Drivers of organisational change and development………………………………. 10

The “end of certainty”………………………………………………………………. 10

Technological change and innovation………………………………. 12

Domestic reform agenda………………………………………………….. 12

Globalisation……………………………………………………………………. 12

An ethical imperative………………………………………………………………… 12

The Australian response………………………………………………………………….. 13

Organisational responses…………………………………………………………………. 14

Shaky foundations?………………………………………………………………………… 17

Structure of this document………………………………………………………………. 18

CHAPTER THREE : LITERATURE REVIEW……………………………………… 20

A brief history                                                                                              20

The Early Era: Scientific management………………………………………….. 20

The Middle Era: Behavioural management……………………………………. 21

The Present Era : Hybrid management…………………………………………. 22

The contingency approach……………………………………………………. 23

The emergence of organisational change and development……………………. 23

An orientation of the different OC&D methods…………………………….. 25

Old wine in new bottles?……………………………………………………………. 25

The role of culture in organisational change and development………….. 26

Transferability of OC&D practices across national cultures………………. 29

The efficacy of organisational change and development…………………… 31

Organisational change and development in Australia……………………………. 32

Business Process Re-engineering………………………………………………………. 33

Total Quality Management………………………………………………………………. 35

CHAPTER FOUR : RESEARCH QUESTIONS AND METHODOLOGY.. 37

The Research Questions………………………………………………………………….. 37

Scope and approach……………………………………………………………………….. 38

Methodology                                                                                               38

Phase 1 : Literature search………………………………………………………….. 38

Phase II: Analysis of case studies…………………………………………………. 40

Testing research question one…………………………………………………….. 41

Testing research question two…………………………………………………….. 41

CHAPTER FIVE : PRELIMINARY FINDINGS……………………………………. 43

Phase I: Literature Search………………………………………………………………… 43

The revised approach…………………………………………………………… 43

Phase II: Analysis of case studies………………………………………………………. 46

Item 1: Case Type…………………………………………………………………….. 47

Item 2: Primary change method used……………………………………………. 49

Item 3: Industry……………………………………………………………………….. 50

Item 4: Number of organisations covered……………………………………… 51

Item 5: Statistical methods used…………………………………………………… 53

Items 6, 7, and 8 : Success scores…………………………………………………. 55

CHAPTER SIX : RESULTS AND DISCUSSION…………………………………… 58

Research Question One………………………………………………………………….. 59

Have OC&D initiatives in Australian organisations over the past decade been successful?……….. 59

Research Question Two:…………………………………………………………………. 60

Are some OC&D methods more successful in Australia than others?……………… 60

Descriptive statistics: Average success rate of different methods….. 60

ANOVA: Analysis of Variance between methods……………………… 61

Question Three:…………………………………………………………………………….. 64

Is OC&D more successful in particular sectors than others?………………………… 64

Limitations of this research……………………………………………………………… 65

CHAPTER SEVEN : IMPLICATIONS AND FUTURE RESEARCH DIRECTIONS         68

Possible Reasons?………………………………………………………………………….. 69

Preferred orientation?………………………………………………………………… 69

Workforce characteristics?………………………………………………………….. 71

National culture?………………………………………………………………………. 71

Participation and empowerment?………………………………………………… 72

Short and concise…………………………………………………………………………… 74

Future Research Direction……………………………………………………………….. 74

APPENDIX A : Case Studies forming the formal data set………………………….. 75

APPENDIX B : Data table for ANOVA analysis……………………………………… 87

CHAPTER ONE : BACKGROUND

“In 1993 our organisation spent two and a half million dollars on becoming a ‘learning organisation’.  We hired a consultant and held workshops in every state and regional office.  It was a waste of money that would have been better spent on public health.  Nothing seems to have changed, we have the same problems today that we had then, and we are about to embark on yet another round of organisational change and development.  Will we ever learn?”

Comment from a regional director of the Australian Department of Health and Community Services, 25/7/1999

Organisational change and development comes in many shapes sizes and acronyms, from BPR, TQM, and JIT to productivity reforms, downsizing, upskilling, broad-banding and so on.  More than a third of Australian organisations have introduced some form of organisational change over the past decade (Vickery & Wurzbury 1998) yet there appears to be a groundswell of opinion that little has changed (Kerka 1995, Savery 1996, Snell & Chak 1998).  

Internationally and increasingly, academics are questioning the efficacy of organisational change methods, with some describing them as ‘fads’ (Carson et al 1999), ‘fashion statements’ (Abrahamson 1996)  and ‘a waste of time and money’ (Deakins & Makgill 1997). 

It seems that many Australians believe that despite the rhetoric of employee empowerment and participation, organisational change and development initiatives work solely in the interests of the shareholders at the expense of the employees (Savery 1998).

Australia has been pushing over the last decade for productivity reform, yet most people believe they can still improve their productivity (Savery 1998) and that minimal benefits have been realised from these changes (Mescal 1994).

These anecdotes accord with this author’s experience of organisational change in the Northern Territory public sector, and have been echoed in the sentiments of many people with whom I have spoken during the collation of this dissertation.

Australia is not alone in this regard. Professor Geert Hofstede, an internationally respected scholar and organisational change practitioner suggests that, no matter what flavour of organisational change is favoured, unless certain conditions are present the same problems will continue to manifest (Hofstede 1999).  Abrahamson (1996) suggests that organisations the world over are prey to the whims of management theory ‘fashion designers’ in the same way that people are quarry to changing dress fashions.  The latest management fashions, according to Abrahamson, do little to change the underlying problems of organisations, but rather they dress them up in the latest vogue.

‘Do we know what works and what doesn’t in Australia, and why?’

So what is really going on in Australian organisations?  Are we, as Abrahamson (1996) suggests, simply following one management fashion after another, to no great effect?  Are we, as Mescal (1994:9) suggests, turning to institutions “other than conventional religion for sources of meaning in our lives”?  Are the perceptions that organisational change and development initiatives fail, valid?  Do we know what works and what doesn’t in Australia, and why?

Certainly Australian industries have faced a barrage of ‘new’ organisational change methods over the past two decades.  Emphasis in the early 1980s was on quality, following the emergence of a vast literature on Total Quality Management, Quality Assurance Programs, Quality Circles, and the like.  These methods imported management techniques primarily developed in Japan and which are believed to have successfully shifted Japan to economic prosperity in the post World War Two years. 

In the early 1990’s Business Process Re-engineering with its North American accent stepped into the limelight, urging for more clever use of technology in organisations, to “obliterate rather than automate” inefficient processes. Australia embraced BPR with vigour, convinced that its apparent ‘radicalness’ from other methods would yield the quick and magnificent gains it promised – claims of thirty percent reductions in costs; eighty percent reduction in delivery time; and sixty percent reduction in inventory were not uncommon (Fitzpatrick et al 1999).

1995 heralded a sustained push for Australian firms to become Learning Organisations, based on the notion that only those organisations that can learn and adapt faster than the rate of change will survive in an increasingly turbulent and competitive global economy (Revans 1991). This new approach was supported by the Australian government via the Karpin report (1995) and drew from British, American, and Australian schools of thought.

In the late 1990s the focus shifted slightly again, this time towards Benchmarking and Best Practice programs.  Best Practice aims to examine business practices from all over the world, and to adopt within Australian organisations those practices that are identified as “the world’s best”.

In parallel to these past two decades of change, or perhaps as a result, the Australian people have witnessed the commercialisation of public utilities such as power, water, and telecommunications facilities; governments everywhere are outsourcing, downsizing, and restructuring; enterprise bargaining has replaced union-led employment awards, and the concept of job security has taken on a much shorter-term focus than in previous decades.

‘little is known about the efficacy of the methods of change’

Despite Australia’s apparent love affair with organisational change, little is known about the efficacy of the methods of change in an Australian context.  Some academics have focussed on examining the efficacy of a particular method, such as Sohal’s studies on TQM (Sohal 1994, 1995a, 1995b, 1995c, 1996) but even in these instances the scope of the studies is limited to a specific method within a specific industry – TQM and manufacturing, in the case of Sohal.  Little compilation has been done in Australia across industry sectors, or across different change methods, and there appears to be no comparison between methods. We know very little about whether organisational change actually works in Australia, or about what works and what doesn’t.

This dissertation is an attempt to bridge this gap.  It seeks to discover whether organisational change and development in Australia works, whether some methods work better than others, and whether some industry sectors have greater success implementing organisational change than others.  

This research relies on existing literature to tell the story of organisational change in Australia.  It draws from existing case studies and compares common elements across sectors and across methods.

It is hoped that these findings will inform significantly to the current debate on change and development within Australian organisations.  It is also hoped that the findings will contribute to establishing a clear research agenda that will lead, ultimately, to the future health of Australia’s organisations and their employees: to the focussing of Australia’s resources on developing organisations that are not only internationally competitive but also socially ethically and environmentally sustainable.

CHAPTER TWO : INTRODUCTION

What is organisational change and development?

Organisational change and development is a broad concept that can include strategic, structural, and behavioural dimensions (OECD 1996).  Organisational change can be a very complex phenomenon occurring across multiple layers of an organisation, with strategic innovations disseminating from the corporate level and work reorganisation and technology innovations occurring at the operational level.  Or it can be a single specific intervention into a single specific aspect of organisation, such as the outsourcing of a non-core business function or the re-engineering of a specific business process. (OECD 1997). 

“a planned process of increasing organisational effectiveness.”

Definitional problems bedevil the issue of exactly what organisational change and development is.  For example, considerable confusion exists in the literature between the terms ‘organisational change’, ‘organisation development’ (commonly referred to OD), and ‘organisational development’.  Traditionally, organisation development refers to a specific set of interventions for achieving change whereas organisation­al development refers to the whole and coordinated process of achieving change using many tools, including OD techniques.  Many writers have questioned whether  differences exist between the various terms and techniques (eg Sohal 1995a, Shein 1999) with some suggesting that the majority are simply repackaged versions of existing techniques (eg Abrahamson 1996). The United Nations Development Programme has its own label for organisational development, terming it “Institution Building”.  Institution Building is taken to include “any activities or interventions within an organisation or within a system of organisations that are designed to make it better at doing what it is meant to do” (UNDP 1996:15).

Orr et al (1999) state that “ the terms and descriptions used in the literature for organisational changes vary, but the overall objective is the same, to increase organisational effectiveness and to ensure the future of these organisations by establishing more efficient operations” (p.657).

Therefore, to avoid semantic confusion I have fused these terms into an all encompassing ‘organisational change and development’ (OC&D), applying my own definition of “a planned process of increasing organisational effectiveness.”  In this way, organisational change and development would encompass any planned organisational change or development activity provided that the motive is to increase organisational effectiveness. 

There is no one definitive measure of ‘increased organisational effectiveness’, and in any case its interpretation would vary from industry to industry, and according to the particular circumstances, culture, economic situation and so on of the organisation undergoing the change. Measurements of increased organisational effectiveness might for example include reduced production costs, increased market share, and lower employee turnover (Vickery & Wurzbury 1998).  In some instances it could simply mean organisational survival in a time of  increased competition or turbulent change (Montuori 2000). Yuchtman & Seashore (1967) propose that “organizational effectiveness be defined as the ability of the organization, in either absolute or relative terms, to exploit its environment in the acquisition of scare and valued resources” (p.898). Child (1995) reviews the relationship between organisational structure and performance measurement and concludes that “in whatever perspective organisational performance is placed, an adequate definition is problematic” (p.417). 

Drivers of organisational change and development

It is neither possible nor indeed necessary to identify all of the specific reasons for Australian organisations undertaking OC&D initiatives: reasons vary according to industry, geographic location, economic and political climate and so on.  However, we can broadly observe that OC&D is usually a response to changes in the greater environment.  Some of these changes include, but are by no means limited to, the general issues identified in
Figure 1.

The “end of certainty”

In trying to explain the complex forces of change, Toffler (1980) talks of the “third wave” which will sweep through society and reshape everything that we consider familiar.  The first and second waves, according to Toffler, were the agricultural and industrial revolutions.  The changes envisioned in the third wave are not going to be incremental in nature, but are predicted to be quantum jumps accompanied by great upheaval. This prediction is echoed by Mike Pedler (1997) who suggests, somewhat tongue in cheek, that the world is reaching the end of certainty, that “shortish period in history when we believed we could control the world and manage our affairs” (p.xxix).  The post-modernist extension is that we have entered the era that marks the end of era-style development: uni-directional progress as such no longer exists, and change and development are now multi-directional and paradoxical (Pedler et al 1997).

An ethical imperative

The motivation for change can also be viewed from an ethical imperative. According to Pedler et al (1997) and the United Nations (UNDP 1995, 1996, 1997) organisations must not be restrict themselves to issues of the bottom line: growing concerns over the environment, questions of ethical business behaviour, and the growing inequality between rich and poor nations also serve as drivers of change in our increasingly interconnected world.

‘it is not sustainable to operate in a way that is technically elegant but ethically or environmentally incompetent’

Management accountability for the secondary costs of their organisation’s actions, described by some writers as social and economic disintegration (Blunt 1995), is called for.  From a ‘whole earth’ perspective, it is not sustainable in the long term for organisations to operate in a way that is technically elegant but ethically or environmentally incompetent (UNDP 1995, 1996).

The vulnerability of organisations to fall behind the rapid pace of this third wave of change is said to be reasonably evidenced by the mortality rate amongst Fortune 500 companies: Pascale (1991) notes that by 1987 two thirds of Fortune 500 companies from 1982 – only five years later – had slipped from their pinnacle. 

The Australian response

Throughout the 1980s and 1990s the Australian government introduced significant reforms in an attempt to improve Australia’s productivity and international competitiveness.

One arm of these reforms focussed on government-sponsored training initiatives designed to build skills and competencies in Australian organisations, for example through initiatives such as competency based training, the introduction of a vocational training system, and the introduction of a training guarantee levy designed to increase spending on training (Laffety & Roan 1999).

Another arm of reform focussed on industrial relations.  In response to mounting pressure from employer groups, unions, and trade partners, in 1988 the Commonwealth government launched a sweeping review of the centralised Australian industrial award system that had governed the terms and conditions of Australian employees since 1904 (Smith & Hayton 1999).  The reforms became known as Award Restructuring, and, guided by the overarching Structural Efficiency Principle, led in 1991 to the introduction of enterprise bargaining which allowed organisations to strike employment agreements with their employees as an alternative to union-negotiated award agreements.  This gave Australian organisations – and arguably workers – a previously unexperienced level of flexibility in terms of negotiating employment conditions and, some authors claim, brought with it greater job insecurity and employee turnover (Abbott 1996; Kramar 1998).

Another strand of productivity and efficiency reforms has been economic deregulation, through the gradual dismantling of tariff protection systems and deregulation of key areas of the economy (Smith & Hayton 1999).  These developments have exposed Australian organisations to the rigours of international competition as well as a higher level of domestic competition.

‘Enterprising Nation provides the blueprint for taking Australian enterprises into the 21^st century’

Following on from these reforms, the Australian government appointed an industry task force, chaired by David Karpin, to investigate the leadership and management skills required to bring about permanency of reforms and constructive changes within Australian enterprises. The resulting report, “Enterprising Nation: renewing Australia’s managers to meet the challenges of the Asia-Pacific Century” (Karpin 1995) provides numerous recommendations which form the blueprint for taking Australian enterprises into the 21^st century.  To comply with this government-endorsed blueprint, Australian organisations need to pursue four key areas of organisational change and development:

1. To develop a positive enterprise culture;
2. To develop best practice programs;
3. To recognise, value, and capitalise on diversity;
4. To become learning organisations

Clearly, to achieve these goals, organisational change and development must continue within Australian organisations for many years to come.

Organisational responses

The impact of government-led reforms on Australian organisations should not be underestimated.  In a thorough investigation of a decade of industrial reform in Australia – that is, the working relationship between industry, organisations, and employees – Davis & Landsbury (1996) observe that “the decade since the unveiling of the federal government’s policy paper on industrial democracy and employee participation [1986 – 1996] has witnessed some of the most fundamental industrial relations reforms this century” (p.5).

Australian organisations have experienced extraordinary change over the last ten years: a 1991 survey of 2,353 Australian workplaces found that 86 percent of managers reported experiences of major change and restructuring in the previous two years, including reorganised management structures, changed senior management personnel, and significantly restructured work practices (Callus et al., 1991). In 1993, 25 percent of all Australian businesses in manufacturing and 14 percent in non-manufacturing implemented advanced management techniques, changed management or workplace structures, or adopted enterprise bargaining (ABS 1995).

‘management and organisational theories for ‘organisations struggling to control their destiny’ abound’

An article from the Business Review Weekly paints a somewhat frenetic picture of  Australian organisations’ responses to these myriad of changes:

“Organisations are struggling to gain control of their destiny.  They exhibit an overwhelming fear and desperation as they realise what worked in the past can no longer assure their future prosperity”

(James 1998:38).

Whether this ostensible desperate reaching paints an accurate picture or not, there is certainly no shortage of proposed solutions: management and organisational theories for ‘organisations struggling to control their destiny’ abound.

Some of these approaches include (and are by no means limited to):

• quality circles
• business process reengineering
• organisational culture change
• employee gain-sharing
• employee empowerment
• organisational restructuring
• best practice
• networked organisations
• total quality management
• learning organisations

The pace does not seem to be slowing, with a veritable army of ‘newer’ and ‘better’ change concepts marching across the pages of the academic literature every year.  Concepts such as “spherical organisations” (Miles & Snow 1999); “eco-organisations” (Shareef 1999); “sibling organisations” (Bagshaw 1998); “enlightened organisations” (Griffey, 1998);  “autogenic crisis” (Barnett & Pratt 2000), “cladistics” (McCarthy & Ridgeway 2000),  “conscious organisations” (Heaton & Harung 1999) and others litter the literature ad infinitum. 

“We are like a big fish that has been pulled from the water”

This thirst for change can be linked to the Chinese saying about the fish, quoted in the introduction of Lester C Thurow’s classic “The Future of Capitalism” (1994):

“We are like a big fish that has been pulled from the water and is flopping wildly to find its way back in.  In such a condition the fish never asks where the next flip or flop will bring it.  It senses only that its present position is intolerable and that something else must be tried”

Anonymous Chinese saying

Whether or not Australian organisations are “flopping” from one change method to another, one thing is certain: techniques to help organisations discern between the different methods are lacking, and empirical evidence to support their efficacy is in short supply.

‘the majority of research based evidence supports the notion that OC&D in Australia has been largely ineffective’

For example, this author was not able to locate a single research based study that supports the notion that organisational changes implemented in Australia over the past two decades in response to economic and industrial reforms have indeed achieved the aims of those reforms, that is, improved productivity and efficiency and increased international competitiveness.  As will be demonstrated in later chapters, the majority of research based evidence in fact supports the notion that OC&D in Australia has been largely ineffective.

Shaky foundations?

This is perhaps to be expected for there is no shortage of academic literature that highlights inconsistencies in organisational development and change theory, suggesting that the theoretical foundations underlying the majority of OC&D techniques are shaky indeed. 

Blunt & Jones (1997) speak of a considerable body of empirical and  theoretical evidence that points to serious limitations of organization development techniques.  Nevertheless, they say, these limitations “have not prevented these ideas from assuming the status of organizational imperatives in the eyes of many practitioners and theorists in Western societies – to the point where, according to Sinclair (1992), notions such as teamworking have become tyrannical and their denial tantamount to heresy” (p.9-10).

As will be discussed shortly, culture is reported widely to play an important role in the success or otherwise of organisational change initiatives.  However, Clark (1999) reviews two decades of literature on the subject and reports that detailed reviews of the cross-cultural management /organisation literature “have consistently highlighted major methodological, epistemological, and theoretical deficiencies” (p.521)

Closer to home, Schaafsma (1996) reports considerable gaps between the recommendations contained within The Karpin Report, and research-based evidence to support those recommendations.  This is surprising given that the Karpin Report purports to provide the blueprint for Australian industry to “meet the challenge of the Asia-pacific Century” (Karpin 1995).

‘it is not before time to suggest that we in Australia stop to ask why’

The reasons for this inconsistency between research based evidence and actual practice in Australian organisations are unclear.  It could be that Australian organisations are simply following normal scientific herding instincts (Kuhn 1962); it could be that pressures to conform to the latest orthodoxy are so strong that, as Carey (1981) found, evidence falsifying the ruling paradigm may be routinely ignored or discounted.  It could be that the reputation of managers relies on them towing the line of particular government or industry taskforce chairs, or it could be, as Abrahamson (1996) suggests, that organisations fall prey to the glitzy rhetoric and slick marketing of management ‘fashion setters’. 

Or it could be that we are like the proverbial fish out of water.

Whatever the reason, it is not before time to suggest that we in Australia stop to ask why.  This answer to ‘why’ is beyond the scope of this research, however it does seek to lead and inform the debate.

Structure of this document

So far, the introduction has proposed a new definition of organisational change and development and provided some background information on what it is.   It has discussed some reasons why companies undertake OC&D, and has examined this issue from a specifically Australian perspective.  It has briefly looked at how the Australian government and Australian organisations have responded to changes in the international and domestic arenas, and has highlighted inconsistencies between academic research and these organisational responses.

The rest of this work will examine evidence from within Australia to determine the efficacy or otherwise of organisational change and development initiatives over the past decade.  The remainder is structured as follows:

Chapter Three: Literature Review

          Provides a brief overview of the history of organisational change and development, highlights some of the major schools of thought and conflicting arguments contained within the literature, and proposes a model by which to assess the orientation of the numerous OC&D techniques.  Draws attention to a lack of evidence to support (or not) the efficacy of OC&D in Australia.

Chapter Four: Research Questions and Methodology

          Identifies three research questions: (1) have OC&D initiatives in Australian organisations over the past decade been successful?, (2) are some OC&D methods more successful in Australia than others?, and (3) is OC&D more effective in particular sectors than others?   Defines the scope and approach of the research to be undertaken and proposes a methodology for testing the research questions.

Chapter Five: Preliminary Findings

          Using a series of pie charts, sets out some of the preliminary findings of the research.

Chapter Six: Results and Discussion

          Uses statistical methods to more fully explore the research questions.  Finds that OC&D is only partially successful in Australia and that this does not differ between industry sectors.  Also finds strong indications that some methods consistently out-perform others.  Identifies a number of limitations of this study.

Chapter Seven: Implications and Future Research Directions

          Explores the implications of the findings in terms of future research directions.  Criticises change practitioners for not being more judicious in the application of change methods, as well as researchers for relying on international evidence rather than undertaking studies in a specifically Australian context.  Recommends a number of clear directions for future research.

CLICK HERE TO DOWNLOAD A PDF OF THE FULL RESEARCH STUDY AND FINDINGS.