In Search of Excellence

In Search of Excellence is a book written by Tom Peters and Robert H. Waterman Jr. First published in 1982, it sold three million copies in its first four years, and was the most widely held monograph in the United States from 1989 to 2006. The book explores the art and science of management used by several companies in the 1980s.

Background

Context

In 1982, at the time of publication of In Search of Excellence, America was looking to Japan as the rising economic force. American businesses were studying Japanese management techniques and looking to learn from their successes. In Search of Excellence went against this trend, by focusing on American companies and studying what made the most successful American companies successful.

Origins

In Search of Excellence did not start out as a book, as Tom Peters explained when interviewed in 2001 to mark the 20th anniversary of the book's publication. In the same interview, Peters claims that he and Waterman were both consultants on the "margins" of McKinsey, based in the San Francisco office.

In 1977, McKinsey director Ron Daniel launched two projects; the first and major one, the Business Strategy project, was allocated to top consultants at McKinsey's New York City corporate HQ and was given significant resources, but could not manage to effectively implement strategy.

Peters states that directly after graduating with a PhD from Stanford and returning to McKinsey, Daniel handed him a "fascinating assignment." Motivated by the new ideas coming from Bruce Henderson's Boston Consulting Group, Peters "was asked [by Daniel] to look at 'organization effectiveness' and 'implementation issues' in an inconsequential offshoot project nested in McKinsey's rather offbeat San Francisco office." While Daniel's first project was focused on Business Strategy, this second project was concerned with Organization, which Peters defined as involving "the structure-and-people side." This "Organization" project was seen as less important, according to Peters in a Fast Company interview.

Despite being described as "marginal," the project "had an infinite travel budget that allowed [Peters] to fly first class and stay at top-notch hotels and a license from McKinsey to talk to as many cool people as [he] could all around the United States and the world." Peters admits that "There was no carefully designed work plan. There was no theory that I was out to prove. I went out and talked to genuinely smart, remarkably interesting, first-rate people." In addition to Karl Weick and Einar Thorsrud, Peters notes that Douglas McGregor's theory of motivation known as Theory X and Theory Y was directly influential on the direction of the project.

In a 1978 article, "Symbols, Patterns and Settings," Peters argued that "shifting organizational structure" and "inventing new processes"—structure and system, respectively—were only two tools of organizational change. Peters then outlines eight "mundane" tools that every manager has at their fingertips. He described this article as a "tentative presentation" and "the first public expression of these ideas."

In 1979, McKinsey's Munich office requested Peters to present his findings to Siemens, which provided the spur for Peters to create a 700-slide two-day presentation. Word of the meeting reached the US and Peters was invited to present also to PepsiCo, but unlike the hyper-organised Siemens, the PepsiCo management required a tighter format than 700 slides, so Tom Peters consolidated the presentation into eight themes. These eight would form the chapters of In Search of Excellence.

In 1980, Waterman joined Peters, and—along with Waterman's friends in academia Tony Athos and Richard Pascale—came together at a two-day retreat in San Francisco to develop what would become known as the McKinsey 7S Framework, the same framework that would organize In Search of Excellence. In June 1980, Peters published an op-ed in the Manager's Journal section of the Wall Street Journal titled "The Planning Fetish." In this article, he "stressed the importance of execution and dismissed the whole idea of strategy." As strategy was McKinsey's main operation at the time, this was seen as a "frontal assault" on the company, leading Mike Bulkin, the head of the New York office, to demand that Daniel fire Peters.

The primary "innovative" theme that under-girded what would become In Search of Excellence was that "structure is not organization." This also happened to be the title of a 1980 journal article authored by Bob Waterman, Tom Peters, and Julien Phillips in which they argue that the "picture of the thing is not the thing...An organizational structure is not an organization."

In December 1981, Peters left the company, after agreeing to a fifty percent royalty split with McKinsey. Waterman stayed at the firm for three more years, but received no royalties from In Search of Excellence.

Key ideas

Part 1: Approach and method

In the first chapter of the book, Peters and Waterman introduced the background for the book and their research methods.

Peters and Waterman were concerned with how organizations were organized and managed. They wondered whether structure follows strategy, as Alfred Chandler had suggested. To address this, their perspective was that:

"...any intelligent approach to organizing had to encompass, and treat as interdependent, at least seven variables: structure, strategy, people, management style, systems and procedures, guiding concepts and shared values (i.e. culture), and the present and hoped-for corporate strengths or skills."

They used these seven "variables" to create a visual framework, which became known as the McKinsey 7S Framework. They then used their 7S framework as a lens through which to evaluate organizational excellence. They conducted in-depth interviews with leaders at 43 "excellent" publicly traded companies using this lens. They then reduced the set of emerging insights from these interviews down to eight "themes."

Part 2: Intellectual orientation

The second chapter of the book, "The Rational Model," introduces and then critiques the rationalist approach to management. An example of the rationalist mindset is reproduced below:

"Professionalism in management is regularly equated with hard-headed rationality … It doesn't teach us to love customers. It doesn't instruct our leaders in the rock-bottom importance of making the average Joe a hero and a consistent winner … It doesn't show, as Anthony Athos puts it, that ‘good managers make meanings for people, as well as money." (p. 29)

By contrast, a more "social" form of management takes into account the realities of what really motivates people. This set of real human motivations is explored in the third chapter, "Man Waiting for Motivation."

The fourth chapter puts these concerns into a historical context, exploring the evolution of management theories between 1900 and the time of publication of the book in the early 1980s. The latest era of management is characterised as more "social" than "rational," meaning that real human motivations drive business goals and activities. It also more "open" than "closed," meaning that outside forces such as market pressures can shape the evolution of structure and organisation within a firm. This leads to an increasing concern with the ongoing evolution of an organization, and the role of culture in maintaining and shaping an organization.

Ultimately, these chapters would be seen in today's terms as advocating for leadership over management. Leaders articulate values and purpose, and achieve buy-in to vision and values from employees.

These chapters set the foundation for the rest of the book, which address eight core themes for the book.

Part 3: The eight characteristics of excellent companies

Peters and Waterman found eight common themes which they argued were responsible for the success of the chosen corporations. The book devotes one chapter to each theme.

1. A bias for action – a preference for doing something—anything—rather than sending a question through cycles and cycles of analyses and committee reports
2. Staying close to the customer – learning customers' preferences and catering to them
3. Autonomy and entrepreneurship – breaking the corporation into small companies and encouraging them to think independently and competitively (aka, "split-up")
4. Productivity through people – creating in all employees the awareness that their best efforts are essential and that they will share in the rewards of the company's success
5. Hands-on, value driven – insisting that executives keep in touch with the firm's essential business
6. Stick to the knitting – remaining with the business; "The company knows best."
7. Simple form, lean staff – few administrative layers, few people at the upper levels
8. Simultaneous loose–tight properties – fostering a climate where there is dedication to central values of the company combined with tolerance for all employees who accept these values

Discussion

The "Drucker gap"

Before and after In Search of Excellence, Peter Drucker was probably the preeminent management theorist. Drucker presaged and covered similar perspectives to Peters and Waterman's approach to management theory, for example in Drucker's 1954 book The Practice of Management.

Peters first read Drucker's The Effective Executive in 1968. Peters claims that when writing In Search of Excellence, he was "pissed off" at Peter Drucker:

"But I did have an agenda. My agenda was this: I was genuinely, deeply, sincerely, and passionately pissed off! ... Who was I pissed off at? At Peter Drucker, for one. Today, everybody acts as if Peter Drucker has always been one of those who gets it. Go back and read Concept of the Corporation. Peter Drucker may be an Austrian, but he's more German than the Germans when it comes to hierarchy and command-and-control, top-down business operation. Take a look at the business bible according to Peter Drucker, and you'll see. Organizations are about organization! You vill be in your place! That was the received order of the day. So, in my mind, Peter Drucker was the enemy. A good enemy, but still the enemy."

After publication of In Search of Excellence, Peters re-read a wider body of Drucker's work, and commented:

"'I had considered at least a little of what Bob Waterman and I had written in In Search of Excellence to be new,' Peters said ... 'But upon rereading [Drucker's] "The Practice of Management" book, to my amazement . . . I found everything we had written (was) in some corner or other of that book.'"

Peters's alleged confession of "faked data"

In December 2001, Fast Company published an article entitled "Tom Peters's True Confessions". Most of the "confessions" were humorously self-deprecating remarks (In Search of Excellence had been "an afterthought... a hip-pocket project that was never supposed to amount to much"). One of them, however, used the term "faked data:"

This is pretty small beer, but for what it's worth, okay, I confess: We faked the data. A lot of people suggested it at the time. The big question was, How did you end up viewing these companies as "excellent" companies? A little while later, when a bunch of the "excellent" companies started to have some down years, that also became a huge accusation: If these companies are so excellent, Peters, then why are they doing so badly now? Which I'd say pretty much misses the point.

[In] Search [of Excellence] started out as a study of 62 companies. How did we come up with them? We went around to McKinsey's partners and to a bunch of other smart people who were deeply involved and seriously engaged in the world of business and asked, Who's cool? Who's doing cool work? Where is there great stuff going on? And which companies genuinely get it? That very direct approach generated a list of 62 companies, which led to interviews with the people at those companies. Then, because McKinsey is McKinsey, we felt that we had to come up with some quantitative measures of performance. Those measures dropped the list from 62 to 43 companies. General Electric, for example, was on the list of 62 companies but didn't make the cut to 43—which shows you how "stupid" raw insight is and how "smart" tough-minded metrics can be.

Were there companies that, in retrospect, didn't belong on the list of 43? I only have one word to say: Atari.

Was our process fundamentally sound? Absolutely! If you want to go find smart people who are doing cool stuff from which you can learn the most useful, cutting-edge principles, then do what we did with Search: Start by using common sense, by trusting your instincts, and by soliciting the views of "strange" (that is, nonconventional) people. You can always worry about proving the facts later.

BusinessWeek ran an article about Fast Company's article. As related by BusinessWeek, the article was actually written by Fast Company founding editor Alan M. Webber, based on a six-hour interview with Peters. Peters reviewed and approved the article prior to publication, but the actual phrase "we faked the data" was Webber's, and Peters had not actually used these words during the interview. BusinessWeek quoted Peters as saying, "Get off my case. We didn't fake the data." According to BusinessWeek, Peters says he was "pissed" when he first saw the cover. "It was his [Webber's] damn word," he says. "I'm not going to take the heat for it."

Criticisms

Market performance of the "excellent" companies

In Search of Excellence is not widely regarded as being great at predicting future success for individual "excellent" companies. NCR, Wang Labs, Xerox and others did not produce excellent results in their balance sheets in the 1980s. However, the set of "excellent" companies studied, as a whole, still outperformed the market. A 2002 analysis in Forbes found that:

"Over a five-, 10- or 20-year period, the Excellence index—an unweighted basket of the 32 public companies among Peter's and Waterman's 43—substantially outperformed the Dow Jones Industrial Average and the broader S&P 500. Since October 1982, when the book was published, the companies on the authors' list earned an average total return of 1,305%, or 14.1% annually. This return outdistanced the DJIA companies, which earned an average annual return of 11.3%, and the S&P, whose companies earned an average annual return of 10.1%."

In an article in Fast Company, Peters remarked that the criticism that "If these companies are so excellent, Peters, then why are they doing so badly now," in his opinion "pretty much misses the point."

The halo effect

The research methodology employed by the authors of this book is also severely criticized by Phil Rosenzweigh in his book The Halo Effect as the "Delusion of Connecting the Winning Dots". Rosenzweigh opines that it was not possible to identify the traits that make a company perform simply by studying already-performing companies as Peters and Waterman did, similar to a retrospective cohort study without a control group.

Legacy

Reception

Almost 40 years after its original publication In Search of Excellence remains a widely read classic and an influential book for leaders and managers. Inc. magazine reported that by March 1999, In Search of Excellence had sold over 4.5 million copies. A 2002 panel of experts convened by Forbes rated In Search of Excellence as the most influential business and management book from the decades between 1980 and 2000.

Impact

Tom Peters has identified several key contributions from the book, that changed the field of future management books.

1. A reorientation from a focus on strategy to recognising the importance of "soft" aspects of business like culture and people.
2. The use of research and the novel use of specific case studies to bring ideas about management to life.
3. Bringing management ideas to a broader audience than previous theorists like Peter Drucker had reached.

In Search of Excellence also opened up the way for further research and publication around excellence in business. Jim Collins's books Built to Last and its prequel Good to Great are widely known influential later works in this genre.