Too Big To Think: Why Prestigious Institutions Stopped Generating Good Ideas

In the late 1960s the founder of Boston Consulting Group, Bruce Henderson, divided his company into three color-coded teams: red, blue, and green.  These teams were instructed to compete against each other.  He hoped that the experiment would generate new ideas for how to run a consulting firm.

It worked.  In a few years, Bill Bain and his blue team prototyped a new fee model that enabled them to increase fees and deepen partnerships with clients. Unfortunately, Bain also decided that he should start his own firm.  In 1971, he left the company to start Bain & Company and took many of Boston Consulting Group’s top leadership with him.  

After spending seven years working in the strategy consulting industry, it’s impossible to imagine this kind of experiment taking place in those same firms.  Today’s top three strategy consulting firms, McKinsey, Bain, and Boston Consulting Group, are large organizations that haven’t seen a serious competitor in more than thirty years and are known more for scandals and 100-page PowerPoint “decks” than being the source of innovative and creative new ideas.  

During Bill Bain’s time at Boston Consulting Group and then at Bain & Company, however, strategy consulting firms were a central force in creating, shaping, and popularizing many of the most important business ideas of the 20th century.  

At some point, these firms became “too big to think” which occurred to me after I decided to leave the industry after nine years to work on my own in 2017.  Despite fewer resources and no “brand” to lean on, I’ve been able to unlock a level of creativity and idea generation as an independent researcher and writer of words on the internet that is ]beyond what I felt possible at any point in my previous path.  

How is this possible and what does it mean about where ideas might emerge in the future?

These Firms Were Generative and Important

If we look back at some of the ideas popularized in consulting firms, we might laugh.  Strategy is important!  Listen to your customers!  Pay attention to the “soft side” of organizational change!  Maximize shareholder value!

Bruce Henderson, Founder of BCG

Yet until they emerged in the consulting-academic-industrial idea complex during the 1960s to 1980s, these ideas were not common knowledge.  These ideas also did not show up out of thin air either.  They were the product of unique cultures which put tremendous value on the discovery of new ideas.  The In Lords of Strategy, author Walter Kiechel III described McKinsey’s embrace of ideas in building a “knowledge culture” in the late 1970s:

As part of building the “knowledge culture,” its consultants began grinding out staff papers—often twenty pages long, based on experience with clients and internal debates, and some suitable for repurposing as Harvard Business Review articles. Within five years, they had turned out twenty-three, which bore titles such as “Strategic Market Segmentation,” “Competitive Cost Analysis,” and even “The Experience Curve as a Strategy Tool.

Again – these ideas sound obvious if you have some experience in the business world.  The phrase “shareholder value” probably doesn’t even incite a reaction in even the most cold-blooded CEO.  Yet from the 1960s to the 1980s, there was a hunger for new ideas in the business world and in society at large.  A great example of this is Milton Friedman’s multi-part “Free to Choose” series that aired to millions in the United States on television in 1980.  The series explored capitalism and markets and was enthusiastically consumed by people in the US and then millions more around the world throughout the decade. 

As company leaders turned these ideas into profits, finding and implementing better ideas became an organizing assumption in business.  The economist Tyler Cowen has made a convincing augment that spreading this assumption via consulting firms setting up shop abroad has been one of the best things that have happened for global growth and prosperity. 

One example of this happening in practice was Boston Consulting Group using “experience curve” work to help Texas Instruments (TI) change how it thought about planning for the future.  By modeling how learning effects would lower the costs of manufacturing electronics over time, they helped TI develop a plan to lower prices in an attempt to align with those savings such that they could capture a larger market share of the emerging personal calculator space.  Although this approach did lead to some price wars, TI was able to grow its calculator sales from 3 million to 45 million in only five years from 1971 to 1975.  This put the company in a position to invest in many other areas of the business, helping them tripe overall revenue in six years from 1973 to 1979.

It was exciting to be working at a consulting firm during this time.  There was so much low-hanging fruit and so many ideas to test.  If you were someone driven by the discovery of new ideas, working at a consulting firm likely wasn’t going to slow you down.

Now, that is no longer true.  Consulting firms have become too big to think.  As a former McKinsey consultant Rohit Krishnan argued to me, “first these firms did intellectual arbitrage, then data arbitrage, and now it’s mostly effort arbitrage.”

Instead of bringing new ideas into the business world, consultants now serve the role of effort-on-demand for executives who are trying to extract another 1% out of the business.  

And yet, I would admit that they are still some of the best places to work.  This raises a much deeper issue: if consulting firms are no longer great places to discover and generate new ideas, where can you go if you care about such a thing?  

The short answer is the internet and it’s something I explore.  But first, we need to understand the forces undermining creativity and idea generation in these institutions.

No space for weirdos

In 1974 Tom Peters landed a job at McKinsey & Company.  He had done a Ph.D. at Stanford in organizational behavior and was obsessed with how you could use statistical analysis to learn about the success of businesses.  For the next several years, Peters existed on the margins of the firm.  Here’s how he reflected on his experience in 2001: 

We were out in the San Francisco office, far, far away from McKinsey headquarters, tucked inside an office that never made any money but that was well-known for its weirdness. In fact, there were always people in other parts of McKinsey who would reach a point in their analyses where they would try to get us involved because they wanted us to do our “San Francisco thing.” We were the closest thing McKinsey had to hippies — hippies in black suits.

Peters was a weirdo, one who went on to write one of the most influential business books of all time.  That book, In Search of Excellence, sold more than 5 million books and helped to usher in a new way of thinking about how companies should improve.  

If Peters was hired into a consulting firm today, he wouldn’t last long.

As organizations grow, they inevitably add rules, structure, layers, and policies to manage the complexity.  This comes with many tradeoffs, many of which are worth it, but it’s quite hard to scale effectively while still cultivating an environment where weirdos can thrive.    

When Peters was at the firm in the late seventies, it has less than 500 people.  McKinsey now has about 30,000 people and adds the equivalent of Peter’s McKinsey every couple of months. There’s no agenda against allowing people like Peters to thrive, it’s just an unintended side effect of structure and scale.

In my last firm before working for myself, I spent almost two years working on a project that probably could have taken about a month.  What took so long?  The company was filled with countless high-ranking and important people, all of which had veto power to shut down any minor change.  The political scientist Francis Fukuyama called this a “vetocracy” – a system where no individual can gain enough power to make a decision.  

This is the key point.  No single person is deciding that creativity should be eradicated but as a byproduct of systems needed to manage large-scale enterprises, the result is the same.  

When Peters reflected back on writing his book, he said, “I had no idea what I was doing when I wrote Search. There was no carefully designed work plan. There was no theory that I was out to prove.” 

As consulting firms and other institutions have become bigger, individuals are nudged to stop being weird and to conform.  

The Slow Subtle Death Of Creativity

At the individual level, the nudge to stop being weird happens in subtle ways that are hard to see in the moment.

Early in my career, I worked in operations, spending time on manufacturing floors doing process engineering, and then as a strategy consultant helping companies implement design and operational improvement programs.  The most popular approach that almost everyone used was called “lean,” inspired by the systems invented at Toyota in the 20th century.  I became obsessed with this, reading almost every book I could find on the subject.  I was inspired by how Toyota’s approach started with genuine respect for people and an obsession with cultivating a culture that incentivized teamwork and problem-solving.  I had a desire to combine some of my experience working with front-line operators on manufacturing floors with this human-first approach.  Yet the executives we worked with often didn’t want to do the hard messy work of changing cultures.  So you give them tools, dashboards, and easy-to-understand systems that can be rolled out globally and tell yourself that one day, you will do it differently.

Of course, that day never comes.  You get raises and promotions and try to forget that you had that deeper impulse to try new ideas and to push boundaries.  

You Can Sell Everything As Long As It’s Top-Down Change

Example of top-down transformation framework we used

While working at Boston Consulting Group, leading a small research team on organizational transformation, I became fascinated by the ideas related to complex adaptive systems.  This is a way of thinking about organizations where you pay attention to what is happening at the lower levels of the organization and put more emphasis on competition, slack, and a healthy amount of chaos.

I wanted to inject these ideas into our models of what we were pitching to clients but wasn’t finding much interest around me.  Why change things that were working?  If you can sell $10 million using the existing approaches, why go deeper?

In terms of the firm’s continued growth, this made sense.  Consulting firms sell top-down solutions to senior executives because they can be sold for a lot of money and they also served the purpose of giving those executives a platform to channel their unbridled career ambitions.  Looking at organizations through the lens of complex adaptive systems would explicitly acknowledge that the people paying us were not as important as we made them seem.  

In other words, it would leave consulting firms without a business model.

And so almost anyone who decides to stay working in these firms or even leave to a similarly large company or other institutions decides that this is the way things have to be.  They give up what David Deutch argued was the significantly unique thing about humans, “our ability to create new explanations,” and decide that the payoffs from staying on a certain career track are better than the payoffs to following curiosity and possibility.

Exit To More Creative Pastures

New spaces are emerging

It took me a damn long time to figure out that being in environments where I could channel a passion for ideas and my natural curiosity was vital for me.  

Luckily, the internet has given me a second space beyond traditional work to do this.  I started sharing my own writing while still employed in consulting.  First I wrote on quora and then with a little more courage, publicly on LinkedIn.  While I was terrified of being mocked by colleagues for doing this, no one said anything.  Instead, I befriended a wide range of people trapped like me – Chief Talent Officers, Partners at consulting firms, senior executives, founders, and many people in HR – all hungry for better explanations.   I wasn’t crazy.

Knowing these others existed inspired me to start exploring and finding my own path.  As I became a better writer and went deeper into the ideas I was curious about, I slowly started to realize the costs I had been paying on my path, which made it easy to become self-employed and not have the expectation that I needed to match my own salary.  The upsides in terms of who I might become were priceless

Right now there are millions of people in too big to think organizations who feel stifled and uninspired.  They want to explore ideas, experiment and create.  Some people might point to exceptional individuals that seem to make it work in these environments but for every person that is able to express themselves within too big to think institutions, there are undoubtedly dozens more that have given up or are on their way to doing so.  Most people have responsibilities and the enormous task of carving your own path just isn’t worth the risk of giving up a nice paycheck that affords a certain kind of life. 

Increasingly, there are offramps.  One example is Justin Murphy’s Indie Thinkers community, which is attempting to create a home for hyper-curious humans who want to explore specific topics and teach others on the internet, especially from Academic backgrounds.  Another is Dave Perell’s Write of Passage, which positions itself as a place to learn to write on the internet but I think is better thought of as the world’s best simulation of learning how to make friends, share ideas, create a business, and build a life on the internet.  Finally, Anna Gat’s Interintellect is a digital home for modern public intellectuals where people hold three-hour in-depth digital “salons” around new and old ideas. 

What do these three of these have in common? They were created by people who decided to exit traditional institutions in search of spaces where they could more actively unleash their creativity and pursuit of interesting ideas in their lives.

It took me a while to realize I was like them.

Death By A Thousand Cuts

In 2007 when I was graduating from college, I didn’t have a sense that I could do anything except work for a big company like every adult around me.  My first job was for General Electric. The job was fine but the GE that had been in the mix of generating interesting new ideas of the 1980s had ceased to exist.  

Consulting was a dramatic improvement and when I joined McKinsey & Company at the age of 23, I joined a research office where I spent every day with a hyper-curious set of people specializing in a wide range of industries, geographies, and functions.  The lunchroom conversations were exciting and for the first time, I had the sense that I was exactly where I was supposed to be.

But eventually, most of us, including me, willingly sacrificed our curiosity in exchange for job opportunities, promotions, and the ability to keep going on impressive career paths.

This is how consulting firms and other organizations become too big to think.  There is no seriousness czar outlawing curiosity but the pressures of bureaucracy, long career paths, and large companies force those tradeoffs as part of how things work.

Eventually, the tradeoffs became too much for me and I hit eject on a path that made sense.  To some people, this seemed insane and probably still seems insane.  They are likely running the math that I’ve probably lit on fire more than a million dollars of income.  Yet my own calculus values my sense of aliveness and curiosity at a much higher billing rate than my consulting firms sold my man-hours. 

Yet when I made those first few friends through my writing, I gained confidence that a different path was possible.  In 2015, the paths to making a living outside of a traditional job were not quite as clear but more groundwork was being laid.  I followed a group of early adopters who had dared to carve their own path without the ease of social media, ubiquitous internet connection, and high-quality digital devices.  Over time, the costs of embarking on a path like mine will go lower and lower and we’ll get to a point where there will be a mass exodus of under-inspired curious humans from too big to think institutions.  

In the next twenty years, the most interesting ideas will emerge from people willing to carve new paths, join and invest in emerging institutions, share their ideas publicly, and those willing to find the others.  They will come from weirdos like me, writing directly to you, the lovely people of the internet.

The Shrinking Idea Dividend

If we are to buy my argument that consulting firms are “too big to think” we have to grapple with the fact that these firms are growing at impressive rates much faster than the rest of the economy.

I’d argue that the ideas that they helped generate in the second half of the 20th century laid the groundwork for their current success.  Because of how innovative and generative they were, they generated an enormous positive halo effect of trust and respect among the current generation of business leaders and that continues to pay a lucrative dividend.  Yet the reality on the ground is that they are no longer being hired for their ability to be innovative – they are increasingly doing non-strategic work, including implementation, data analytics, managing project management offices, and complex technology transformations.

It shouldn’t surprise us then that interest in the two most famous consulting firms, McKinsey & Company, and Boston Consulting Group has steadily declined as seen by google trend data for more than twenty years.

Today you are more likely to be able to recognize a McKinsey Partner that has spent time in jail than any specific idea that has come out of the firm in the last twenty years.

At one point, consulting firms really did have a corner on the best ideas.  They were tightly linked with Academia and other industry leaders who were shaping the beginnings of the modern industrial economy.  If you were a company and wanted the “best thinking” you hired a consulting firm.  

It took me a while to realize that despite talking a lot about ideas, consulting firms didn’t play a meaningful role in helping to generate them anymore.  Most of what we did in the industry was to repurpose ideas created decades ago or repackage ideas that were emerging out of the technology industry.  

As information has become hyper-available, you no longer hire a consulting firm for ideas or data processing leverage, you hire them to quickly tap into hyper-driven young people who want to spend long hours on certain problems with the information that is readily available.

Which leaves us with a question: where are ideas emerging?

While the digital has become increasingly important in our lives, people continue to discount the internet as a serious space for ideas.  From my vantage point, I’m increasingly blown away by the caliber of people who are deciding that they are willing to give up the safety, security, and prestige of impressive career paths in order to put their passion for ideas and creativity first.  

I’ve been able to tap into a dynamic exchange of ideas that is far more inspiring and energizing than I experienced inside the best consulting firms in the world.

Ideas have always emerged from individuals but as the industrial economy took over the world, we increasingly thought that it needed to happen within formal institutions.  This is no longer true.  Increasingly, the best ideas are formally or informally linked to digital communities, Venture Capital startup incubators, personal newsletters, or even emergent idea webs on Twitter.  If you want to seriously engage with ideas, getting tenure at HBS, becoming a partner at McKinsey, or working as a writer at Fortune is likely going to undermine that mission more than help.  

This is exactly what Tom Peters tells us even in his reflection on his career in consulting:

Don’t always bet on the little guy, but do always bet against headquarters. Because headquarters politics will invariably and inevitably “bland up” and then kill any worthwhile project.

A bet against headquarters is a bet against any firm that is too big to fail.

What will always be true is that ideas will always emerge in the same old way: out of the minds of creative individuals who like “finding things out” as curious human Richard Feynman once said. 

It’s up to you to figure out if you are one of those curious humans and decide whether or not you are in a place that’s too big to think or not.

Thanks to Malcolm Ocean and Rohit Krishnan for very useful edits to this piece!

About Paul Millerd

Paul is a writer, creator, and curious human that is passionate about how people can reimagine their relationship with work to do things that matter. He published The Pathless Path in 2022.