In the 1980s, it seemed we were on the cusp of a revolution in management. American business exited the 1970s in terrible shape. Bureaucracy was discredited. Technocracy was, too: "best practice" was derived from people performing narrowly defined tasks in rigid processes that yielded poor quality products at a high cost. There was a call for more employee participation, engagement, and trust. Tom Peters was strutting the stage telling us about excellence and heroizing the empowered employee (you may remember the yarn about the FedEx employee who called in a helicopter to get packages out of a snowbound location). Behavioural stuff was in the ascendency. We were about to enter a new era in management.
Until we weren't.
Behaviourally centric techniques - including Agile - are still fringe movements in management, not mainstream practice. The two Freds - Frederick the Great (his organization model for the Prussian military defines most modern organizations) & Frederick Taylor (scientific management) - still rule the management roost. Frederick the Great organized his military like a machine: a large line organization of specialists following standardized procedures using specialized tools, with a small staff organization of process & technical experts to make the line people more productive. Frederick Taylor defined and measured performance down to the task level. We see this today, even in tech firms: large, silo'd teams of specialists, sourced to lowest-common-denominator position specs, with their work process optimized by Six Sigma black belts. Large organizations are no different than they were 30, 50, 75 years ago.
What happened?
First, it's worth looking at what didn't happen.
1. The shift from manufacturing jobs to service jobs was supposed to give rise to networks of independent knowledge workers collaborating to achieve business outcomes. It's true that many modern service jobs require more intellectual activity than manufacturing assembly line jobs of the past. However, just like those manufacturing jobs, modern service jobs are still fragmented and specialized. Think about policy renewal operations at insurance companies, or specialized software developers working in silos: they are information workers, but they are on an information assembly line, doing piecework and passing it onto the next person.
"[Big companies] create all these systems and processes - and then end up with a very small percentage of people who are supposed to solve complex problems, while the other 98% of people just execute." Wall Street Journal, 24 December 2007.
The modern industrial service economy has a few knowledge workers, and lots and lots of drones. It's no different from the manufacturing economy of yore.
2. Microcomputing was expected to change information processing patterns of businesses, enabling better analysis and decision support at lower levels of the organization. Ironically, it had the opposite effect. Microcomputers improved the efficiency of data collection and made it easy to consolidate operational data. This didn't erode centralized decision making, this brought it to a new level.
Second, there are things that have reinforced the command-and-control style of management.
1. "Business as a machine" - a set of moving parts working in coordination to consistently produce output - remains the dominant organizational metaphor. If we're going to have organizations of networked information workers, we have to embrace a different metaphor: the organization as a brain. Machines orchestrate a handful of moving parts that interact with each other in predefined, repetitive patterns. Brain cells connect via trillions of synapses in adaptable and complex ways. The "networked organization" functions because its members develop complex communication patterns. Unfortunately, it is much harder to explain how things get done in a network organization than it is in a machine organization: general comprehension of neuroscience hasn't improved much in the past 25 years, whereas it is easy for people to understand the interplay of specialized components in a simple machine.
2. Service businesses grew at scale, and the reaction to scale is hierarchy, process, and command & control. As I've written previously, the business of software development hasn't been immune to these pressures.
3. The appetite for operational data has increased significantly. A 2007 column in the WSJ pointed out that management by objective and total quality management have been replaced by a new trend: management by data. Previous management techniques are derided by the data proponents as "faith, fear, superstition [or] mindless imitation".
4. Service businesses (e.g., business process outsourcing) moved service jobs to emerging market countries where, owing to economic and perhaps even cultural factors, command and control was easily applied and willingly accepted.
5. In the last 12 years, debt has been cheap - cheaper than equity. In 1990, 2 year and 10 year Treasurys were paying 8%. In 2002, the 2 year paid 3.5% and the 10 year paid 5%. Today (September 2013), they're paying < 0.5% and 2.64%, respectively. When debt is cheap, CFOs swap equity for debt. When we issue debt, we are committing future cash flows to interest payments to our bondholders. And unlike household debts, most corporate debt is rolled-over. To make the debt affordable we need to keep the interest rates low, which we influence by having a high credit rating. Stable cash flows and high credit ratings come from predictable business operations. As more corporate funding comes from debt instead of equity, it puts the squeeze on operations to be predictable. With predictability comes pressure for control. Those new management practices that emerged to empower individuals and teams advertise themselves as providing better "flexibility", not "control". They are anathema to businesses with financing that demands precise control.
6. In the past decade, corporate ownership has been concentrated in fewer and fewer hands. This has happened through equity buybacks (in 2005-8 this was usually funded with debt, since 2009 it's just as likely to be funded with excess cash flow) and dual-class share structures (Groupon, Facebook, News Corp, etc.)
7. The external concentration of ownership coincided with internal concentration of decision making. Speaking from experience, around 2006 the ceiling on discretionary spending decisions dropped precipitously in many companies. In most large companies, a v-level manager used to be able to make capital decisions up to $1m. Empirically, I've seen that drop to $100k in most large firms.
8. The notion of "best practice" has been in vogue again for at least a decade.
9. Recessions in 2001 (when businesses reigned in unrestrained tech spending) and 2008 (when businesses reigned in all spending) tightened belts and increased operational scrutiny.
10. I also suspect that business education has shifted toward hard sciences like finance and away from soft sciences like management. The study of org behaviours was core b-school curriculum in the 1980s. It appears this has moved into a human resources class, which emphasize org structures. This treats organizational dynamics as a technical problem, not a behavioural one. I haven't done much formal research in this area, but if it's true, it means we've created a generation of business executors at the cost of losing a generation of business managers.
What does all this mean?
The Freds will continue to dominate management practice for the foreseeable future. Corporate profitability and cash flows have been strong, especially since the 2008 financial crisis. That, twined with ownership and decision-making authority concentrated in fewer hands, means that there is no incentive to change and, more importantly, there is actually disincentive to do so. Among middle managers, the machine metaphor offers the path of least effort and least resistance. It also means that when large companies adopt alternative approaches to management & organization at scale - for example, when large corporates decide to "go Agile" - the fundamental practices will be co-opted and subordinated to the prevailing command-and-control systems.
This isn't to say that alternative approaches to management are dead, or that they have no future. It is to say that in the absence of serious upheaval - the destabilization / disruption of established organizations, or the formation of countervailing power to the trends above - the alternatives to the Freds will thrive only on the margins (in pockets within organizations) and in the emerging (e.g., equity-funded tech start-up firms).
This leads to a more positive way of looking at it: it isn't that the day of post-Fred management & organization has come and gone, it's that it is yet to come. The increasing disruption caused by technology in everything from retail to education to NGOs will defy command and control management.
But that still begs the question: after the disruption, when the surviving disruptors mature and grow, will they eventually return to the Freds?
