Multiple layers of authority overlap both horizontally (different people and committees engage with the same issue) and vertically (many decisions are liable to review by some other body). The lack of focus in decision making results in an absence of executive authority; while professional management is subject to random amateur interference. In consequence, able people are not easily attracted to management roles; and so the amateurs view the professionals with often justified and frequently reciprocated contempt.-- John Kay, How a proud corporate history can lead to poor governance
A business has many sub-organizations. They may be functional (sales, accounting, etc.) or regional (EMEA, Americas, etc.), or specialized (e.g., product). A company may consist of all three: regional sales and marketing teams, working with multiple product teams, all of whom share corporate-level finance and legal. We want the organization structure to balance customer service with operating efficiency: completely autonomous divisions offer high customer service at a cost to our customers or to our profitability, while completely silo'd organizations squeeze every penny of efficiency at a cost to customer service. Competitive customer service at the lowest cost of execution lies somewhere in between these extremes.
Captive IT faces the same challenge as the rest of the business: organize to provide the greatest level of service while containing costs. IT generally organizes around technical roles: we have infrastructure people, database people, helpdesk people, software development people, and so forth. We also have specializations therein: we have ERP developers, who are not the same as our front-end developers, who are not the same as our legacy system developers. And none of them are quite the same as our QA people.
These sub-specializations are fairly well entrenched because career path is generally associated with role, and even specific technologies. For one thing, deeper expertise in a narrow set of technologies will command a higher salary than shallow expertise in a wide range of technologies. For another, a manager is unlikely to promote a promising member of the web development team to be tech lead of an ERP team because the technical knowledge is not transferable.
IT leaders structure their organizational hierarchy with this in mind. For utility functions, this works just fine: provisioning hardware and e-mail accounts is the same no matter who the user is. But in strategic software development, the business domain influences technical implementation, so IT needs tighter alignment with the business. The specialist IT structure is mapped to more granular business customers in a matrix: we form delivery teams to support a business line or a specific product owned by the business, but each team's tech lead reports to a VP of engineering within the tech organization.
Like everybody else, IT is under cost pressure. The greater the pressure, the more likely IT leadership is to make something into a shared service, which translates into fewer people owning multiple responsibilities for multiple teams. Quality assurance, project management, and techops, for example, can provide greater service when paired with a business partner, but become organizations unto themselves (with fewer staff) as a means of creating cost efficiencies.1
From an organizational perspective, this makes things somewhat untidy within IT because we have a matrix (shared services) within a matrix (development teams organized by business line, but reporting up to an engineering leader). If there is a shared service within a shared service, such as DBAs being part of techops, we have a matrix within a matrix within a matrix. It's turtles, all the way down.
Add to this the recent phenomenon of product organizations. A product hierarchy - common but by no means exclusive to tech firms - is chartered to elevate users (people who use the software) and customers (those who pay the bills for those who use the software) in ways that subject matter experts and software engineers are not able to. As the proxy for consumers and buyers, they influence priorities and packages of functionality. But product owners don't necessarily have strong footing in either the business domain or software development. In practice, they act as a referee between SMEs and engineers. At best they're an emerging function clawing at opportunity from a different perspective; at worst they're another level of intermediation in the decision making process.
And so we have the situation described above by John Kay: no small number of people being brought to bear on a problem, but a structural inability to get results.
With no defined power structure, the vacuum is filled by people who turn non-executive roles into a near full-time occupation. [...] Petty politicians enjoy the feeling of being at the centre and jostle for power; the power they seek is not the ability to get things done but the negative power that comes from “no decision without me”. Secrecy about matters of no significance bolsters their sense of self-importance.-- Ibid.
Instead of better business alignment, we have fragmented ownership and competing priorities and agendas. Matrices create, rather than alleviate, impediments to getting things done.
In Part II, we'll look at executive disenfranchisement in the matrix organization.
1 It's worth noting that the decision to make something a shared service is frequently justified as a means of promoting "best practices". Functions like QA or devops that are fragmented into separate product teams will show inconsistent performance; consolidating them into a single function should, in theory, be a step toward making them more consistent (ignorant of technical, asset or personnel restrictions on that). The irony is that somehow, "best practices" - whatever that's supposed to mean - will compensate for the fact that a critical function is deliberately being starved of investment.