Since 2006, I've written multiple blog posts, a few articles and self-published an e-book on governing investments in strategic software. Software development is unique from most every other office-based occupation because it is the conversion of capital into intangible assets by way of human effort, at a level of effort that remains labor intensive to this day. Although other white-collar occupations such as back-office accounting can be labor intensive, their costs flow through to the income statement as SG&A rather than depreciation. Blue-collar occupations such as manufacturing labor can be capitalized (the labor that goes into building a car is a cost that is factored into finished goods inventory, which is a balance sheet phenomenon), but decades of capital investment in robotics have reduced the labor intensity of most manufacturing functions. True, many firms (particularly nascent tech firms) expense their software development labor costs; yet development is still a capital allocation function if they're spending investor capital or retained earnings rather than cash flow from operations to finance the development. Restated, software development is discretionary spend of balance sheet cash that could be directed to other investments or returned to investors.
Being an act of capital allocation, it has always struck me odd that most corporate captive IT organizations and tech firms self-govern the transformation of capital into software. In practice, software development governance is clubby, even more clubby than most corporate boards where the CEO has hand-picked the directors. For purposes of ceremony, IT governance is heavy-handed in reporting, but light touch in execution, loaded as it typically is with vendor reps with sales revenue on the line, delivery managers with bonuses on the line and business sponsors with promotions on the line. This isn't an environment where hard questions are tolerated, let alone asked. Some years ago, I was working with a regulated healthcare company replatforming its customer-facing solutions at all locations worldwide. It's target state one month after launch was to have 5 locations running the new software in parallel with legacy, processing 0.15% of corporate transactions, with another 5 locations ready for go-live the following month. In the event, the software had to be shut down after two locations were live for a few days because the new platform was dispensing client instruction incompatible with and contradictory to the healthcare products being provided. Sixty days after go-live, no locations were operating on the new platform, there was no resolution for this potentially life-endangering flaw, and therefore no path to production for the initial 2, or 5, or 10 locations, let alone any of the thousands of locations beyond that. The PMOs self-reported program status to the governance board? Green.
"We're all ok here" has been the modus operandi for IT governance for the past two decades. A lot of this is a function of the fact that despite the dot-com bust and the 2008 financial crisis, the first two decades of the 2000s have been characterized by overall global economic growth. Mid-way through 2020, we are now in a period of economic uncertainty. COVID-19 has initiated socioeconomic change that will have lasting effect on consumer behaviors, business practices and public policy. Uncertainty remains as the nature and trajectory that containment and combatitive policies to defeat the virus remain unknown. A lengthy period of containment and combat won't simply harden new business patterns such as work-from-home policies and consumer preference for take-out dining: it will unleash entirely new patterns as socioeconomic pressures build concomitant with human frustration at restrictions and seasonal change incompatible with policy relaxation.
The uncertainty factor matters because it creates new pressures for captive IT and tech firms alike to show diligence with every dollar spent. No matter the exposure to socioeconomic disruption your company now faces and balance sheet strength your company had entering into this crisis, the CEO was not and is not too keen on capital spend right now. While there are good cases to be made for investments in strategic software, particularly those that are reasonable preparations for how a society and economy function in the future, the tolerance for both wasteful allocation and misallocation is low.
Which brings us to the question of software delivery governance. If governing software delivery was hard during stable economic times, it is that much much harder during unstable economic times. Weak governance is highly vulnerable to threats old and new that cannot be swept under the rug worn threadbare by sudden economic convulsions. There are three threats worth highlighting: overspend, vendor fraud, and employee misalignment.
The first threat is little to no tolerance for overspending. I've written previously that CFOs earn their stripes on Wall Street by creating the appearance of control. Control arises from operational predictability: predictable operations create stable cash flows that finance healthy dividends and buoy the credit rating. In good times, perhaps the chief accountant was able reclassify a lot of maintenance work as capital improvement to bury that $2m IT project overrun a few years ago, and a late-year sales flourish buried that $4m overrun last year. In a sustained downturn, there's no accounting trickeration or sales windfall to come to the rescue. Not to mention, the CEO really wants to tell Wall Street that despite the economic upheaval, the company results were so good and the indicators so overwhelmingly positive, we're increasing the dividend. No CIO in their right mind will jeopardize the CEO's victory lap.
The second threat is an increase in fraud. Vendors have always exported overhead labor from their income statement to that of their customers. At a tactical level, vendor overhead employees are written into client contracts in loosely-defined roles that permit them to bill a few hours every billing cycle. Vendor direct labor rates are slightly inflated as a means of deflecting attention from the fact that vendor direct labor hours are significantly inflated. Surcharges such as "vendor administration services" - which amount to vendors charging their customers to make certain that timesheets are entered, invoices are generated and payments are collected - are added as service fees. And, of course, vendors cheerfully enter into one-way contracts to deliver CEO vanity projects of "re-imagination" and "market disruption" with no clawback clauses for results or outcomes. There is also the more pedestrian version where vendors enter into one-way contracts that create disincentive for improvement and (perversely) create incentive for perpetual failure: e.g., a vendor that charges per bug fix incident has no incentive to remediate the underlying cause, and actually has incentive to perpetuate the underlying cause.
These are all phenomenon that happen in good times, the product of lax contract administration, asymmetric knowledge of software delivery that favors the vendor over the buyer, and the starry-eyed "true north" ambitions of buyers. The boss wants to do this, we're not a software firm, procurement was told to get the deal done, and the contract gives little room for pushback, so everybody goes along for the ride. In unstable times, the pressures for vendors to inflate contracts to meet revenue targets, stave off layoffs, and sign obligation-light contracts to create flexibly tappable cash flow intensify that much more. Vendors have many more problems to try to export to their customers.
The third threat to look out for is the mismatch between individual and organizational goals. I wrote about this in 2013 in a two-part series on conflict. The phenomenon materializes as contrived complexity: obfuscated domain knowledge, jealously protected relationships, and unnecessarily complicated code are all examples of acts of individual self-preservation of employment that undermine effective governance of a strategic software investment. The board member who everyone knows is beholden to the knowledge of the governed will be played. If there is nobody with the domain knowledge, relationships, and algorithmic know-how to call bullshit on the executors, then the bullshitters will rule the day every day.
Today, governance is increasingly being put to the test, specifically at a time when governance standards have become institutionally lax and have been so for a long period of time. Governance will tighten, as described by John Maynard Keynes in his book The Great Crash of 1929:
In depression all this is reversed. Money is watched with a narrow, suspicious eye. The man who handles it is assumed to be dishonest until he proves himself otherwise. Audits are penetrating and meticulous. Commercial morality is enormously improved.
Keynes' statement that "all of this is reversed" is all well and good, but the fact is that regulators always lag the regulated, and governance is a regulatory phenomenon. By the time regulators catch up with the regulated, the damage is done to regulators' reputations for control, scrutiny and alignment. Rest assured that your successor will bask in the glory of being a more diligent investor, a more scrupulent auditor, and a more competent leader than you were.
If you prefer not to allow your career gravestone to be a platform for your successor to dance upon, what does it take to know that your governance practices are up to the task?
It's worth pointing out that you never will. Governance is something that is only evident by its absence. In addition to being intangible, there is no infallibility to governance: in practice, even the most prudent governance structures fail to detect problems, while the most lax governance structures suffer no damage from problems that do not metastasize. Governance is not a precise science. Governance is only obvious when it succeeds as a counterfactual (hindsight makes clear a crisis was intentionally avoided, such as investing in Bernie Madoff's fund in the summer of 2008) or when it fails (a fund piles in all of its money into Madeoff's fund in the summer of 2008).
That said, what should IT be doubling-down on to increase the effectiveness of its governance practices?
- Strengthen the cost audit function. Ask people everywhere from the accounting fraternity to the program management office to procurement to increase the scrutiny of every dollar that goes out the door for a strategic software investment, whether for badged employee payroll, vendor services, cloud services, or licensed product. The objective is to identify non-value-generative chargebacks. Either the person responsible for the charge can answer "what value did we get for this" or they cannot. The point isn't to spot administrative errors, but inexplicable costs and patterns of charges that point to budget leakage or legitimately high costs that point to accelerated budget drainage, and to do so while threat to budget can be contained.
- Strengthen the solution audit function. Engage a captive technical audit function or an external vendor with no connection whatsoever to a program of work to perform audits and peer reviews of tests, code, and team practices. Is the code written to a degree of simplicity that it is comprehensively testable, and easily maintainable by other developers? Can the piece-parts of the solution be tested in isolation and without an environment, or do they require expansive infrastructure and consistent data fixtures? Are there transitive dependencies that create false testing positives or negatives? Are integrations validated through contract tests before arriving in an environment? Is the team working in very narrow silos of domain knowledge or technical expertise? Are some people acting as bottlenecks? Have the number of hand-offs increased to complete requirements or fix defects? Have the number of defects increased, or has the time to resolution of them increased? Keeping a close eye on both the what and the how provides an early warning to delays in delivery or higher cost of maintenance than planned.
- Increase the threshold of acceptance of management status reports. In medicine, patient testing is frequently done to confirm or deny a threshold of "no evidence of disease." This involves taking a sampling of cells and subjecting them to tests to ascertain a percentage present per the statistically significant sample, and a trend of counts of samples made over time. A higher standard of acceptance is "no evidence of disease", but that's an impractical threshold as 100% testing of cells would be fatal to the patient. In strategic software investing, the lower threshold of "no evidence of failure" is a proxy of convenience to the board: if the program manager says the status is green and the summary indicators support that, the program manager has given no reason not to believe the status is green, but not reason to believe that it well and truly is. Program governance must raise the threshold on program management to provide "evidence of no failure". That means program management conclusively demonstrates that effort is not passing for results, tasks are not masquerading as requirements, and work is not being deferred that conflates "developer complete" with "user acceptance".
- Be an activist investor. As a board member, get your own data, form your own contra hypotheses, and engage in direct, constructive interrogation at board meetings. As a board member, you can talk to users, shadow teams, analyze delivery data, look at code and run analyses over that code. Nothing else will provide more effective leadership in these times from a governance role than investor activism that respectfully challenges those on the line creating a strategic software asset.
- Have at least one independent director on the governing board. Break up traditional governance structures by having independent directors with no connection to solution sponsorship, vendor representatives, or delivery managers. With no political, financial or emotional investment in any one or any thing, independent directors are better positioned to provide critical insight of the state and trajectory of the investment, as well as clinical recommendations for potential change.
Governance of strategic software investments does well to apply the standard of US <-> Soviet arms reductions treaties in the 1980s of "trust, but verify." Creating a no-trust governance environment is counterproductive. Low-trust environments are extremely dysfunctional and chaotic, and at best offer an anti-pattern for governance. That clubby team of vendor reps and ambitious managers doesn't perform well when at each other's throats like the crew of the ISS Enterprise in Star Trek: TOS. But a board driven by an activist chair and flanked by an independent director with a penchant for healthy skepticism creates an environment where trust is earned and re-earned, rather than tacitly given or outright kicked to the curb. That sets the tone for the board, delivery leaders and delivery partners, a tone especially valuable during a time when control is at a premium and trust is in rather short supply.
