Fail Early, Fail Often... Or Maybe Not?
A Catastrophe of Start-Up Mantras
Welcome to Polymathic Being, a place to explore counterintuitive insights across multiple domains. These essays take common topics and investigate them from different perspectives and disciplines to come up with unique insights and solutions.
Today's topic takes the recent OceanGate catastrophe and demonstrates how it is quite literally the hardware manifestation of Silicon Valley software start-up mentalities. We’ll explore how these simple tropes lead to terrible behaviors, myopic designs, and are behind the vast majority of failures of start-ups to get out of the seed rounds of funding.
“Fail early, fail often.”
“Minimum Viable Product.”
“Building the airplane while flying it.
“Ship it, fix it, ship it again.”
“Move fast and break things.”
Behind each of these cute design slogans are millions of horror stories of myopic leaders, sub-optimal designs, terrible decisions, products that can’t scale, systems architectures that look like bubblegum and bailing twine, and failed start-ups. It’s the bane of simple tropes that hide complex systems.
Worse is that I hear them all the time. Just the other day I got pushback from a member of a client team as I was leading them through my Systems Innovation process. Complaining about the time it took to do the systems analysis he said, “Make it work, then make it right, and always make it fast.”
I just asked: “Make what work? And please define ‘right.’” He sat there with no answer that solved the direct, let alone the more complex problem set and so I brought the attention back to Systems Thinking. When we finished the exersice we’d defined the whole system, captured measures of outcome, effectiveness, and performance, and had four courses of action with go/no-go gates to quickly adapt toward the best solution. We defined the ‘What’ and then what ‘Right’ looks like.
There’s an old program management axiom that states:
“There’s never enough time to do it right the first time, but there’s always time to go back and do it over and over and over again.”
Yet behind each of those slogans above is also a great grain of truth. Yes, you should be willing to accept failure. Certainly, you should be able to come up with a demonstration minimum viable product. There ought to be a continuous integration, continuous deployment Lean software model. Of course, we should move fast and sometimes things will break. But no, never build the airplane while flying it… that’s just dumb on all levels!
These slogans, however, get distilled down to tropes, and tropes are misapplied the vast majority of the time. What starts as a motivation becomes an excuse for piss-poor planning, reactive leadership, and a lack of design. We covered some of this in Taoism and Proofs of Concepts which can be summarized as “Design first; then code.” These tropes also make up a foundation for many organizations’ Functional Stupidity. They are easy shortcuts that gloss over reality, end up being notoriously difficult to execute, and have no place with unskilled leadership.
The Consequences
OceanGate’s Titan is a recent example of what happens when you apply these tropes to a hardware design. That submersible was literally the manifestation of Silicon Valley software start-up slogans applied to critical hardware designs heading into incredibly difficult environments.
For perspective, more people have been in space than have been at the depth that Titan was going down to. It’s not a forgiving environment like deploying a software product to the cloud where a catastrophic failure means your website won’t load. These slogans create enough bad designs and mountains of tech debt for software; they’re horrific for most other things if applied without context.
OceanGate’s CEO, Stockton Rush, lived up to his name as he pushed his team to get something, anything out the door to prove the concept quickly. The errors are legion from using a wireless gaming controller, to expired carbon fiber (an already dumb material to use) to not using experienced submariner expertise because they’re old and white and not ‘inspirational.’
However, his young and inspirational safety engineer was fired for raising concerns in previous years and a litany of other risks were ignored as they rushed forward. It’s the exact same behavior that I’ve seen on program after program in company after company all with some form of justification from the ‘Build it while you fly it’ type mantras.
Back to Software
Yes, that was a hardware issue on critical safety equipment in what looks like a one-off situation. I hope it’s a one-off because most companies that work in these extreme spaces might go slow, they might be expensive, and they might be risk-averse, but they don’t have this level of catastrophe. Yet I’ve been on hundreds of software design reviews which exhibit the exact same behaviors, often supported by those slogans.
This is where we need to balance the equation. Those slogans are useful, they are motivating, they invite experimentation and risk-taking and these are all good things.
And they also lead to proofs of concept that don’t scale, they encourage leadership to just continue to iterate on fractional improvements vs. systemic and transformative innovation, they result in mountains of tech debt, and justify rarely, if ever, learning from their failures. It’s what underpins the Programmers’ Credo:
“We do these things not because they are easy, but because we thought they were going to be easy.”
It’s the balance that’s the hardest part. The intent of this essay isn’t to provide a perfect answer but to provide a perfect example of these software slogans applied to another domain. If it seems absurd for OceanGate then it’s equally absurd, just less catastrophic in many other areas. It’s also the fundamental reason why over 70% of software companies don’t make it out of their seed rounds. That balance is also the key trait of skillful leaders. Since it is a balance, there’s not one simple answer either. It truly takes a systems perspective across multiple dimensions to balance.
The behaviors that pervert these slogans are something we’ve looked at with The Successfully Unsuccessful, Lazy Leadership, Systems Thinking, Face the Fear, Embrace the Divergents, Maturing Product Process and People for Start-up Success, and others. The solution is much harder but right now I’ll suffice to provide a clear example that we can use as a case study.
Summary
OceanGate provides a painful example of how slogans can get misapplied very quickly. It’s a perfect example due to the undeniable consequence of a litany of bad leadership and design decisions. What I hope everyone walks away from this essay with is that, while people might not die from the same behaviors elsewhere, that does not make them any less dumb.
Fundamentally, the pressure the OceanGate CEO put on his team to rush is nothing compared to the pressure he’s feeling right now. How useful does “Fail early, fail often,” “Minimum Viable Product,” “Building the airplane while flying it, “Ship it, fix it, ship it again,” or “Move fast and break things” sound right now?
Yes, challenge the status quo, and don’t get locked down in bureaucracy. Also ensure that you are balanced with cross-disciplinary, systems thinking to ensure you’ve got the full perspective on design, measures of performance, safety, and timely execution. It’s this challenging balance between process anarchy and process paralysis where the true innovation lies.
Enjoyed this post? Hit the ❤️ button above or below because it helps more people discover Substacks like this one and that’s a great thing. Also please share here or in your network to help us grow.
Polymathic Being is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Further Reading from Authors I really appreciate
I highly recommend the following Substacks for their great content and complementary explorations of topics that Polymathic Being shares.
- Business Leadership Insights
- Great Life Tips and Tricks
- Practical AI
The Muse by
Fun, Witty, Insight- Great little nuggets to learn from
- by Great round up of AI topics in the media and tech reporting.
Looking for other great newsletters and blogs? Try The Sample
Every morning, The Sample sends you an article from a blog or newsletter that matches up with your interests. When you get one you like, you can subscribe to the writer with one click. Sign up here.
I think about how China's Wanxiang corp bought A123 Systems (lithium-ion batteries) and Fisker Automotive's tech after both went bankrupt.
Both received millions in American taxpayer dollars to develop their tech. But then no more was forthcoming. They failed on their first "try", and there was no more government or investor willingness to fund a second, so they closed shop. The Chinese swooped in and scooped up those distressed assets, meaning the US lost the tech.
Meanwhile, China provided loans, investments, and incentives to Wanxiang to move it to China and spend the better part of a decade developing it before it became profitable.
China's strategy is expensive and often wasteful. It sometimes fails completely. We like to criticize them for "picking winners." But I wonder to what degree there's a fail-based iteration process that we're less tolerant, or which our system does not support, which costs us in the long run.
This doesn't directly touch on your point, which was a good one. Why rush into execution on a shaky foundation?
But their tech was good, just not quick enough to scale and become profitable.
As always I appreciate you avoid binary thinking. The question isn’t whether “fail fast” is right or wrong, it is in which circumstances it should be applied. Website design? Maybe, with caveats. Submarine construction? No.
“OceanGate’s CEO, Stockton Rush, lived up to his name”
I love this kind of witty writing!