Paper Trail

The Unshakeable Law of Software and the AI Revolution

March 20, 202617:49Paper Trail

This episode explores Joel Spolsky's "unshakeable law" from his 2000 essay, arguing that rewriting software from scratch is the "single worst strategic mistake" a company can make. Listeners will learn why this approach leads to the loss of valuable "embedded knowledge" and can be disastrous, as exemplified by the cautionary tale of Netscape's downfall and the "second-system effect." The discussion highlights the enduring relevance of these principles in software development, even in an AI-driven world.

Key Takeaways

Detailed Report

The Unshakeable Law of Software and the AI Revolution

Building software often tempts developers with the idea of starting fresh, scrapping old, 'clunky' code for a pristine new version. However, according to Joel Spolsky's seminal 2000 essay, "Things You Should Never Do, Part I," this impulse is "the single worst strategic mistake" a company can make. Spolsky's "unshakeable law" posits that throwing away existing code means discarding invaluable 'embedded knowledge'—years of accumulated bug fixes and problem-solving wisdom. This principle, initially laid out over two decades ago, remains profoundly relevant, even as artificial intelligence transforms the landscape of software development.

The Peril of the Blank Slate

Spolsky's core argument against rewriting code from scratch is blunt: what appears to be a 'mess' often contains critical, hard-won knowledge. Programmers, he noted, are often convinced they can build something better and cleaner, but in doing so, they lose the history of problems solved. It's akin to demolishing a library to build a new one, only to lose all the books and the wisdom they contain.

This strategic pitfall resonates with other well-documented phenomena in software development:

  • The Second-System Effect: Described by Frederick P. Brooks Jr. in *The Mythical Man-Month*, this effect highlights how architects, after building a successful first system, often become overconfident and attempt to incorporate too many features into a second system, leading to bloat and complexity.
  • The Moving Target Problem: A fundamental flaw of large-scale rewrites, this problem arises because the old system must continue to operate and evolve while the new one is being built. New features added to the old system must then be incorporated into the new, creating a constantly expanding scope and a perpetually delayed launch.

Cautionary Tales from History

The history of software development is littered with examples that validate Spolsky's warning:

  • Netscape's Downfall: In the mid-90s, Netscape Navigator dominated the browser market. Facing fierce competition from Microsoft's Internet Explorer, Netscape management made the fatal decision to rewrite their browser from scratch, aiming for Netscape 6. This rewrite took nearly three years, during which Netscape couldn't release major updates. Its market share plummeted from 86% in 1996 to just over 50% by late 1997, and by the time Netscape 6 launched in 2000, it was a commercial and critical failure, accelerating the company's demise.
  • Digg v4: A pioneering social news aggregator, Digg launched a complete redesign and rewrite, 'Digg v4,' in 2010. Plagued with bugs and perceived as favoring large publishers, it alienated its user base, who largely migrated to Reddit. The rewrite effectively killed the company.
  • Friendster's Struggle: One of the earliest social networks, Friendster embarked on a series of redesigns and rewrites to address scaling issues. This stalled development, leading to notoriously slow page load times and creating a vacuum that competitors like MySpace and Facebook quickly filled.

These examples underscore the timeless dangers of underestimating complexity, freezing feature development, and alienating users through a 'big bang' rewrite.

AI as a Catalyst for Incremental Improvement

The advent of artificial intelligence, particularly Large Language Models (LLMs), is fundamentally changing how developers interact with and improve existing code. AI-powered tools like GitHub Copilot and Amazon CodeWhisperer are acting as intelligent 'refactoring partners,' automating many of the tedious and error-prone tasks that Spolsky advocated for doing incrementally.

These tools enable developers to:

  • Automate Refactoring: Highlight confusing code and use natural language prompts to suggest cleaner, more descriptive versions, and even generate unit tests to ensure functionality remains intact.
  • Perform Security Scans: Identify vulnerabilities in existing code and provide generative AI-powered suggestions for fixes.

The efficiency gains are remarkable. Reports indicate that generative AI can accelerate tech modernization timelines by 40-50%. Organizations using AI-driven refactoring have reported a 40% improvement in code maintainability, a significant reduction in new technical debt, 40% faster code review cycles, and 60% fewer regression bugs.

AI as a 'Software Archaeologist'

One of the biggest barriers to modernizing legacy systems is simply understanding what they do. These systems are often poorly documented, with crucial business logic embedded in the code, and that knowledge often departs with original developers. AI is proving invaluable as a 'software archaeologist,' capable of:

  • Automated Documentation: Analyzing vast, complex codebases to generate human-readable explanations of modules in plain English, drastically reducing the learning curve.
  • Dependency Mapping: Untangling complex dependencies and creating visual maps of system interactions.
  • Inferring Intent: LLM agents can generate natural-language summaries for thousands of functions, making system functionality explorable through semantic browsers.

This automated comprehension phase significantly lowers the risk for any modernization effort, whether refactoring or a potential rewrite.

The Enduring Limits of AI in Rewrites

Despite AI's power, the report concludes that it does not eliminate the core strategic risks Spolsky identified. While AI can dramatically accelerate the *coding* portion of a rewrite, it doesn't solve the 'moving target problem,' the danger of a feature freeze, or the loss of implicit knowledge. Experts caution that current AI models have significant limitations:

  • Lack of Domain Context: AI knows *what* the code does, but not *why* certain trade-offs were made or the specific business rules behind them. It's like an inexperienced junior programmer rewriting a complex system they don't fully understand.
  • 'Confidently Wrong' Output: AI can produce plausible-looking code that passes some tests but contains subtle bugs, performance regressions, or security vulnerabilities. As Martin Fowler noted, AI output should be treated as "a pull request from a rather dodgy collaborator" requiring rigorous human review.
  • Architectural Incapacity: AI is not yet capable of making complex architectural decisions or understanding the trade-offs involved in designing a new system. Experiments in early 2025 to have leading AI models completely rewrite multi-file legacy applications failed to produce complete, functional results.

Human supervision and judgment remain absolutely critical. AI excels at heavy lifting and boilerplate code, but not deep strategic thinking.

The 'Ship of Theseus' Approach: Bending the Law

The most promising development is a new hybrid approach: the 'Ship of Theseus' strategy. This involves progressively replacing parts of a legacy system with new, modernized components. The new code initially acts as a proxy, redirecting to the old system. Feature by feature, new implementations are built and swapped in, while old code is slowly deprecated and deleted.

This incremental rewrite, supercharged by AI, avoids the 'stop the world' danger of a full rewrite. The system remains operational, continuously delivering value, and allowing for ongoing learning and course correction. AI tools are leveraged at every step to understand old components, generate new code, translate business logic, and create tests.

AI isn't enabling a 'big bang' rewrite; it's enabling a faster, safer, continuous modernization strategy that honors Spolsky's core wisdom. It transforms the rewrite from a high-stakes gamble into a managed process of renewal, leveraging AI's speed and scale while adhering to the strategic prudence of incremental change. Spolsky's law, therefore, isn't broken by AI; it's bent and upgraded for a new era of software development.

Show Notes

The Unshakeable Law of Software and the AI Revolution

Source Materials

  • Research Prompt: An inquiry into the continued relevance of Joel Spolsky's "unshakeable law" against rewriting software from scratch, particularly in the context of modern AI-driven development.

References & Resources

  • Joel Spolsky: Software developer, writer, and co-founder of Stack Overflow, known for his influential essays on software development.
  • Things You Should Never Do, Part I: A seminal 2000 essay by Joel Spolsky arguing that rewriting software from scratch is "the single worst strategic mistake" a company can make.
  • Netscape Navigator: A pioneering web browser that dominated the market in the mid-1990s before its decline following a strategic rewrite attempt.
  • Microsoft Internet Explorer: A web browser developed by Microsoft, which gained market share by being bundled with Windows, ultimately surpassing Netscape Navigator.
  • The Mythical Man-Month: A classic 1975 book on software engineering by Frederick P. Brooks Jr., which introduced concepts like the "second-system effect."
  • Digg v4: A controversial 2010 redesign and rewrite of the social news aggregator Digg, which led to significant user backlash and the platform's decline.
  • Reddit: A social news aggregation, content rating, and discussion website that saw significant growth following Digg's missteps with v4.
  • Friendster: One of the earliest social networking sites, which struggled with scaling and development issues, creating an opening for competitors like MySpace and Facebook.
  • GitHub Copilot: An AI-powered code suggestion tool that assists developers by generating code snippets and refactoring suggestions in real-time.
  • Amazon CodeWhisperer: An AI coding companion that provides real-time code recommendations, security scans, and generative AI-powered fixes.
  • McKinsey & Company: A global management consulting firm whose reports highlight the efficiency gains from generative AI in tech modernization.
  • Martin Fowler: A prominent author and speaker on software development, quoted for his cautious view on AI-generated code, comparing it to a "dodgy collaborator."
  • Forte Group: A software development company mentioned for its use of LLM agents to analyze and document large legacy codebases.

Glossary

  • Embedded Knowledge: The accumulated understanding, bug fixes, and solutions built into existing software code over time. When code is rewritten from scratch, this valuable, often undocumented, knowledge is lost.
  • Second-System Effect: A phenomenon described by Frederick P. Brooks Jr. where, after building a successful first system, developers tend to over-engineer a subsequent system by trying to incorporate too many features, leading to complexity and delays.
  • Moving Target Problem: A strategic challenge in large-scale software rewrites where the existing system continues to evolve with new features and requirements, causing the scope of the new system to constantly expand and delay its completion.
  • Technical Debt: The implied cost of additional rework caused by choosing an easy or quick solution now instead of using a better approach that would take longer. It accumulates over time and can make software harder to maintain.
  • Refactoring: The process of restructuring existing computer code without changing its external behavior. Its purpose is to improve nonfunctional attributes of the software, such as readability, maintainability, and complexity.
  • Large Language Models (LLMs): A type of artificial intelligence model trained on vast amounts of text data, capable of understanding, generating, and processing human language, and now increasingly used for code.
  • Unit Tests: Small, isolated tests written by developers to verify that individual units or components of a software application work as intended, ensuring changes don't introduce new bugs.
  • Legacy Code: Source code that is no longer actively supported or maintained, or code inherited from older systems. It is often difficult to understand, modify, and integrate with modern technologies.
  • COBOL: (Common Business-Oriented Language) An older, high-level programming language primarily designed for business, finance, and administrative systems. Many critical legacy systems still run on COBOL.
  • Java: A widely-used, object-oriented programming language known for its "write once, run anywhere" capability, often used for cloud-native applications.
  • Regression Bugs: Software bugs that appear in a previously working feature after a change or update has been made to the code, indicating a step backward in functionality.
  • Ship of Theseus: A philosophical thought experiment about identity, applied in software development to describe a continuous, incremental modernization strategy where parts of a system are progressively replaced while the system remains operational, eventually transforming it entirely.

Full Transcript

HostSo, imagine this: you're building a groundbreaking piece of software. It’s working, but it's a bit clunky, full of patches, maybe not the prettiest code. And you think, "You know what? Let's just scrap it. Start fresh, build it right this time."
ExpertAnd if you're a software developer, that thought is probably the most seductive, dangerous idea you'll ever have. Because according to one of the most foundational essays in software engineering, doing that is "the single worst strategic mistake" a company can make.
HostThe single worst? That's a pretty bold claim, especially coming from an essay written back in 2000. But the evidence presented in this report suggests that not only was Joel Spolsky, the author, right then, but his "unshakeable law" might still hold true even in our AI-driven world.
ExpertIt's a fascinating look at what we think of as progress. We're talking about a world where AI can practically write code itself, and yet, the fundamental human and strategic pitfalls of software development seem to stubbornly persist.
HostSo, let's start at the beginning. This idea, this "law," comes from an essay titled "Things You Should Never Do, Part I," by Joel Spolsky. What was his core argument back in 2000?
ExpertHis core argument was really blunt: never rewrite your code from scratch. He said it's a "seductive but deeply flawed fantasy." Programmers, he argued, look at existing code and see a "mess," and they're convinced they can build something better, cleaner, faster, without all the old mistakes.
HostAnd he says that's flawed. Why? Because what looks like a mess actually contains something incredibly valuable?
ExpertPrecisely. Spolsky's key insight was this concept of "embedded knowledge." He wrote, and I'm quoting here, "When you throw away code and start from scratch, you are throwing away all that knowledge. All those collected bug fixes. Years of programming work." It's like demolishing an old, cluttered library because you think you can build a new, modern one faster, but in doing so, you lose all the actual books and the wisdom they contain.
HostThat makes a lot of sense. You're not just getting rid of bad code, you're getting rid of the history of problems solved. And the prime cautionary tale he used was Netscape. For those who might not remember, Netscape was *the* browser of the early internet.
ExpertAbsolutely. In the mid-90s, Netscape Navigator dominated, sometimes capturing 80-90% of the browser market. They were the commercial internet. But then Microsoft's Internet Explorer came along, bundled for free with Windows, and the competition got fierce.
HostAnd that's when Netscape made their fatal strategic error, according to Spolsky.
ExpertThat's right. Instead of incrementally improving their existing browser, Netscape management decided to scrap it all and rewrite Netscape Communicator 4.0 from the ground up, aiming for a new, standards-compliant version, Netscape 6. The plan was noble, the execution disastrous.
HostHow disastrous are we talking? What was the timeline here?
ExpertWell, the rewrite took nearly three years. In the rapidly evolving internet economy of the late 90s, three years was an eternity. During that time, Netscape couldn't release any major new versions, essentially freezing their product while Microsoft was iterating relentlessly. The market share numbers tell the story: Netscape went from 86% in 1996 to just over 50% by the end of '97. By September 1998, Internet Explorer had overtaken them.
HostWow. And when Netscape 6 finally launched in November 2000, what happened?
ExpertIt was a commercial and critical failure. Slow, buggy, unstable. By then, IE's market share exceeded 80%. Spolsky's essay, published months before the official release, was prophetic. The rewrite didn't save Netscape; it accelerated its demise. It was a multi-billion dollar business destroyed, in part, by this single strategic mistake.
HostThat's a stark example. But Spolsky wasn't just talking about Netscape. This "never rewrite" idea resonates with other well-documented phenomena in software development, like the "second-system effect."
ExpertExactly. This concept was first described by Frederick P. Brooks Jr. in his seminal 1975 book, *The Mythical Man-Month*. He noticed that after building a successful first system, architects often get overconfident. They want to incorporate *every* feature they deferred the first time around, leading to an over-engineered, bloated, and overly complex second system.
HostSo, the first system is usually lean because you're cautious, but the second one is where you get ambitious, trying to do too much.
ExpertPrecisely. Brooks called it "dangerous." And Netscape's attempt to build a revolutionary new browser from scratch, incorporating a host of new architectural ideas, is a classic example of this second-system effect playing out. They were emboldened by their initial success, but that ambition led to their downfall.
HostAnd then there's the "moving target problem." This sounds like something that would plague any long-term development project.
ExpertIt's a fundamental strategic flaw of the "big rewrite." While you're busy building your shiny new system, the old one can't just stop. It still needs to serve existing customers. New features still need to be added to remain competitive. This creates a debilitating split. You're effectively running two parallel development tracks.
HostSo, any new feature added to the old system during the rewrite has to be incorporated into the new one as well. That sounds like a scope nightmare.
ExpertIt is. It leads to duplicated effort and a constantly expanding scope for the rewrite project, making it a seemingly endless development cycle. The new system is perpetually behind the evolving market needs. It’s a longer, harder, and more failure-prone path than many anticipate.
HostYou'd think after Netscape, companies would learn this lesson. But the report highlights that these strategic dangers are timeless, with examples from well *after* Spolsky's essay. Digg v4 and Friendster are called out.
ExpertAnd they are excellent examples. Digg was a pioneering social news aggregator, but in 2010, they launched "Digg v4," a complete redesign and rewrite. It was plagued with bugs, removed popular features, and was perceived as favoring large publishers. The backlash was swift. Users fled to Reddit, which was much smaller at the time. The rewrite essentially killed the company.
HostThat's incredible. So, a redesign, a rewrite, can actually be a corporate killer. What about Friendster?
ExpertFriendster, one of the earliest social networks, struggled with scaling. They embarked on a series of redesigns and rewrites, which stalled development. The site became notoriously slow—page load times sometimes exceeded 40 seconds! This created a massive vacuum, a window for competitors like MySpace and Facebook to swoop in and capture the market. They were chasing a perfect new system while failing to evolve their working one.
HostSo, the core warnings from Spolsky—underestimating complexity, the feature freeze, alienating users—they're still relevant today. It really does seem like an unshakeable law. But this report also brings in the major disruptor: AI. How does AI change this picture?
ExpertThis is where it gets really interesting. AI, particularly Large Language Models, are fundamentally transforming how developers interact with and improve existing code. AI-powered tools are now acting as intelligent "refactoring partners," automating many of the tedious and error-prone tasks that Spolsky advocated for doing incrementally.
Host"Refactoring partner." So, not rewriting, but improving the existing code structure while preserving its functionality. Things like renaming variables, extracting duplicated code, simplifying logic.
ExpertExactly. In 2000, this kind of large-scale, incremental refactoring was manual, laborious. Today, AI tools like GitHub Copilot and Amazon CodeWhisperer are integrated directly into a developer's workflow. A developer can highlight confusing legacy code, give a natural language prompt – "Refactor this to be more descriptive" – and Copilot suggests a cleaner version. It can even generate unit tests to ensure the refactoring doesn't break anything.
HostThat sounds like a game-changer for tackling technical debt. CodeWhisperer does similar things, right?
ExpertYes, CodeWhisperer offers similar real-time code suggestions. A notable feature is its ability to perform security scans on existing code, identify vulnerabilities, and then provide generative AI-powered suggestions to fix them. So, these tools are directly addressing Spolsky's point about incrementally improving code, but doing it at a speed and scale he couldn't have imagined.
HostAnd the report cites some pretty impressive numbers from 2025 and 2026 regarding efficiency gains.
ExpertThey are remarkable. McKinsey reported that generative AI can accelerate tech modernization timelines by 40-50%. One logistics company apparently cut their modernization timeline by more than half by using AI to transform legacy COBOL modules into a cloud-native Java environment.
HostWow. COBOL to Java. That's a significant leap. And what about technical debt, that accumulation of quick fixes and compromises?
ExpertOrganizations using AI-driven refactoring have reported a 40% improvement in code maintainability and a significant reduction in new technical debt. Some estimates suggest AI can reduce the time required for complex modernization by 40% to 50%. Plus, improved quality: 40% faster code review cycles and 60% fewer regression bugs.
HostThose are staggering figures. It sounds like AI is making the incremental approach, the one Spolsky championed, not just wise, but incredibly efficient. But this leads to the central question: if AI can do all this, is it bending Spolsky's law? Does it make the "big rewrite" less dangerous?
ExpertThis is where the report gets very nuanced. One of the biggest barriers to modernizing or even replacing a legacy system is simply *understanding* what it does. These systems are often poorly documented, with crucial business logic embedded in the code, and often, that knowledge leaves when the original developers do.
HostSo, AI becomes a "software archaeologist." It can dig through decades-old code and make sense of it?
ExpertPrecisely. LLMs can analyze vast, complex codebases and perform tasks like automated documentation, generating human-readable explanations of modules in plain English. This dramatically reduces the learning curve. They can also untangle complex dependencies, creating visual maps of how different parts of the system interact.
HostThat's huge. A high-level architectural view that might have taken a human team months to piece together.
ExpertAnd AI can even infer the "intent" behind the code. The report mentions Forte Group using LLM agents to generate natural-language summaries for over 12,000 functions in a legacy insurance platform, making the system's functionality explorable through a semantic browser. This automated comprehension phase significantly lowers the risk for *any* modernization effort, whether it's refactoring or a potential rewrite.
HostSo, AI helps you understand the old system. And if it can write new code, does that mean it makes a full rewrite less dangerous now? Can AI just rewrite the whole thing for you?
ExpertThe answer, according to the report, is still a nuanced "not really." While AI can dramatically accelerate the *coding* portion of a rewrite, it doesn't eliminate the core strategic risks Spolsky identified. You still have the moving target problem, the danger of a feature freeze, and the loss of implicit knowledge.
HostSo, AI doesn't solve the strategic challenges, just the tactical coding ones.
ExpertExactly. And experts caution that current AI models still have significant limitations. They often lack domain context – they know *what* the code does, but not *why* certain trade-offs were made, or the specific business rules behind it.
HostIt's like asking a brilliant but inexperienced junior programmer to rewrite a complex system they don't fully understand.
ExpertA very apt analogy. And AI models can also be "confidently wrong." They can produce plausible-looking code that passes some tests but contains subtle bugs, performance regressions, or security vulnerabilities. Martin Fowler is quoted saying AI output should be treated as "a pull request from a rather dodgy collaborator" that requires rigorous human review.
Host"A dodgy collaborator." I love that. So, you can't just trust it implicitly. And what about high-level architectural decisions?
ExpertThe report notes that AI isn't yet capable of making those complex architectural decisions or understanding the trade-offs involved in designing a new system. An experiment in early 2025 to have leading AI models completely rewrite a multi-file legacy application actually failed. They couldn't process the entire codebase or produce a complete, functional rewrite.
HostSo, the consensus is that AI is a powerful assistant, but human supervision and judgment remain absolutely critical. It excels at the heavy lifting, the boilerplate, the code translation, but not the deep thinking.
ExpertThat's right. And this leads us to the most promising development in this space: a new hybrid approach that really bends, but doesn't break, Spolsky's law. The report calls it the "Ship of Theseus" approach.
HostThe Ship of Theseus, like the philosophical thought experiment where if you replace every plank of a ship, is it still the same ship?
ExpertPrecisely. In software, it means progressively replacing parts of the legacy system with new, modernized components. The new code acts as a proxy, initially redirecting to the old system. Then, feature by feature, the new implementation is built and swapped in, while the old code is slowly deprecated and deleted.
HostThis sounds like an incremental rewrite, but supercharged by AI.
ExpertThat's exactly what it is. It avoids the "stop the world" danger of a full rewrite, because the system remains operational and can continue to deliver value. It's a gradual process, allowing for continuous learning and course correction. And AI tools are used at every step to understand the old components, generate new code, translate business logic, and create tests.
HostSo, AI isn't enabling a "big bang" rewrite; it's enabling a faster, safer, continuous modernization strategy that actually honors Spolsky's core wisdom.
ExpertThat's the key takeaway. It transforms the rewrite from a single, high-stakes gamble into a continuous, managed process of renewal. It's the best of both worlds: leveraging AI's speed and scale while adhering to the strategic prudence of incremental change.
HostThis has been a fascinating journey, from the early internet to the cutting edge of AI. So, to wrap up, what are the absolute key insights listeners should take away from this discussion?
ExpertI'd say first, Spolsky's core strategic principles are still very much alive. The dangers of the "second-system effect," the "moving target problem," and losing embedded knowledge in legacy code are timeless risks in software development.
HostAnd the second insight?
ExpertAI hasn't invalidated those strategic warnings, but it *has* fundamentally changed the mechanics of how we deal with legacy code. Automated, large-scale refactoring makes the incremental improvement Spolsky advocated for faster, cheaper, and more efficient than ever.
HostSo, it's about *how* you do it, not *if* you should do it.
ExpertExactly. And a huge part of that "how" is AI's third contribution: understanding legacy systems. AI's ability to rapidly analyze, summarize, and document complex codebases drastically lowers the risk and cost of engaging with old code, making previously daunting projects much more manageable.
HostAnd finally, the future of software modernization isn't a binary choice between refactor or rewrite.
ExpertAbsolutely. The modern, AI-supercharged approach is a hybrid one: continuous, incremental modernization using the "Ship of Theseus" strategy. It avoids the catastrophic risk of a "big bang" rewrite while still achieving the goal of a modern, maintainable system. AI acts as an accelerator for this safer, more pragmatic approach.
HostIt's almost like AI has given Spolsky's law an upgrade. It bends, but it hasn't broken.
ExpertA very precise way to put it.
HostSo, for listeners thinking about their own projects, whether it's code, or even just processes in their own lives, perhaps the question isn't whether to start over, but how to continually and incrementally improve what's already working. And how much hidden knowledge might be embedded in the systems we're quick to dismiss as a "mess."