Scaling for Ghosts: 7 Microservices, 47 Users, and the Trap of Resume-Driven Development

The Case of the Over-Engineered Startup

A seasoned engineer, with eight years of experience in high-scale systems, joined a pre-seed startup tasked with building a project management tool for remote teams. Given total autonomy, the engineer proceeded to construct a full-blown, enterprise-grade distributed system. This elaborate setup included seven distinct microservices, a Kubernetes cluster on AWS, Kong API Gateway, Jenkins CI/CD, a sharded PostgreSQL database, a 3-node sharded Redis cluster, Kafka for event-driven messaging, Prometheus, Grafana, and an ELK stack, complete with circuit breakers and exponential backoff.

Remarkably, this system, designed for 100,000 users, was built for a startup that, after three months of development and a month in beta, had only 47 total users. The author, writing anonymously under the pseudonym "Production Systems" in *Stackademic*, candidly described it as a "crime scene" of over-engineering, built for "ghosts" rather than actual customers.

The Trap of Resume-Driven Development

This incident is not an isolated case but exemplifies a broader behavioral phenomenon: Resume-Driven Development. The core thesis is that the software engineering job market heavily rewards familiarity with complex, distributed systems. An engineer with "Deployed and maintained a 7-service Kubernetes cluster with Kafka event streaming" on their resume commands a significantly higher market premium than someone who simply "Maintained a single Spring Boot monolith."

The engineer themselves confessed, "I had 8 years of experience at companies with scale problems... I knew how to build systems that survive Black Friday." They were not optimizing for the startup's immediate needs but for their own perceived career trajectory, effectively using the pre-seed startup as a paid sandbox. This creates a fundamental conflict: the company needs rapid iteration and user acquisition, while the engineer is incentivized to build complex infrastructure that looks impressive on a resume, regardless of its actual utility to the business.

Tangible Costs of Over-Engineering

The consequences of building for "ghosts" were substantial.

Financial and Operational Drain

During the fourth month, this hyper-scalable system designed for 100,000 users was handling extremely low traffic. The database had minimal active connections, the Kafka cluster processed very few messages, and the Redis cache hit rate was negligible. It was, as described, a Rolls-Royce engine idling in a traffic jam.

For 47 users, the monthly AWS bill represented a significant cost per user, per month, purely in cloud infrastructure. For a pre-seed startup with limited runway, this was a critical drain on resources.

Human Capital Cost

The most chilling metric was the human capital cost. The engineer admitted to spending more than half their work week debugging infrastructure—wrestling with Kubernetes and Kafka—rather than building features or engaging with users.

In Month 5, the founder confronted the engineer, asking, "We need a frontend engineer. Can you handle backend alone?" The engineer's response: "Uh… maybe? I'm spending most of my time on infrastructure." The founder reasonably pushed back: "Why? We have 47 users. Serious question. What are you working on?" The founder's perspective was clear: "We need features to get users. Not infrastructure for users we don't have." This highlights that a startup's most precious resource is developer hours, and burning that time on unnecessary complexity is akin to suffocating the company.

Cargo-Culting and Premature Optimization

The report also points to the "cargo-culting" of buzzwords. Kafka, for instance, was developed by LinkedIn to handle trillions of messages per day. Here, it was used to manage to-do list updates for 47 people, generating very few messages per hour—the architectural equivalent of using a commercial jetliner to cross the street.

This is part of a broader industry culture susceptible to "cargo-cult programming," where developers mimic the outward forms of successful systems without understanding *why* those systems were built that way. Tech vendors and cloud providers aggressively market complex architectures, and startup engineers often prematurely adopt solutions detailed in hyperscaler engineering blogs (the "Netflix/Uber effect"). However, Netflix adopted microservices to solve organizational scaling problems with thousands of engineers, not just technical traffic problems. As the report aptly states, "If you have fewer users than can fit on a standard city bus, you do not have a distributed systems problem. You have a customer acquisition problem."

The Path to Simplicity: A Successful Rewrite

Following the confrontation and a moment of clarity, the engineer pitched a total rewrite. In just three weeks during Month 6, they merged all seven microservices into a single Spring Boot application with a PostgreSQL database. Kafka was replaced with simple PostgreSQL queues, Redis was removed entirely, and Kubernetes was deleted. The entire simplified system was deployed on Railway, a Platform-as-a-Service.

The results were staggering: the monthly AWS bill plummeted significantly, deploy times were drastically reduced, and debugging time dropped substantially. Most critically for a startup, feature velocity doubled. The author's initial conclusion was that "Constraints Are Gifts. A monolith forces you to keep things simple. A single database forces you to think about data design. Constraints force good decisions."

Beyond the Monolith: Nuance and Trade-offs

While the outcome was positive, the idea that "a monolith forces good decisions" is a convenient and arguably simplistic takeaway. A sloppy developer can build a disastrous, tightly-coupled, unmaintainable "Big Ball of Mud" monolith just as easily as they can build messy microservices. The monolith simply limits the *blast radius* of bad decisions by removing network boundaries, latency, and serialization pain; it doesn't magically cure poor design principles.

This critique aligns with a broader industry trend of high-profile companies abandoning microservices when operational overhead outweighs benefits. Martin Fowler, a renowned software architect, has long advocated for a "Monolith First" strategy, noting that most successful microservice stories started with a monolith that grew too big. Our case study is a textbook example of the "serious trouble" that can arise from building a microservice system from scratch.

Even Amazon Prime Video, a poster child for cloud microservices, saw its Video Quality Analysis team move from a distributed, serverless microservices architecture back to a single-process monolith, resulting in a 90% reduction in infrastructure costs and massively improved scaling. Similarly, Segment, a customer data platform, consolidated over 150 microservices into a "distributed monolith" due to the operational overhead of updating shared libraries. Kelsey Hightower, a prominent Kubernetes advocate, has also stated, "Microservices will not fix a bad Monolith." These examples reinforce that architectural choices are trade-offs, not universal truths, and a "monolith-first" approach is often pragmatic, deferring complexity until absolutely necessary.

The User's Perspective and Future Heuristics

Perhaps the most poignant realization from the engineer's story is what the actual users experienced throughout this architectural odyssey. The 47 users noticed "zero difference" from the initial complex architecture. They never saw the seven microservices, Kubernetes, or Kafka, and they never cared about the architecture. Users subscribe to a product for a problem solved, not for its impressive Kubernetes manifest.

The engineer's eventual heuristic, after this painful lesson, was to design for "Now + One Order of Magnitude." If you have 50 users, design for 500, not 100,000. This is a simple, practical rule for scaling responsibly—building just enough to meet current and immediate future needs, rather than chasing hypothetical future scale.

Ultimately, the engineering culture needs to shift its rewards system. We need to celebrate the delivery of value and problem-solving, not just the deployment of complex, buzzword-heavy infrastructure. The opportunity cost of an engineer spending a significant portion of their week wrestling with Kafka consumer lag instead of building features that would actually acquire customers is a hard but crucial number for engineering leaders to quantify.