The Pragmatic Engineer: Backstage: an Open-Source Developer Portal

👋 Hi, this is Gergely with a 🔒 subscriber-only issue 🔒 of the Pragmatic Engineer Newsletter. In every issue, I cover challenges at big tech and high-growth startups through the lens of engineering managers and senior engineers.

Backstage: an Open-Source Developer Portal

A deepdive into what Backstage is, why it is gaining rapid popularity across large tech companies, and alternatives to it.

Over the last year, I have talked with software engineers at a variety of companies (Netflix, Grab, Wealthsimple, QuintoAndar, Wayfair). They all kept mentioning the same tool name: Backstage. All these companies were either planning, or in the process of adopting, Backstage as their developer portal.

After looking further, I observed that, although only released in 2020 in public, Backstage has seen surprisingly large adoption at larger tech companies. Other adopters include American Airlines, Booking.com, Brex, DAZN, Epic Games, Expedia, Glovo, HelloFresh, Monzo, PagerDuty, Splunk, Siemens, Trendyol, Twilio VMware, Wise, and hundreds of others.

I decided to look more into this topic. To do so, I initially contacted the most well-known Backstage SaaS provider, Roadie, for their insights, talked with an engineer from the team which created Backstage, and got in touch with Backstage adopters.

In this issue, we cover:

1. The need for a developer portal. Why do tech companies need a developer portal, and at what stage does this become necessary?

2. The history of Backstage. How did it start, and where is it today?

3. What is Backstage, and how does it work? An overview of the main parts: the software catalog, software templates, TechDocs, and other plugins.

4. Why was Backstage open sourced? Backstage could be considered a competitive advantage for Spotify. Why did they open source it?

5. Spotify’s version of Backstage. Spotify operates arguably the most advanced version of Backstage. What additional features have they built, and how do they use their developer portal?

6. Getting started with Backstage. How do you adopt the tool? A case study from RD Station and advice from Roadie.

7. Alternatives to Backstage. If you’re looking for a developer portal, what other alternatives do you have?

This issue mentions several vendors related to developer portals. As per my ethics policy, I strive to provide an independent viewpoint and disclose any affiliations, should I have them. I have no affiliations with any vendors mentioned in this article.

As a note, this article contains a lot of images, and some email clients will cut off the bottom of the issue. Read this article online to enjoy it uninterrupted.

Read the article online

1. The need for a developer portal

Let’s take the typical path of a tech startup. A technical founder builds a prototype of a web app, hosting the code on GitHub, running it on AWS. The project gets traction, and the founder gets investment and hires a team.

The team now builds a mobile app and starts to get more customers. They begin moving away from Lambdas to managing their own EC2 instances. The mobile team adds feature flags to easily roll back features, and all teams add monitoring, alerting, and error/crash reporting.

Fast forward to when the team is at 30 engineers, split between 5 Product teams, with the first Platform engineering team about to be born. The company is trying to get the timing of this team right.

How does the engineering team operate regarding the 'non-coding' work? They probably have 1 or more wiki pages that list:

• The various services and codebases, and their owners

• Steps to execute common operational tasks like adding more server capacity and debugging infrastructure issues

• Links to SaaS providers they use for things like feature flags and observability, perhaps with some basic documentation on finding the relevant things

This setup still works for a small-enough team. However, as the engineering team grows, and the number of services, at one point:

• Several of the new services won’t be added to this wiki page.

• The documentation on the 'how to' tasks will become outdated.

• The most productive engineers can recall what system to go to, to get something specific done. 

• A major challenge for a new joiner engineer becomes unpacking a lot of the tribal knowledge about developer tools and processes.

Engineering teams these days increasingly make use of the Cloud and various SaaS services for developer tasks. There’s an explosion of interfaces to learn, tools to use, and places to find service information. When a team has too many of these tools, and the company wants to cut down on the tribal knowledge they need to get around, as a developer: they will consider putting a developer portal in place.

This is where a developer portal like Backstage comes in. Spotify went through this type of same journey and faced the same challenges.

2. The history of Backstage

In 2014, Spotify was at around 600 employees, and over 100 engineers, but growing quickly. Even at that time, the engineering team, they observed issues with their developer experience. Martina Iglesias Fernández is currently the CTO of Roadie. At the time, she was a lead backend engineer at Spotify, and she described these challenges at the time:

• A service sprawl. Spotify was spinning up new services weekly, or even more frequently.

• Duplicated work. Two teams would be building a similar service.

• Fragmentation. It was hard to tell what was deployed to production..

• Lack of ownership. Some services were becoming “unowned” after the person who created them changed teams or left.

Backstage was born in response to these issues, with the project starting in 2014, called System Z. Here’s how Martina described the approach of the company:

“Backstage was conceived inside Spotify at a time when the company was experiencing rapid growth. New engineers were joining every week, and with them, the number of microservices was increasing.

Collectively, Spotify’s engineering team realized that it was becoming more difficult to know what we had running in production, and who was responsible for each component.

Eventually, one of the Platform teams decided to build a catalog of all microservices and to model systems and components in it. The catalog was called System-Z.

System-Z started as a place where Spotify teams could register microservices and their metadata. There would be a link to the code, the name of a team who owns it, and the name of the product owner. Eventually it grew to include information about its relation to other components, and groups of microservices began to be organized into cohesive systems.

As the usage of System-Z grew, we realized it was also the perfect place for us to put frontend UIs for developer infrastructure tools. So many tools that previously only had a CLI were extended to also have a UI frontend inside System-Z.

This huge growth of usage and features led to a rewrite of System Z in 2017, to what is known as Backstage today.”

Spotify initially built Backstage for their own use cases. The company saw good success by adopting System Z and extending it to what would become Backstage. They observed the following results by 2020:

• A 55% decrease in onboarding time for their engineers. Onboarding time was measured by time until the 10th pull request.

• 280 engineering teams inside Spotify used Backstage.

• 2,000+ backend services, 300+ websites, 4,000+ data pipelines, and 200+ mobile features onboarded to Backstage.

The company realized that it was building something that similar companies could use, and company leaders decided to open source it in 2020. By the end of 2020, Backstage was accepted to the Cloud Native Computing Foundation.

As of today, Backstage is considered to be an incubating project, which is a stable project used successfully in production. More than 600 companies use Backstage to power their developer portals.

3. What is Backstage and how does it work?

Backstage is a platform for building developer portals. Its open-source version comes with a few components out of the box:

• The software and services catalog

• Software templates

• TechDocs

• Other plugins

Backstage is very much a “Do It Yoursel”' (DIY) Platform. So you can expect to either put in a lot of work to get it up and running (and working for your needs) or to turn to a vendor that already has almost-ready-to-go configurations. In this sense, it is similar to most open-source tooling offerings, where you have the self-hosted “DIY” version and vendors offering managed services with additional features.

The Software and Services Catalog

The catalog lists services, websites, and libraries. However, this catalog can also track entities like:

• APIs

• Domains

• Locations

• Resources: for example, S3 buckets

• Systems

• Users

• Groups: teams of users

Team pages. Ownership is a core concept of Backstage. Every catalog item has exactly one owner. The concept of ownership makes it easy to see which group or user owns certain components, like services, and what components a certain group owns. Owners can be individual users, though they are most commonly groups (teams).

A neat thing with team pages is how a team page can have a “Docs” tab, serving as a team wiki where the team can add things like how to contact the team, their roadmap, runbooks, and so on.

Components pages. For each component, which are most frequently services, APIs, and libraries, you can customize the component’s page with widgets. So, for example, for services, these pages can show health metrics, outages, documentation, and reports.

Some neat plugins include: 

• PagerDuty Backstage plugin: to sync ongoing incidents and display who is oncall

• GitHub Actions plugin: to visualize the builds

• Sentry plugin: to visualize errors and warnings

• Snyk plugin: to get application security insights for the service

• Lighthouse plugin: the audit results for a website

• Linking to operational tools and dashboards, such as monitoring and alerting, documentation, and so on

Each service can define tabs to display more details or allow triggering of actions. Commonly used plugins, operating as tabs, are:

• Security: displays results of dependency security alerts, vulnerability scans, security reports, and so on

• CI: shows build status and the build log to help debug or re-trigger the build.

• API: shows the definition of the API, if the service exposes one and displays endpoints to help you understand how to use them

• Documentation: TechDocs for the component or service 

• Pull requests: displays the currently open ones, as well as the closed ones

• Kubernetes: reports the status of clusters and how to manage them, including across different Cloud providers, providing visibility into multi-Cloud in one place, and answers questions like how many pods are running in production, the regions where they are based, or if there are unhealthy pods

Software Templates

The Scaffolder shows how to create new services/components from Backstage. When developers want to create a new API, service, website, or any other component, they could do it from Backstage, assuming there’s a Scaffolder set up for it.

A scaffold is a template, which can be a series of steps to collect details before generating the scaffolded version.

As engineering teams grow, scaffolding becomes a more pressing need. If you are a small team, every engineer can set up their own services. However, if you have more than a dozen engineering teams that are creating new services, libraries, and websites, ideally all these new things could follow similar approaches when it comes to things like:

• The structure of the project

• Logging and error reporting

• Common dependencies used

“The golden path to production” is a popular term to describe what a well-defined approach to scaffolding could help achieve.

Scaffolding is not just about standardization, but it is about getting started quickly. With scaffolding, a new joiner can create a new service/website/library with confidence, knowing how to follow the company best practices from the start.

In short, scaffolding works like this:

• Collect values from the user, as a series of steps.

• Inject these values into an existing codebase template.

• Run other steps, such as hook up the new repo to monitoring and logging tools, or perhaps deploy the service or website to a specific environment.

Backstage adopters frequently first utilize software templates as the first developer portal feature. This is because software templates are very powerful by themselves, and Backstage does not need to have the whole engineering team onboarded to make them useful.

On the other hand, the software and services catalog only becomes useful after onboarding the existing services, websites, libraries, and other components, for which onboarding takes a longer time.

Tech Docs

Documentation is written using Markdown and is stored in the repository with the code. Backstage then parses these .md documents and makes them searchable across the system. Docs written alongside the code are automatically rendered in the “Docs” tab for any component or service. Neat!

Backstage adopters have told me that TechDocs require some additional tweaking to enable when you are running Backstage for yourself. 

The idea of having documentation within Backstage sounds good, in theory. However, in practice, most companies already have a place to store documentation, and it’s not in Backstage. So to avoid duplicate documentation t, those companies would need to commit to use Backstage as documentation. If they don’t, then they have documentation living in two different places!

TechDocs, because of this duplication and the additional setup required, are adopted less than software templates and the software and services catalog, based on my conversations with Backstage adopters.

Other plugins

Backstage is a modular developer portal, and plugins are the backbone of it. The framework comes with Core plugins built by Spotify, and a host of additional plugins for developer tools. Here’s a screenshot of some of the available plugins:

A plugin I cannot not mention is the TechRadar plugin. In the issue Consolidating Technologies, we already touched on a ‘tech radar’ process, and as I wrote:

Staff software engineer Frank de Jonge shared the process he implemented at FinTech startup Mollie, a European competitor to Stripe, valued at $6.5B at its last fundraising round:

“We put a process in place based on ‘tech radar.’ We started by listing the core technologies that teams used. The process described key steps and contacts for moving a technology through the stages of assessment → trial → general adoption. For each step, we described what they need to provide and who to involve.” 

Here is how the Tech Radar plugin looks like, within Backstage:

In a nice pollination of ideas across tech, ThoughtWorks created the concept of the tech radar, Zalando created a neat visualization they use, and the Spotify team created the Tech Radar plugin.

How does Backstage work, under the hood?

Under the hood, Backstage is a collection of entities and plugins. Each entity is described by a YAML configuration file. For example, this is how a service entity could be described:

The above entity describes a service and lists several widgets, including a Sentry one, a TechDocs one, PagerDuty, Snyk, and Lighthouse. It lists the owner of the service, its lifecyle, and the APIs it provides.

In the YAML, dependencies can also be listed, helping build a dependency graph.

And… that’s about it! Backstage is written mostly in TypeScript, and it’s open source. The power of Backstage lies in the plugins.

4. Why was Backstage open sourced?

Backstage is a powerful developer portal that has served Spotify very well. Even today, Spotify’s internal Backstage experience is years ahead of the open-source version.

That made me wonder: why did Spotify open source Backstage? Is having a great developer portal not a competitive advantage for the company, allowing them to move faster? Playing devil’s advocate: why share this ‘advantage’ with the world to use?

Martina Iglesias Fernández explained that there were two main reasons: hiring and attracting talent, and not wanting to bet on an internal-only technology that would became outdated, and force Spotify to do a painful migration, later on:

“Before Kubernetes came to be, Spotify had developed a similar container solution internally and used it with many of its services. Once it became apparent that Kubernetes would dominate the market, Spotify had to take the expensive decision of deprecating their solution and migrating all their services to Kubernetes.

After investing significant resources in building out Backstage, Spotify did not want to repeat the same mistake of releasing their technology too late and having to redo their internal developer portal.”

Gustav Söderström, Chief Research and Development Officer at Spotify, noted 4 more reasons in the podcast Spotify: A Product Story:

“1. Although many companies can benefit from Backstage, it isn't going to make us or our competitors much better at being an audio company.

2. Releasing Backstage would show the tech world the kind of innovative products [Spotify] is building behind the scenes. And new hires could already be familiar with the workflow, making onboarding even quicker.

3. Making Backstage's source code freely available, we let anyone add their own plugins to the marketplace, plugins that everyone, including us, could use. We were effectively opening up our engineering team to contributions from the entire world.

4. If Backstage were as useful to everyone else as we hoped it would be, it could potentially be adopted industry wide, which would mean we wouldn't need to switch it out later on, when someone else inevitably set a new standard for solving this problem, avoiding the type of very expensive migration that we face when we move to the cloud.”

I found it interesting for Gustav to reflect on how a developer portal was not the core competency of Spotify. Spotify is an audio company, first and foremost, and it competes with other audio companies. It would not make sense for them to open up any tools that could help their direct competitors gain ground on them. However, Backstage does not fall in this category.

Open sourcing as a way to reduce long-term maintenance costs is an interesting approach, and I’d argue that perhaps not as many companies consider it. Spotify is still a heavy investor in Backstage (many of the contributors working at Spotify), but they are no longer the only ones, thus dividing the cost.

The part on Kubernetes becoming the market leader and Spotify deciding to move over, leaving their in-house technology behind, also raises an interesting thought:

Could it be that open-source developer tools are more likely to become market leaders? The Developer Portal space has plenty of alternatives for companies building their service catalogs similar to Backstage. However, none of them are open source or owned by a neural entity like the Cloud Native Computing Foundation.

I’d wager that the CNCF ownership will greatly help the adoption of projects like Backstage, the same way it has for the likes of Kubernetes (container management), Prometheus (monitoring), or Vitess (database).

5. Spotify’s version of Backstage

The open-sourced version of Backstage has several differences to what Spotify uses:

• Spotify’s internal Backstage is deeply integrated with the company’s services.

• Spotify uses currently not yet open-sourced Backstage features like creating Data Pipelines.

• All tools and ad-hoc infrastructure are already integrated into Spotify’s Backstage portal, making it truly a central place for engineers.

In short, Spotify has a Backstage version that is a true “one-stop shop” developer portal, thanks to additional features that are either tightly coupled to Spotify, or not yet open sourced. OSS adopters could, in theory, build a similar experience. However, in practice, adopters do not tend to do this.

Spotify uses Backstage for:

• Squad ownership and metrics: each squad has their ownership visualized and browsable.

• Service catalog: this lists all services and makes it easy to create new ones.

• Website catalog: Spotify has more than 300 websites (think: lots of microsites), and these can be discovered in one place.

• Workflows: creating, running, and managing them

• Data endpoints: discovering and accessing 4,000+ data endpoints across the company

• App features: the ability to turn feature flags on and off

• Dashboards: to define custom dashboards for squads to use

• GCP projects: to view and manage these

• Storage: to manage and request storage at a team-level

• Manage and visualize upgrades and migrations

• Communicate best practices 

The Spotify team shared how they use Backstage in this video. Below are a few visualizations of some of the more interesting approaches they use:

6. Getting started with Backstage

A common misconception with Backstage is that the product can be used as is. This is not exactly true. To use Backstage, you need to pick and choose the features you want to use and then configure them. You might also find it helpful to build some custom plugins that integrate with your internal systems.

There is no “official” Backstage support, as Backstage is open source. However, SaaS providers like Roadie have stepped up, and their business model is to provide support to get started with Backstage, including for setup, hosting, upgrades, and Platform management.

Some consulting companies also help with setting up Backstage,including Frontside, RedHat, and ThoughtWorks. 

Adopting Backstage at RD Station

RD Station claims to be the largest SaaS company in Latin America, with more than 40,000 customers. The company employs about 1,000 people, and about 130 of them are software engineers. I talked with tech lead Rogerio Angeliski and asked why they looked for a solution like Backstage. Rogerio told me:

“Our first use case for Backstage was the service catalog. By the time we grew to 100 software engineers, we had the ‘Google Sheets disease’ for keeping track of our services and domains. Although we are not using a full microservice architecture, we are close enough, meaning lots of services, and lots of APIs.

Another one of our pain points was how it took 2–3 months to bootstrap a new service and get it to production. We thought Backstage’s Software Templates function could help with this.”

Here is a screenshot of RD Station’s Backstage setup, showing the ‘“devtools’” team:

So how did they set up Backstage? Did they use a vendor or go at it themselves? From Rogerio:

“We decided to self-host because we had a hard requirement to use a Rails scaffolder module, which I created.

Early on, before the 1.0 release, upgrading to a new Backstage version was very painful. However, since the 1.0 release, upgrades are very smooth, and the upgrade-helper tool helps a lot as well.

Right now, we are using the Service Catalog and Software Templates. In the future, we want to enable the TechDocs and Searching functions. However, right now we don’t feel that we need these just yet.”

What plugins does RD Station use, or build? Rogerio told me they use:

• GitHub plugins

• The Rollbar plugin for error logging and tracking

• The Tech Insights plugin for production review 

• Custom plugins the team built for Slack integration

And as for the future? Here’s a design of where the team would like to get with Backstage, after building more plugins to make this plan a reality:

Building plugins

Building plugins is a key part to creating the kind of Backstage experience that a team or company envisions. Most Backstage adopters I talked with have either built custom plugins for their needs, or are planning to do more of them, like RD Station expects to do so.

Creating a plugin is pretty straightforward: run a backstage-cli command to scaffold a new plugin, and write the business logic of the plugin using TypeScript. The Backstage-cli generates a scaffolded plugin that has a helpful example component, as well as unit tests.

To take a look at the source code of some of these production plugins:

• GitHub actions plugin

• Rootly plugin: a plugin displaying Rootly services, incidents in Backstage

• List of all plugins in the Backstage directory

I talked with a software engineer who is building Backstage plugins at Netflix who said how building a plugin and building a web app are not too different:

“A Backstage plugin is just another react app from my perspective. While I see the value in Backstage, the plugins we can build are only as good as our internal systems and the data those systems expose. So investing in Backstage will help with visibility, but it will not make any internal systems better, magically.”

How Roadie sets up a new Backstage instance

I asked the Roadie team how they typically go about onboarding a new company onto Backstage. Here’s what they shared about their process. This can be a useful outline of steps even if you are an engineer who is planning to make a pitch to your team or company, even if you don’t plan to use the support of a vendor. Onboarding to something like Backstage is a lot of time and effort. It’s worth estimating the effort it will take, and how long it can go. What Roadie does:

1. Proof-of-concept pitch. Working with the in-house Platform/Productivity/DevOps team, Roadie puts a pitch together on what they would like the portal to look like once it is done. This usually takes a few days. For teams that will host Backstage internally, the pitch could take a few weeks, as the developer teams figure out how to set up Backstage on their infra.

2. Approval of the proof-of-concept. The stakeholder, usually someone in engineering leadership, approves this first version to go ahead.

3. Integrations and plugins. The in-house Platform/Productivity/DevOps team sets up the integrations and plugins in Backstage, which they need for their features. Self-hosted Backstage adopters install dependencies using the yarn package manager, and they configure the integrations by hand. When using a hosted solution like Roadie, those steps are abstracted away with a setup UI.

4. Early adopter teams. Once the portal is set up, the in-house Platform/Productivity/DevOps team usually onboards a few early adopter teams to use Backstage and provides feedback. They iterate, based on what they hear.

5. Announcement to the whole organization. When the early adopter teams are successfully onboarded, the Platform/Productivity/DevOps team usually announces the availability of Backstage and asks other teams to onboard.

6. Improvement of the Developer Portal! As teams are onboarded, the Platform/Productivity/DevOps team usually looks to add more tools into the portal to make the lives of engineers easier. This usually means building integrations or plugins. Self-hosted customers also need to keep Backstage up-to-date, but if it runs on a vendor like Roadie, the vendor takes care of this.

Full onboarding typically takes at least 3 months, the Roadie team told me. By full onboarding, Roadie means that the entire organization - usually at or above 100 engineers - is ready to use the portal. A lot depends on what features organizations onboard. It’s common for companies onboard to use the service catalog first, but not the Scaffolder at first, or the other way around.

Other Ways to Get Started

Here are additional resources to get started with Backstage:

• The official Backstage site. Documentation and getting started.

• Introduction to Backstage course by the Linux Foundation.

• Spotify Learn: Backstage. A step-by-step guide created by Spotify.

And a few interesting talks about Backstage:

• Leading Architectural Change with Backstage. Andy Hoffman, DevOps manager at Cribou giving an overview on how Backstage can address most challenges that come with adopting Kubernetes.

• Introduction to Backstage (OSS) version. An overview by Roadie.

• Introduction to Backstage (at Spotify). An overview on how Spotify uses Backstage, internally.

• Backstage Demos: A collection of tutorials by Spotify (2020–2022).

• Keynote: 10x Effectiveness of Dev Teams with Backstage: Roadie Tuite David explains his journey with Backstage from 2020.

7. Backstage alternatives

If you’d like to set up a developer portal, outside of taking an open-source solution like Backstage, you can use a vendor that specializes in providing SaaS solutions outside of the box. 

Here are some better-known ones:

• Cortex: Founded in 2019, the company offers a ready-made Developer Portal with a catalog that has capabilities similar to Backstage. The Platform uses a Scaffolder built on top of Cookiecutter (a Python library). This approach can make it trickier to express some Cloud-native workflow operations. Cortex also does not allow custom integrations into the developer portal, while Backstage does. Cortex offers Scorecards to enforce best practices across teams and a Query Builder to query data sources.
  
  The company raised $15M Series A funding in November 2021. The company has several well-known customers like Adobe, Confluent, Opendoor, and others.

• OpsLevel: Founded in 2018, the company offers a ready-made developer portal with a catalog. Similar to the Scorecards feature at Cortex, it offers Service Maturity checks to ensure compliance within services. It supports repositories and services, but it is not as flexible in expanding to any type of entity, e.g., metrics, migrations, storage, GCP profiles, like Backstage does. OpsLevel also uses a Scaffolder based on Cookiecutter.
  
  The company also raised $15M Series A funding in March 2022. The company has several well-known customers like Auth0, Fastly, and Under Armour. 

• Port: Founded in 2022, Port offers a no-code developer portal and similar capabilities to Backstage. Their Scaffolder is a facade to connect your own endpoints, unlike Backstage that supports custom Scaffolder actions. Port is the most recent player in this field.
  
  The company raised $7M seed funding in November 2022. The company lists a handful of scaleups like Run.ai (Series C), Rookout (Series B), and Neosec (Series A) as its customers. The company offers a neat self-service live demo of their offering, without needing to register. It’s a refreshing approach, as almost all other providers require you to schedule a sales demo.

Opting for a SaaS over Backstage has its advantages. A definite advantage when opting for a SaaS is that the onboarding and setup tends to be much easier, the tools are often more opinionated, and the user experience can be more polished than in the case of Backstage. Because these vendors need to make a good case for their pricing, they are frequently ahead in areas like compliance offerings to “enforce” certain practices.

The biggest advantage of choosing Backstage is minimizing vendor lock-in, as the “vendor” is Backstage, an open-source, community-owned Platform. Even if you choose a SaaS vendor to manage hosting, you have the option to move to self-hosting and modify the code yourself, or even fork the project if you need to.

The biggest downside of going with the vendor tends to be reflections on the upsides of Backstage. While the ease of setup and the user experience could be better, you’re more likely to be locked in with a vendor.

Even without detailing the alternatives in this developer platform space, it’s clear that this area is attracting a lot of investment: there’s a growing demand from companies to buy or build solutions for the problem of too many kinds of internal developer tools.

Other examples of open-source, Backstage-like developer tools

Although not direct competitors, a few other open-source developer tools are worth a mention, which overlap with Backstage functionality.

Clutch is a solution originally developed by Lyft, designed to improve developers’ experiences and operational capabilities. Clutch helps with things like:

• Managing cloud-native infrastructure and preventing accidents during rollouts by programmatically catching mistakes and enforcing configurations.

• Complementing infrastructure-as-a-code, creating a home for live, runtime changes and guided workflows.

• Securing by design: supports SSO, role-based access control (RBAC), built-in auditing with sinks for Slack, extensive logging and reporting.

Clutch integrates especially well with open-source Edge and service proxy Envoy, also built by Lyft. When used with Envoy, Clutch allows for using the Chaos experimentation framework out of the box.

How does Clutch compare to Backstage? The Clutch team describes the differences as this:

• The company views Backstage to be more tailored toward frontend engineers, and it sees backend support in early stages.

• Clutch positions itself as offering a suite of tools for developing maintainable backend and frontend workflows, which it claims support richer patterns.

My take is that Clutch feels much more of a Backstage competitor as a “developer portal for engineering tools” - but one that has no ambition to take on areas like service catalogs. Clutch is focused on defining and operating workflows, and it seems to excel at this, especially for organizations utilizing Envoy. Clutch doesn’t have broad ambitions to become the go-to portal for things outside of workflows, though.

Hygieia by Capital One is a tool to make CI/CD pipelines more intuitive, more accessible, and more efficient. The tool offers in-depth visualizations for these pipelines. These dashboards can be built for different audiences: for example, for an engineering team visualizing things they care about or higher-level dashboards for executives:

Takeaways

Developer portals don’t replace your existing tools: they pull together information and help you find the right tools more easily.

Referring to Backstage as “one thing” can be confusing. Patrick Valenzuela is an engineering manager, and an adopter of Backstage. He shares his observation on how it’s easier to talk about parts of Backstage, rather than the whole thing:

“I’ve found that referring to Backstage as a single thing is not as helpful. Backstage is easier understood when broken down into:

1. The developer portal. Software templates are one example of a developer portal, but you can build custom plugins that become very powerful. For example, we plan to build and consolidate more tools in the Backstage UI. An example is a custom plugin that shows global deploy history, where engineers could have the option to revert deployments.

2. The catalog of components. This is the list of services, websites, libraries, APIs, and so on.”

Backstage does help organize things, but by itself, it won’t improve developer productivity. The real power of Backstage comes when the existing developer tools make the work of engineers much easier already. Then, Backstage can surface these features.

Backstage does bring a somewhat opinionated way to think about ownership of everything, which I find both refreshing and much-needed. However, to make full use of Backstage, you’ll most likely need to build custom plugins to expose data, actions, and visualizations that engineers need to use, day to day.

Backstage being owned by the CNCF makes it a safe bet to consider adopting. Once an engineering organization passes a few hundred engineers, having something akin to a developer portal becomes essential.

Several vendors on the market offer these solutions. The big differentiator with Backstage is that it is not a vendor, and not even an open-source project controlled by a vendor. It’s owned by the Cloud Native Computing Foundation, which is as good of a guarantee on neutrality as it gets in the open-source world.

I talked with an engineer at a Big Tech who said thattheir company is in the process of onboarding to Backstage. This engineer said that their company would be wary of adopting a platform for something so crucial as developer tools, if it was built by a recently founded developer tools startup. This is because it’s unclear if that company will still be around in a few years: but the developer portal needs to be.

Backstage, by being in the incubating phase of the CNCF, looks like it’s here to stay for a long time, and it is a non-risky choice to adopt. The project is very active in being developed: just in the past 30 days, 114 authors published 893 commits to the main branch.

Backstage solves a problem that companies with ~100 engineers or services start to have. Talking with companies that have adopted Backstage, I did not find any organization onboarding before they hit the roughly 100-engineer mark. This is because, until you have small-enough teams, and not too many services, Backstage only solves the problems that are not your biggest.

Backstage addresses these issues:

• Service discoverability: making it easy to find services in organizations where there are too many of these.

• Scaffolding of services/websites/components: helping standardize how new services/sites/components are created.

So while Backstage has been helpful for larger organizations, it is unlikely to be a sensible choice for smaller companies or in organizations where service discoverability and scaffolding are not real problems.

Backstage requires significant investment to deliver its full value and benefits. Every Backstage adopter engineer and manager I talked with shared that you need to put in a lot of work to get proper value out of Backstage.

Backstage sounds amazing when people first hear about it, and the Spotify demos sometimes feel too good to be true. However, Backstage’s OSS version is a far cry from the capabilities that Spotify has built: because Spotify put several more engineering years of investment into building out their own plugins.

Also, while Backstage defines an option and structure for ownership, it is still down to the organization to implement this ownership. Will you use automation and LDAP groups? Will you put a Technical Program Manager-led program in place to assign ownership to services and maintain this? Backstage itself can help highlight and solve for ownership gaps, but it won’t solve this for you.

I hope this overview was useful. To stay in the know for what’s going on with Backstage, subscribe either to the Backstage Monthly newsletter by Spotify or the Backstage Weekly newsletter by Roadie, or join Backstage’s Discord channel.

Thanks to Patrick Valenzuela and Rogerio Angeliski for their input and review of this article.

Special thanks to the Roadie team, David Tuite, Jorge Lainfiesta, Martina Iglesias Fernández, for helping with input and reviews for this article. Roadie was the second most active corporate contributor to Backstage after Spotify in 2022, and the company’s business is about helping organizations adopt and operate Backstage.

This post is only for paying subscribers of The Pragmatic Engineer. This email is intended for a single recipient, but occasional forwarding is totally fine.

Like & Comment

As a subscriber, you also have access to 🔒 subscriber-only resources for engineering managers and engineers.

Know someone who would benefit from the newsletter? Consider gifting a subscription.

Give a gift subscription

Like

Comment

Share

© 2023 Gergely Orosz
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe