In 2025, the gap between high-performing engineering teams and the rest is widening. The secret? A streamlined developer experience. But how do you eliminate friction, reduce cognitive load, and empower developers to ship code faster? The answer is an Internal Developer Platform (IDP). This guide will walk you through building your very first IDP from the ground up, using two industry-standard, open-source powerhouses: Spotify's Backstage for the developer portal and Kubernetes for the underlying orchestration. We'll move from core concepts to a practical, hands-on implementation, giving you the foundation to revolutionize your team's workflow. By the end of this article, you will have a functional IDP prototype and a clear roadmap for scaling it to meet your organization's unique needs, putting you at the forefront of the platform engineering movement.
Part 1: The 'Why' - Core Concepts of Modern Platform Engineering
What is an Internal Developer Platform (IDP)?
An IDP is more than a collection of CI/CD pipelines and scripts; it's a curated, paved 'golden path' for software development within your organization. It codifies best practices for security, observability, and deployment into a self-service platform. The primary goals are to foster developer self-service, allowing engineers to provision infrastructure, create services, and access tools without filing tickets or waiting for an operations team. This standardization accelerates developer onboarding, as new hires can become productive within hours, not weeks. By abstracting away the underlying complexity of cloud-native infrastructure, an IDP drastically reduces cognitive load. Developers can focus on writing business logic instead of wrestling with YAML configurations, IAM roles, and network policies, minimizing the costly effects of context switching.
Why Backstage is the Heart of Your Developer Portal
Born out of Spotify's need to manage microservices at scale, Backstage is an open-source framework for building developer portals. It has become the de-facto standard for the IDP user interface layer, providing a unified experience on top of your existing tools. Its power lies in three core, extensible features:
- The Software Catalog: This is the single pane of glass for all your software assets—microservices, libraries, websites, and even ML models. By ingesting metadata from
catalog-info.yamlfiles in your repositories, it creates a searchable, filterable inventory that tracks ownership, dependencies, and key APIs. It answers the question, 'Who owns this service and where can I find its documentation?' - Software Templates (The Scaffolder): This feature allows you to codify service creation. Instead of cloning a repo and performing manual setup, developers use a simple UI to scaffold a new, production-ready service in minutes. Templates can create a repository, set up CI/CD pipelines, provision a database, and register the new component in the Software Catalog automatically.
- TechDocs (Documentation-as-Code): TechDocs solves the problem of stale documentation by treating it like code. It renders Markdown files stored alongside your source code within the Backstage UI. This ensures documentation is versioned, reviewed in pull requests, and always accessible right next to the component it describes.
Why Kubernetes is the Engine for Your Platform
Kubernetes has won the container orchestration war, providing a robust, API-driven foundation for modern applications. For an IDP, it serves as the universal abstraction layer for infrastructure. Its declarative nature is the key to this synergy. You define the desired state of your application in a manifest file, and Kubernetes' control plane works relentlessly to make that state a reality. This model allows a platform team to offer complex capabilities (e.g., 'deploy this service with high availability across three zones') through a simple, standardized API. For the IDP, Kubernetes is the reliable engine that takes the output of a Backstage template—a new containerized service—and handles the complex tasks of scheduling, scaling, networking, and self-healing, providing a consistent deployment target regardless of the underlying cloud provider.
Part 2: The 'How' - Prerequisites and Environment Setup
Your Toolkit: What You'll Need
Before we begin, ensure you have the following tools installed and configured on your local machine:
- Software: Docker Desktop (or another OCI-compliant container runtime),
kubectl(the Kubernetes command-line tool),helm(the Kubernetes package manager), and Node.js v18+ (we'll usenpx, which is included). - Infrastructure: A running Kubernetes cluster. For this guide, we recommend Minikube for a simple, local setup. Alternatives include Kind, K3s, or a free-tier cluster from a cloud provider like GKE, EKS, or AKS.
- Accounts: A GitHub account (or GitLab/Bitbucket). We will use this to store our component source code and templates.
Step 1: Preparing Your Kubernetes Cluster
If you're using Minikube, getting a cluster running is a one-line command. This will create a single-node Kubernetes cluster inside a local virtual machine or container.
minikube start --driver=dockerOnce the command completes, verify that your kubectl context is correctly pointing to the new cluster and that the node is ready.
kubectl get nodes# Expected Output:# NAME STATUS ROLES AGE VERSION# minikube Ready control-plane 2m v1.28.3Step 2: Installing a Standalone Backstage App
Backstage provides a command-line utility to bootstrap a new standalone application, which includes a pre-configured frontend and backend. Run the following command in your terminal:
npx @backstage/create-app@latestFollow the prompts, giving your app a name (e.g., my-idp). Once it's finished, navigate into the new directory (cd my-idp). You'll see a monorepo structure, with the most important directories being packages/app (the frontend) and packages/backend. The root app-config.yaml file is where you'll do most of your configuration. Start the application by running:
yarn devThis will start the frontend (typically on port 3000) and backend (port 7007) concurrently. Open http://localhost:3000 in your browser to see your new, empty Backstage portal.
Part 3: Building Your Core IDP Features
Feature 1: Integrating the Kubernetes Plugin to View Workloads
Let's connect Backstage to our cluster to provide visibility into running services. First, install the frontend and backend plugins:
yarn --cwd packages/app add @backstage/plugin-kubernetes
yarn --cwd packages/backend add @backstage/plugin-kubernetes-backendNext, configure the connection in app-config.yaml. We'll add a section that tells Backstage how to find and authenticate with our Minikube cluster.
# app-config.yaml
kubernetes:
serviceLocatorMethod:
type: 'multiTenant'
clusterLocatorMethods:
- type: 'config'
clusters:
- name: minikube
url: https://192.168.49.2:8443 # Replace with your minikube IP: `minikube ip`
authProvider: 'serviceAccount'
skipTLSVerify: true # For local development only
serviceAccountToken: "..." # We will get this in the next stepTo get a service account token, you need to create a ServiceAccount and a ClusterRoleBinding in Kubernetes with view-only permissions. This is a crucial security step. Then, extract the token and place it in your config. For brevity, we'll skip the detailed RBAC setup, but in production, you must use a token with least-privilege access. Finally, to link a component in the catalog to its resources in Kubernetes, add an annotation to its catalog-info.yaml file:
# catalog-info.yaml
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: my-example-service
annotations:
backstage.io/kubernetes-id: my-example-serviceAfter restarting Backstage, the component's page will now have a 'Kubernetes' tab showing its associated Deployments, Pods, and Services.
Feature 2: Creating a 'Hello World' Microservice with Software Templates
Now, let's empower developers to create new services. We'll build a simple template for a Node.js 'Hello World' application. The heart of a template is the template.yaml file, which defines the UI form and the steps to execute.
# template/template.yaml
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
name: nodejs-hello-world
title: Node.js Hello World
description: Creates a simple Node.js service with a Dockerfile and Kubernetes manifest.
spec:
owner: platform-team
type: service
parameters:
- title: Fill in some steps
required:
- component_id
- owner
properties:
component_id:
title: Name
type: string
description: Unique name of the component
owner:
title: Owner
type: string
description: Owner of the component
steps:
- id: fetch-base
name: Fetch Base Template
action: fetch:template
input:
url: ./content
values:
component_id: "{{ parameters.component_id }}"
owner: "{{ parameters.owner }}"
- id: publish
name: Publish
action: publish:github
input:
repoUrl: github.com?owner=my-org&repo={{ parameters.component_id }}
- id: register
name: Register
action: catalog:register
input:
repoContentsUrl: "{{ steps.publish.output.repoContentsUrl }}"
catalogInfoPath: '/catalog-info.yaml'In a sub-directory named content, you would place your template files, such as a Dockerfile, a simple index.js, and a Kubernetes deployment.yaml. These files can use templating syntax like {{ values.component_id }} which will be replaced by the user's input from the form. To make this template available in Backstage, add its location to app-config.yaml under catalog.locations. Once registered, you'll find 'Node.js Hello World' in the 'Create...' section of Backstage. Running it will guide a user through the form, create a new GitHub repository with the templated files, and register the new service in the Software Catalog, all in one automated workflow.
Feature 3: Automating Documentation with TechDocs
Live documentation is the final piece of our core IDP. TechDocs requires a 'builder' to generate static HTML from Markdown and a storage location to serve it from. For local development, we can use Docker for the builder and the local filesystem for storage. Configure this in app-config.yaml:
# app-config.yaml
techdocs:
builder: 'local'
generator:
runIn: 'docker'
publisher:
type: 'local'With the backend configured, you just need to tell Backstage where to find the documentation for a given component. This is done with another annotation in the component's catalog-info.yaml file.
# In the repository for my-example-service
# catalog-info.yaml
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: my-example-service
annotations:
backstage.io/techdocs-ref: dir:.This annotation tells TechDocs to look for documentation in the same repository, starting at the root (dir:.). Now, create a directory named docs in your component's repository and add an index.md file. In the same repository, create an mkdocs.yml file to configure the documentation site. After pushing these changes, the 'Docs' tab in Backstage for that component will render your Markdown files into a beautiful, searchable documentation site, ensuring your docs live and evolve with your code.
Part 4: Scaling and Next Steps for Your 2025 IDP
Treating Your Platform as a Product
The most successful IDPs are treated not as internal projects, but as products with internal developers as their customers. This requires a product-management mindset. Your top priority should be gathering feedback through surveys, interviews, and analytics to understand your developers' biggest pain points. Use this data to build a public roadmap for your platform, prioritizing features that deliver the most value, such as integrating CI/CD visibility to show build statuses or adding cost-tracking plugins. Crucially, this work cannot be a side-of-the-desk effort. A dedicated, empowered platform team is essential for the long-term success, maintenance, and evolution of the IDP.
Essential Security Considerations
As your IDP becomes more powerful, security becomes paramount. Here are three key areas to focus on:
- Authentication and Authorization: Move beyond anonymous access. Integrate an identity provider like Okta, Azure AD, or GitHub OAuth into your Backstage frontend. Use the
permission-frameworkbackend plugin to define fine-grained authorization policies (e.g., only members of the 'SRE' group can create new database templates). - Secrets Management: Never hard-code secrets like API keys or database credentials in
app-config.yamlor template files. The Backstage backend should load secrets from environment variables or a dedicated secrets manager like HashiCorp Vault or AWS Secrets Manager. For templates, integrate with these tools to securely inject secrets during the scaffolding process. - Kubernetes RBAC: The service account used by the Kubernetes plugin must adhere to the principle of least privilege. Create a dedicated
ClusterRolewith only theget,list, andwatchpermissions required for the plugin to function. Avoid using a cluster-admin token at all costs, especially in production environments.
Future-Proofing: What's Next in Platform Engineering?
The platform engineering space is evolving rapidly. To keep your IDP at the cutting edge in 2025 and beyond, consider these forward-looking integrations:
- AI-Assisted Workflows: Explore how AI can enhance the developer experience within your IDP. This could range from a chatbot that helps developers find the right documentation to an AI assistant that suggests optimizations for a Kubernetes manifest generated by a template.
- Integrated FinOps and Cost Management: Give developers direct visibility into the cost of the infrastructure they consume. Integrate plugins like Backstage's Cost Insights to display cloud spend per team or service, fostering a culture of cost accountability.
- Deeper Observability and Security Scanning: Go beyond just listing Kubernetes objects. Integrate with your observability stack (e.g., Prometheus, Grafana, Dynatrace) to display real-time service health metrics. Likewise, integrate with security scanners (e.g., Snyk, Trivy) to show vulnerability reports directly on a component's page in the catalog, shifting security left.
Conclusion
You've successfully laid the groundwork for a modern Internal Developer Platform. By combining Backstage's user-centric portal with the declarative power of Kubernetes, you've built a system that can create, track, and manage software components—the first step in building 'golden paths' for your developers. Remember, an IDP is a journey, not a destination. You've built a powerful foundation. The real value comes from iteratively adding features that solve your team's most significant pain points, directly boosting productivity and developer happiness. What will you integrate next? Share your progress in the comments below, and start exploring the vast ecosystem of Backstage plugins to take your platform to the next level.