Join us
@dexter0195 ă» May 29,2023 ă» 6 min read ă» 480 views
Discover in this story how to boost the productivity of your engineers with an Internal Developer Platform and what are the key features to make it successful.
Discover in this story how to boost the productivity of your engineers with an Internal Developer Platform and what are the key features to make it successful.
Automation, a sad storyâŠ
Once upon a time đŠ, when your company was small enough so that a handful of people was building and running all the software and the infrastructure, the operations were very smooth and it was very easy to ship something to production, your teammate (most probably the cofounder) would sit right next to you and deploy your application with you, or they would even give you full access to do it yourself (security was not yet a concern at that time).
Then, as the company grows, more and more people are starting to come in and bring value by building more performant but also more complex systems. Now, in order to deploy your application you need a skilled team of engineers who knows how the whole system works and that know how to keep it running and in good shape, a big applause to the infra team!
But the company keeps growing and growing and your infra team needs to find an ally in order to keep up with the always-rising number of requests. Here is where Automation starts to kick in.
Lots of new tools are introduced to ease the day-to-day tasks of the infra team, and they can now deploy things with a couple of commands using Infrastructure as Code.
Sure, automation is already a good start, but we wouldnât unlock its full potential if we still need to rely on someone to know how and where to kick in that automation. Let me explain why.
Most of the time infrastructure automation is done thanks to CD pipelines, which given an input in a specific format from a GitHub Pull Request will perform some actions in order to deploy things with the right configurations. The issue with pipelines is that you need to know âwhereâ and âhowâ to trigger them, which in an always-growing environment can become challenging. Youâve probably seen people organizing meetings to do pull requests together with the DevOps/SRE, or frustrated developers who canât get approval for their PR as itâs not in the right format or syntax is wrong.
All this has to end!
It should be easier for people to perform those operations in full autonomy and they shouldnât spend their valuable time searching and learning the specific YAML format needed to deploy things.
This is where Internal Developer Portals come to rescue us.
What is an Internal Developer Portal (IDP)?
A Developer Portal is a go-to place for developers when they need to learn about their ecosystem and contribute to it. Developer Portals exist to balance Developer Autonomy and Discoverability: it allows teams to keep working independently, choosing their preferred tools, and deploying frequently, but expose their work so everyone can explore and benefit from their microservices, APIs, libraries, websites, pipelines, and algorithms.
What are the main features to look into when choosing an IDP solution?
There are many alternatives of IDPs on the market: open-source, paid, and free. My objective here is not to give you a benchmark on which is the best but to give you the tools to choose the most successful for your organization according to what I believe are the most important features.
A successful IDP needs to serve three purposes:
The catalog is a central repository that contains all of the APIs, SDKs, and other resources that are available to developers within an organization. The catalog typically provides developers with a comprehensive overview of the available resources, including documentation, sample code, and other relevant information. It may also include features such as search functionality, filtering, and sorting to help developers quickly find the resources they need.
What is important to look at in this component is how easily you can get meaningful data into the catalog. This includes both how to better represent the entities and their relations and from which sources you can easily pull information from.
There are many alternatives in the market but we can summarize them in two types:
The first relies on making direct calls to the components ( cloud providers, Kubernetes API, ⊠) or to some GitOps repo in order to pull the data, while the latter leverage API to push new data from various sources.
My 2 cents here, Push (API-based) approach is more flexible, allowing you to push data from CI pipelines or custom exporters that you can easily write in your preferred language.
Scaffolder is a tool that helps developers to quickly create new applications or projects by generating a starting point for their code. This can include a basic application structure, pre-configured settings and sample code that developers can build upon. Scaffolder may also provide guidance and best practices for developers to follow, helping to ensure that applications are built in a consistent and scalable manner.
Most of the IDPs allow you to make use of GitOps patterns in order to provision a new repository or a new resource in an already existing environment thanks to underlying IaC. Most of them make use of Cookiecutter or a similar approach.
This is a capability that not all the IDP offers, but that I think is fundamental to really empower developers to run their own infrastructure without granting them access to production directly.
Day-2 Operations can go from âRestart a Serviceâ to âDeploy a temporary test environmentâ or to âCreate a new DBâ.
The important thing to look at in this feature is how easily you can hook your preferred automation to the actions in the IDP. For example, you might want to leverage GH action to perform the automation action, Terraform, or whatever tool is in your day-to-day operations.
Once your ecosystem of services starts to flourish it will be difficult to keep track of what is the production readiness or the security posture of a service. Here is where the Catalog, thanks to the data it gathers can give you visibility over these important KPIs.
Most of the IDPs allow you to either pull data from external sources or use the information on the Catalog and evaluate them as part of what they call âScorecardsâ. Scorecards allow you to define what are the different levels of maturity you expect from a service by linking it to levels like âBronzeâ, âSilverâ and âGoldâ. In this way, it becomes very easy to focus your effort on getting everyone on the same Golden Path.
There are many choices to make around automation and Internal Developer Portal. Most of the time youâll be able to find and follow the patterns and best practices of people that already took the path leading to this amazing journey. But many times youâll be also wandering in uncharted territory and on top of that youâll need to be able to make this new concept, work well with the preexisting infrastructure automation and processes of your organization.
So I wish you good luck on this journey, I hope you find this article insightful and please feel free to share your experiences in the comments section.
Ah and make sure to follow me on Medium for more articles!đ ->
Credits:
Join other developers and claim your FAUN account now!
Site Reliability Engineer, Shift Technology
@dexter0195Influence
Total Hits
Posts
Only registered users can post comments. Please, login or signup.