exanubes
Q&A

Pulumi Overview #1 Pulumi: Managing Cloud Infrastructure as Code

This article is part of a series

  1. Pulumi: Managing Cloud Infrastructure as Code
  2. Pulumi: Configuration and secrets
  3. Pulumi: Creating a custom component resource

Pulumi is an open source tool for managing cloud infrastructure as code. It supports multi-region, multi-account and even multi-cloud deployments. Unlike AWS CDK, Pulumi does not compile the infrastructure code to a CloudFormation template. Instead, it uses Terraform and the cloud provider’s native SDK to provision and manage cloud resources. It uses the desired state model for deciding what changes need to be made by comparing the current and desired states.

TOC

Installation

To start using Pulumi, you will need their CLI tool which has the Pulumi engine embedded in it. You can install it using homebrew

brew install pulumi

Pulumi Cloud

Pulumi Cloud is their SaaS offering which acts as the backend for the Pulumi CLI. It stores the state of the infrastructure and provides a way to share the state with other team members. You can sign up for a free account at Pulumi Cloud . Alternatively, you can also create your own custom backend or self-host Pulumi Cloud , however, that’s an Enterprise level solution at the time of writing.

Community

I’d say Pulumi’s community is still relatively small but then again, the IaC space is also quite small and specialized. It did, however, manage to amass nearly 20k stars on GitHub which suggests it is quite popular and an appreciated tool. The trend of Software Developers being responsible for infrastructure is picking up pace, similar to how DBA role was absorbed by developers/programmers, especially in smaller enterprises and devshops.

I think that’s why we’re seeing more and more IaC solutions in mainstream programming languages rather than Domain Specific Languages like HCL for Terraform, so the space will probably grow in size in the coming years.

The official place for discussion is the Pulumi Community Slack and many of the packages are open sourced and maintained on a community driven pulumiverse GitHub account where you can contribute. Pulumi also provides tools for creating your own custom Providers and Packages .

Pulumi AI

This is a great tool and a source of information about Pulumi. They have a very well-trained model that can generate a lot of useful code albeit with some limitations and can be frustratingly slow – still faster than trying to google the answer though. Nonetheless, it is an invaluable resource for learning Pulumi and getting started with whatever resources you need to create.

When searching for information about Pulumi, often times the top links are cached Pulumi AI answers

Getting Started

To create a new project, you can use the Pulumi CLI

pulumi new [template-name]

Should you not enter a template name, the CLI will prompt you to either select a template or write an AI prompt that will generate a starter project for you.

You can select the language you want to use, the cloud provider and the project name. At the time of writing, Pulumi supports TypeScript, Python, Java, .NET, Golang and YAML.

The featured providers are AWS, Azure, GCP and Kubernetes but there are over 150 packages available for different cloud providers and services. Some of the more popular ones could be Cloudflare, Docker, Auth0, Datadog, Grafana, Alibaba Cloud but there are many more available . The beauty of Pulumi is that you can relatively easily create your own packages using existing providers or even write your own completely from scratch.

Project structure

Let’s see what a typical Pulumi project looks like. I’ll be using TypeScript for this example but the structure is similar for other languages.

Project Configuration

The project configuration is stored in the Pulumi.yaml file. This file contains the project name, description, runtime and language information. You can also use it to define a strongly typed configuration schema for your project or change the project structure to your liking.

By default, Pulumi will use index.ts for the entry point of the program but you can change it by setting the main property in the Pulumi.yaml file. You can find an exhaustive list of possible Project configuration options in the documentation .

I wrote a dedicated article about config in pulumi projects if you’re interested

Stack Configuration

A Pulumi programme can be split into stacks which is simply a logical separation of concerns. When creating a project, you can either name your own stack or Pulumi will use dev as the default stack name and create a Pulumi.dev.yaml file in the project root. This file contains the stack configuration and can be used to override the project configuration.

Config in Pulumi projects can be as simple as key value pairs and as complicated as nested objects and arrays. You can also use it to store encrypted secrets either with the pulumi-managed encryption key, a passphrase or a secrets manager of your choice. Currently, you can choose between AWS, Azure, GCP and HashiCorp.

Entry Point

When first creating a project, Pulumi will create an index.ts file in the root of the project. This file is the entry point of the program and contains the code that will create the resources.

Pulumi programme’s structure is synchronous even though provisioning resources is an asynchronous operation and resources are often provisioned in parallel where possible. When using those values in other resources, Pulumi will automatically wait for the resource to be created before proceeding – they call it “lifting” – and it is based on their Output<T> type.

When lifting the value is not possible, you can use .apply() on any of the available properties of a resource e.g., bucket.bucketDomainName.apply( domain => console.log(domain)). This is property specific though, should you need to wait for multiple values, you can use pulumi.all([val1, val2]).apply(([prop1, prop2])=>...).

When creating a string value, you can use pulumi.interpolate`` tag function to interpolate values from other resources or configuration and pulumi's jsonStringify and jsonParse utilities for objects

Stacks

As mentioned previously, Pulumi uses stacks to handle different development stages like prod, dev, qa or create ephemeral environments for feature branches. You can manage stacks with the Pulumi CLI using the pulumi stack command.

If used by itself, it will print your current stack’s information. You can also use it to create a new stack, switch to a different stack, list all stacks or delete a stack.

pulumi stack ls
pulumi stack init [stack-name]
pulumi stack select [stack-name]
pulumi stack rm [stack-name]
pulumi stack help

Outputs

When creating resources, you can define outputs that will be available to other resources or the user. You can define outputs in the entry file by simply exporting them.

export const bucketName = bucket.bucket;

Everytime you run pulumi up or pulumi preview, the outputs will be printed to the console. They’re also visible in the Pulumi Cloud dashboard or you can use pulumi stack or pulumi stack output command to view them.

Outputs are not only useful for easy access to the resource data but can also be used in other Pulumi programmes. You can use the StackReference class to reference another stack’s output.

import * as pulumi from '@pulumi/pulumi';
const stack = new pulumi.StackReference('org/project/stack');
const stackOutput = stack.getOutput('property');

You can read more about stacks in the documentation’s stack section .

Pulumi v. AWS CDK

Pulumi and AWS CDK are similar in that they both allow you to define cloud infrastructure using programming languages. However, there are some key differences between the two.

  • Pulumi supports multiple cloud providers, while AWS CDK is specific to AWS – technically it can be integrated with other providers via the Constructs Library, but practically it’s not
  • Pulumi uses the cloud provider’s SDK or Terraform to create and manage resources, while AWS CDK compiles the infrastructure code to CloudFormation templates
  • Pulumi provides a fairly low-level API for provisioning resources which makes it more verbose as you have to define everything yourself whereas AWS CDK provides many batteries-included constructs that abstract away a lot of the boilerplate code and lowers the entry threshold for beginners
  • CDK obviously benefits from being the official AWS tool which cannot be ignored. It would seem that the official tool would have a larger community, but I’m not so sure about it. Pulumi has almost double the stars on GitHub which of course doesn’t mean that the community is larger but at least it’s more appreciative of the tool and that could mean more engagement from the community. So even though, the community size is undecided it would seem that developer satisfaction is definitely on Pulumi’s side. Still, CDK probably has more resources available online, and it has a tight integration with the AWS ecosystem like the CDK Migrate Service for exporting the existing infrastructure to CDK code.
  • AWS CDK requires you to have a significant understanding of CloudFormation because it inherits its strengths and weaknesses which often lead to hard-to-debug errors that cannot be solved by simply analyzing the CDK code itself

Conclusion

Pulumi offers a flexible and powerful approach to managing cloud infrastructure as code, supporting multi-cloud deployments with direct SDK usage. Its intuitive CLI and extensive language and provider support make it accessible to developers of all skill levels, while features like stack-specific configuration and outputs enhance project scalability and maintainability. Despite similarities to AWS CDK, Pulumi’s granular control and growing community solidify its position as a versatile solution for infrastructure automation in today’s cloud development landscape.

Personally, I find Pulumi to be a great tool and have not used CDK since. I find it more flexible and easier to use, especially when creating custom resources. Even though it is more verbose and a bit harder to use at first, I appreciate the control it gives me and it strengthens my AWS knowledge as I have to provision everything myself and cannot rely on the “magic” constructs to do all the heavy-lifting for me.

FAQ