Terraform is one of the most popular Infrastructure as Code tools for good reason. It supports all of the major public cloud platforms (and much more via a plethora of providers), its HCL language is declarative and easy to read while also offering useful constructs like loops, and it offers first-class support for code modularity and packaging.
Modules are simply Terraform code in a subdirectory. They have their own variables, resources, and outputs (and if you’re doing them well, their own documentation). If you’re using Terraform, you’re almost certainly using modules. At enterprise scale, the real question is how your organization is using modules to eliminate redundancies and drive reuse.
We’ve seen plenty of teams create their own modules with grand intentions of sharing them across their organization but those plans usually fall flat. Modules serve as useful tools for code organization within a team, but without a central clearinghouse of modules (like CloudFormation’s Registry), it’s hard to get widespread adoption. Fortunately, Terraform makes it much easier to share infrastructure code than CloudFormation.
Git
Terraform can import modules from any Git (or Mercurial) repository. This is the most common approach we see for sharing modules in enterprises. It’s easy to understand and set up. And as long as the module repositories are highly available, it scales well. But there are some considerations for making this approach effective.
First, use a single GitHub organization as the home for all of your module repositories. This makes it easier to discover modules and helps eliminate redundant modules.
Second, institute a common CI/CD pipeline for modules. (This applies to all methods for sharing modules.) The pipeline should enforce basic checks such as formatting (terraform fmt), syntax validation (terraform validate), and linting. Modules should have user-friendly documentation to make it easier for customers to incorporate the modules into their code. Terraform-docs is an example of a tool that can help automate this process.
Provide releases of Terraform modules by using Git tags with semantic version numbers. This lets consumers precisely specify their dependencies and avoid unexpected changes when modules are updated, sidestepping the biggest weakness of CloudFormation’s modules implementation.
HTTPS, S3, and GCP Buckets
You can serve modules from any webserver over HTTPS or out of an AWS S3 or GCP Cloud Storage bucket. Using a public cloud object store can make it easier to provide more durability and availability. With all three approaches you need to publish artifacts with a version in their name so that consumers can use the specific version of the module they want.
Public Registry
Terraform’s public registry is a catalog that contains a vast number of modules and providers both from Hashicorp, its verified partners, and anyone else who wants to publish their module. One of the benefits of the registry is discoverability. It’s easy to search for modules and quickly see how to use them. On the discoverability scale, it’s far better than the HTTPS/bucket approach and reasonably better than a GitHub organization.
The public registry is a good starting point and Hashicorp has an interest in keeping it highly available. But for production use it’s less enticing. Ideally, you should be caching all of your dependencies internally to avoid an issue where those dependencies are unknowingly changed or become unavailable. Review public modules before use the same as you would code from any other source such as npm or PyPI.
Using the registry also allows you to specify a module version with constraints. This provides more flexibility over embedding the version in a Git source string such as ?ref=v1.2.0 however, being too lax could lead to breakage.
Private Registry
The public registry is convenient, but not without its downsides. An improvement is to store your organization’s trusted and shared modules in a registry that is only available to your organization. Hashicorp’s Terraform Cloud service provides that private repository, but you are still reliant on an external dependency to fetch your modules. Depending on your organization, this could be an acceptable trade-off.
But for enterprise that aren’t willing to make that trade-off, Terraform Enterprise is a self-hosted version of Terraform Cloud. If you’re a large enterprise, this option deserves consideration for the level of control it provides.
Beyond being private registries, both products manage state for deployments, mitigating one of Terraform’s weakest points compared to CloudFormation and ARM. It also provides integration with Hashicorp’s Sentinel policy-as-code tool to provide additional controls around what Terraform does.
Finally, you can build your own private registry that implements the Terraform Registry API. Popular artifact managers Artifactory and Nexus are also working on support for serving as a Terraform registry, but it’s not supported yet.
If you want to learn more about scaling your IaC approach, I wrote an article explaining some general guidelines and available tools. SingleStone helps companies of all sizes, from start-up to Fortune 500, with getting into the cloud and using it to effectively drive real business agility—and results. If your company is ready to manage its infrastructure code like a high-performing organization, we can help.
