Terraform Module Design for Reusable Multi-Cloud Infrastructure

Module Composition Over Monolith Modules

I've spent the last four years building Terraform modules for organizations that operate across AWS, GCP, and Azure simultaneously. The single biggest mistake I see teams make is creating one giant module that tries to handle every cloud provider through conditional logic. It doesn't work. Not at any meaningful scale.

The better approach? Small, composable modules with well-defined interfaces. Think of them like functions in a programming language — each one does exactly one thing, accepts inputs through variables, and exposes outputs that other modules consume.

The Interface-First Design Pattern

Before writing any resource blocks, I start with the variables.tf and outputs.tf files. This isn't some academic exercise. When three different teams need to consume your networking module, they'll discover every gap in your interface design within the first week.

Here's what a well-designed module interface looks like for a multi-cloud networking component:

# variables.tf
variable "cloud_provider" {
  type        = string
  description = "Target cloud: aws, gcp, or azure"
  validation {
    condition     = contains(["aws", "gcp", "azure"], var.cloud_provider)
    error_message = "Supported providers: aws, gcp, azure."
  }
}

variable "network_config" {
  type = object({
    cidr_block       = string
    subnet_count     = number
    enable_nat       = bool
    private_subnets  = bool
  })
}

variable "tags" {
  type    = map(string)
  default = {}
}

Notice the validation block on the provider variable. That's not optional. Without it, someone will pass "AWS" (uppercase) and spend an hour debugging why the conditional logic downstream produces empty resource blocks.

Conditional Resources with Provider Wrappers

The count and for_each meta-arguments are your primary tools for multi-cloud conditional logic. But don't sprinkle them directly through a monolith module. Instead, create provider-specific sub-modules and call them conditionally from a root module.

# main.tf (root module)
module "aws_network" {
  source = "./modules/aws-vpc"
  count  = var.cloud_provider == "aws" ? 1 : 0

  cidr_block      = var.network_config.cidr_block
  subnet_count    = var.network_config.subnet_count
  enable_nat      = var.network_config.enable_nat
  private_subnets = var.network_config.private_subnets
  tags            = var.tags
}

module "gcp_network" {
  source = "./modules/gcp-vpc"
  count  = var.cloud_provider == "gcp" ? 1 : 0

  cidr_block      = var.network_config.cidr_block
  subnet_count    = var.network_config.subnet_count
  enable_nat      = var.network_config.enable_nat
  private_subnets = var.network_config.private_subnets
  labels          = var.tags
}

This pattern keeps each provider's implementation isolated. When AWS releases a new VPC feature, you're modifying one sub-module, not hunting through conditionals scattered across 800 lines of HCL.

Output Normalization Across Providers

Here's where things get tricky. AWS calls it a "subnet ID." GCP calls it a "subnetwork self_link." Azure calls it a "subnet resource ID." Your consuming modules shouldn't need to care about these differences.

I've found the most reliable pattern is a normalized output object:

# outputs.tf
output "network" {
  value = {
    id          = try(module.aws_network[0].vpc_id, module.gcp_network[0].network_id, module.azure_network[0].vnet_id, "")
    subnet_ids  = try(module.aws_network[0].subnet_ids, module.gcp_network[0].subnet_ids, module.azure_network[0].subnet_ids, [])
    nat_gateway = try(module.aws_network[0].nat_id, module.gcp_network[0].router_id, module.azure_network[0].nat_gateway_id, "")
    provider    = var.cloud_provider
  }
}

The try() function chains are ugly. I won't pretend otherwise. But they're explicit about what's happening, and they fail loudly when all branches return nothing. I'll take ugly-but-debuggable over clever-but-mysterious every time.

Versioning Strategy for Shared Modules

Semantic versioning isn't just a suggestion here — it's the mechanism that prevents one team's infrastructure update from breaking another team's deployment at 2 AM.

Pin your module sources to specific tags, not branches:

module "networking" {
  source  = "git::https://github.com/org/terraform-modules.git//networking?ref=v2.3.1"
  # NOT ref=main — that's asking for trouble
}

Our team discovered the hard way that referencing the main branch means every terraform plan could pull a different module version. One developer's plan shows no changes while another's shows a complete network rebuild. The debugging session that followed cost us most of a Thursday afternoon.

Breaking Change Protocol

Any change that modifies a variable's type, removes a variable, changes a resource's address (triggering destroy/recreate), or alters output structure is a major version bump. No exceptions. I've watched teams try to sneak breaking changes into minor versions. The result is always the same: an incident ticket and a postmortem.

Testing Multi-Cloud Modules

You can't meaningfully test multi-cloud modules without actually deploying resources. Unit testing HCL with mock providers tells you the syntax is valid. It doesn't tell you whether your GCP subnet CIDR calculation actually produces routable addresses.

Our testing pipeline runs Terratest against all three providers in parallel. Each test deploys the module, validates the outputs, runs connectivity checks, and tears everything down. The full suite takes about 12 minutes. That's not fast, but it catches real issues — like the time our Azure NAT gateway module silently skipped creating route table associations when the subnet count exceeded four.

Structure your test files to mirror the module's provider split:

tests/
  aws_network_test.go
  gcp_network_test.go
  azure_network_test.go
  helpers_test.go

Each test file handles its own provider authentication and cleanup. Shared helpers extract common assertions like "can ping across subnets" or "NAT gateway routes traffic correctly." Don't share Terraform state between tests — that path leads to flaky teardowns and orphaned resources costing money overnight.

Registry Publishing and Documentation

If your module isn't in a registry with auto-generated docs, it doesn't exist. Engineers won't read your README. They'll open the Terraform registry, scan the input table, copy the example, and move on.

The private registry from Terraform Cloud handles this well enough, though the documentation generation has quirks. It pulls descriptions from variable blocks and output blocks, so those descriptions need to be written for humans, not for linters. "The CIDR block for the VPC" tells the reader nothing. "IPv4 CIDR block (/16 to /28) for the primary network — smaller blocks limit subnet count" actually helps them make a decision.

For organizations that can't use Terraform Cloud, a GitHub Pages site generated from terraform-docs works. It's more maintenance, but it gives you control over layout and searchability that the built-in registry can't match.

Practical Module Boundaries

The hardest design decision isn't technical. It's political. Where you draw module boundaries determines which team owns which piece of infrastructure. I've settled on this rough heuristic after several reorganizations:

Networking modules own everything up to the subnet level. Compute modules own instances, containers, and load balancers. Data modules own databases, caches, and queues. Security modules own IAM, firewall rules, and encryption keys. Each module type maps to a team that understands that domain deeply.

Cross-cutting concerns like tagging, naming conventions, and region selection live in a shared "conventions" module that every other module calls. It's small — maybe 50 lines of locals — but it prevents the naming drift that makes cloud bills impossible to parse six months later.

The alternative — letting every team define their own tagging — means you'll eventually find production resources tagged "env=prod", "environment=production", "stage=prd", and "tier=p" all in the same account. Ask me how I know.

Error Handling in Multi-Cloud Modules

Terraform doesn't have try-catch blocks. When a resource creation fails in AWS, you can't catch that error and fall back to a GCP resource instead. What you can do is validate inputs early and provide clear error messages that tell the operator what went wrong without requiring them to read provider-specific error codes.

Custom validation rules in variable blocks are your first line of defense. Check CIDR formats, ensure subnet counts don't exceed AZ counts, verify that naming conventions match organizational standards. Every validation rule you add is one fewer support ticket from a team that passed bad inputs and got a cryptic provider error 200 lines deep in the plan output.

variable "network_config" {
  type = object({
    cidr_block  = string
    subnet_count = number
  })

  validation {
    condition     = can(cidrhost(var.network_config.cidr_block, 0))
    error_message = "cidr_block must be a valid IPv4 CIDR notation (e.g., 10.0.0.0/16)."
  }

  validation {
    condition     = var.network_config.subnet_count >= 1 && var.network_config.subnet_count <= 8
    error_message = "subnet_count must be between 1 and 8."
  }
}

For post-deployment validation, I add a null_resource with a local-exec provisioner that runs health checks against newly created infrastructure. It's not elegant, but it catches configuration issues that only manifest after resources exist — like a VPC peering connection that's "active" according to AWS but doesn't actually route traffic because the route table associations are wrong.

Documentation as Module Interface

We generate documentation automatically from variable descriptions, output descriptions, and a README template using terraform-docs. The CI pipeline fails if any variable or output is missing a description. This seems heavy-handed until you realize that undocumented modules get used incorrectly, and incorrect usage generates support burden that costs more than the five seconds it takes to write a description.

Our convention: variable descriptions answer "what does this control and what are the constraints?" Output descriptions answer "what is this value and where would you use it?" Both should be actionable, not just labels. Compare "The VPC ID" (useless) with "VPC ID — pass to compute modules as their network target" (tells you what to do with it).