Helm Chart Design Patterns for Multi-Environment Deployments

Beyond helm create: Charts That Don't Break in Production

Every Helm chart starts with helm create, and that's usually where the problems begin. The scaffold gives you a deployment, a service, an ingress, and a service account — a reasonable starting point. But the default templates make assumptions that don't survive contact with real multi-environment deployment workflows.

I've maintained Helm charts that deploy the same application across dev, staging, production, and disaster recovery environments. The patterns I'll cover here emerged from years of watching charts break in specific environments because someone hardcoded a value that should've been parameterized, or over-parameterized something that should've been a sensible default.

Values File Hierarchy for Multi-Environment

The most common pattern I've seen work well is a layered values structure. You start with a values.yaml that contains production defaults — because production is where things need to work, and it should be the path of least resistance. Environment-specific overrides live in separate files:

charts/my-app/
├── Chart.yaml
├── values.yaml              # production defaults
├── values-dev.yaml          # dev overrides
├── values-staging.yaml      # staging overrides
├── values-dr.yaml           # disaster recovery
└── templates/

The deployment command becomes:

# Dev deployment
helm upgrade --install my-app ./charts/my-app   -f charts/my-app/values.yaml   -f charts/my-app/values-dev.yaml   -n dev

# Production — just the base values
helm upgrade --install my-app ./charts/my-app   -n production

This approach means production gets the defaults. You don't need to remember to pass the production values file. If someone runs helm upgrade without specifying any values file, they get production settings. That's intentional — I'd rather someone accidentally deploy with production resource requests in dev than with dev resource requests in production.

Structuring the Values Schema

Don't make every field configurable. I've seen charts with 200+ values where changing one thing requires understanding the relationship between six different keys. Instead, group values by concern and only expose what actually changes between environments:

# values.yaml — production defaults
replicaCount: 3

image:
  repository: registry.internal/my-app
  tag: ""  # set by CI/CD
  pullPolicy: IfNotPresent

resources:
  requests:
    cpu: 500m
    memory: 512Mi
  limits:
    cpu: "2"
    memory: 1Gi

autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 10
  targetCPUUtilization: 70

env:
  LOG_LEVEL: "info"
  DB_POOL_SIZE: "20"

secrets:
  existingSecret: "my-app-secrets"

Then the dev override touches only what differs:

# values-dev.yaml
replicaCount: 1

resources:
  requests:
    cpu: 100m
    memory: 256Mi
  limits:
    cpu: 500m
    memory: 512Mi

autoscaling:
  enabled: false

env:
  LOG_LEVEL: "debug"
  DB_POOL_SIZE: "5"

Template Patterns That Prevent Outages

There's a specific pattern for Deployment templates that I've seen prevent a whole class of incidents. Always define a maxUnavailable and maxSurge in your rolling update strategy, and make them environment-aware:

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: {{ .Values.deployment.maxUnavailable | default "25%" }}
      maxSurge: {{ .Values.deployment.maxSurge | default "25%" }}

In dev, you might want maxUnavailable: 100% for fast deploys. In production, maxUnavailable: 0 with maxSurge: 1 means you never lose capacity during a rollout. That slow, careful rollout has saved us from bad deploys more than once — the health check catches the problem before the old pods are terminated.

The _helpers.tpl Patterns Worth Keeping

The default _helpers.tpl from helm create defines some useful named templates, but there are a few additions I always make. First, a standard set of labels that includes the chart version, the app version, and a managed-by annotation:

{{- define "my-app.labels" -}}
helm.sh/chart: {{ include "my-app.chart" . }}
app.kubernetes.io/name: {{ include "my-app.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/component: {{ .Values.component | default "backend" }}
{{- end }}

Second, a checksum annotation that forces pod restarts when ConfigMaps or Secrets change:

template:
  metadata:
    annotations:
      checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
      checksum/secret: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}

Without that checksum, updating a ConfigMap doesn't trigger a pod rollout. Your new config sits there doing nothing until the next deploy touches the Deployment spec. I've watched teams debug "why isn't my config change taking effect" for an hour before realizing the pods weren't restarted.

Handling Secrets Without Storing Them in the Chart

Never put actual secret values in your values files, even if the repo is private. The pattern I use is to reference external secrets by name:

{{- if .Values.secrets.existingSecret }}
envFrom:
- secretRef:
    name: {{ .Values.secrets.existingSecret }}
{{- end }}

The actual secrets get created by your secrets management system — Vault with the external-secrets operator, AWS Secrets Manager, or even a manually created Secret in each namespace. The chart just needs to know the name. This separation means your chart repo doesn't contain any sensitive data, and different environments can use different secrets backends.

Library Charts for Shared Standards

When you're managing 15+ microservices that all need the same patterns — health checks, security contexts, resource defaults, PodDisruptionBudgets — a library chart saves enormous duplication. Create a chart with type: library in Chart.yaml, define your standard templates, and have application charts depend on it:

# library chart: charts/standard-app/Chart.yaml
apiVersion: v2
name: standard-app
type: library
version: 1.2.0

Application charts import it as a dependency and call the library's templates. When you need to update the standard PodDisruptionBudget across all services, you change it once in the library chart, bump the version, and update the dependency in each app chart. It's not automatic — you still need to update each consumer — but it's vastly better than copy-pasting the same YAML block into 15 charts.

Testing Charts Before They Hit Production

Helm has a built-in test framework that almost nobody uses. You can create test pods that run after installation and verify the deployment is working:

# templates/tests/test-connection.yaml
apiVersion: v1
kind: Pod
metadata:
  name: {{ include "my-app.fullname" . }}-test
  annotations:
    "helm.sh/hook": test
spec:
  containers:
  - name: wget
    image: busybox
    command: ['wget', '--spider', 'http://{{ include "my-app.fullname" . }}:{{ .Values.service.port }}/healthz']
  restartPolicy: Never

Run helm test my-app after deployment, and it'll spin up the test pod, hit the health endpoint, and report success or failure. I combine this with helm lint in CI — lint catches syntax issues, template rendering with --dry-run catches logic errors, and post-deploy tests catch runtime issues.

For more thorough validation, kubeval or kubeconform validates rendered templates against the Kubernetes OpenAPI schema. This catches things like using a field name that doesn't exist in your target cluster's API version — something that helm lint won't flag because it doesn't know your cluster's schema.

Versioning Strategy

Separate chart version from app version. The chart version (in Chart.yaml) tracks changes to the templates, defaults, and chart structure. The app version tracks which version of your software the chart deploys. Bump the chart version when you change templates. Set the app version dynamically in CI/CD by passing --set image.tag=... or updating Chart.yaml's appVersion before packaging.

This separation matters because you often need to change deployment configuration without changing the application — adding a new environment variable, adjusting resource limits, or fixing a template bug. Those changes deserve their own version number in the chart, independent of application releases.

Chart Dependency Management

When your application depends on external services like Redis or PostgreSQL, you have two choices: include the dependency chart as a subchart, or treat it as an external dependency that's deployed separately.

For production, I strongly prefer external dependencies. Bundling PostgreSQL as a subchart means helm uninstall deletes your database. That's exactly the kind of accident that multi-team organizations can't afford. External dependencies get their own lifecycle, their own backups, and their own team responsible for them.

For dev and testing environments, subcharts are convenient. You can spin up the entire stack with one command. Use a conditional in your Chart.yaml dependencies section with a condition field like redis.enabled, then set that to true in values-dev.yaml and false in production values. Dev gets a bundled Redis. Production connects to the managed Redis cluster.

Rollback Strategy

Helm keeps release history by default - the last 10 revisions. When a deployment goes wrong, helm rollback my-app reverts to the previous release. But there are gotchas.

A rollback rerenders the templates with the previous release's values. If your templates reference ConfigMaps or Secrets that were updated outside of Helm, the rollback restores the Deployment but not the ConfigMap. Your rolled-back pods might be running old code with new configuration, which can cause its own problems.

The safer rollback strategy is to redeploy the previous known-good image tag rather than using helm rollback. This goes through the full deployment pipeline - CI, values update, Argo CD sync - and produces a clean state rather than a partial revert.

Pre-Commit Validation

Catch chart errors before they reach the cluster. I add these checks to the CI pipeline for any PR that modifies chart files: helm lint validates chart structure, helm template with kubeconform validates rendered templates against the Kubernetes OpenAPI schema, and kubent (kube-no-trouble) catches deprecated API versions that will break on your next cluster upgrade. These checks take 10 seconds to run and have prevented dozens of deployment failures in my experience.