Pod Security Standards and Admission Control in Multi-Tenant Clusters

Q: What is Pod Security Standards Levels?

Kubernetes defines three security profiles. privileged is unrestricted — anything goes. baseline blocks known privilege escalation paths while remaining compatible with most workloads. restricted follows security hardening best practices and breaks many applications that weren't designed for it. For most production namespaces, baseline with warn.

Multi-Tenant Security Starts at the Pod Spec

Pod Security Standards replaced PodSecurityPolicies in Kubernetes 1.25, and the migration wasn't optional — PSPs are gone. But many teams migrated by slapping the "privileged" level on every namespace just to stop the warnings, effectively disabling security enforcement. That defeats the entire purpose.

In a multi-tenant cluster where different teams deploy different workloads, you need admission control that prevents one team's misconfigured deployment from compromising shared infrastructure. Here's how I've set that up across clusters with 20+ teams sharing the same control plane.

Pod Security Standards Levels

Kubernetes defines three security profiles. privileged is unrestricted — anything goes. baseline blocks known privilege escalation paths while remaining compatible with most workloads. restricted follows security hardening best practices and breaks many applications that weren't designed for it.

For most production namespaces, baseline with warn on restricted violations is the sweet spot. Here's what that looks like:

apiVersion: v1
kind: Namespace
metadata:
  name: team-alpha-production
  labels:
    pod-security.kubernetes.io/enforce: baseline
    pod-security.kubernetes.io/enforce-version: v1.28
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/warn-version: v1.28
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/audit-version: v1.28

This configuration enforces baseline — pods that violate baseline rules won't be created. It warns on restricted violations — developers see a warning in their kubectl output but the pod still gets created. And it audits restricted violations — they show up in the API server audit log for security teams to review.

What Baseline Actually Blocks

The baseline level prevents the most dangerous pod configurations. It blocks pods that run as privileged (securityContext.privileged: true), pods that add dangerous Linux capabilities (SYS_ADMIN, NET_RAW, etc.), pods that use hostNetwork, hostPID, or hostIPC, pods that mount hostPath volumes pointing at sensitive paths, and pods with containers running as root when runAsNonRoot isn't set.

These aren't edge cases. I've seen production deployments from major vendors that set privileged: true in their default manifests because it was easier than figuring out which capabilities they actually needed. The baseline level catches these before they hit your cluster.

Admission Controllers Beyond PSS

Pod Security Standards handle the basics, but production multi-tenant clusters need more granular controls. This is where policy engines like OPA Gatekeeper or Kyverno come in.

I've used both. Kyverno is my preference for teams new to admission control because policies are written in YAML, not Rego. Here's a Kyverno policy that requires all images to come from your internal registry:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-internal-registry
spec:
  validationFailureAction: Enforce
  rules:
  - name: check-image-registry
    match:
      any:
      - resources:
          kinds:
          - Pod
    validate:
      message: "Images must come from registry.internal"
      pattern:
        spec:
          containers:
          - image: "registry.internal/*"
          initContainers:
          - image: "registry.internal/*"

This prevents anyone from deploying a container image pulled from Docker Hub or any other public registry. Combined with image scanning in your CI pipeline, it ensures only vetted images run in your cluster.

Enforcing Resource Requests

Another critical policy for multi-tenant clusters: require resource requests on all containers. Without this, a team can deploy a workload with no requests, and the scheduler treats it as needing zero resources. The pod gets scheduled, consumes actual CPU and memory, and causes resource contention that's invisible to the scheduler.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-resource-requests
spec:
  validationFailureAction: Enforce
  rules:
  - name: check-resources
    match:
      any:
      - resources:
          kinds:
          - Pod
    validate:
      message: "CPU and memory requests are required"
      pattern:
        spec:
          containers:
          - resources:
              requests:
                cpu: "?*"
                memory: "?*"

Namespace Isolation Architecture

Each tenant team gets their own namespace with a standardized set of policies. I use a namespace provisioning controller that creates all the required resources when a new team is onboarded:

The namespace itself with Pod Security Standards labels. A ResourceQuota scoped to the team's allocation. LimitRanges with sensible defaults. Network policies that deny all ingress/egress by default (the team's deployment manifests explicitly allow what they need). RBAC bindings giving the team admin access within their namespace but nothing outside it.

The key insight is that these resources form a boundary — not individual controls. A team can do whatever they want inside their namespace, subject to the policies. They can't affect other namespaces, can't access the control plane, and can't escalate privileges beyond what the pod security standard allows.

Handling Exceptions

Every multi-tenant cluster has workloads that genuinely need elevated privileges. Monitoring agents need hostPath access. CNI plugins run as privileged. Some legacy applications need NET_RAW for health checks that use ICMP.

Don't create exceptions by relaxing the namespace-level policy. Instead, use a separate namespace with elevated permissions and restrict who can deploy to it. In Kyverno, you can create policies that apply only to specific namespaces or that exclude specific service accounts:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-privileged-to-system
spec:
  validationFailureAction: Enforce
  rules:
  - name: block-privileged
    match:
      any:
      - resources:
          kinds:
          - Pod
    exclude:
      any:
      - resources:
          namespaces:
          - kube-system
          - monitoring
    validate:
      message: "Privileged pods only allowed in kube-system and monitoring"
      pattern:
        spec:
          containers:
          - =(securityContext):
              =(privileged): false

This allows privileged containers only in kube-system and monitoring. Every other namespace is blocked. The team that manages monitoring infrastructure has RBAC access to the monitoring namespace; application teams don't.

Audit and Compliance Reporting

The audit log labels from Pod Security Standards feed into your compliance reporting. Every restricted violation is logged with the pod spec, the namespace, the user who created it, and the specific violation. If your compliance team asks "are any workloads running as root?", you can query the audit log for restricted violations with reason runAsNonRoot.

Kyverno additionally generates PolicyReport resources that summarize violations per namespace. These integrate with tools like Policy Reporter, which provides dashboards and Slack notifications for policy violations. In my clusters, a Slack notification fires whenever someone deploys a pod that violates the restricted profile. It doesn't block the deployment (baseline is enforced, not restricted), but it creates visibility for the security team.

Runtime Security Beyond Admission

Admission control prevents bad configurations from entering the cluster. But what about workloads that pass admission checks and then behave maliciously at runtime? A container that's allowed to run as non-root can still make system calls that it shouldn't.

Seccomp profiles restrict which system calls a container can make. Kubernetes 1.27+ includes a default seccomp profile (RuntimeDefault) that blocks dangerous syscalls like keyctl, personality, and userfaultfd. Enable it cluster-wide through a policy engine or by setting the securityContext seccompProfile type to RuntimeDefault on each pod.

For workloads that need specific syscalls beyond the default profile, create a custom seccomp profile and deploy it to nodes via a DaemonSet or the Security Profiles Operator. The operator simplifies this by managing profile distribution across nodes automatically.

AppArmor and SELinux

AppArmor (on Ubuntu/Debian nodes) and SELinux (on RHEL/CentOS nodes) provide mandatory access control at the OS level. They restrict file access, network operations, and capabilities beyond what Kubernetes security contexts control.

In practice, enabling the default AppArmor profile (runtime/default) catches most container escape attempts. Custom profiles are useful when you need to restrict a container to specific filesystem paths - for instance, preventing a web server container from reading anything outside /app and /tmp.

Network Policy Integration

Pod security and network policy work together. A comprehensive multi-tenant security posture includes both. The pod security standards prevent privilege escalation at the container level. Network policies prevent unauthorized communication at the network level. Without both, there's a gap - a non-privileged container can still exfiltrate data to an external endpoint if there are no egress restrictions.

The minimum viable network policy set for a multi-tenant namespace: deny all ingress and egress by default, then allow specific communication paths. Don't forget the DNS egress policy - without it, no pod in the namespace can resolve DNS names, and every service-to-service call fails with a cryptic connection error. I've seen this mistake cause 30-minute outages during network policy rollouts.

Combine this with the pod security standards from earlier in this article, and you have a defense-in-depth approach: admission control prevents dangerous configurations, runtime security restricts system calls, and network policy constrains communication. No single layer is sufficient, but together they make it significantly harder for a compromised workload to affect other tenants.

Compliance Automation

For organizations that need to demonstrate compliance with frameworks like SOC 2 or ISO 27001, the combination of Pod Security Standards, network policies, and a policy engine provides auditable evidence. Each denied admission is logged. Each policy violation generates a PolicyReport. Each network policy denial can be tracked via flow logs.

Build automated compliance reports that pull from these data sources. A monthly report showing zero PSS violations, no unauthorized network flows, and no policy exceptions that weren't approved through the change management process satisfies most auditors. The key is automation - if generating the report requires manual effort, it won't get done consistently.