Configure Kubernetes Pod Security Standards with admission controllers for policy enforcement

Verify cluster requirements

Check that your Kubernetes cluster supports Pod Security Standards. This requires Kubernetes 1.25 or later with the PodSecurity admission plugin enabled.

kubectl version --short
kubectl api-resources | grep podsecurity

Verify the PodSecurity admission controller is enabled:

kubectl get --raw /api/v1 | jq '.resources[] | select(.name=="namespaces")'

Create test namespaces with different security levels

Create namespaces to demonstrate different Pod Security Standard enforcement levels.

apiVersion: v1
kind: Namespace
metadata:
  name: test-privileged
  labels:
    pod-security.kubernetes.io/enforce: privileged
    pod-security.kubernetes.io/audit: privileged
    pod-security.kubernetes.io/warn: privileged

apiVersion: v1
kind: Namespace
metadata:
  name: test-baseline
  labels:
    pod-security.kubernetes.io/enforce: baseline
    pod-security.kubernetes.io/audit: baseline
    pod-security.kubernetes.io/warn: baseline

apiVersion: v1
kind: Namespace
metadata:
  name: test-restricted
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

Apply the namespace configurations:

kubectl apply -f test-privileged-ns.yaml
kubectl apply -f test-baseline-ns.yaml
kubectl apply -f test-restricted-ns.yaml

Configure cluster-wide Pod Security Standards

Set default Pod Security Standards for all namespaces by configuring the admission controller. Create an admission configuration file:

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
name: PodSecurity
  configuration:
    apiVersion: pod-security.admission.config.k8s.io/v1beta1
    kind: PodSecurityConfiguration
    defaults:
      enforce: "baseline"
      enforce-version: "latest"
      audit: "restricted"
      audit-version: "latest"
      warn: "restricted"
      warn-version: "latest"
    exemptions:
      usernames: ["system:serviceaccount:kube-system:*"]
      runtimeClasses: []
      namespaces: ["kube-system", "kube-public", "kube-node-lease"]

Test baseline policy enforcement

Create a pod that violates baseline security standards to test enforcement:

apiVersion: v1
kind: Pod
metadata:
  name: privileged-test
  namespace: test-baseline
spec:
  containers:
  - name: test-container
    image: nginx:1.25
    securityContext:
      privileged: true
    ports:
    - containerPort: 80

Attempt to create the pod:

kubectl apply -f privileged-pod.yaml

This should fail with a Pod Security Standards violation. Create a compliant pod instead:

apiVersion: v1
kind: Pod
metadata:
  name: compliant-test
  namespace: test-baseline
spec:
  containers:
  - name: test-container
    image: nginx:1.25
    securityContext:
      allowPrivilegeEscalation: false
      runAsNonRoot: true
      runAsUser: 1001
      capabilities:
        drop:
        - ALL
    ports:
    - containerPort: 8080

kubectl apply -f compliant-pod.yaml

Install OPA Gatekeeper for advanced policies

Install OPA Gatekeeper to implement custom admission policies beyond Pod Security Standards:

kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/release-3.14/deploy/gatekeeper.yaml

Wait for Gatekeeper components to be ready:

kubectl wait --for=condition=Ready pod -l control-plane=controller-manager -n gatekeeper-system --timeout=90s
kubectl wait --for=condition=Ready pod -l control-plane=audit-controller -n gatekeeper-system --timeout=90s

Create Gatekeeper constraint templates

Define a constraint template to enforce resource limits on all pods:

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequiredresources
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredResources
      validation:
        openAPIV3Schema:
          type: object
          properties:
            limits:
              type: array
              items:
                type: string
            requests:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredresources
        
        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          not container.resources.limits
          msg := "Container missing resource limits"
        }
        
        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          not container.resources.requests
          msg := "Container missing resource requests"
        }
        
        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          required := input.parameters.limits[_]
          not container.resources.limits[required]
          msg := sprintf("Container missing required resource limit: %v", [required])
        }
        
        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          required := input.parameters.requests[_]
          not container.resources.requests[required]
          msg := sprintf("Container missing required resource request: %v", [required])
        }

kubectl apply -f resource-limits-template.yaml

Apply resource limit constraints

Create a constraint using the template to enforce CPU and memory limits:

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredResources
metadata:
  name: must-have-resources
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    excludedNamespaces: ["kube-system", "kube-public", "kube-node-lease", "gatekeeper-system"]
  parameters:
    limits: ["cpu", "memory"]
    requests: ["cpu", "memory"]

kubectl apply -f require-resources-constraint.yaml

Create image security policies

Define a template to restrict container images to trusted registries:

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8sallowedrepos
spec:
  crd:
    spec:
      names:
        kind: K8sAllowedRepos
      validation:
        openAPIV3Schema:
          type: object
          properties:
            repos:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8sallowedrepos
        
        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
          not any(satisfied)
          msg := sprintf("Container image <%v> is not from an allowed repository", [container.image])
        }
        
        violation[{"msg": msg}] {
          container := input.review.object.spec.initContainers[_]
          satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
          not any(satisfied)
          msg := sprintf("Init container image <%v> is not from an allowed repository", [container.image])
        }

Apply the template and create a constraint:

kubectl apply -f allowed-repos-template.yaml

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedRepos
metadata:
  name: repo-must-be-trusted
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
      - apiGroups: ["apps"]
        kinds: ["Deployment"]
    excludedNamespaces: ["kube-system", "kube-public", "kube-node-lease", "gatekeeper-system"]
  parameters:
    repos:
      - "docker.io/library/"
      - "gcr.io/"
      - "quay.io/"
      - "registry.k8s.io/"

kubectl apply -f allowed-repos-constraint.yaml

Configure security context constraints

Create a template to enforce security context requirements:

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8ssecuritycontext
spec:
  crd:
    spec:
      names:
        kind: K8sSecurityContext
      validation:
        openAPIV3Schema:
          type: object
          properties:
            runAsNonRoot:
              type: boolean
            readOnlyRootFilesystem:
              type: boolean
            allowPrivilegeEscalation:
              type: boolean
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8ssecuritycontext
        
        violation[{"msg": msg}] {
          input.parameters.runAsNonRoot
          container := input.review.object.spec.containers[_]
          not container.securityContext.runAsNonRoot
          msg := "Container must set runAsNonRoot to true"
        }
        
        violation[{"msg": msg}] {
          input.parameters.readOnlyRootFilesystem
          container := input.review.object.spec.containers[_]
          not container.securityContext.readOnlyRootFilesystem
          msg := "Container must set readOnlyRootFilesystem to true"
        }
        
        violation[{"msg": msg}] {
          not input.parameters.allowPrivilegeEscalation
          container := input.review.object.spec.containers[_]
          container.securityContext.allowPrivilegeEscalation != false
          msg := "Container must set allowPrivilegeEscalation to false"
        }

kubectl apply -f security-context-template.yaml

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sSecurityContext
metadata:
  name: security-context-required
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    excludedNamespaces: ["kube-system", "kube-public", "kube-node-lease", "gatekeeper-system"]
  parameters:
    runAsNonRoot: true
    readOnlyRootFilesystem: true
    allowPrivilegeEscalation: false

kubectl apply -f security-context-constraint.yaml

Test policy enforcement with compliant workloads

Create a deployment that meets all security requirements:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: secure-app
  namespace: test-restricted
spec:
  replicas: 2
  selector:
    matchLabels:
      app: secure-app
  template:
    metadata:
      labels:
        app: secure-app
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 1001
        fsGroup: 2001
      containers:
      - name: app
        image: docker.io/library/nginx:1.25
        ports:
        - containerPort: 8080
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 1001
          capabilities:
            drop:
            - ALL
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "200m"
            memory: "256Mi"
        volumeMounts:
        - name: tmp
          mountPath: /tmp
        - name: cache
          mountPath: /var/cache/nginx
      volumes:
      - name: tmp
        emptyDir: {}
      - name: cache
        emptyDir: {}

kubectl apply -f secure-deployment.yaml

Monitor policy violations and audit logs

Check Gatekeeper violations and constraint status:

kubectl get constraints
kubectl describe k8srequiredresources must-have-resources
kubectl describe k8sallowedrepos repo-must-be-trusted

View audit logs for policy violations:

kubectl logs -l control-plane=audit-controller -n gatekeeper-system
kubectl get events --field-selector reason=FailedCreate

Set up monitoring with Prometheus

Configure monitoring to track policy violations. If you have Prometheus installed, you can link it to existing monitoring tutorials like monitor Kubernetes cluster with Prometheus Operator.

apiVersion: v1
kind: Service
metadata:
  name: gatekeeper-audit-metrics
  namespace: gatekeeper-system
  labels:
    app: gatekeeper-audit
spec:
  ports:
  - name: metrics
    port: 8888
    targetPort: 8888
  selector:
    control-plane: audit-controller

kubectl apply -f gatekeeper-metrics-service.yaml

Create policy exception workflows

Configure exemptions for specific workloads that need elevated privileges:

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sSecurityContext
metadata:
  name: security-context-with-exemptions
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    excludedNamespaces: ["kube-system", "kube-public", "kube-node-lease", "gatekeeper-system", "monitoring"]
  parameters:
    runAsNonRoot: true
    readOnlyRootFilesystem: false
    allowPrivilegeEscalation: false

You can also exempt specific workloads by labels:

kubectl label namespace monitoring policy-exemption=true