Running as root means if an attacker escapes the container, they have root access to the host. Risks include container breakout to host filesystem, binding to privileged ports, and capability escalation. Fix by: setting runAsNonRoot: true in pod security context, using a non-root user in the Dockerfile (USER instruction), and ensuring the image builds with a non-root user. Also set allowPrivilegeEscalation: false and drop all capabilities with capDrop: ALL.

First, stop the bleeding: block the vulnerable image version in your CI/CD admission control (OPA Gatekeeper or Kyverno). Identify all affected services via your image registry tags and deployment inventory. Prioritize by exposure (internet-facing vs internal) and data sensitivity. Build and push fixed images for the highest-priority services, test, and deploy. For lower-priority services, schedule into sprint planning. Set up automatic vulnerability scanning on new image pushes to catch this earlier. Consider a "golden image" strategy where security hardens a base image centrally.

Use image signing and verification: sign images with Cosign or Notary during the build pipeline, then verify signatures at admission time using a policy controller (Kyverno or OPA Gatekeeper). Store signing keys in a KMS (AWS KMS, Google Cloud KMS, HashiCorp Vault). Enable admission control to reject unsigned images. Use short-lived tokens for CI/CD authentication rather than long-lived credentials. Audit all image pull events. Implement a software supply chain bill of materials (SBOM) to track what went into each image.

Seccomp restricts syscalls a container can make at the kernel level — the most granular control but requires knowing which syscalls an application needs. AppArmor works at the application level, restricting capabilities and file access paths — easier to use for known application profiles. SELinux works at the system level, labeling files and processes — most powerful but complex to configure. In practice: Docker defaults ship with a sensible seccomp profile blocking dangerous syscalls. For Kubernetes, seccomp via securityContext.seccompProfile and AppArmor via container.apparmor.security.beta.kubernetes.io are the common paths. SELinux is typically used at the host level.

Runtime detection tools like Falco monitor syscall behavior and flag anomalous activity: a shell spawning inside a container, unexpected network connections, writing to sensitive paths like /etc/ or /root/. Sysdig captures system calls for deeper analysis. Network monitoring detects exfiltration attempts via unusual outbound traffic. Integrate these with your SIEM or alerting system. Also monitor container restart counts, unexpected process trees (kubectl top pods showing unusual CPU), and node-level indicators like new SSH keys in /root/.ssh/.

Trivy is the default choice for most teams — it has a large vulnerability database, is fast, and integrates with most CI systems with minimal configuration. Grype is stronger when you need native SBOM support or want to scan SBOMs you already generated with Syft. Grype handles SBOM inputs directly while Trivy requires Syft as a separate step for SBOM generation. If your pipeline already generates SBOMs, Grype avoids adding another tool. If you want the simplest setup with the most baked-in integrations, Trivy wins.

Cosign generates a key pair, signs your container image digest, and stores the signature as an OCI artifact in your registry alongside the image. To enforce verification in Kubernetes, deploy Kyverno or OPA Gatekeeper as an admission controller. Create a policy that queries the registry for a valid Cosign signature before allowing the pod to start. If the image is not signed or the signature is invalid, the admission controller rejects the pod. Store the Cosign private key in a KMS (AWS KMS, Google Cloud KMS, HashiCorp Vault) — never in the pipeline itself. Rotation involves re-signing all images with the new key and updating the policy.

Identify the specific paths the application needs to write to at runtime. Common candidates are /tmp, /var/log, and /run. Create emptyDir volumes mount at those paths in your pod spec. Set readOnlyRootFilesystem: true globally, then selectively mount writable volumes to the specific locations the application requires. For log directories, mount an emptyDir or a mounted NFS share. This approach gives you the security benefit of a read-only root while allowing legitimate writes where needed. You can also use a tmpfs mount for temporary files.

Pod Security Standards define three levels — privileged, baseline, and restricted — that cluster operators can enforce cluster-wide. Pod Security Admissions (built into Kubernetes 1.25+) is the controller that enforces those standards at the namespace level via labels. PSPs were the previous mechanism but were deprecated because they required a mutating admission webhook and had security gaps. To enforce restricted PSS on a namespace, label it pod-security.kubernetes.io/enforce: restricted. The PSA checks runs before pods are scheduled, rejecting those that violate the policy. This replaces the old PSP webhook approach with something simpler and more maintainable.

Linux capabilities split the power of root into fine-grained units — CAP_NET_RAW lets a process send raw packets, CAP_SYS_TIME lets it set the system clock. Running as root in a container does not give all capabilities by default because Docker drops many, but not all. Dropping ALL capabilities and then adding back only what your application needs follows the principle of least privilege. To determine what your app needs: run it under seccomp with a permissive profile while monitoring which syscalls fire (use strace or Falco), then build a restrictive seccomp profile from that baseline. For capabilities, start with nothing and add them one at a time while testing functionality. CAP_NET_BIND_SERVICE is commonly needed for processes binding to ports below 1024.

Never bake secrets into images or pass them as plain-text environment variables — both end up in image layers and container logs. The Kubernetes-native approach is to use a secrets management tool like HashiCorp Vault with the Vault Secrets Operator or the CSI Secret Store driver, which mounts secrets as files in the container filesystem without ever exposing them as env vars. For Azure, use Key Vault with the AKV CSI provider. Alternatively, use Kubernetes external secrets with AWS Secrets Manager or GCP Secret Manager. The key principle: secrets should be injected at runtime from an external store, never baked into the image at build time.

First, isolate the affected pod — prevent it from scheduling new work while you investigate. Capture the running container state: docker inspect for the container config, docker diff to see filesystem changes from the image, and docker logs for stdout/stderr. Extract the container's process tree with docker top and network connections with docker exec netstat or similar. Take a snapshot of the container's memory with docker checkpoint if your runtime supports it. Pull the image and compare it to the expected image digest. Preserve logs and audit trails before letting the pod restart or scaling it out.

Network policies in Kubernetes are namespace-scoped and act as a firewall for egress and ingress traffic per pod. Label your namespaces and pods, then create a NetworkPolicy that selectors the appropriate pods. For a frontend-backend setup: the frontend policy allows ingress from the ingress controller only, the backend policy allows ingress from the frontend only. Common mistakes: forgetting that network policies are additive within a namespace (a pod with no policy is fully accessible), not accounting for DNS resolution (pods need to communicate with kube-system for DNS), and applying policies only to named namespaces without understanding that pods in unlabeled namespaces can still reach your services.

An SBOM (Software Bill of Materials) is a structured inventory of every package and dependency in your container image — the equivalent of an ingredient list. It matters because when a new vulnerability drops (like Log4Shell), you query your SBOM database to identify every affected image in minutes rather than scanning each one individually. To generate one: use Syft to scan your image and produce an SPDX or CycloneDX SBOM. Store the SBOM alongside the image in your registry or in a separate artifact store. In CI/CD, generate the SBOM after building the image, store it as a build artifact, and integrate it with your vulnerability scanner so Grype or Trivy can correlate CVE data with your exact package versions.

Start with the default Falco rule set and run in audit mode — Falco logs warnings but does not block. Collect a week of alerts and identify which rules fire hundreds of times per day. Those rules are noise in your environment. Suppress them by creating exceptions in falco config. Then identify which alerts represent genuine security signals by correlating with known incidents. Keep those rules. Review quarterly — rule effectiveness changes as your workload changes. The goal: security engineers should be able to investigate every Falco alert that fires in a day. If they cannot, you have too much noise and will miss real incidents.

hostPath volumes let a container read/write files on the host node's filesystem. An attacker who escapes the container and has access to hostPath-mounted directories can read sensitive host data, write cron jobs to the host, or modify kubelet configuration. Alternatives: use Kubernetes ConfigMaps for configuration files, Secrets for credentials (via CSI or Vault), emptyDir for temporary storage, or PersistentVolumeClaims with appropriate access modes for persistent data. If you must use hostPath for system-level access (like the node's crictl socket for a CNI plugin), restrict it with PSP or PSS to only the specific service accounts that need it, and document why it is required.

Never pull by tag alone — tag mutability means node:18-alpine today is not the same image in six months. Pin to a specific digest in your Dockerfile: FROM node@sha256:abc123.... Scan every image in CI before it is used, even if it comes from a trusted registry like Docker Hub. Use a VEX (Vulnerability Exploitability eXchange) document to communicate which CVEs in your dependencies are not exploitable in your context. For critical workloads, maintain a hardened "golden image" base that your security team audits and signs with Cosign. Pull from official sources only and verify the registry's image signature when available.

Build-time scanning (Trivy, Grype) catches known vulnerabilities in your dependencies and base image layers before they reach production. It is preventive and deterministic — given the same image, it produces the same results. Runtime monitoring (Falco) catches anomalous behavior that is not in any vulnerability database: misconfiguration attacks, zero-days, and attacker behavior specific to your environment. You need both. Build-time scanning prevents known CVEs from reaching production. Runtime monitoring catches everything else — the attacks that exploit misconfigurations or vulnerabilities that have no CVE yet. Without runtime monitoring, a zero-day exploit of a medium-severity CVE will sail through because no scanner flags it.

Layer ordering matters: put instructions that change most frequently at the end of the Dockerfile so that cache invalidation does not rebuild sensitive layers. Never put secrets in RUN commands — they appear in the layer history. Use multi-stage builds so the final image contains only the runtime artifacts, not the build toolchain (which may include source code or build secrets). Set appropriate file permissions in the Dockerfile (chmod only what is needed). Remove package manager caches, package lists, and temporary files in the same layer that installs them. Validate that the final image does not contain shell, package managers, or debugging tools unless explicitly needed at runtime.

In your CI pipeline, run trivy image --exit-code 1 --severity HIGH,CRITICAL after building and before pushing. If it exits with code 1, the build fails and the image is not pushed. Set appropriate severity thresholds — blocking on HIGH/CRITICAL is common; blocking on LOW/MEDIUM is too noisy for most teams. When a build fails due to a critical CVE, you have a few paths: update the affected dependency to a patched version (preferred), rebuild from a patched base image, apply a vulnerability exception with risk acceptance if the CVE is not exploitable in your context, or implement a compensating control like runtime monitoring to catch exploitation attempts. Never ignore critical CVEs without documented risk acceptance.