Fix: CreateContainerConfigError

CreateContainerConfigError means the kubelet tried to assemble your container’s configuration and could not, because something the pod references does not exist. Almost always that is a ConfigMap, a Secret, or a specific key inside one that the pod expects through envFrom, env.valueFrom, or a volume. The container never starts, so there are no logs; the answer is in the pod’s events.

Warning  Failed  kubelet  Error: configmap "app-config" not found
Warning  Failed  kubelet  Error: couldn't find key DATABASE_URL in Secret prod/app-secrets

What this error means

Before the container runtime can create a container, the kubelet gathers everything the spec asks for: environment variables from ConfigMaps and Secrets, values pulled from specific keys, and mounted config volumes. If any referenced object is absent, or a named key is not present in an object that does exist, the kubelet cannot build a valid container config and marks the pod CreateContainerConfigError. This is strictly a configuration resolution failure that happens before the container is created, which is what separates it from CreateContainerError (the runtime failing to create an otherwise configured container) and CrashLoopBackOff (a container that started and then kept exiting). Because nothing ran, kubectl logs is empty and kubectl describe pod is where the real message lives.

Common causes

• The pod references a ConfigMap or Secret by a name that was never created, or was renamed, in this namespace.
• The object exists but a specific key named in valueFrom.configMapKeyRef or secretKeyRef is missing.
• The ConfigMap or Secret lives in a different namespace than the pod, so it is invisible to it.
• A typo in the ConfigMap or Secret name, or in the key name, in the pod spec.
• A deploy that shipped the workload before the ConfigMap or Secret it depends on, so ordering left the reference dangling.
• An invalid envFrom or valueFrom block that points at the wrong resource type.

How to fix it

Stabilize

1. If a rollout put the workload into this state, roll back to the last revision that started cleanly so traffic keeps flowing while you fix the reference:

kubectl rollout undo deployment/<name>

Diagnose

1. Read the pod’s events. The Failed event names the exact missing object or key:

kubectl describe pod <pod-name> -n <namespace>

2. Check whether the referenced ConfigMap and Secret actually exist in the pod’s namespace:

kubectl get configmap,secret -n <namespace>

3. Read the pod spec to see exactly what it references, so you match names and keys precisely:

kubectl get pod <pod-name> -n <namespace> -o yaml | \
  grep -A3 -iE 'configMapKeyRef|secretKeyRef|configMapRef|secretRef'

4. If the object exists, confirm the specific key the pod wants is present in it:

kubectl get configmap app-config -n <namespace> -o jsonpath='{.data}'
kubectl get secret app-secrets -n <namespace> -o jsonpath='{.data}'

Fix

1. If the object is missing, create it in the correct namespace. For a ConfigMap:

kubectl create configmap app-config \
  --from-literal=APP_ENV=production -n <namespace>

2. For a Secret with a missing key:

kubectl create secret generic app-secrets \
  --from-literal=DATABASE_URL='postgres://...' -n <namespace>

3. If the name or key was a typo, correct the reference in the pod or deployment spec so it matches the real object, then apply:

kubectl apply -f deployment.yaml

4. Once the object or key exists, delete the stuck pod so the controller recreates it and the kubelet can resolve the config:

kubectl delete pod <pod-name> -n <namespace>

How to prevent it

• Deploy ConfigMaps and Secrets before, or together with, the workloads that reference them, so an object is never missing at pod start.
• Keep a workload and its config in the same manifest or Helm chart so names and keys stay in sync.
• Validate references in CI (with a tool like kubeconform or a policy check) so a dangling ConfigMap or Secret reference fails before it reaches the cluster.
• Use consistent naming and avoid renaming a ConfigMap or Secret without updating every reference in the same change.
• Watch for the pod phase in your rollout checks so a CreateContainerConfigError blocks the deploy rather than surfacing in production.

When to call a senior engineer

Call for help when the referenced object clearly exists yet the pod still will not start, when namespace, RBAC, or secret injection tooling interact in confusing ways, or when a broken rollout has production down and you need it unblocked now. Erzon engineers can read the failure back to the exact missing reference, fix the config and the deploy ordering that caused it, and set validation so this error stops reaching your cluster.