Home / Tutorials
TutorialsThe exact error, and how to fix it.
Real production error messages, one per page, each with the real cause and a step-by-step fix from the engineers who handle these for a living. Search the exact text of your error. If you are mid-incident, every page tells you what to do first and when DIY stops being safe.
Database
canceling statement due to statement timeout A query ran longer than statement_timeout allows, so PostgreSQL killed it. The timeout is doing its job. The real question is why the query got slow.
Fix this error → PostgreSQL · Degradedcould not resize shared memory segment ... : No space left on device Parallel query workers exchange data through POSIX shared memory in /dev/shm, and the container gave them only 64MB. Grow /dev/shm or shrink the per gather memory.
Fix this error → Redis · DegradedCROSSSLOT Keys in request don't hash to the same slot In Redis Cluster, multi-key commands need every key in the same hash slot. Use hash tags to co-locate related keys, or avoid cross-slot multi-key ops.
Fix this error → Redis · Production downERR max number of clients reached Redis hit its maxclients limit and every new connection is turned away. Here is how to kill the stale clients, find who leaked them, and raise the ceiling safely.
Fix this error → MySQL · Production downERROR 1040 (HY000): Too many connections MySQL has hit max_connections and is rejecting clients. Get in on the reserved slot, kill the leak, then size the pool right.
Fix this error → MySQL · Auth blockedERROR 1045 (28000): Access denied for user MySQL matched your connection to a user@host account and the credentials failed for that pair. The host half of the account is the part everyone forgets to check.
Fix this error → MySQL · Production downERROR 1114 (HY000): The table is full Writes are failing because something ran out of room: the disk, a tablespace, or a temporary table in memory. Find which one before you delete anything.
Fix this error → MySQL · DegradedERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction A transaction waited past innodb_lock_wait_timeout for a row lock another transaction is holding. Find the blocker, then keep transactions short.
Fix this error → MySQL · DegradedERROR 1206 (HY000): The total number of locks exceeds the lock table size One statement tried to lock more rows than MySQL had room to track. Batch the write, grow the InnoDB buffer pool, or convert MyISAM tables to InnoDB.
Fix this error → MySQL · DegradedERROR 1213: Deadlock found when trying to get lock; try restarting transaction Two transactions locked rows in opposite orders and neither could proceed, so InnoDB killed one. The deadlock report names the exact queries and locks involved.
Fix this error → MySQL · DegradedERROR 1226 (42000): User has exceeded the 'max_user_connections' resource One database account has hit its personal connection cap. The server may be fine; that user cannot open another session until existing ones close or the limit is raised.
Fix this error → MySQL · DegradedERROR 2006 (HY000): MySQL server has gone away The connection to MySQL was dropped mid-query. Usually a timeout, an oversized packet, or a server that restarted. Diagnose before you retry.
Fix this error → MySQL · Production downERROR 2013 (HY000): Lost connection to MySQL server during query The client was mid query when the connection vanished. A crashed server, a fired timeout, an oversized packet, and a network drop all produce this same message. The server error log tells them apart.
Fix this error → PostgreSQL · Production downERROR: could not extend file "base/16384/24576": No space left on device The data volume is full and PostgreSQL cannot write. Free space carefully, because deleting the wrong file corrupts the cluster.
Fix this error → PostgreSQL · DegradedERROR: could not serialize access due to concurrent update Two transactions touched the same rows under a strict isolation level, and PostgreSQL aborted one to protect consistency. The fix is a retry loop, not a config change.
Fix this error → PostgreSQL · DegradedERROR: deadlock detected Two transactions each hold a lock the other needs. PostgreSQL killed one to break the cycle. Here is how to stop the cycle forming.
Fix this error → MySQL · Production downERROR: Got error 28 from storage engine Error 28 is the operating system's ENOSPC: a disk is full. The culprit is the data directory or the tmpdir where big sorts and temp tables spill.
Fix this error → PostgreSQL · Production downFATAL: remaining connection slots are reserved for non-replication superuser connections Every ordinary connection slot is taken. Your app cannot connect at all, and the usual cause is a connection leak or missing pooler, not a database failure.
Fix this error → PostgreSQL · Auth blockedFATAL: role "app" does not exist The server you reached has no role by that name. Either the role was never created on this cluster, or you are connecting to a different cluster than you think.
Fix this error → PostgreSQL · Production downFATAL: sorry, too many clients already PostgreSQL has hit max_connections and is refusing every new connection. Restore service, then find the leak.
Fix this error → PostgreSQL · DegradedFATAL: terminating connection due to conflict with recovery A hot standby cancelled your query because replaying WAL would have removed rows the query still needed. Choose between replica lag, primary bloat, or moving the query.
Fix this error → MySQL · DegradedGot a packet bigger than 'max_allowed_packet' bytes A single statement or row exceeded the packet size limit, so MySQL refused it. The limit exists on both server and client, and both must be raised for the fix to stick.
Fix this error → Redis · DegradedLOADING Redis is loading the dataset in memory Redis restarted and is reading its dataset back into memory, refusing commands until it finishes. The bigger question is why Redis restarted and how long the load will take.
Fix this error → Redis · Production downMISCONF Redis is configured to save RDB snapshots but is currently not able to persist on disk The last background save failed, so Redis stopped accepting writes on purpose. Find why BGSAVE fails (disk, memory, or permissions) and writes come back.
Fix this error → Redis · Production downOOM command not allowed when used memory > 'maxmemory' Redis hit its maxmemory limit and the noeviction policy is rejecting every write. Here is how to free memory now and stop it recurring.
Fix this error → PostgreSQL · Degradedout of shared memory / increase max_locks_per_transaction The shared lock table is full. Some transaction is taking far more object locks than the server was sized for, and partitioned tables are the usual suspect.
Fix this error → Redis · Production downREADONLY You can't write against a read only replica Your application is writing to a Redis replica, which refuses by design. After a failover this means clients are still pointed at the old master. Fix the routing, not the replica.
Fix this error → PostgreSQL · DegradedSSL SYSCALL error: EOF detected The encrypted connection to PostgreSQL died mid conversation. The client only sees EOF; the real story is in the server log, the kernel log, or the network path between them.
Fix this error →Operating system
bash: fork: retry: Resource temporarily unavailable The kernel is refusing to create new processes. You have hit a process, thread, or PID limit. Here is how to get a shell back and find the cause.
Fix this error → Linux · DegradedENOSPC: System limit for number of file watchers reached A dev server or watcher asked the kernel to watch more files than inotify allows, so it fails with a misleading ENOSPC. Raising the watch limit fixes it.
Fix this error → Linux · Production downerror while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory A required shared library is missing or not on the dynamic loader's path, so the binary cannot start at all. Find the library, install it, and refresh the cache.
Fix this error → Linux · Production downfork: Cannot allocate memory The kernel refused to fork a new process, so even simple commands fail. It is memory or PID exhaustion, and you may need an existing shell to dig your way out.
Fix this error → Linux · Production downNo space left on device The volume is full, or you are out of inodes. Find the space before you start deleting, because deleting the wrong thing makes it worse.
Fix this error → Linux · Production downOut of memory: Killed process 12345 (mysqld) total-vm:..., anon-rss:..., file-rss:... The kernel ran out of memory and killed a process to survive. Find what consumed the RAM before it happens again.
Fix this error → Linux · Production downSegmentation fault (core dumped) The kernel killed the process for touching memory it does not own. The crash is in the core dump, and reading its backtrace beats guessing every time.
Fix this error → Linux · DegradedToo many open files A process ran out of file descriptors, so every new connection or file open fails. Raising the limit buys time; finding what holds thousands of descriptors is the fix.
Fix this error → Linux · Production downtouch: cannot touch 'file': Read-only file system Writes are failing because the filesystem is mounted read-only, and on a production box that usually means the kernel demoted it after a disk error. Check dmesg before remounting.
Fix this error →Cloud
403 AuthorizationFailure (Azure Storage) The storage account rejected an authenticated request, most often because firewall or private endpoint network rules block the caller, not because the credentials are wrong.
Fix this error → Azure · Degraded429 TooManyRequests (Azure) Azure is throttling your requests at some layer: ARM, a service limit, or Cosmos DB request units. The Retry-After header tells you when to come back; the layer tells you what to fix.
Fix this error → Google Cloud · Degraded429: Rate Limit Exceeded (GCP) You exceeded a Google Cloud quota or per resource rate limit. The fix is three steps: back off and retry, find the exact quota in the console, then raise it or spread the load.
Fix this error → Azure · Production down502 Bad Gateway (Azure App Service) The App Service front end forwarded the request to your app and got nothing valid back. The app crashed, hung past the platform timeout, or never came up after a restart or swap.
Fix this error → AWS · Production down503 Service Temporarily Unavailable (ALB: no healthy targets) The load balancer is up but every target in the group failed health checks, so there is nowhere to send traffic. Fix one target and service returns; the health check reason tells you what to fix.
Fix this error → Google Cloud · Production down503: The service is currently unavailable (GCP) Something between the caller and the workload refused or failed the request. The first question is whose 503 it is: Google's API, your load balancer, or your own backend.
Fix this error → AWS · Auth blockedAccessDenied: User is not authorized to perform this operation Some layer of AWS policy evaluation denied the call. The error names the principal and action; the fix is finding which of the five policy layers said no.
Fix this error → AWS · Deploy blockedCannotPullContainerError: pull image manifest has been retried 5 time(s): failed to resolve ref ECS cannot pull your container image. Almost always the image reference, the network path to the registry, or the execution role. Here is how to tell which.
Fix this error → Google Cloud · Production downcould not connect to server: Connection refused Is the server running on host "127.0.0.1" and accepting TCP/IP connections on port 5432? Connection refused means nothing was listening where the app knocked. With Cloud SQL that is almost always the Auth Proxy not running, IAM blocking it, or the network path to the instance. Here is the checklist.
Fix this error → AWS · DegradedEncoded authorization failure message: <encoded blob> AWS hid the real denial reason inside an encoded blob. Decode it with sts decode-authorization-message and the JSON names the exact statement, action, and resource that said no.
Fix this error → Azure · Production downError 40613: Database 'mydb' on server 'myserver' is not currently available. Please retry the connection later. Azure SQL says the database is temporarily unavailable. Often a transient failover, sometimes a real resource cap. Retry correctly, then find out which.
Fix this error → AWS · DegradedExpiredToken: The provided token has expired. Temporary credentials from STS, SSO, or an assumed role hit their expiry. The quick fix is to re-login or re-assume; the real fix is letting the SDK refresh credentials instead of caching a token.
Fix this error → Google Cloud · Degradedgoogleapi: Error 403: Quota exceeded for quota metric 'Queries' and limit 'Queries per minute' of service 'compute.googleapis.com' for consumer 'project_number:123456789012', rateLimitExceeded Google rejected the call because the project ran into a quota. It is either a per-minute rate quota or a hard allocation quota, and the fix is different for each. Here is how to tell them apart and clear both.
Fix this error → AWS · DegradedInsufficientInstanceCapacity AWS temporarily has no physical capacity for that instance type in that Availability Zone. This is not your quota; the fix is another AZ, an equivalent type, mixed instances, or a capacity reservation.
Fix this error → AWS · DegradedInvalidClientTokenId: The security token included in the request is invalid. AWS does not recognize the access key ID in the request. The cause is a deleted, disabled, or wrong-partition key, or stale credentials, and the fix starts with confirming who you actually are.
Fix this error → AWS · DegradedMalformedPolicyDocument: ... AWS rejected a policy because its JSON is structurally invalid: a bad ARN, missing Version, wrong Principal, or a condition typo. Validate it with the policy tools and the message points to the field.
Fix this error → AWS · DegradedOptInRequired: You are not subscribed to this service. AWS is blocking the call because something is not opted in: a disabled region, a missing Marketplace or service subscription, or an account that is not fully activated. Enable the right one and retry.
Fix this error → Google Cloud · Auth blockedPERMISSION_DENIED: The caller does not have permission The API authenticated the caller and then refused it. The trap is that the caller is often not the identity you think it is. Here is how to identify the real principal, the missing permission, and the layer that blocked it.
Fix this error → AWS · DegradedProvisionedThroughputExceededException: The level of configured provisioned throughput for the table was exceeded DynamoDB is throttling you. Sometimes it is real capacity, but very often it is a hot partition. Here is how to tell and fix both.
Fix this error → AWS · DegradedRequestTimeTooSkewed: The difference between the request time and the current time is too large Your machine's clock is off from AWS by more than 15 minutes, so SigV4 signature validation fails. Sync the host clock with NTP and the calls start working again.
Fix this error → AWS · DegradedSlowDown: Please reduce your request rate. (S3 503) S3 is throttling you, usually because too many requests hit the same key prefix at once. Retries with backoff stabilize it, key design fixes it.
Fix this error → Azure · DegradedSNAT port exhaustion Your app's outbound connections are failing intermittently because it ran out of SNAT ports. The cure is connection reuse plus a real outbound path like NAT Gateway, not a restart.
Fix this error → AWS · Deploy blockedStack:arn:... is in ROLLBACK_COMPLETE state and can not be updated. A stack whose very first create failed lands in ROLLBACK_COMPLETE, which is a terminal state you cannot update. The fix is to correct the template, delete the stack, and recreate it.
Fix this error → AWS · DegradedTask timed out after 3.00 seconds (Lambda) Your function hit its configured timeout and Lambda killed it mid execution. The default is only 3 seconds; the fix is finding what it was waiting on, not just raising the limit.
Fix this error → AWS · Production downThe instance could not be created because storage is full (RDS) Your RDS instance has no free storage, so writes are failing or the instance sits in STORAGE_FULL. Adding allocated storage is the fix; knowing what filled it prevents the rerun.
Fix this error → Google Cloud · Production downThe user-provided container failed to start and listen on the port defined by the PORT environment variable Cloud Run started your container but nothing began listening on $PORT before the startup deadline. Usually the app binds the wrong port or address, or crashes during boot.
Fix this error → AWS · DegradedThrottlingException: Rate exceeded AWS is rate-limiting your API calls. The fix is correct exponential backoff plus jitter, and reducing the call rate at the source.
Fix this error → AWS · Deploy blockedUPDATE_ROLLBACK_FAILED An update failed, then the rollback failed too. The recovery is continue-update-rollback, skipping only the truly stuck resources, then reconciling those by hand before deploying again.
Fix this error →Infrastructure as code
Error: Backend configuration changed Your backend block differs from what Terraform last initialized. You must re-init, but the wrong flag can strand your state. Here is when to reconfigure and when to migrate.
Fix this error → Terraform · Deploy blockedError: Cycle: ... Two or more resources reference each other, so Terraform cannot order them. Here is how to see the loop with terraform graph and break it, including the security group classic.
Fix this error → Terraform · Deploy blockedError: Error acquiring the state lock The remote backend lock is held by another run. Usually a crashed apply left it behind. Here is how to confirm nothing is live and release the lock without corrupting state.
Fix this error → Terraform · Deploy blockedError: Instance cannot be destroyed A resource has prevent_destroy = true and your plan would destroy or replace it. This guard usually protects databases. Here is how to find why, and whether to override it.
Fix this error → Terraform · Deploy blockedError: Invalid count argument ... the "count" value depends on resource attributes that cannot be determined until apply Your count comes from a value Terraform will not know until apply, such as the length of a computed list. count must be known at plan time. Here is how to base it on known inputs.
Fix this error → Terraform · Deploy blockedError: Invalid for_each argument ... the "for_each" value depends on resource attributes that cannot be determined until apply Your for_each keys come from a value Terraform will not know until apply, such as an ID from another resource. Keys must be known at plan time. Here is how to fix the map.
Fix this error → Terraform · Deploy blockedError: Provider produced inconsistent final plan The provider returned a value during apply that differed from the plan. This is almost always a provider bug, not your HCL. Here is how to work around it and report it.
Fix this error → Terraform · Deploy blockedError: Saved plan is stale Your saved plan file no longer matches current state, so Terraform will not apply it. Common in CI where plan and apply are separate stages. Here is how to regenerate cleanly.
Fix this error →Network
1024 worker_connections are not enough An Nginx worker ran out of connection slots and started dropping requests. Raising worker_connections is step one, but the OS file descriptor limit has to rise with it.
Fix this error → Nginx · Degraded413 Request Entity Too Large A request body was bigger than Nginx allows. The default client_max_body_size is 1 MB, and the limit can be set at several layers, so find the one that is actually rejecting you.
Fix this error → HTTP · Degraded429 Too Many Requests Something between your client and the origin decided you sent too many requests. The fix is finding which limiter fired, honoring Retry-After, and smoothing the bursts that trip it.
Fix this error → HTTP · Degraded499 Client Closed Request The client hung up before your backend finished answering. A 499 is never sent to anyone; it is Nginx telling you that responses are slower than callers are willing to wait.
Fix this error → Nginx · Production down502 Bad Gateway Nginx could not get a valid response from your backend. The 502 page is generic, but the Nginx error log names the exact failure. Start there.
Fix this error → HTTP · Degradedcertificate verify failed: self signed certificate in certificate chain Your client received a certificate chain ending in a CA it does not trust, almost always a corporate TLS proxy or a private CA. The fix is trusting that CA explicitly, never turning verification off.
Fix this error → HTTP · Degradedconnect ETIMEDOUT <ip>:<port> Silence, not rejection. Something between you and the target is dropping packets, or the host is gone. Refused and timed out point at different culprits.
Fix this error → HTTP · Production downERR_SSL_PROTOCOL_ERROR The browser and the server failed to complete a TLS handshake at all. Something on port 443 is not speaking valid, modern TLS, and openssl s_client will tell you what.
Fix this error → HTTP · Production downError 502 Bad Gateway (Cloudflare) Cloudflare reached out to your origin and got a broken or missing response. The branding on the error page tells you whose side failed, and it is usually the origin.
Fix this error → DNS · DegradedError: getaddrinfo ENOTFOUND <hostname> The hostname did not resolve to an IP address. It is a name problem, not a connection problem. Start with the hostname printed in the error, exactly as printed.
Fix this error → HTTP · Degradedhas been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource The browser made the request, got the response, and refused to hand it to your JavaScript because the server never said the origin was allowed. Fix is server side, always.
Fix this error → DNS · Production downNXDOMAIN DNS answered that the domain name does not exist at all. Either the record is genuinely missing, the wrong nameservers are answering, or a stale negative answer is cached.
Fix this error → HTTP · Degradedrecv() failed (104: Connection reset by peer) The remote end sent a TCP RST instead of closing cleanly. Somebody crashed, timed out, or reused a connection the other side had already abandoned.
Fix this error → Nginx · Degradedsocket() failed (24: Too many open files) while connecting to upstream An Nginx worker hit its open file descriptor limit and could not create a new upstream socket. Raise worker_rlimit_nofile and the systemd LimitNOFILE, not just worker_connections.
Fix this error → DNS · Production downstatus: SERVFAIL The resolver tried to answer and failed. Usually broken authoritative servers or a DNSSEC validation failure. The +cd flag tells you which in one query.
Fix this error → DNS · Production downTemporary failure in name resolution The host could not get an answer from any DNS resolver at all. This is not a missing record; it is a broken path between the machine and its resolver.
Fix this error → HTTP · Degradedupstream prematurely closed connection while reading response header from upstream Your app closed the connection before answering. Look for a crash, an OOM kill, a timeout shorter than the request, or a keepalive connection the proxy reused after the app already closed it.
Fix this error → Nginx · Degradedupstream sent too big header while reading response header from upstream Your backend answered, but its response headers were larger than the Nginx proxy buffer. Users see a 502, and the fix is either bigger buffers or smaller headers.
Fix this error →Scaling
0/3 nodes are available: 3 Insufficient cpu. preemption: 0/3 nodes are available The scheduler cannot fit your pod on any node because CPU requests exceed what is free. Fix the requests or add capacity.
Fix this error → Kubernetes · Production down0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims The pod cannot schedule because its PersistentVolumeClaim never bound to a volume. Fix the StorageClass or provisioner and the PVC binds.
Fix this error → Kubernetes · Production downBack-off restarting failed container app in pod ... reason: CrashLoopBackOff A container keeps crashing and Kubernetes keeps restarting it with growing backoff. CrashLoopBackOff is a symptom. Here is how to find the real crash.
Fix this error → Kubernetes · Production downCreateContainerConfigError The kubelet cannot build the container config because a referenced ConfigMap, Secret, or key is missing. The pod events name exactly what it cannot find.
Fix this error → Kubernetes · Production downErrImagePull The kubelet just failed to pull your container image and the pod cannot start. The Failed event under kubectl describe pod holds the exact registry error.
Fix this error → Kubernetes · Degradedexceeded quota: requests.cpu, requested: ... limited: ... The namespace ResourceQuota has no room left for the CPU your new pods request, so the API server refuses to create them. Deploys and scale-ups silently stall.
Fix this error → Kubernetes · Production downImagePullBackOff Kubernetes tried to pull the container image, failed, and is now backing off between retries. The pod events name the exact pull failure. Start there.
Fix this error → Kubernetes · DegradedReadiness probe failed: HTTP probe failed with statuscode: 503 The kubelet probed your health endpoint and got a 503, so the pod was pulled out of load balancing. The question is why your own app says it is not ready.
Fix this error → Kubernetes · DegradedState: Terminated Reason: OOMKilled Exit Code: 137 The container exceeded its memory limit and the kernel killed it. Exit code 137 means SIGKILL from an OOM. Here is how to right-size it.
Fix this error → Kubernetes · Production downThe node was low on resource: ephemeral-storage. Pod evicted The kubelet hit its disk pressure threshold and killed pods to reclaim space. Find what filled the node's disk before the evictions cascade to other nodes.
Fix this error → Nginx · Degradedupstream timed out (110: Connection timed out) while reading response header from upstream Nginx gave up waiting on the backend and returned 504. The timeout is the symptom. The fix is almost always upstream, not Nginx.
Fix this error →Authentication
AADSTS700016: Application with identifier 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' was not found in the directory 'contoso.onmicrosoft.com' Entra ID cannot find an app registration matching the client ID you sent, in the tenant you sent it to. Usually a wrong client ID or wrong tenant.
Fix this error → Authentication · Auth blockedinvalid_grant: The provided authorization grant is invalid, expired, or revoked The token endpoint rejected the credential you exchanged: a used or expired authorization code, a dead refresh token, a mismatched redirect_uri, or clock skew. Identify which grant type failed before touching anything.
Fix this error → Authentication · Auth blockedJsonWebTokenError: invalid signature The token's signature does not match what the verifying key computes, so the verifier and the signer disagree about the key. Decode the header, identify the signer, and compare keys.
Fix this error → Authentication · Auth blockedSSL certificate problem: unable to get local issuer certificate The client cannot build a trust chain to a known root. Almost always a missing intermediate cert or an outdated CA bundle. Do not just disable verification.
Fix this error → Authentication · Auth blockedTokenExpiredError: jwt expired The token is past its exp claim, so verification fails. Expiry is by design. The fix is correct refresh handling, not longer-lived tokens.
Fix this error → Authentication · Auth blockedx509: certificate has expired or is not yet valid A TLS certificate in the chain is expired or not yet valid, or the system clock is wrong. Confirm the real cause before you regenerate anything.
Fix this error → Authentication · Auth blockedx509: certificate signed by unknown authority The client cannot trace the server certificate to a CA it trusts. Either the server is not sending its full chain, or the client is missing the root. Do not disable verification.
Fix this error →About these tutorials.
Are these real error messages?
Yes. Every page is built around a verbatim error string you actually see in production, from PostgreSQL, MySQL, Linux, AWS, Azure, Kubernetes, and common auth and TLS stacks. Search the exact text of your error and, if it is here, you get the real cause and the real fix, not a generic "have you tried restarting" thread.
Can I follow these fixes myself?
That is exactly what they are for. Each page walks the fix step by step, in the order a senior engineer would work it: stabilize first, diagnose, fix properly, then prevent it recurring. Every page also marks the point where DIY stops being safe, so you know when to keep going and when to call.
My error is not listed. Can you still help?
Yes. The list grows from the failures we see most, so a gap is a useful signal, not a dead end. Paste your exact error into the contact form and a senior engineer will read it and reply within one business hour with a first diagnosis. Triage is free, whether or not the error ever becomes a page here.
Your error not here, or fix not holding?
Paste the exact error into the form and a senior engineer replies within one business hour. Triage is free.
Book a fix