Fix: ERR max number of clients reached

Redis returns ERR max number of clients reached when the number of connected clients hits the maxclients limit, so every new connection is accepted, told the error, and closed. Existing connections keep working, which makes this failure deceptive: long-lived services hum along while anything that reconnects, autoscales, or deploys fresh instances is locked out. The cause is almost never legitimate traffic. It is usually a connection leak in an application client or a file descriptor cap that quietly lowered the limit.

What this error means

Every Redis connection is one client and one file descriptor. The maxclients directive caps how many Redis will hold at once, with a default of 10000. When the cap is reached, new TCP connections are not queued; each one immediately receives the error and is disconnected:

(error) ERR max number of clients reached

There is a second, sneakier version of the same problem. At startup, Redis checks the process file descriptor limit. It needs maxclients plus 32 descriptors (the 32 are reserved for internal use such as persistence files and the event loop). If the OS limit is lower, Redis reduces maxclients to fit and logs it. A box with the common ulimit -n 4096 default runs with an effective maxclients of 4064, no matter what the config file says.

Unlike some databases, Redis reserves no superuser connection slot, so once the limit is hit you cannot open a fresh admin session either. You need an already-open connection, or you work from the application side.

Common causes

• A connection leak: application code creates Redis clients per request or per job and never closes them, so connections accumulate until the cap.
• Pool math that no longer adds up: each service instance holds a pool of, say, 50 connections, and autoscaling to 200 pods quietly demands 10000.
• Serverless functions (Lambda, Cloud Functions) opening a new connection per invocation with nothing to close them promptly.
• A low file descriptor limit that shrank the effective maxclients far below the configured value at startup.
• No timeout set (the default is 0, never disconnect idle clients), so dead and forgotten connections are kept forever.
• A crash-restart loop in a client service that abandons connections faster than TCP keepalive reaps them.

How to fix it

STABILIZE first. You need connections back, and you need an open session to work from.

1. If you still have any working connection (an app shell, an existing monitoring session), use it. Check the state:

redis-cli INFO clients
redis-cli CONFIG GET maxclients

connected_clients at or near maxclients confirms the diagnosis.

2. Raise the ceiling at runtime to restore service. This works up to what the current file descriptor limit allows.

redis-cli CONFIG SET maxclients 20000

If Redis refuses because the fd limit is too low, or you have no live connection at all, go to the leaking application instead: restart the worst offender service, which closes its leaked connections at the TCP level and frees slots within seconds. Restarting Redis itself also works but drops every client and, if persistence is off, the data.

3. Reap idle connections. Set an idle timeout so Redis closes clients that have sent nothing for 5 minutes:

redis-cli CONFIG SET timeout 300

DIAGNOSE. Find out who owns the connections.

redis-cli CLIENT LIST

Each line shows addr (the client IP and port), age, idle (seconds since last command), name, and cmd (last command run). Aggregate by IP to find the leaker:

redis-cli CLIENT LIST | awk '{print $2}' | cut -d= -f2 | cut -d: -f1 | sort | uniq -c | sort -rn | head

One host holding thousands of connections with high idle values is your leak. Kill its stale clients by address or one by one by id:

redis-cli CLIENT KILL ADDR 10.0.3.17:52344
redis-cli CLIENT KILL ID 4711

On Redis 7.4 and later you can also cull everything older than a threshold in one command:

redis-cli CLIENT KILL MAXAGE 3600

Check the Redis log for the startup warning about reduced maxclients. Confirm the process limit on the host:

cat /proc/$(pgrep -x redis-server)/limits | grep "open files"

FIX. Address both layers.

In the application, use one long-lived client or pool per process, created at startup, never per request. Cap the pool at a size that, multiplied by your maximum instance count, stays well under maxclients. In serverless code, create the client outside the handler so it survives across invocations. Name your connections so the next incident is diagnosable in seconds:

# in each service, at client init
CLIENT SETNAME orders-api

On the server, persist a deliberate limit and make the OS support it. In redis.conf:

maxclients 20000
timeout 300
tcp-keepalive 300

And in the systemd unit for redis-server:

[Service]
LimitNOFILE=65536

Then systemctl daemon-reload and restart Redis in a maintenance window. On managed services (ElastiCache, Azure Cache) maxclients is fixed per node size, so the fix is pool discipline plus a bigger node or a cluster.

How to prevent it

• Alert on connected_clients at 80 percent of maxclients so you act before the ceiling.
• Enforce pooling in a shared client wrapper: one pool per process, explicit max size, health checks on checkout.
• Do the multiplication whenever autoscaling limits change: pool size times max instances must stay under maxclients with margin.
• Keep timeout and tcp-keepalive set so dead peers and abandoned connections are reaped automatically.
• Set CLIENT SETNAME in every service so CLIENT LIST maps connections to owners instantly.
• Verify the effective maxclients in the startup log after every host or image change, not just the value in redis.conf.

When to call a senior engineer

Call for help when connections climb again after every restart and the leak hides somewhere in a fleet of services, when you cannot get any admin session into Redis to triage, or when fixing it properly means introducing pooling across a codebase that opens clients ad hoc. Erzon engineers can trace every connection to its owning service, patch the leaking client code, and set limits and alerts so the ceiling never surprises you again.