Fix: connect ETIMEDOUT

connect ETIMEDOUT means your process sent TCP SYN packets to the target and got nothing back before the timeout. Not an acceptance, not a rejection, nothing. This is a different animal from ECONNREFUSED, where the host actively answers that no one is listening. Silence means the packets are being dropped somewhere between you and the target, the target is unreachable, or the target is too overloaded to complete handshakes. Each of those has a distinct signature, and you can usually separate them with nc and two minutes of looking.

What this error means

In a Node.js stack trace it looks like this, and other runtimes and tools report the same condition as “Connection timed out”:

Error: connect ETIMEDOUT 10.0.1.23:5432
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1595:16) {
  errno: -110,
  code: 'ETIMEDOUT',
  syscall: 'connect',
  address: '10.0.1.23',
  port: 5432
}

A TCP connection starts with a three-way handshake: your side sends SYN, the target replies SYN-ACK, you send ACK. ETIMEDOUT means the SYN-ACK never arrived. The kernel retransmitted the SYN several times with growing delays, gave up, and returned the error. Note that DNS already succeeded, since the error carries an IP address. The question is purely why that IP and port do not answer, and firewalls that drop (rather than reject) are the most common answer in cloud environments, because dropping is the default behavior of security groups.

Common causes

• A security group, network ACL, or host firewall silently dropping the traffic, usually because the source IP or port is not in an allow rule.
• Wrong port in configuration, on a host whose firewall drops instead of rejecting unknown ports, so you get a timeout rather than a refusal.
• Unreachable host: the instance is stopped, the IP is stale after a redeploy, or there is no route (missing VPC peering, missing NAT gateway, wrong subnet).
• An overloaded upstream whose accept queue is full, so the kernel drops incoming SYNs during traffic peaks.
• Asymmetric routing or a middlebox (NAT, conntrack) dropping the return path, which looks identical from the client side.

How to fix it

STABILIZE first. If a production dependency is timing out, check for the trivial reversible causes before deep diagnosis: was a security group edited, did a deploy change the target IP, did the target instance stop. Reverting the last change to the path is often the fastest recovery. If the upstream is overloaded, reduce pressure by lowering client concurrency or failing fast, because piling retries onto a full accept queue makes it worse.

DIAGNOSE. First, reproduce the failure from the machine that is failing, with a bounded timeout:

nc -vz -w 5 10.0.1.23 5432

Three outcomes, three directions:

• Connection timed out: confirms the silent drop or unreachable host. Continue below.
• Connection refused: the host is up and reachable, nothing listens on that port. Check the port number and whether the service is running. This is a different error page.
• succeeded!: the path works now. Your problem is intermittent, skip to the overload checks.

For HTTP targets, curl gives the same split with more detail:

curl -v --connect-timeout 5 http://10.0.1.23:8080/health

Next, establish whether the host is reachable at all:

ping -c 3 10.0.1.23
traceroute -n 10.0.1.23

Ping is often blocked, so a failed ping proves nothing by itself, but a successful ping with a failed connect is informative: the host is alive and something is filtering that specific port. Traceroute that dies at a consistent hop shows where the path ends.

Then verify the target from the inside, if you can reach it another way (SSH, SSM, console):

sudo ss -ltn | grep 5432        # is anything listening on the port?
sudo iptables -L -n -v          # host firewall rules
netstat -s | grep -i listen     # "SYNs to LISTEN sockets dropped" and
                                # "times the listen queue of a socket overflowed"

If nothing is listening, fix the service or the port. If the listener exists and the overflow counters climb while you watch, the upstream is shedding connections under load. If the listener exists and counters are flat, the drop is in the network path: work outward through host firewall, security group, network ACL, and routing.

In AWS terms, the checklist is: the target’s security group must allow inbound on the port from the client’s security group or CIDR, both subnets’ NACLs must allow the traffic and the ephemeral return ports, and the route tables must connect the two subnets (peering, transit gateway, or IGW/NAT for public paths). VPC Reachability Analyzer will pinpoint the blocking component if manual review stalls.

FIX. Match the fix to what you isolated:

• Firewall or security group block: add an inbound rule on the target allowing the port from the client’s security group (preferred over raw IPs, since instances change). Remember NACLs are stateless and need the ephemeral return range open too.
• Wrong port: correct the configuration and add a startup connectivity check so a bad port fails the deploy instead of timing out in production.
• Unreachable host: start the instance, fix the stale IP by using a DNS name or service discovery instead of hardcoded addresses, or add the missing route or peering connection.
• Overloaded upstream: scale the service or make accept faster. Raise the listen backlog in the application and the kernel ceiling (net.core.somaxconn) if the backlog is genuinely too small, but treat rising overflow counters as a capacity signal first, not a tuning knob.
• Middlebox drops: check conntrack exhaustion (conntrack -S or the nf_conntrack count vs max) on NAT hosts and firewalls in the path, and raise the table size or reduce connection churn with keepalives.

Confirm the fix the same way you reproduced the failure, from the same source machine:

nc -vz -w 5 10.0.1.23 5432

How to prevent it

• Reference security groups, not IP addresses, in allow rules, so replacements and autoscaling do not silently fall outside the rules.
• Use DNS names or service discovery for dependencies, never hardcoded IPs that go stale on redeploy.
• Set explicit, short connect timeouts in clients (a few seconds), separate from request timeouts, so failures surface fast and retries are controlled.
• Monitor listen queue overflows and conntrack usage on busy services, and alert before they saturate.
• Test connectivity from the deployment environment in CI or a smoke test after infrastructure changes, since security group edits rarely announce themselves.

When to call a senior engineer

Call for help when the timeout is intermittent and load-correlated, when traffic crosses VPCs, VPNs, or NAT layers and no single team owns the whole path, or when packet captures are the next step and nobody on hand reads them fluently. Erzon engineers can trace the SYN from client to listener, find the hop that eats it, and fix the rule, route, or capacity problem so connections stop dying in silence.