Tech · PRACTICAL GUIDE
OpenSSH ControlMaster stuck? Prove the socket before deleting it
Resolve the effective ControlPath, ask the master whether it is alive, separate stale paths from hung transports and path failures, then repair only the proven boundary.
A ControlMaster failure is not one problem. The path can resolve somewhere unexpected, exceed a Unix-domain socket limit, live on a filesystem that cannot host the socket correctly, name a dead master, or still reach a local master whose remote transport is hung. Deleting the pathname first destroys evidence and can race a live process.
Use an evidence-first sequence: freeze the effective configuration with ssh -G, validate the private ControlPath and its length, ask that exact socket with ssh -O check, then choose exit, unlink, relocation or keepalive repair from the observed boundary. Finish by proving a new master is reused by a second client.
1. Freeze the alias and effective configuration
Start with the same alias and command context that failed. Run ssh -G alias and record hostname, user, port, proxyjump, controlmaster, controlpath and controlpersist. OpenSSH applies command-line settings first, then user configuration, then the system file, and uses the first value obtained for most parameters; inspecting only the last matching stanza can therefore identify the wrong socket.
Do not expand a different alias by hand. Tokens such as %h, %p, %r and %C are resolved from the effective connection, so two names that appear equivalent can still map to different masters. Keep the captured output with the incident evidence.
2. Validate the path before testing liveness
Confirm that the resolved ControlPath sits in a directory owned by the SSH user and not writable by other users. Keep it unique with %h-%p-%r or, when length is a concern, %C. Check every parent directory for search permission and the final directory for write permission; a missing or unwritable parent is a path failure, not a stale socket.
Measure the encoded pathname. Linux exposes a 108-byte sun_path array for pathname sockets, while portable implementations can be shorter, so leave margin rather than targeting one exact maximum. A short private runtime directory avoids deeply nested home, container or CI workspace paths without weakening ownership.
3. Ask the exact master
Run ssh -O check alias using the frozen configuration. If you must test a captured path directly, use ssh -S "$socket" -O check host with the same effective host, user and port. A successful response identifies a running master and normally reports its process ID; save that output before changing anything.
Classify the result instead of matching one error string across every OpenSSH build. No pathname points back to expansion or setup. A pathname that refuses or cannot contact a master is a stale-candidate, but first verify the owner, file type, alias and any process that still has the socket open. A successful check moves the investigation to transport health, not deletion.
4. Separate a stale pathname from a hung transport
If the master answers and you intend to stop it, use ssh -O exit alias and confirm that the process and pathname disappear. Do not unlink a pathname under a responsive master: existing clients may continue while new ones create a competing master. If no master answers, the path belongs to the expected user and no matching process or open socket remains, archive the evidence and remove only that proven stale pathname.
A responsive control socket proves only the local control channel. It does not prove that the master can still exchange traffic with the server. Configure ServerAliveInterval together with ServerAliveCountMax when a bounded disconnect is required; unlike TCPKeepAlive, these encrypted protocol messages detect a server that stops responding through an otherwise established transport.
5. Repair path and filesystem placement
If creation fails on an overlay or container workspace, reproduce once in a short private runtime directory on a local filesystem. One documented CI case succeeded after moving ControlPath away from overlayfs. Treat that as a placement diagnosis, not proof that every overlay behaves the same. Preserve %C or another destination-unique token so different hosts, ports and users cannot collide.
Prefer a user-owned 0700 directory such as a suitable $XDG_RUNTIME_DIR subdirectory. If /dev/shm is used for a bounded test, create a private subdirectory and verify ownership and mode; never place predictable bare sockets in a shared world-writable directory. Record cleanup behavior for ControlPersist and for abrupt job termination.
6. Prove repair and define the limit
Start one verbose connection and capture the line showing multiplexing setup. In a second process, connect through the same alias and verify reuse, then run ssh -O check again. Compare a fresh ssh -G capture with the original so the acceptance result proves the intended path and tuple, not an accidental fallback to a non-multiplexed session.
Accept the repair only when the private path is creatable, the exact master answers, a second client reuses it, exit or persistence behaves as designed, and the remote command succeeds. This workflow covers pathname-based ControlMaster sockets and sampled OpenSSH clients; abstract sockets, platform-specific launchers, security modules, proxies and vendor patches may require additional evidence.
