Technology · PRACTICAL GUIDE
Python subprocess timed out but children keep running? Audit the process group and pipes
Trace the direct child, descendants, session, process group and inherited pipe writers before choosing terminate, group signaling or recursive cleanup.
A timeout proves only that one waiting operation exceeded a deadline. It does not prove that the work equals one PID, that every descendant stayed in one group, or that captured stdout and stderr have no other writers. Those are separate ownership questions.
The safe repair is to label one launch, capture the topology and pipe state, then use the narrowest platform primitive that owns the work. Escalate only after a bounded grace period and verify both process exit and pipe closure through the real launcher.
Freeze the timeout evidence before cleanup
Add a launch token and record the wrapper PID, direct child PID, PPID chain, executable, command, start time, session ID, process-group ID and whether shell=True or capture_output was used. Capture this before killing the leader because reparenting can erase the original ancestry.
Record separately whether the leader is alive, which descendants remain and whether communicate is waiting on an open pipe. A dead direct child with a blocked reader is a pipe-ownership failure, not proof that Python ignored the timeout.
Choose the ownership unit, not just a PID
If the launched program is truly one process, the direct-child timeout contract may be enough. If it launches workers, shells or build tools, define whether your application owns a POSIX process group, a recursive descendant set, a Windows job or only the direct child.
Do not signal a group derived from an unverified PID. Record the group leader and membership at launch, and keep unrelated long-lived services outside that group. A daemon that calls setsid or otherwise escapes requires a different supervisor boundary.
Use a POSIX group only when you created it
For a command whose whole tree belongs to this request, start_new_session=True can create a fresh session and group before exec. On timeout, send the chosen graceful signal to the verified group, wait for a bounded grace period, then escalate remaining group members and reap the direct child.
Avoid preexec_fn in threaded applications for this job; the documented start_new_session and process_group parameters exist to cover supported setup cases. Recheck membership because descendants may deliberately create another session and leave the group.
Treat inherited pipes as a separate shutdown gate
Captured pipes close only after every process holding a writer closes it. shell=True, worker pools and helper programs can leave a grandchild with stdout or stderr even after the immediate child is dead, so a second communicate call can still wait.
Preserve partial TimeoutExpired output, close or terminate the proven writers, and then drain the pipe. Do not solve a pipe hang by discarding all output unless that loss is an explicit product decision with its own test.
Use a platform branch and prove the result
On platforms or launchers without a suitable owned group, an optional psutil path can snapshot recursive descendants, terminate them, wait, then kill survivors while checking process identity. Its documentation warns that ancestry can be lost if an intermediate process disappears, so capture early.
Replay the real shell, service, CI runner or worker launcher. Force timeout, then prove the direct child and owned descendants are gone, captured pipes close within the bound, partial output is handled as designed, the exit is reaped and a sentinel unrelated process remains alive.
