Technology · PRACTICAL GUIDE
Linux says “fork: Cannot allocate memory”? Prove the failing boundary before rebooting
Free RAM is not a diagnosis. Preserve the failed fork context, prove that the PID namespace is alive, compare cgroup memory events and effective limits, then read the host commit ledger before adding swap or restarting.
Free RAM is not a diagnosis. Preserve the failed fork context, prove that the PID namespace is alive, compare cgroup memory events and effective limits, then read the host commit ledger before adding swap or restarting.
fork can return ENOMEM because kernel memory is tight or because the target PID namespace has lost its init process. A cgroup can hit memory.max even while the host still has available memory.
1. Freeze the exact fork failure and execution context
Record the timestamp, parent PID, service or container identity, exact command and the syscall errno before rebooting. Keep the first journal or trace line and the cgroup and namespace paths. A shell message alone does not identify which boundary rejected the child.
Do not begin with drop_caches, a global OOM trigger, unlimited memory or a host reboot. Those actions can remove the failing state, harm unrelated workloads and make a later success impossible to attribute.
2. Prove that the PID namespace can still create children
The fork manual documents a distinct ENOMEM branch when the init process of the target PID namespace has terminated. Compare the failing parent namespace links with a known-good process and verify that the namespace still has a live PID 1 before treating the message as host memory exhaustion.
If this branch is proven, repair the container or namespace lifecycle so a supervised init remains alive. Adding swap, raising MemoryMax or killing a random high-RSS process does not restore a dead namespace.
3. Audit the service cgroup and its ancestor limits
Resolve the failing PID to its cgroup, then capture memory.current, memory.max, memory.high, memory.swap.current and memory.events or memory.events.local before and after one controlled attempt. Read parent slices too: the effective limit may be stricter above the service.
A rise in max or oom without host-wide pressure is workload-local evidence. oom_kill can stay zero for allocations that fail without invoking the killer. Preserve the event delta instead of assuming that the absence of an OOM-kill line clears the cgroup.
4. Compare host commit and physical-pressure evidence
Read MemAvailable, swap state, CommitLimit and Committed_AS together with pressure and kernel logs. Under strict overcommit, allocation can fail when committed address space reaches the permitted ledger even if a free-memory snapshot looks comfortable.
Separate memory ENOMEM from process-count failures. RLIMIT_NPROC, threads-max, pid_max or pids.max normally lead fork to EAGAIN; record the actual errno before changing task limits. This prevents a second, unrelated limit from being tuned on guesswork.
5. Repair one boundary and repeat the same start
For a dead namespace, restore supervised namespace init. For a cgroup boundary, fix the leak or right-size the narrow unit and ancestor policy. For host commit or physical pressure, remove the proven consumer, change workload sizing, or add capacity under an explicit rollback plan.
Repeat the same start at least twice while watching the same namespace identity, cgroup counters and commit ledger. Accept only when the child starts, the expected supervisor owns it, no unexplained event counter rises, no hidden OOM kill occurs and the repair survives a normal restart.
