Tech · PRACTICAL GUIDE
Python log rotation lost lines? Prove the inode switch and record union
Use one rotation owner, audit the open descriptor against the pathname, and verify numbered records exactly once across the rotated and active files.
A successful logrotate exit and a newly created app.log do not prove that a Python process stopped writing to the renamed inode. WatchedFileHandler checks the target device and inode before each emit on Unix-like systems, so verify that behavior at the descriptor, pathname and record levels.
Use one rotation owner. Pair WatchedFileHandler with external rename-and-create rotation, or use one internal rotating handler, but do not stack both mechanisms. Then number records around the boundary and accept the result only when the old and new files form one exact sequence without gaps or duplicates.
1. Choose exactly one rotation owner
For a service whose logrotate policy renames app.log and creates a new app.log, attach one WatchedFileHandler to the logger. Do not also attach RotatingFileHandler or TimedRotatingFileHandler to that same destination. The community duplicate-record case is a configuration conflict, not a reason to add a second watcher.
On Windows, or on a filesystem whose device and inode semantics do not behave like the tested local Unix filesystem, do not assume this method applies. Run the acceptance test on the real mount and process account before deployment.
2. Capture the descriptor and pathname identity
Immediately before rotation, flush the handler and record os.fstat(handler.stream.fileno()) for the open descriptor plus os.stat(log_path) for the pathname. Device and inode should identify the same object. Also record the service user, parent-directory permissions and the intended owner and mode of the replacement file.
This baseline distinguishes a stale descriptor from a missing or inaccessible replacement path. The CPython issue shows that reopen can fail when the target directory disappears; a rotation command succeeding cannot guarantee the application will reopen successfully.
3. Rotate by rename and create, then emit once
Rename app.log to app.log.1 and create a fresh app.log with the reviewed ownership and permissions before the next application write. The logrotate manual documents create as the step that makes a new file immediately after rotation. Keep the rotated file available until the audit finishes.
Emit one numbered sentinel after rotation and flush. Then compare the new pathname inode with the handler descriptor. Pass this layer only when they match and the rotated file still has the pre-rotation inode. If the descriptor still matches app.log.1, stop and fix the path, permissions or filesystem behavior.
4. Prove every numbered record exactly once
Write a bounded sequence before and after the boundary, such as seq-001 through seq-003 before rotation and seq-004 through seq-006 after it. Read both files completely. The union must equal the expected set, every expected value must occur once, and no unrecognized value may appear.
The local CPython 3.12.14 fixture placed seq-001 through seq-003 in the renamed inode and seq-004 through seq-006 in the new inode. Missing and duplicate sets were empty; the post-emit descriptor inode equaled the new pathname inode. Preserve the fixture output with the deployment evidence.
5. Reject shortcuts that hide the boundary
Do not treat copytruncate as an equivalent lossless fix. logrotate explicitly warns that data may be lost in the interval between copying and truncating. It may be a compatibility choice for a program that cannot reopen, but its documented loss window must be accepted and tested as a separate design.
Do not compare only total line counts. One missing record and one duplicate record can leave counts unchanged. Do not delete the rotated file before reading it, and do not infer success from the new path alone.
6. Record the limits of the result
This test proves the observed rename-and-create boundary for the tested Python version, filesystem, service account and configuration. It does not prove future rotations, remote filesystem semantics, container volume behavior or Windows compatibility.
Repeat the bounded test after changing Python, the handler, logrotate policy, mount type, permissions or service account. For production monitoring, alert on reopen exceptions and unexpected descriptor-to-path mismatches, then preserve both files for incident analysis.
