Technology · PRACTICAL GUIDE
Python SpooledTemporaryFile touched disk or read empty? Audit rollover and offset
Separate hidden fileno rollover from an EOF cursor before handing a spooled upload to a library or subprocess.
SpooledTemporaryFile has two separate surprises: a tiny payload may touch disk because a consumer asks for fileno(), and a correct payload may arrive empty because the stream was handed over at EOF. Treat rollover and cursor position as separate axes.
Use only public operations to identify the rollover trigger, rewind before the handoff, choose a bytes-or-descriptor contract, and test both exact delivery and storage policy at the real consumer boundary.
1. Separate storage state from stream position
Capture the exact object handed to the consumer, the configured max_size, bytes written, current tell() value and whether any code requests fileno(). A spool can still be in memory while its position is at EOF, or it can be on disk while preserving that same EOF position. These are independent facts.
Do not use private _file or _rolled fields as a production contract. Reproduce with public operations, record temporary-directory activity in the test environment, and wrap only the consumer boundary you own so the call that requests a descriptor is visible.
2. Name the rollover trigger
Python documents three paths to disk: the size grows beyond a nonzero max_size, code calls fileno(), or code calls rollover() explicitly. The default max_size=0 does not mean “spill every write”; in current CPython it disables the size trigger, while fileno() and rollover() still force disk backing.
Audit the real library, framework or subprocess path instead of assuming the payload is too large. A descriptor-only API can call fileno() behind an innocent-looking handoff and roll over even a few bytes. If disk is forbidden, a descriptor consumer is incompatible with that policy.
3. Reset the public cursor before handoff
After writing, call seek(0) before a consumer reads the same file object. The CPython issue that reported empty subprocess stdin was resolved this way: the process received a valid stream whose position was already at the end. Rollover does not promise to rewind it.
For FastAPI UploadFile, use await upload.seek(0) in async code, or the underlying public file method in synchronous code when the API contract permits it. Record which component owns the final close and never perform a second blind read without an explicit rewind.
4. Choose a handoff contract
If the consumer accepts bytes, a pipe, BytesIO or normal read result can preserve a no-disk policy. If it requires a real descriptor, call rollover() deliberately, confirm temporary space and permissions, seek(0), hand off the object, then close it in the documented owner.
Do not convert the spool to a permanent file by reading private attributes. When durable output is required, copy to a caller-selected destination with an explicit name, fsync or atomic-replace policy as needed; a temporary file can disappear when closed and is not a persistence interface.
5. Test both axes at the real boundary
Write a distinctive payload, assert tell() after the write, rewind, invoke the real library or subprocess, and verify the exact bytes or parsed result. Run one below-threshold case and one forced-rollover case so behavior is not inferred from size alone.
Add a test double that records calls to fileno() and a temporary-directory observation in an isolated environment. The acceptance gate is two-part: the consumer receives the complete payload, and disk use matches the declared policy. Close the spool and assert cleanup separately.
