Technology · PRACTICAL GUIDE
Python says “dictionary changed size during iteration”? Prove who changed it before copying or locking
Separate same-loop mutation, concurrent writers, re-entrant callbacks and runtime-owned registries, then choose a repair whose semantics you can test.
RuntimeError: dictionary changed size during iteration is a boundary signal: a live mapping was structurally modified while something was traversing it. The message does not tell you whether the mutator was the same loop, another thread, a callback, or a registry owned by Python or a dependency.
Preserve the traceback, prove the mapping owner and mutator, choose snapshot, split-phase mutation, shared synchronization, or an upstream fix, then validate data invariants under the failing interleaving.
1. Freeze the traceback and name the mapping
Keep the complete traceback, Python and package versions, thread or task name, and a stable description of the mapping. Identify whether the failing frame is your loop, deepcopy, weakref, sys.modules, an executor, or a dependency. Do not start by replacing every .items() call.
Record whether the code is iterating d, d.keys(), d.values(), or d.items(), and whether the structural change is an insertion or deletion. Updating the value for an existing key is a different semantic question, but callbacks executed during that update can still add or remove keys.
2. Reproduce one mutator class at a time
First run the loop with all callbacks and background writers disabled. If it still fails, inspect the loop body and functions it calls for delete, pop, clear, setdefault on a missing key, or insertion. If failure appears only under load, instrument every write path with mapping identity, operation, key hash, thread or task, and timestamp.
Do not assume a traceback in deepcopy or sorted means those functions created the mutation. They may be traversing a live mapping while another thread, weak-reference callback, cleanup hook, comparator, logger, signal handler, or destructor changes it.
3. Choose snapshot or split-phase mutation deliberately
For an application-owned mapping where a point-in-time view is acceptable, iterate list(d), list(d.items()), tuple(d), or d.copy() as appropriate. A shallow snapshot stabilizes the key traversal; it does not clone mutable values or make later compound reads consistent.
When the loop decides which keys to remove or add, prefer two phases: collect decisions from an unchanged view, then apply mutations afterward. For destructive consumption, a documented popitem loop can express the intended semantics more clearly than mutating a live view.
4. Make ownership real for concurrent or re-entrant code
If multiple threads may touch the mapping, route all reads that require consistency and every structural write through the same Lock or RLock, or move ownership to one thread and communicate with a queue. Creating a lock, or holding it only around the reader, does not synchronize writers that ignore it.
For callbacks and observers, do not hold an arbitrary lock while calling unknown user code. Take a bounded snapshot under the ownership boundary, release it, then invoke callbacks; or queue side effects for a second phase. Document whether callbacks see a snapshot or the newest state.
5. Treat runtime-owned registries as dependency incidents
If the traceback ends in sys.modules, weakref, ProcessPoolExecutor, a plugin manager, or another library-owned registry, first reproduce on a supported current release and inspect the project issue or changelog. Preserve a minimal reproducer and the exact fixed version. Avoid patching private dictionaries or swallowing RuntimeError and retrying an arbitrary side effect.
Acceptance should run a stress test that forces the old interleaving, verify no keys are lost or duplicated, check ordering assumptions explicitly, confirm callbacks run exactly once, and prove any snapshot staleness is intended. A run that merely avoids the exception is not enough.
