Restic reports missing packs: build a repair ledger before changing the repository

1. Stop mutations and preserve the first evidence

Disable scheduled backup, forget and prune jobs for this repository. Do not keep chipping away at a failed prune, manually delete a pack, rebuild an index, unlock an active process or start snapshot repair while the failure is still being classified. A lock error and a confirmed stale lock are separate questions.

Save the exact restic version, command, exit status, stdout and stderr. The first failure can contain a pack ID or backend error that later commands no longer display. Redact repository URLs, usernames and paths before posting logs publicly; never include passwords, tokens or cloud credentials.

Create a full repository copy if possible. The official minimum fallback is a copy of the index and snapshots folders so a failed repair can be rolled back. A second repository for new backups can reduce operational pressure, but it is not proof that the damaged repository is repaired.

2. Capture the machine summary without merging away stderr

Configure the repository and password through your existing secure method, then run the check appropriate to your cost and time budget. A default check validates structure; `--read-data` reads every pack and can incur substantial time and bandwidth. Subset checks are useful maintenance, not a substitute for full coverage when you need to close an incident.

Keep stdout and stderr separate because the documented JSON interface puts the final summary on stdout and error objects on stderr. One simple POSIX-shell capture is shown below. The exit-code line must immediately follow the check command.

```sh restic version > restic-version.txt restic check --json > check-summary.json 2> check-errors.jsonl printf '%s\n' "$?" > check-exit.txt ```

Confirm that `check-summary.json` contains one complete summary object. Retain `check-errors.jsonl` even if the summary looks sufficient; free-form message text may change, so automation should branch on documented fields and preserve the original error evidence for humans.

3. Build a read-only decision ledger

Save the complete code below as `restic_check_ledger.py`. It reads only the saved summary, refuses a final-component symlink or non-regular input and reads at most 1 MiB plus one sentinel byte before refusing oversize data. It explicitly decodes UTF-8 and rejects duplicate JSON keys. It does not read credentials, contact a backend or change the evidence or repository.

```python #!/usr/bin/env python3 """Validate a saved `restic check --json` summary and print a read-only ledger.""" import argparse import json import re import sys from pathlib import Path PACK_ID = re.compile(r"^[0-9a-f]{64}$") def unique_object(pairs): result = {} for key, value in pairs: if key in result: raise ValueError("duplicate JSON property") result[key] = value return result def load_summary(path: Path) -> dict: if not path.is_file() or path.is_symlink(): raise ValueError("input must be a regular file, not a symlink") with path.open("rb") as stream: raw = stream.read(1024 * 1024 + 1) if len(raw) > 1024 * 1024: raise ValueError("input exceeds the 1 MiB audit limit") try: value = json.loads(raw.decode("utf-8"), object_pairs_hook=unique_object) except (UnicodeDecodeError, json.JSONDecodeError) as exc: raise ValueError("input is not one complete UTF-8 JSON document") from exc if not isinstance(value, dict) or value.get("message_type") != "summary": raise ValueError("requires the stdout summary object from restic check --json") errors = value.get("num_errors") if "broken_packs" not in value: raise ValueError("broken_packs is required, including when it is null") packs = value["broken_packs"] if packs is None: packs = [] # Restic v0.19.1 encodes an empty Go slice as JSON null. index = value.get("suggest_repair_index") prune = value.get("suggest_prune") if type(errors) is not int or errors < 0: raise ValueError("num_errors must be a nonnegative integer") if not isinstance(packs, list) or any( not isinstance(item, str) or not PACK_ID.fullmatch(item) for item in packs ): raise ValueError("broken_packs must contain complete lowercase pack IDs") if len(packs) != len(set(packs)): raise ValueError("broken_packs contains duplicates") if type(index) is not bool or type(prune) is not bool: raise ValueError("repair suggestions must be booleans") return { "numErrors": errors, "brokenPacks": packs, "suggestRepairIndex": index, "suggestPrune": prune, } def classify(summary: dict) -> dict: if summary["brokenPacks"]: status = "BROKEN_PACKS" next_step = "follow the exact restic check instruction for repair packs" elif summary["suggestRepairIndex"]: status = "INDEX_REPAIR_SUGGESTED" next_step = "follow the exact restic check instruction for repair index" elif summary["numErrors"]: status = "OTHER_ERRORS" next_step = "stop and review saved stderr or ask the restic support channel" elif summary["suggestPrune"]: status = "PRUNE_OPTIONAL_AFTER_HEALTH_REVIEW" next_step = "treat prune as cleanup, not damage repair" else: status = "NO_ERRORS_REPORTED" next_step = "retain evidence and perform a representative restore rehearsal" return { "status": status, **summary, "mutationApproved": False, "nextStep": next_step, "requiredBeforeMutation": [ "preserve full stdout, stderr, exit status and restic version", "freeze backup, forget and prune jobs", "copy the repository or at least index and snapshots when possible", "investigate storage or hardware faults before repair", ], } def main() -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("summary", type=Path, help="saved stdout summary JSON") args = parser.parse_args() try: ledger = classify(load_summary(args.summary)) except (OSError, ValueError) as exc: print(json.dumps({"status": "INCONCLUSIVE", "reason": str(exc)})) return 2 print(json.dumps(ledger, indent=2, sort_keys=True)) return 0 if ledger["status"] == "NO_ERRORS_REPORTED" else 1 if __name__ == "__main__": raise SystemExit(main()) ```

Run `python3 restic_check_ledger.py check-summary.json > repair-ledger.json`. Exit 0 is reserved for `NO_ERRORS_REPORTED`; the actionable non-clean categories exit 1, and malformed or unsupported input exits 2 as `INCONCLUSIVE`. Saving stdout with shell redirection creates a new report; choose a new filename and keep the original files unchanged.

Restic v0.19.1's inspected check implementation initializes `broken_packs` as an empty Go slice that can encode as JSON `null`. The parser accepts `null` and `[]` as no listed broken packs, but rejects an absent key or malformed IDs. The 19 synthetic tests cover all five outcomes, these null/array forms, precedence, duplicate keys, UTF-16 refusal, bounded reads, symlinks, booleans, malformed IDs and CLI exits 0/1/2 with unchanged input bytes. Preserve the original check exit status separately. This is not a sandbox against path races or a live repository diagnosis.

4. Interpret the five ledger outcomes

`BROKEN_PACKS` means the documented summary lists complete damaged pack IDs. Preserve the list and follow the exact `repair packs` instruction from the same check/version. The official guide says this takes precedence over its generic index-repair section because pack repair also updates the index.

`INDEX_REPAIR_SUGGESTED` means check explicitly set `suggest_repair_index` and did not list broken packs in the saved summary. `OTHER_ERRORS` means the documented flags do not resolve the failure; inspect saved stderr and ask restic support rather than guessing a repair command.

`PRUNE_OPTIONAL_AFTER_HEALTH_REVIEW` is cleanup, not damage repair. Unreferenced packs can lead to a prune suggestion without being the same condition as missing needed packs. `NO_ERRORS_REPORTED` is the summary’s report, not a restore test or proof that every historical check covered all data.

Status | What it establishes | Mutation boundary BROKEN_PACKS | Check listed damaged pack IDs | Use only the exact repair-packs instruction after backup and fault review INDEX_REPAIR_SUGGESTED | Check requested index repair | Repair only after the preserved-copy gate OTHER_ERRORS | Failure is unresolved by documented flags | Stop and review stderr/support PRUNE_OPTIONAL_AFTER_HEALTH_REVIEW | Cleanup may be useful | Do not treat prune as repair NO_ERRORS_REPORTED | This check summary reports no errors | Run a representative restore rehearsal

5. Fix the cause before changing repository metadata

Repeated corruption points to storage, transport, memory, filesystem or hardware trouble until investigated. Repairing metadata while the backend continues truncating or changing files can damage the repository again. Record backend type, restic version and the first observable storage fault for support.

Do not infer that every unexpected-size pack should be deleted manually. Current official guidance tells users to follow `check` and `repair packs` when damaged packs are detected. Historical forum workarounds belong to their version and failure context, not a universal recipe.

If ongoing backups cannot pause, initialize a separate repository through your normal approved process and retain the damaged repository offline for investigation. Do not reuse its URL or credentials in public examples, and do not call the replacement a repair.

6. Choose only the branch that check actually requested

For listed damaged packs, use the `repair packs` command and IDs that the saved check output directs you to use. For an explicit index-repair suggestion without listed broken packs, use the documented index-repair path. Do not run both reflexively, and do not repair a healthy index “just in case.”

After the index is correct, rerunning the original backup tasks can restore missing blobs that still exist in the source data. This can also heal older snapshots that reference the same content. It cannot recreate source bytes that no longer exist, nor does a successful new snapshot prove all older snapshots are complete.

Re-run a structural check after the repair or backup-healing step. If errors remain, keep the next result as a new evidence set rather than replacing the original. A changed error set is progress evidence, not permission to erase the incident history.

7. Map affected files before accepting snapshot data loss

Use `restic repair snapshots --dry-run` only after the index is correct and backup healing has been attempted where possible. Review every `would save new snapshot` message and record the original snapshot, affected path, available source copy and decision owner.

The dry-run is an impact map. It does not repair data, and it does not approve `--forget`. Snapshot repair removes inaccessible content from rewritten snapshots; adding `--forget` also removes the original damaged snapshots. That is an explicit data-loss decision, not routine cleanup.

Pause if important paths are listed, the dry-run output is incomplete, source copies have not been searched, or no one is authorized to accept the loss. Preserve the old snapshots and ask the project’s support channel when the outcome is uncertain.

8. Prove the repaired path with both check and restore

The official closing check is a full `restic check --read-data` that ends with no errors. On a large or costly backend, plan the window and cost; a series of random percentage subsets does not guarantee eventual complete coverage.

Also restore a representative snapshot to a new empty target. Include at least one high-value directory, a large file and a small metadata-heavy tree, then compare application-level readability or independent hashes where available. Do not restore over the only source copy.

Close the ledger only when the final check, restore rehearsal, snapshot/path inventory and repository copy are all identified. Record what remains untested. “No errors” and “the files we need can be restored” are related but separate claims.

Frequently asked questions

Can I run prune to remove the error? Not when needed packs are missing or other errors remain. Prune is cleanup; follow the exact repair instruction from check first.

Does `repair index` recover missing file bytes? No. It corrects the index. Rerunning backups may reintroduce content that still exists at the original source.

Why save stderr if I have the JSON summary? The documented error objects are written to stderr and preserve failure detail. Branch on stable summary fields, but keep the original errors for diagnosis.

Is `repair snapshots --dry-run` destructive? The dry-run is for impact review. The later repair writes replacement snapshots, and `--forget` removes the originals; keep those decisions separate.

Does a clean structural check prove every pack byte is intact? No. Full pack verification requires `--read-data`; a representative restore adds a separate user-level recovery test.