ArchiveSweep

Low-level design

OOP and SOLID class design for machine-coding LLD: interfaces, invariants, in-memory state, and concurrency you can implement and test.

• ArchiveSweep

  3 studies
  1. 1.ArchiveSweep (File Deduplication) LLD - Spec, Pipeline & ConceptsMachine-coding LLD: recursively scan a directory tree, skip symlinks, collapse hard links to one inode, then find byte-identical duplicate groups with a staged pipeline (size -> sample prefix -> streaming SHA-256 -> byte verify).
  2. 2.ArchiveSweep - Streaming SHA Solution & TestsFull runnable stdlib implementation: archive_sweep(root, workers) with scan, sample, stream hash, byte verify, ThreadPoolExecutor, and unittest coverage for duplicates, near-misses, symlinks, hard links, and empty files.
  3. 3.ArchiveSweep - Concurrency, Symlinks & Hardlink Edge CasesWhich stages parallelize safely, how to aggregate errors without losing groups, symlink policy, hardlink reclaim math, and how to explain the design without overselling hash equality.