ZJX 1.21 STREAMWALL: The Archive Stayed the Same. The Memory Profile Transformed.

What Happened

ZJX 1.21 STREAMWALL changes how the archive is produced and verified, not what the archive contains. The ZJX1 container, manifest, metadata profile, codec selection, archive bytes, and restored file semantics remain unchanged when limits are sufficient for the deterministic algorithm.

Silesia is the proof that this distinction matters operationally. The same 211.9 MB corpus that previously reached 8.58 GB RSS and failed without an archive now completes into a retained, byte-compatible 53.0 MB archive under a 1 GiB host ceiling and 768 MiB ZJX ceiling. Verification passes, and extraction restores the exact source tree while peaking at only 19.3 MB RSS.

That outcome moves Silesia from a scaling limit into an evidence product. Enterprise retention, large structured logs, repository history, and mixed-asset trees no longer need to justify an unbounded memory allowance to keep ZJX ' s compression behavior.

Why It Mattered

Archive trust is a resource question as much as a byte question. A technically impressive ratio means little if the operation that produces it can only run on hosts with unbounded memory or if verification itself needs the same large reservation. Operators need three properties at once: the archive is deterministic, the restore is exact, and the resource cost is explicit before the job starts.

STREAMWALL makes resource cost an admission decision. Every major allocation and spool write is checked against a governed working-memory limit and a planned spool envelope. If the work cannot fit, ZJX rejects it with the stable classifications MemoryBudgetExceeded or SpoolBudgetExceeded before host enforcement becomes the mechanism. That turns a potential process-kill into a legible policy event.

The practical consequence is that structured workloads become deployable memory stories. The twelve selected Greyforge lanes - telemetry, compact JSONL, sharded NDJSON, workflow artifacts, agent traces, repository fixtures, and Siglum portfolio scales - all reproduce their ZJX 1.2 digests under bounded ceilings while reporting materially lower memory profiles.

The Architectural Response

Bounded operation is not a single optimization. It is a coherent engine replacement that keeps ZJX1 semantics while removing the places where payload bytes became retained RAM.

ZJX now operates behind an operation-scoped resource governor that owns working-memory limits, spool limits, reserved codec state, and observed peaks. A single reusable input buffer replaces whole-file retention. Earlier raw chunks move to a file-backed spool so later references can replay them without holding the entire corpus. Solid candidates, structured transforms, and decoded replay use governed spools rather than capturing complete streams in memory. Verification and extraction hash and measure incrementally while streaming.

Single-worker synchronous stages provide the first backpressure mechanism. Deterministic candidate ordering, tie-breaking, codec arguments, and environment isolation are preserved so buffered boundaries do not alter archive bytes. Where the historic one-shot encoder was required for byte identity, ZJX keeps it through bounded file-backed mapping rather than replacing its output.

The result is operationally legible: Silesia and multi-gigabyte inputs can be attempted under explicit ceilings, spool expectations are planned before launch, and a rejected job names its budget class instead of disappearing into a host kill.

What Changed

The evidence change is broad. Every required release-success criterion passes. The preferred 512 MiB Silesia stretch target is the single unmet gate - the exact legacy xz candidate evaluation conservatively reserves 705,794,571 bytes of fixed state, even though observed total RSS is already below 512 MiB.

Public corpora reproduce deterministic output. Calgary, Canterbury, enwik8, and enwik9 each produce the same bytes and SHA-256 as ZJX 1.2. Enwik8 becomes the clearest practical-corpus win: the same 24.8 MB archive completes in 408.264 s versus 453.532 s - 9.98% faster - while encode RSS falls from 1,103 MB to 733 MB. Calgary and Canterbury track their 1.2 times at 101.96% and 100.65%. Enwik9 is 6.07% slower at 333 s versus 314 s with byte-identical output.

The validation packet confirms that bounded execution does not trade correctness for pragmatism. The full suite records 28 of 28 Zig tests, 129 Python integration checks with one optional skip, 37 focused Arena tests with 14 parameterized subtests, smoke PASS, and scrub PASS. Frozen compatibility seals 103 files - 102 indexed artifacts plus their checksum index at SHA-256 b82356c409433b712bdd9cf9c548c8203236e5a0bf40eddbfc9d8f866dfa2cc7. Chunk reads at every tested byte width produce byte-identical archives. The five-ceiling memory matrix produces one digest from 256 MiB through 8 GiB host limits. Final canonical campaigns report zero timeouts and zero OS OOM events.

What Comes Next

STREAMWALL closes the bounded-memory architecture, not an entire product horizon. The present limit on the Silesia stretch target is compatibility codec state, not retained payload. Temporary spool capacity becomes the next capacity surface: large inputs succeed because they can stream to governed disk, and very large inputs may ultimately be spool-bound rather than memory-bound.

For teams evaluating ZJX, the next decision is narrower than the headline. Select a retention lane where structured bytes and exact restore discipline matter - telemetry, governed JSONL, sharded NDJSON, repository snapshots, or ordered Siglum-scale history - then validate that lane under the ceilings the deployment can offer. The archive will report whether it fits before the host does.

ZJX remains a Linux-first lossless archive prototype. STREAMWALL makes it an archive that can be operated, not only measured.