birnpack — a from-scratch single-file CM compressor built under a strict "no copying" rule

Hi all — first post here. I'd like to share a hobby research project and would love honest testing/feedback from this community.

  

  **birnpack** is a lossless compressor in a single C file (~1,600 lines): a hand-evolved context-mixing model (logistic mixing of ~14 predictor inputs per bit, hashed byte contexts, two match models, indirect

  bit-history contexts, an SSE/APM stage, and an x86 branch-target prefilter for executables). Everything predicts raw bytes directly.

  

  The unusual part is the rule it was built under: **never call, link, or re-implement an existing compressor** (no zlib/lzma/zstd/flac, no LZ77 copied from anywhere), and **never decode a container format** (no

  JPEG/deflate/CABAC unpacking — recompressors were explicitly forbidden). Whatever gains exist had to come from modelling raw bytes. Lossless was gated mechanically: every change had to survive a byte-exact

  round-trip over a 17-file corpus, or it was reverted.

  

  Full disclosure: the model mechanics were evolved in an AI-assisted research loop — but under mechanical honesty guards (full-corpus byte-exact gate on every change, a clone detector against re-labelled

  variants, and a watchdog that killed anything calling or imitating an external compressor). I verified the results independently. Happy to discuss the setup.

  

  **enwik8** (measured on Linux, 16-core, single file, symmetric coder):

  

xz -9        24,865,252    (122 s)

bzip2 -9     29,008,758    (5 s)

birnpack     30,294,831    (enc 24.2 s, dec 24.3 s, verified byte-exact)

gzip -9      36,445,248    (5 s)

  So on pure text it lands between gzip and bzip2 — respectable for "no LZ, no borrowed code", but nothing record-breaking, and far from paq8-class. Where it does better is **mixed real-world files**: on my

  17-file corpus (office docs, CAD text, JPEG/HEIC, ELF binaries, logs, C source) it beat gzip -9 on **every single file** (overall ratio 0.539), e.g.:

ELF executable  139 KB:  birnpack 46,850   vs gzip -9 61,924

shared library  680 KB:  birnpack 177,715  vs gzip -9 272,702

text log        293 KB:  birnpack 26,308   vs gzip -9 38,972

STL mesh        2 MB:    birnpack 77,963   (gzip far behind)

  Already-compressed formats (JPEG/HEIC) shrink only ~1–3 % — expected, since format decoding was forbidden. Speed is ~4 MB/s each way (context mixing; that's the price).

  Code (MIT): https://github.com/ingo6/birnpack — `make && make test` runs a byte-exact round-trip self-test. I'd genuinely appreciate results on your own corpora, broken edge cases, and any thoughts on the model.