Evict oversized Zstd compression and decompression contexts instead of returning them to `RecyclingPool`.

Manage `ZSTD_CCtx` and `ZSTD_DCtx` handles in `ZstdWriterBase` and `ZstdReaderBase` via `RecyclingPool::RawGet()` and `RawPut()`, returning a context to the pool only when `ZSTD_sizeof_CCtx() <= 8 MiB` or `ZSTD_sizeof_DCtx() <= 4 MiB`. Compression contexts at the default window up to level 8 (~7.7 MiB), small pledged blocks, and streaming decompression up to `windowLog == 21` (~2.5 MiB) remain pooled, while larger contexts (such as unpledged compression at level >= 9 or with `windowLog >= 22` at level >= 7, and `windowLog >= 22` streaming decompression) are freed on completion, `Reset()`, or destruction instead of staying pinned in the LIFO pool across subsequent small streams.

PiperOrigin-RevId: 1018771307
4 files changed
tree: 05febd73239c578a25a7ecb6dc777138421b31f8
  1. .github/
  2. doc/
  3. python/
  4. riegeli/
  5. tf_dependency/
  6. .bazelrc
  7. configure
  8. CONTRIBUTING.md
  9. LICENSE
  10. MANIFEST.in
  11. MODULE.bazel
  12. README.md
README.md

Riegeli

Riegeli/records is a file format for storing a sequence of string records, typically serialized protocol buffers. It supports dense compression, fast decoding, seeking, detection and optional skipping of data corruption, filtering of proto message fields for even faster decoding, and parallel encoding.

See documentation.

Status

Riegeli file format will only change in a backward compatible way (i.e. future readers will understand current files, but current readers might not understand files using future features).

Riegeli C++ API might change in incompatible ways.