In `SerializedMessage{Reader,Rewriter}`, if `src->SupportsSize()`, then reduce
the limit of the internal `LimitingReader` to the source size early, rather than
when the source size is reached. This lets `OnLengthDelimited()` fail early if
the length, which is untrusted, exceeds the remaining size.

Remove `SerializedMessage{Reader,Rewriter}::OnLengthUnchecked()`.
`OnLengthDelimited()` is almost as efficient, and is safer for processing
untrusted input given the above. Use `OnLengthDelimited()` in `OnString()` etc.

Clarify that `SerializedMessage{,Backward}Writer::WriteLengthUnchecked()`
verifies that the length does not exceed 2GiB. Remove a redundant check before
calling `WriteLengthUnchecked()`.

Limit parsed message lengths to 2GiB instead of 4GiB in
`SerializedMessage{Reader,Rewriter}`.

Remove unused `SerializedMessageBackwardWriter::CopyFieldFrom()`.

PiperOrigin-RevId: 808517120
9 files changed
tree: a7bf7b602b3a5e1191f0f911681e67d981f1a77f
  1. doc/
  2. python/
  3. riegeli/
  4. tf_dependency/
  5. .bazelrc
  6. configure
  7. CONTRIBUTING.md
  8. LICENSE
  9. MANIFEST.in
  10. MODULE.bazel
  11. README.md
README.md

Riegeli

Riegeli/records is a file format for storing a sequence of string records, typically serialized protocol buffers. It supports dense compression, fast decoding, seeking, detection and optional skipping of data corruption, filtering of proto message fields for even faster decoding, and parallel encoding.

See documentation.

Status

Riegeli file format will only change in a backward compatible way (i.e. future readers will understand current files, but current readers might not understand files using future features).

Riegeli C++ API might change in incompatible ways.