tree 03384a8a7996fa448641e15fceed3f06d899b7a4
parent 5868b2023771228e24cddc490d6fa32ed82f62d7
author A Googler <no-reply@google.com> 1760475000 -0700
committer Copybara-Service <copybara-worker@google.com> 1760475042 -0700

Prevent OOM caused by loading large files into memory.

Previously when building Android APK zip archives, the source files are read entirely into memory as byte arrays. So it cannot support files larger than 2 GB and will crash with OOM when allocating the arrays.

The change adds the support for an arbitrary InputStream of unknown size which will work for large files.

To minimize the impact to the build speed, when a file is small enough, the whole file is still read into memory to avoid repeated disk I/O. Large files are read in a streaming fashion to limit their memory consumption at the cost of more disk I/O.

Currently the boundary between small and large size is set to 512 MB empirically. The Java byte arrays can support up to 2 GB, but loading such large files into memory can also cause a high memory pressure for the JVM, so I didn't set it to 2 GB.

PiperOrigin-RevId: 819382752
Change-Id: Id0b96172e1674b4a3556a61538c957fa599c5c79