fix(pypi): fetch retained wheel archives lazily (#4316) URL-resolved wheels currently retain both the downloaded archive and extracted files even when a build only consumes the Python library. Download the archive temporarily for extraction and remove it afterward, while preserving the existing `whl` and `whl_file` targets through a lazy sibling `<repo>_whl` repository. Ordinary Python builds do not fetch that sibling repository. Apply matching patches in the separate wheel-file repository and extract from its patched archive. This keeps both original and patched archives out of the extracted repository. Preserve caller-owned wheel inputs, using a distinct `_patched_whl` repository for their patched replacements to avoid collisions with existing source repositories. Source-built wheels and direct `whl_archive` URL calls retain their current archive behavior. Both download paths share checksum validation, authentication, and URL/environment substitution. Extend the existing wheel-library integration test with a generated wheel served over HTTP. Separate Bazel invocations check archive cleanup, lazy repository creation, both wheel targets, matching and nonmatching patches, preservation of caller-owned inputs, and the full `pip.parse(uv_lock = ...)` path. Retain the existing pip-backend, source-distribution, direct `whl_archive`, and dependency-wiring assertions in the same integration target. Add analysis coverage for archive ownership and exclusion from runtime files. Gazelle's module mapping still consumes wheel archives and therefore materializes their archive repositories. Changing that consumer to use extracted libraries is separate follow-up work. Validation: - All 31 bzlmod example tests pass, including an assertion that the public wheel archive contains the same patched metadata as the extracted library. Both `bazel vendor //...` and building all targets with the resulting vendor directory pass locally. - All 24 affected Bazel test targets pass with `--config=fast-tests`, including the consolidated `//tests/integration:whl_library_test_bazel_self` target and its five test methods. The nested integration target is run with tag filters overridden and local test concurrency limited to one. - The storage analysis suite passes under bzlmod. Under Bazel 7.4.1 WORKSPACE mode, the bzlmod-specific ownership assertion is skipped and runtime archive exclusion passes. - All applicable pre-commit hooks pass, including buildifier, Ruff, deleted-package generation, and Gazelle. - The NumPy 2.3.3 PoC imports successfully without materializing an archive repository; explicitly building its `whl` target materializes the archive. Its extracted repository uses 57.74 MiB instead of 73.61 MiB allocated on the test filesystem. Shared download caches are outside that measurement. Implements the delete-by-default / optional-archive-access suggestion in https://github.com/bazel-contrib/rules_python/issues/3091#issuecomment-3074481951. Preserves the wheel-consumer functionality introduced in #364 and fixed in #458. Related to #3968 and complements the extraction-reuse work in #3856.
This repository is the home of the core Python rules -- py_library, py_binary, py_test, and related symbols that provide the basis for Python support in Bazel. It also contains package installation rules for integrating with PyPI and other indices.
Documentation for rules_python is at https://rules-python.readthedocs.io and in the Bazel Build Encyclopedia.
Examples live in the examples directory.
The core rules are stable. Their implementation is subject to Bazel's backward compatibility policy. This repository aims to follow semantic versioning.
The Bazel community maintains this repository. Neither Google nor the Bazel team provides support for the code. However, this repository is part of the test suite used to vet new Bazel releases. See How to contribute page for information on our development workflow.
requirements.txt is how users have been defining dependencies for a long time. We support this to support legacy usecases or package managers that we don't support directly. Any additional information that we need will be retrieved from the SimpleAPI during the bzlmod extension evaluation phase. Then it will be written to the MODULE.bazel.lock file for future reuse. We have plans to support uv.lock file directly. uv is recommended for generating a fully locked requirements.txt file and we do provide a rule for it.py_binary, py_test rules should scale to large monorepos and we work hard to minimize the work done during analysis and build phase. What is more, the space requirements for should be minimal, so we strive to use symlinks rather than extracting wheels at build time. This means that for different configurations of the same build, we are not extracting the wheel multiple times thus scaling better over the time. From 2.0 onwards we are creating a virtual env for each target by creating an actual minimal virtual environment using symlinks. We plan on creating the traditional site-packages layout in the future by default.rules_python and this has resulted in a few PEPs supported within pure starlark - PEP440, PEP509.Common misconceptions:
rules_python has to keep backwards compatibility with google3. Whilst this might have been true in the past, rules_python is an open source project and any compatibility needs should come from the community - we have no requirement to keep this compatibility and are allowed to make our decisions. However, we do want to keep backwards compatibility as long as possible to not upset users with never ending migrations.rules_python is not caching pip downloads. With 2.0, we use Bazel's downloader by default and rely on bazel to provide the repository caching mechanisms. This means that for simpler setups this should result in transparent and scalable caching with the most recent bazel versions unless there are issues in the bazel itself.For detailed documentation, see https://rules-python.readthedocs.io
See Bzlmod support for more details.