fix(stamping): skip build data spawn when --stamp=false (#4160) ## Summary `PyWriteBuildData` is dirty on every build with a cold Skyframe when outputs are not downloaded. That is, once per `py_binary` and `py_test` in the build. When stamping is off, the output is a constant, so the action spawn is pure overhead. ## Root Cause Analysis The output is declared with `_py_builtins.declare_constant_metadata_file`. Constant metadata always reports itself as unchanged, so Bazel cannot validate it from the action cache. It stats the output on disk instead. Under `--remote_download_outputs=minimal` the output is not on disk. The stat fails and the action is dirty. It re-executes, hits the remote cache, is still not downloaded, and is dirty again on the next cold-server build. Setting `--remote_download_regex` to match the file avoids this, at the cost of downloading it, which confirms the stat is the trigger. The use of a constant metadata file is justified (as per the comment in the code). With stamping on, the action reads `volatile-status.txt`, which changes every build. The build data file is in the binary's runfiles, so without it its consumers would invalidate on every build due to timestamps. However, with stamping off, there is no volatile input. There is nothing to shield. Unfortunately, the current implementation keeps the _additional_ cost (constant metadata being always unchanged plus failed `stat` syscalls) for none of the benefits. ## Change When `is_stamping_enabled(ctx)` is false, the content of the file is known at analysis time: ``` TARGET <label> STAMPED FALSE ``` So declare an ordinary output and write it with `ctx.actions.write`. No spawn, and no constant metadata output. Note this is broader than `--nostamp`. `is_stamping_enabled` is also false for the exec configuration and for `stamp = 0`, so tool-config binaries benefit from the fix even when `--stamp=true` is set at the top level. The stamped path is unchanged and keeps its constant metadata output. ## Compatibility The output is byte-identical. The label is written with `str(ctx.label)` to match `"TARGET": str(ctx.label)` in the spawn's env, which yields the canonical `@@//pkg:target` form. `ctx.actions.args().add(ctx.label)` renders the display form and drops the `@@`. After the change, the tests in `tests/build_data/build_data_test.py` pass without any modifications. ## Alternative considered Keeping the spawn and declaring an ordinary output would also fix the dirtiness. Writing at analysis time is lighter, since it needs no subprocess. ## Environment rules_python 2.2.0, Bazel 9.2.0, Linux. Reproduce with `--remote_download_outputs=minimal` against a warm remote cache, then `bazel shutdown` and rebuild.
This repository is the home of the core Python rules -- py_library, py_binary, py_test, and related symbols that provide the basis for Python support in Bazel. It also contains package installation rules for integrating with PyPI and other indices.
Documentation for rules_python is at https://rules-python.readthedocs.io and in the Bazel Build Encyclopedia.
Examples live in the examples directory.
The core rules are stable. Their implementation is subject to Bazel's backward compatibility policy. This repository aims to follow semantic versioning.
The Bazel community maintains this repository. Neither Google nor the Bazel team provides support for the code. However, this repository is part of the test suite used to vet new Bazel releases. See How to contribute page for information on our development workflow.
requirements.txt is how users have been defining dependencies for a long time. We support this to support legacy usecases or package managers that we don't support directly. Any additional information that we need will be retrieved from the SimpleAPI during the bzlmod extension evaluation phase. Then it will be written to the MODULE.bazel.lock file for future reuse. We have plans to support uv.lock file directly. uv is recommended for generating a fully locked requirements.txt file and we do provide a rule for it.py_binary, py_test rules should scale to large monorepos and we work hard to minimize the work done during analysis and build phase. What is more, the space requirements for should be minimal, so we strive to use symlinks rather than extracting wheels at build time. This means that for different configurations of the same build, we are not extracting the wheel multiple times thus scaling better over the time. From 2.0 onwards we are creating a virtual env for each target by creating an actual minimal virtual environment using symlinks. We plan on creating the traditional site-packages layout in the future by default.rules_python and this has resulted in a few PEPs supported within pure starlark - PEP440, PEP509.Common misconceptions:
rules_python has to keep backwards compatibility with google3. Whilst this might have been true in the past, rules_python is an open source project and any compatibility needs should come from the community - we have no requirement to keep this compatibility and are allowed to make our decisions. However, we do want to keep backwards compatibility as long as possible to not upset users with never ending migrations.rules_python is not caching pip downloads. With 2.0, we use Bazel's downloader by default and rely on bazel to provide the repository caching mechanisms. This means that for simpler setups this should result in transparent and scalable caching with the most recent bazel versions unless there are issues in the bazel itself.For detailed documentation, see https://rules-python.readthedocs.io
See Bzlmod support for more details.