blob: 52e619a120d1c04fe3c05f5e432d1079f5e1e9f7 [file] [log] [blame] [view] [edit]
:::{default-domain} bzl
:::
(configuring-toolchains)=
# Configuring Python toolchains and runtimes
This document explains how to configure the Python toolchain and runtimes for different
use cases.
## Bzlmod MODULE configuration
How to configure `rules_python` in your `MODULE.bazel` file depends on how and why
you're using Python. There are four basic use cases:
1. A root module that always uses Python. For example, you're building a
Python application.
2. A library module with dev-only uses of Python. For example, a Java project
that only uses Python as part of testing itself.
3. A library module without version constraints. For example, a rule set with
Python build tools, but defers to the user as to what Python version is used
for the tools.
4. A library module with version constraints. For example, a rule set with
Python build tools, and the module requires a specific version of Python
be used with its tools.
### Root modules
Root modules are always the top-most module. These are special in two ways:
1. Some `rules_python` bzlmod APIs are only respected by the root module.
2. The root module can force module overrides and specific module dependency
ordering.
When configuring `rules_python` for a root module, you typically want to
explicitly specify the Python version you want to use. This ensures that
dependencies don't change the Python version out from under you. Remember that
`rules_python` will set a version by default, but it will change regularly as
it tracks a recent Python version.
NOTE: If your root module only uses Python for development of the module itself,
you should read the dev-only library module section.
```
bazel_dep(name="rules_python", version=...)
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.defaults(python_version = "3.12")
python.toolchain(python_version = "3.12")
```
### Library modules
A library module is a module that can show up in arbitrary locations in the
Bzlmod module graph -- it's unknown where in the breadth-first search order the
module will be relative to other modules. For example, `rules_python` is a
library module.
#### Library modules with dev-only Python usage
A library module with dev-only Python usage is usually one where Python is only
used as part of its tests. For example, a module for Java rules might run some
Python program to generate test data, but real usage of the rules don't need
Python to work. To configure this, follow the root-module setup, but remember to
specify `dev_dependency = True` to the bzlmod APIs:
```
# MODULE.bazel
bazel_dep(name = "rules_python", version=..., dev_dependency = True)
python = use_extension(
"@rules_python//python/extensions:python.bzl",
"python",
dev_dependency = True
)
python.defaults(python_version = "3.12")
python.toolchain(python_version = "3.12")
```
#### Library modules without version constraints
A library module without version constraints is one where the version of Python
used for the Python programs it runs isn't chosen by the module itself. Instead,
it's up to the root module to pick an appropriate version of Python.
For this case, configuration is simple: just depend on `rules_python` and use
the normal `//python:py_binary.bzl` et al. rules. There is no need to call
`python.toolchain` -- `rules_python` ensures _some_ Python version is available,
but more often, the root module will specify some version.
```
# MODULE.bazel
bazel_dep(name = "rules_python", version=...)
```
#### Library modules with version constraints
A library module with version constraints is one where the module requires a
specific Python version be used with its tools. This has some pros/cons:
* It allows the library's tools to use a different version of Python than
the rest of the build. For example, a user's program could use Python 3.12,
while the library module's tools use Python 3.10.
* It reduces the support burden for the library module because the library only needs
to test for the particular Python version they intend to run as.
* It raises the support burden for the library module because the version of
Python being used needs to be regularly incremented.
* It has higher build overhead because additional runtimes and libraries need
to be downloaded, and Bazel has to keep additional configuration state.
To configure this, request the Python versions needed in `MODULE.bazel` and use
the version-aware rules for `py_binary`.
```
# MODULE.bazel
bazel_dep(name = "rules_python", version=...)
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(python_version = "3.12")
# BUILD.bazel
load("@rules_python//python:py_binary.bzl", "py_binary")
py_binary(..., python_version="3.12")
```
### Pinning to a Python version
Pinning to a version allows targets to force that a specific Python version is
used, even if the root module configures a different version as a default. This
is most useful for two cases:
1. For submodules to ensure they run with the appropriate Python version
2. To allow incremental, per-target, upgrading to newer Python versions,
typically in a monorepo situation.
To configure a submodule with the version-aware rules, request the particular
version you need when defining the toolchain:
```starlark
# MODULE.bazel
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
python_version = "3.11",
)
use_repo(python)
```
Then use the `@rules_python` repo in your `BUILD` file to explicitly pin the Python version when calling the rule:
```starlark
# BUILD.bazel
load("@rules_python//python:py_binary.bzl", "py_binary")
py_binary(..., python_version = "3.11")
py_test(..., python_version = "3.11")
```
Multiple versions can be specified and used within a single build.
```starlark
# MODULE.bazel
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.defaults(
# The environment variable takes precedence if set.
python_version = "3.11",
python_version_env = "BAZEL_PYTHON_VERSION",
)
python.toolchain(
python_version = "3.11",
)
python.toolchain(
python_version = "3.12",
)
# BUILD.bazel
load("@rules_python//python:py_binary.bzl", "py_binary")
load("@rules_python//python:py_test.bzl", "py_test")
# Defaults to 3.11
py_binary(...)
py_test(...)
# Explicitly use Python 3.11
py_binary(..., python_version = "3.11")
py_test(..., python_version = "3.11")
# Explicitly use Python 3.12
py_binary(..., python_version = "3.12")
py_test(..., python_version = "3.12")
```
For more documentation, see the bzlmod examples under the {gh-path}`examples`
folder. Look for the examples that contain a `MODULE.bazel` file.
### Other toolchain details
The `python.toolchain()` call makes its contents available under a repo named
`python_X_Y`, where X and Y are the major and minor versions. For example,
`python.toolchain(python_version="3.11")` creates the repo `@python_3_11`.
Remember to call `use_repo()` to make repos visible to your module:
`use_repo(python, "python_3_11")`.
:::{deprecated} 1.1.0
The toolchain-specific `py_binary` and `py_test` symbols are aliases to the regular rules.
For example, `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` are deprecated.
Usages of them should be changed to load the regular rules directly.
For example, use `load("@rules_python//python:py_binary.bzl", "py_binary")` & `load("@rules_python//python:py_test.bzl", "py_test")` and then specify the `python_version` when using the rules corresponding to the Python version you defined in your toolchain. {ref}`Library modules with version constraints`
:::
#### Toolchain usage in other rules
Python toolchains can be utilized in other Bazel rules, such as `genrule()`, by
adding the `toolchains=["@rules_python//python:current_py_toolchain"]`
attribute. You can obtain the path to the Python interpreter using the
`$(PYTHON2)` and `$(PYTHON3)` ["Make"
Variables](https://bazel.build/reference/be/make-variables). See the
{gh-path}`test_current_py_toolchain <tests/load_from_macro/BUILD.bazel>` target
for an example. We also make available `$(PYTHON2_ROOTPATH)` and `$(PYTHON3_ROOTPATH)`,
which are Make Variable equivalents of `$(PYTHON2)` and `$(PYTHON3)` but for runfiles
locations. These will be helpful if you need to set environment variables of binary/test rules
while using [`--nolegacy_external_runfiles`](https://bazel.build/reference/command-line-reference#flag--legacy_external_runfiles).
The original make variables still work in exec contexts such as genrules.
### Overriding toolchain defaults and adding more versions
One can perform various overrides for the registered toolchains from the root
module. For example, the following use cases would be supported using the
existing attributes:
* Limiting the available toolchains for the entire `bzlmod` transitive graph
via {attr}`python.override.available_python_versions`.
* Setting particular `X.Y.Z` Python versions when modules request `X.Y` version
via {attr}`python.override.minor_mapping`.
* Per-version control of the coverage tool used using
{attr}`python.single_version_platform_override.coverage_tool`.
* Adding additional Python versions via {bzl:obj}`python.single_version_override` or
{bzl:obj}`python.single_version_platform_override`.
### Registering custom runtimes
Because the python-build-standalone project has _thousands_ of prebuilt runtimes
available, `rules_python` only includes popular runtimes in its built-in
configurations. If you want to use a runtime that isn't already known to
`rules_python`, then {obj}`single_version_platform_override()` can be used to do
so. In short, it allows specifying an arbitrary URL and using custom flags
to control when a runtime is used.
In the example below, we register a particular python-build-standalone runtime
that is activated for Linux x86 builds when the custom flag
`--//:runtime=my-custom-runtime` is set.
```
# File: MODULE.bazel
bazel_dep(name = "bazel_skylib", version = "1.7.1.")
bazel_dep(name = "rules_python", version = "1.5.0")
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.single_version_platform_override(
platform = "my-platform",
python_version = "3.13.3",
sha256 = "01d08b9bc8a96698b9d64c2fc26da4ecc4fa9e708ce0a34fb88f11ab7e552cbd",
os_name = "linux",
arch = "x86_64",
target_settings = [
"@@//:runtime=my-custom-runtime",
],
urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20250409/cpython-3.13.3+20250409-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz"],
)
# File: //:BUILD.bazel
load("@bazel_skylib//rules:common_settings.bzl", "string_flag")
string_flag(
name = "custom_runtime",
build_setting_default = "",
)
config_setting(
name = "is_custom_runtime_linux-x86-install-only-stripped",
flag_values = {
":custom_runtime": "linux-x86-install-only-stripped",
},
)
```
Notes:
- While any URL and archive can be used, it's assumed their content looks like
a python-build-standalone archive.
- A "version-aware" toolchain is registered, which means the Python version flag
must also match (e.g., `--@rules_python//python/config_settings:python_version=3.13.3`
must be set -- see `minor_mapping` and `is_default` for controls and docs
about version matching and selection).
- The `target_compatible_with` attribute can be used to entirely specify the
argument of the same name that the toolchain uses.
- The labels in `target_settings` must be absolute; `@@` refers to the main repo.
- The `target_settings` are `config_setting` targets, which means you can
customize how matching occurs.
:::{seealso}
See {obj}`//python/config_settings` for flags `rules_python` already defines
that can be used with `target_settings`. Some particular ones of note are
{flag}`--py_linux_libc` and {flag}`--py_freethreaded`, among others.
:::
:::{versionadded} 1.5.0
Added support for custom platform names, `target_compatible_with`, and
`target_settings` with `single_version_platform_override`.
:::
### Using defined toolchains from WORKSPACE
It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example,
the following `MODULE.bazel` and `WORKSPACE` provides a working {bzl:obj}`pip_parse` setup:
```starlark
# File: WORKSPACE
load("@rules_python//python:repositories.bzl", "py_repositories")
py_repositories()
load("@rules_python//python:pip.bzl", "pip_parse")
pip_parse(
name = "third_party",
requirements_lock = "//:requirements.txt",
python_interpreter_target = "@python_3_10_host//:python",
)
load("@third_party//:requirements.bzl", "install_deps")
install_deps()
# File: MODULE.bazel
bazel_dep(name = "rules_python", version = "0.40.0")
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.defaults(python_version = "3.10")
python.toolchain(python_version = "3.10")
use_repo(python, "python_3_10", "python_3_10_host")
```
Note, the user has to import the `*_host` repository to use the Python interpreter in the
{bzl:obj}`pip_parse` and `whl_library` repository rules, and once that is done,
users should be able to ensure the setting of the default toolchain even during the
transition period when some of the code is still defined in `WORKSPACE`.
## Workspace configuration
To import `rules_python` in your project, you first need to add it to your
`WORKSPACE` file, using the snippet provided in the
[release you choose](https://github.com/bazel-contrib/rules_python/releases).
To depend on a particular unreleased version, you can do the following:
```starlark
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
# Update the SHA and VERSION to the lastest version available here:
# https://github.com/bazel-contrib/rules_python/releases.
SHA="84aec9e21cc56fbc7f1335035a71c850d1b9b5cc6ff497306f84cced9a769841"
VERSION="0.23.1"
http_archive(
name = "rules_python",
sha256 = SHA,
strip_prefix = "rules_python-{}".format(VERSION),
url = "https://github.com/bazel-contrib/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(VERSION,VERSION),
)
load("@rules_python//python:repositories.bzl", "py_repositories")
py_repositories()
```
### Workspace toolchain registration
To register a hermetic Python toolchain rather than rely on a system-installed interpreter for runtime execution, you can add to the `WORKSPACE` file:
```starlark
load("@rules_python//python:repositories.bzl", "python_register_toolchains")
python_register_toolchains(
name = "python_3_11",
# Available versions are listed in @rules_python//python:versions.bzl.
# We recommend using the same version your team is already standardized on.
python_version = "3.11",
)
load("@rules_python//python:pip.bzl", "pip_parse")
pip_parse(
...
python_interpreter_target = "@python_3_11_host//:python",
...
)
```
After registration, your Python targets will use the toolchain's interpreter during execution, but a system-installed interpreter
is still used to "bootstrap" Python targets (see https://github.com/bazel-contrib/rules_python/issues/691).
You may also find some quirks while using this toolchain. Please refer to [python-build-standalone documentation's _Quirks_ section](https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html).
## Local toolchain
It's possible to use a locally installed Python runtime instead of the regular
prebuilt, remotely downloaded ones. A local toolchain contains the Python
runtime metadata (Python version, headers, ABI flags, etc.) that the regular
remotely downloaded runtimes contain, which makes it possible to build, e.g., C
extensions (unlike the autodetecting and runtime environment toolchains).
For simple cases, the {obj}`local_runtime_repo` and
{obj}`local_runtime_toolchains_repo` rules are provided that will introspect a
Python installation and create an appropriate Bazel definition from it. To do
this, three pieces need to be wired together:
1. Specify a path or command to a Python interpreter (multiple can be defined).
2. Create toolchains for the runtimes in (1).
3. Register the toolchains created by (2).
The following is an example that will use `python3` from `PATH` to find the
interpreter, then introspect its installation to generate a full toolchain.
```starlark
# File: MODULE.bazel
local_runtime_repo = use_repo_rule(
"@rules_python//python/local_toolchains:repos.bzl",
"local_runtime_repo",
)
local_runtime_toolchains_repo = use_repo_rule(
"@rules_python//python/local_toolchains:repos.bzl",
"local_runtime_toolchains_repo",
)
# Step 1: Define the Python runtime
local_runtime_repo(
name = "local_python3",
interpreter_path = "python3",
on_failure = "fail",
dev_dependency = True
)
# Step 2: Create toolchains for the runtimes
local_runtime_toolchains_repo(
name = "local_toolchains",
runtimes = ["local_python3"],
# TIP: The `target_settings` arg can be used to activate them based on
# command line flags; see docs below.
dev_dependency = True
)
# Step 3: Register the toolchains
register_toolchains("@local_toolchains//:all", dev_dependency = True)
```
:::{important}
Be sure to set `dev_dependency = True`. Using a local toolchain only makes sense
for the root module.
If an intermediate module does it, then the `register_toolchains()` call will
take precedence over the default rules_python toolchains and cause problems for
downstream modules.
:::
Multiple runtimes and/or toolchains can be defined, which allows for multiple
Python versions and/or platforms to be configured in a single `MODULE.bazel`.
Note that `register_toolchains` will insert the local toolchain earlier in the
toolchain ordering, so it will take precedence over other registered toolchains.
To better control when the toolchain is used, see [Conditionally using local
toolchains].
### Conditionally using local toolchains
By default, a local toolchain has few constraints and is early in the toolchain
ordering, which means it will usually be used no matter what. This can be
problematic for CI (where it shouldn't be used), expensive for CI (CI must
initialize/download the repository to determine its Python version), and
annoying for iterative development (enabling/disabling it requires modifying
`MODULE.bazel`).
These behaviors can be mitigated, but it requires additional configuration
to avoid triggering the local toolchain repository to initialize (i.e., run
local commands and perform downloads).
The two settings to change are
{obj}`local_runtime_toolchains_repo.target_compatible_with` and
{obj}`local_runtime_toolchains_repo.target_settings`, which control how Bazel
decides if a toolchain should match. By default, they point to targets *within*
the local runtime repository (triggering repo initialization). We have to override
them to *not* reference the local runtime repository at all.
In the example below, we reconfigure the local toolchains so they are only
activated if the custom flag `--//:py=local` is set and the target platform
matches the Bazel host platform. The net effect is that CI won't use the local
toolchain (nor initialize its repository), and developers can easily
enable/disable the local toolchain with a command line flag.
```
# File: MODULE.bazel
bazel_dep(name = "bazel_skylib", version = "1.7.1")
local_runtime_toolchains_repo(
name = "local_toolchains",
runtimes = ["local_python3"],
target_compatible_with = {
"local_python3": ["HOST_CONSTRAINTS"],
},
target_settings = {
"local_python3": ["@//:is_py_local"]
}
)
# File: BUILD.bazel
load("@bazel_skylib//rules:common_settings.bzl", "string_flag")
config_setting(
name = "is_py_local",
flag_values = {":py": "local"},
)
string_flag(
name = "py",
build_setting_default = "",
)
```
:::{tip}
Easily switching between *multiple* local toolchains can be accomplished by
adding additional `:is_py_X` targets and setting `--//:py` to match.
to easily switch between different local toolchains.
:::
## Runtime environment toolchain
The runtime environment toolchain is a minimal toolchain that doesn't provide
information about Python at build time. In particular, this means it is not able
to build C extensions -- doing so requires knowing, at build time, what Python
headers to use.
In effect, all it does is generate a small wrapper script that simply calls, e.g.,
`/usr/bin/env python3` to run a program. This makes it easy to change what
Python is used to run a program but also makes it easy to use a Python version
that isn't compatible with build-time assumptions.
```
register_toolchains("@rules_python//python/runtime_env_toolchains:all")
```
Note that this toolchain has no constraints, i.e. it will match any platform,
Python version, etc.
:::{seealso}
[Local toolchain], which creates a more full featured toolchain from a
locally installed Python.
:::
### Autodetecting toolchain
The autodetecting toolchain is a deprecated toolchain that is built into Bazel.
**Its name is a bit misleading: it doesn't autodetect anything.** All it does is
use `python3` from the environment a binary runs within. This provides extremely
limited functionality to the rules (at build time, nothing is knowable about
the Python runtime).
Bazel itself automatically registers `@bazel_tools//tools/python:autodetecting_toolchain`
as the lowest priority toolchain. For `WORKSPACE` builds, if no other toolchain
is registered, that toolchain will be used. For Bzlmod builds, `rules_python`
automatically registers a higher-priority toolchain; it won't be used unless
there is a toolchain misconfiguration somewhere.
To aid migration off the Bazel-builtin toolchain, `rules_python` provides
{bzl:obj}`@rules_python//python/runtime_env_toolchains:all`. This is an equivalent
toolchain but is implemented using `rules_python`'s objects.
## Custom toolchains
While `rules_python` provides toolchains by default, it is not required to use
them, and you can define your own toolchains to use instead. This section
gives an introduction to how to define them yourself.
:::{note}
* Defining your own toolchains is an advanced feature.
* APIs used for defining them are less stable and may change more often.
:::
Under the hood, there are multiple toolchains that comprise the different
information necessary to build Python targets. Each one has an
associated _toolchain type_ that identifies it. We call the collection of these
toolchains a "toolchain suite".
One of the underlying design goals of the toolchains is to support complex and
bespoke environments. Such environments may use an arbitrary combination of
{bzl:obj}`RBE`, cross-platform building, multiple Python versions,
building Python from source, embedding Python (as opposed to building separate
interpreters), using prebuilt binaries, or using binaries built from source. To
that end, many of the attributes they accept, and fields they provide, are
optional.
### Target toolchain type
The target toolchain type is {obj}`//python:toolchain_type`, and it
is for _target configuration_ runtime information, e.g., the Python version
and interpreter binary that a program will use.
This is typically implemented using {obj}`py_runtime()`, which
provides the {obj}`PyRuntimeInfo` provider. For historical reasons from the
Python 2 transition, `py_runtime` is wrapped in {obj}`py_runtime_pair`,
which provides {obj}`ToolchainInfo` with the field `py3_runtime`, which is an
instance of `PyRuntimeInfo`.
This toolchain type is intended to hold only _target configuration_ values. As
such, when defining its associated {external:bzl:obj}`toolchain` target, only
set {external:bzl:obj}`toolchain.target_compatible_with` and/or
{external:bzl:obj}`toolchain.target_settings` constraints; there is no need to
set {external:bzl:obj}`toolchain.exec_compatible_with`.
### Python C toolchain type
The Python C toolchain type ("py cc") is {obj}`//python/cc:toolchain_type`, and
it has C/C++ information for the _target configuration_, e.g., the C headers that
provide `Python.h`.
This is typically implemented using {obj}`py_cc_toolchain()`, which provides
{obj}`ToolchainInfo` with the field `py_cc_toolchain` set, which is a
{obj}`PyCcToolchainInfo` provider instance.
This toolchain type is intended to hold only _target configuration_ values
relating to the C/C++ information for the Python runtime. As such, when defining
its associated {external:obj}`toolchain` target, only set
{external:bzl:obj}`toolchain.target_compatible_with` and/or
{external:bzl:obj}`toolchain.target_settings` constraints; there is no need to
set {external:bzl:obj}`toolchain.exec_compatible_with`.
### Exec tools toolchain type
The exec tools toolchain type is {obj}`//python:exec_tools_toolchain_type`,
and it is for supporting tools for _building_ programs, e.g., the binary to
precompile code at build time.
This toolchain type is intended to hold only _exec configuration_ values --
usually tools (prebuilt or from-source) used to build Python targets.
This is typically implemented using {obj}`py_exec_tools_toolchain`, which
provides {obj}`ToolchainInfo` with the field `exec_tools` set, which is an
instance of {obj}`PyExecToolsInfo`.
The toolchain constraints of this toolchain type can be a bit more nuanced than
the other toolchain types. Typically, you set
{external:bzl:obj}`toolchain.target_settings` to the Python version the tools
are for, and {external:bzl:obj}`toolchain.exec_compatible_with` to the platform
they can run on. This allows the toolchain to first be considered based on the
target configuration (e.g. Python version), then for one to be chosen based on
finding one compatible with the available host platforms to run the tool on.
However, what `target_compatible_with`/`target_settings` and
`exec_compatible_with` values to use depends on the details of the tools being used.
For example:
* If you had a precompiler that supported any version of Python, then
putting the Python version in `target_settings` is unnecessary.
* If you had a prebuilt polyglot precompiler binary that could run on any
platform, then setting `exec_compatible_with` is unnecessary.
This can work because, when the rules invoke these build tools, they pass along
all necessary information so that the tool can be entirely independent of the
target configuration being built for.
Alternatively, if you had a precompiler that only ran on Linux and only
produced valid output for programs intended to run on Linux, then _both_
`exec_compatible_with` and `target_compatible_with` must be set to Linux.
### Custom toolchain example
Here, we show an example for a semi-complicated toolchain suite, one that is:
* A CPython-based interpreter
* For Python version 3.12.0
* Using an in-build interpreter built from source
* That only runs on Linux
* Using a prebuilt precompiler that only runs on Linux and only produces
bytecode valid for 3.12
* With the exec tools interpreter disabled (unnecessary with a prebuilt
precompiler)
* Providing C headers and libraries
Defining toolchains for this might look something like this:
```
# -------------------------------------------------------
# File: toolchain_impl/BUILD
# Contains the tool definitions (runtime, headers, libs).
# -------------------------------------------------------
load("@rules_python//python:py_cc_toolchain.bzl", "py_cc_toolchain")
load("@rules_python//python:py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain")
load("@rules_python//python:py_runtime.bzl", "py_runtime")
load("@rules_python//python:py_runtime_pair.bzl", "py_runtime_pair")
MAJOR = 3
MINOR = 12
MICRO = 0
py_runtime(
name = "runtime",
interpreter = ":python",
interpreter_version_info = {
"major": str(MAJOR),
"minor": str(MINOR),
"micro": str(MICRO),
}
implementation = "cpython"
)
py_runtime_pair(
name = "runtime_pair",
py3_runtime = ":runtime"
)
py_cc_toolchain(
name = "py_cc_toolchain_impl",
headers = ":headers",
libs = ":libs",
python_version = "{}.{}".format(MAJOR, MINOR)
)
py_exec_tools_toolchain(
name = "exec_tools_toolchain_impl",
exec_interpreter = "@rules_python/python:none",
precompiler = "precompiler-cpython-3.12"
)
cc_binary(name = "python3.12", ...)
cc_library(name = "headers", ...)
cc_library(name = "libs", ...)
# ------------------------------------------------------------------
# File: toolchains/BUILD
# Putting toolchain() calls in a separate package from the toolchain
# implementations minimizes Bazel loading overhead.
# ------------------------------------------------------------------
toolchain(
name = "runtime_toolchain",
toolchain = "//toolchain_impl:runtime_pair",
toolchain_type = "@rules_python//python:toolchain_type",
target_compatible_with = ["@platforms/os:linux"],
)
toolchain(
name = "py_cc_toolchain",
toolchain = "//toolchain_impl:py_cc_toolchain_impl",
toolchain_type = "@rules_python//python/cc:toolchain_type",
target_compatible_with = ["@platforms/os:linux"],
)
toolchain(
name = "exec_tools_toolchain",
toolchain = "//toolchain_impl:exec_tools_toolchain_impl",
toolchain_type = "@rules_python//python:exec_tools_toolchain_type",
target_settings = [
"@rules_python//python/config_settings:is_python_3.12",
],
exec_compatible_with = ["@platforms/os:linux"],
)
# -----------------------------------------------
# File: MODULE.bazel or WORKSPACE.bazel
# These toolchains will be considered before others.
# -----------------------------------------------
register_toolchains("//toolchains:all")
```
When registering custom toolchains, be aware of the [toolchain registration
order](https://bazel.build/extending/toolchains#toolchain-resolution). In brief,
toolchain order is the BFS-order of the modules; see the Bazel docs for a more
detailed description.
:::{note}
The toolchain() calls should be in a separate BUILD file from everything else.
This avoids Bazel having to perform unnecessary work when it discovers the list
of available toolchains.
:::
## Toolchain selection flags
Currently the following flags are used to influence toolchain selection:
* {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant.
* {obj}`--@rules_python//python/config_settings:py_freethreaded` for selecting
the freethreaded experimental Python builds available from `3.13.0` onwards.
## Running the underlying interpreter
To run the interpreter that Bazel will use, you can use the
`@rules_python//python/bin:python` target. This is a binary target with
the executable pointing at the `python3` binary plus its relevant runfiles.
```console
$ bazel run @rules_python//python/bin:python
Python 3.11.1 (main, Jan 16 2023, 22:41:20) [Clang 15.0.7 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
$ bazel run @rules_python//python/bin:python --@rules_python//python/config_settings:python_version=3.12
Python 3.12.0 (main, Oct 3 2023, 01:27:23) [Clang 17.0.1 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
```
You can also access a specific binary's interpreter this way by using the
`@rules_python//python/bin:python_src` target. In the example below, it is
assumed that the `@rules_python//tools/publish:twine` binary is fixed at Python
3.11.
```console
$ bazel run @rules_python//python/bin:python --@rules_python//python/bin:interpreter_src=@rules_python//tools/publish:twine
Python 3.11.1 (main, Jan 16 2023, 22:41:20) [Clang 15.0.7 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
$ bazel run @rules_python//python/bin:python --@rules_python//python/bin:interpreter_src=@rules_python//tools/publish:twine --@rules_python//python/config_settings:python_version=3.12
Python 3.11.1 (main, Jan 16 2023, 22:41:20) [Clang 15.0.7 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
```
Despite setting the Python version explicitly to 3.12 in the example above, the
interpreter comes from the `@rules_python//tools/publish:twine` binary. That is
a fixed version.
:::{note}
The `python` target does not provide access to any modules from `py_*`
targets on its own. Please file a feature request if this is desired.
:::
### Differences from `//python/bin:repl`
The `//python/bin:python` target provides access to the underlying interpreter
without any hermeticity guarantees.
The [`//python/bin:repl` target](repl) provides an environment identical to
what `py_binary` provides. That means it handles things like the
[`PYTHONSAFEPATH`](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSAFEPATH)
environment variable automatically. The `//python/bin:python` target will not.