docs: add example for a complex multi-platform pypi configuration (#3292)
The core PyPI docs and API reference docs have the basics for setting up
a multi-platform
Bazel build, but there's a lot of cross-referencing and reading between
the lines necessary.
Create a how to guide specifically on how to do it to better explain the
nuances.
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
diff --git a/docs/howto/multi-platform-pypi-deps.md b/docs/howto/multi-platform-pypi-deps.md
new file mode 100644
index 0000000..6cc7f84
--- /dev/null
+++ b/docs/howto/multi-platform-pypi-deps.md
@@ -0,0 +1,194 @@
+:::{default-domain} bzl
+:::
+
+# How-to: Multi-Platform PyPI Dependencies
+
+When developing applications that need to run on a wide variety of platforms,
+managing PyPI dependencies can become complex. You might need different sets of
+dependencies for different combinations of Python version, threading model,
+operating system, CPU architecture, libc, and even hardware accelerators like
+GPUs.
+
+This guide demonstrates how to manage this complexity using `rules_python` with
+bzlmod. If you prefer to learn by example, complete example code is provided at
+the end.
+
+In this how to guide, we configure for using 4 requirements files, each
+for a different variation using Python 3.14 on Linux:
+
+* Regular (non-freethreaded) Python
+* Freethreaded Python
+* Regular Python for CUDA 12.9
+* Freethreaded Python for ARM and Musl
+
+## Mapping requirements files to Bazel configuration settings
+
+Unfortunately, a requirements file doesn't tell what it's compatible with,
+so we have to manually specify the Bazel configuration settings for it. To do
+that using rules_python, there are two steps: defining a platform, then
+associating a requirements file with the platform.
+
+### Defining a platform
+
+First, we define a "platform" using {obj}`pip.default`. This associates an
+arbitrary name with a list of Bazel {obj}`config_setting` targets. While any
+name can be used for a platform (its name has no inherent semantic meaning), it
+should encode all the relevant dimensions that distinguish a requirements file.
+For example, if a requirements file is specifically for the combination of CUDA
+12.9 and NumPy 2.0, then the platform name should represent that.
+
+The convention is to follow the format of `{os}_{cpu}{threading}`, where:
+
+* `{os}` is the operating system (`linux`, `osx`, `windows`).
+* `{cpu}` is the architecture (`x86_64`, `aarch64`).
+* `{threading}` is `_freethreaded` for a freethreaded Python runtime, or an
+ empty string for the regular runtime.
+
+Additional dimensions should be appended and separated with an underscore (e.g.
+`linux_x86_64_musl_cuda12.9_numpy2`).
+
+The platform name should not include the Python version. That is handled by
+`pip.parse.python_version` separately.
+
+:::{note}
+The term _platform_ here has nothing to do with Bazel's `platform()` rule.
+:::
+
+#### Defining custom settings
+
+Because {obj}`pip.parse.config_settings` is a list of arbitrary `config_setting`
+targets, you can define your own flags or implement custom config matching
+logic. This allows you to model settings that aren't inherently part of
+rules_python.
+
+This is typically done using [bazel_skylib flags](https://bazel.build/extending/config), but any [Starlark
+defined build setting](https://bazel.build/extending/config) can be used. Just
+remember to use `config_setting()` to match a particular value of the flag.
+
+In our example below, we define a custom flag for CUDA version.
+
+#### Predefined and common build settings
+
+rules_python has some predefined build settings you can use. Commonly used ones
+are:
+
+* {obj}`@rules_python//python/config_settings:py_linux_libc`
+* {obj}`@rules_python//python/config_settings:py_freethreaded`
+
+Additionally, [Bazel @platforms](https://github.com/bazelbuild/platforms)
+contains commonly used settings for OS and CPU:
+
+* `@platforms//os:windows`
+* `@platforms//os:linux`
+* `@platforms//os:osx`
+* `@platforms//cpu:x86_64`
+* `@platforms//cpu:aarch64`
+
+Note that these are the raw flag names. In order to use them with `pip.default`,
+you must use {obj}`config_setting()` to match a particular value for them.
+
+### Associating Requirements to Platforms
+
+Next, we associate a requirements file with a platform using
+{obj}`pip.parse.requirements_by_platform`. This is a dictionary attribute where
+the keys are requirements files and the value is a platform name. The platform
+value can use a trailing or leading `*` to match multiple platforms. It can also
+specify multiple platform names using commas to separate them.
+
+Note that the Python version is _not_ part of the platform name.
+
+Under the hood, `pip.parse` merges all the requirements (for a `hub_name`) and
+constructs `select()` expressions to route to the appropriate dependencies.
+
+### Using it in practice
+
+Finally, to make use of what we've configured, perform a build and set
+command line flags to the appropriate values.
+
+```shell
+# Build for CUDA
+bazel build --//:cuda_version=12.9 //:binary
+
+# Build for ARM with musl
+bazel build --@rules_python//python/config_settings:py_linux_libc=musl \
+ --cpu=aarch64 //:binary
+
+# Build for freethreaded
+bazel build --@rules_python//python/config_settings:py_freethreaded=yes //:binary
+```
+
+Note that certain combinations of flags may result in an error or undefined
+behavior. For example, trying to set both freethreaded and CUDA at the same
+time would result in an error because no requirements file was registered
+to match that combination.
+
+## Multiple Python Versions
+
+Having multiple Python versions is fully supported. Simply add a `pip.parse()`
+call and set `python_version` appropriately.
+
+## Multiple hubs
+
+Having multiple `pip.parse` calls with different `hub_name` values is fully
+supported. Each hub only contains the requirements registered to it.
+
+## Complete Example
+
+Here is a complete example that puts all the pieces together.
+
+```starlark
+# File: BUILD.bazel
+load("@bazel_skylib//rules:common_settings.bzl", "string_flag")
+
+# A custom flag for controlling the CUDA version
+string_flag(
+ name = "cuda_version",
+ build_setting_default = "none",
+)
+
+config_setting(
+ name = "is_cuda_12_9",
+ flag_values = {":cuda_version": "12.9"},
+)
+
+# A config_setting that uses the built-in libc flag from rules_python
+config_setting(
+ name = "is_musl",
+ flag_values = {"@rules_python//python/config_settings:py_linux_libc": "muslc"},
+)
+
+# File: MODULE.bazel
+pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
+
+# A custom platform for CUDA on glibc linux
+pip.default(
+ platform = "linux_x86_64_cuda12.9",
+ os = "linux",
+ cpu = "x86_64",
+ config_settings = ["@//:is_cuda_12_9"],
+)
+
+# A custom platform for musl on linux
+pip.default(
+ platform = "linux_aarch64_musl",
+ os = "linux",
+ cpu = "aarch64",
+ config_settings = ["@//:is_musl"],
+)
+
+pip.parse(
+ hub_name = "my_deps",
+ python_version = "3.14",
+ requirements_by_platform = {
+ # Map to default platform names
+ "//:py3.14-regular-linux-x86-glibc-cpu.txt": "linux_x86_64",
+ "//:py3.14-freethreaded-linux-x86-glibc-cpu.txt": "linux_x86_64_freethreaded",
+
+ # Map to our custom platform names
+ "//:py3.14-regular-linux-x86-glibc-cuda12.9.txt": "linux_x86_64_cuda12.9",
+ "//:py3.14-freethreaded-linux-arm-musl-cpu.txt": "linux_aarch64_musl",
+ },
+)
+
+use_repo(pip, "my_deps")
+```
diff --git a/docs/pypi/index.md b/docs/pypi/index.md
index c32bafc..1792889 100644
--- a/docs/pypi/index.md
+++ b/docs/pypi/index.md
@@ -11,6 +11,7 @@
With the advanced topics covered separately:
* Dealing with [circular dependencies](./circular-dependencies).
+* Handling [multi-platform dependencies](../howto/multi-platform-pypi-deps).
```{toctree}
lock
@@ -22,6 +23,9 @@
## Advanced topics
```{toctree}
+:maxdepth: 1
+
circular-dependencies
patch
+../howto/multi-platform-pypi-deps
```
diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl
index 6d539a5..2cf20cd 100644
--- a/python/private/pypi/pip_repository.bzl
+++ b/python/private/pypi/pip_repository.bzl
@@ -266,6 +266,9 @@
Those dependencies become available in a generated `requirements.bzl` file.
You can instead check this `requirements.bzl` file into your repo, see the "vendoring" section below.
+For advanced use-cases, such as handling multi-platform dependencies, see the
+[How-to: Multi-Platform PyPI Dependencies guide](/howto/multi-platform-pypi-deps).
+
In your WORKSPACE file:
```starlark
