docs: document how to configure MODULE.bazel for different use cases (#1864)
This largely comes from being regularly tagged on BCR PRs about how a
module should be configured. This info was sort of already in the
getting started docs, but that isn't an obvious place to go and bloated
the what should be simplified steps to getting started.
diff --git a/docs/sphinx/getting-started.md b/docs/sphinx/getting-started.md
index fbecd7b..45d1962 100644
--- a/docs/sphinx/getting-started.md
+++ b/docs/sphinx/getting-started.md
@@ -1,13 +1,18 @@
# Getting started
-The following two sections cover using `rules_python` with bzlmod and
-the older way of configuring bazel with a `WORKSPACE` file.
+This doc is a simplified guide to help get started started quickly. It provides
+a simplified introduction to having a working Python program for both bzlmod
+and the older way of using `WORKSPACE`.
+It assumes you have a `requirements.txt` file with your PyPI dependencies.
+
+For more details information about configuring `rules_python`, see:
+* [Configuring the runtime](toolchains)
+* [Configuring third party dependencies (pip/pypi)](pypi-dependencies)
+* [API docs](api/index)
## Using bzlmod
-**IMPORTANT: bzlmod support is still in Beta; APIs are subject to change.**
-
The first step to using rules_python with bzlmod is to add the dependency to
your MODULE.bazel file:
@@ -15,99 +20,20 @@
# Update the version "0.0.0" to the release found here:
# https://github.com/bazelbuild/rules_python/releases.
bazel_dep(name = "rules_python", version = "0.0.0")
-```
-Once added, you can load the rules and use them:
-
-```starlark
-load("@rules_python//python:py_binary.bzl", "py_binary")
-
-py_binary(...)
-```
-
-Depending on what you're doing, you likely want to do some additional
-configuration to control what Python version is used; read the following
-sections for how to do that.
-
-### Toolchain registration with bzlmod
-
-A default toolchain is automatically configured depending on
-`rules_python`. Note, however, the version used tracks the most recent Python
-release and will change often.
-
-If you want to use a specific Python version for your programs, then how
-to do so depends on if you're configuring the root module or not. The root
-module is special because it can set the *default* Python version, which
-is used by the version-unaware rules (e.g. `//python:py_binary.bzl` et al). For
-submodules, it's recommended to use the version-aware rules to pin your programs
-to a specific Python version so they don't accidentally run with a different
-version configured by the root module.
-
-#### Configuring and using the default Python version
-
-To specify what the default Python version is, set `is_default = True` when
-calling `python.toolchain()`. This can only be done by the root module; it is
-silently ignored if a submodule does it. Similarly, using the version-unaware
-rules (which always use the default Python version) should only be done by the
-root module. If submodules use them, then they may run with a different Python
-version than they expect.
-
-```starlark
-python = use_extension("@rules_python//python/extensions:python.bzl", "python")
-
-python.toolchain(
+pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
+pip.parse(
+ hub_name = "my_deps",
python_version = "3.11",
- is_default = True,
+ requirements_lock = "//:requirements.txt",
)
+use_repo(pip, "my_deps")
```
-Then use the base rules from e.g. `//python:py_binary.bzl`.
-
-#### Pinning to a Python version
-
-Pinning to a version allows targets to force that a specific Python version is
-used, even if the root module configures a different version as a default. This
-is most useful for two cases:
-
-1. For submodules to ensure they run with the appropriate Python version
-2. To allow incremental, per-target, upgrading to newer Python versions,
- typically in a mono-repo situation.
-
-To configure a submodule with the version-aware rules, request the particular
-version you need, then use the `@python_versions` repo to use the rules that
-force specific versions:
-
-```starlark
-python = use_extension("@rules_python//python/extensions:python.bzl", "python")
-
-python.toolchain(
- python_version = "3.11",
-)
-use_repo(python, "python_versions")
-```
-
-Then use e.g. `load("@python_versions//3.11:defs.bzl", "py_binary")` to use
-the rules that force that particular version. Multiple versions can be specified
-and use within a single build.
-
-For more documentation, see the bzlmod examples under the {gh-path}`examples`
-folder. Look for the examples that contain a `MODULE.bazel` file.
-
-#### Other toolchain details
-
-The `python.toolchain()` call makes its contents available under a repo named
-`python_X_Y`, where X and Y are the major and minor versions. For example,
-`python.toolchain(python_version="3.11")` creates the repo `@python_3_11`.
-Remember to call `use_repo()` to make repos visible to your module:
-`use_repo(python, "python_3_11")`
-
## Using a WORKSPACE file
-To import rules_python in your project, you first need to add it to your
-`WORKSPACE` file, using the snippet provided in the
-[release you choose](https://github.com/bazelbuild/rules_python/releases)
-
-To depend on a particular unreleased version, you can do the following:
+Using WORKSPACE is deprecated, but still supported, and a bit more involved than
+using Bzlmod. Here is a simplified setup to download the prebuilt runtimes.
```starlark
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
@@ -130,13 +56,7 @@
load("@rules_python//python:repositories.bzl", "py_repositories")
py_repositories()
-```
-### Toolchain registration
-
-To register a hermetic Python toolchain rather than rely on a system-installed interpreter for runtime execution, you can add to the `WORKSPACE` file:
-
-```starlark
load("@rules_python//python:repositories.bzl", "python_register_toolchains")
python_register_toolchains(
@@ -157,19 +77,10 @@
)
```
-After registration, your Python targets will use the toolchain's interpreter during execution, but a system-installed interpreter
-is still used to 'bootstrap' Python targets (see https://github.com/bazelbuild/rules_python/issues/691).
-You may also find some quirks while using this toolchain. Please refer to [python-build-standalone documentation's _Quirks_ section](https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html).
-
-## Toolchain usage in other rules
-
-Python toolchains can be utilized in other bazel rules, such as `genrule()`, by adding the `toolchains=["@rules_python//python:current_py_toolchain"]` attribute. You can obtain the path to the Python interpreter using the `$(PYTHON2)` and `$(PYTHON3)` ["Make" Variables](https://bazel.build/reference/be/make-variables). See the
-{gh-path}`test_current_py_toolchain <tests/load_from_macro/BUILD.bazel>` target for an example.
-
## "Hello World"
-Once you've imported the rule set into your `WORKSPACE` using any of these
-methods, you can then load the core rules in your `BUILD` files with the following:
+Once you've imported the rule set using either Bzlmod or WORKSPACE, you can then
+load the core rules in your `BUILD` files with the following:
```starlark
load("@rules_python//python:defs.bzl", "py_binary")
@@ -177,5 +88,9 @@
py_binary(
name = "main",
srcs = ["main.py"],
+ deps = [
+ "@my_deps//foo",
+ "@my_deps//bar",
+ ]
)
```
diff --git a/docs/sphinx/index.md b/docs/sphinx/index.md
index 0a9c70f..d7dbb44 100644
--- a/docs/sphinx/index.md
+++ b/docs/sphinx/index.md
@@ -57,6 +57,7 @@
self
getting-started
pypi-dependencies
+configuring
pip
coverage
gazelle
diff --git a/docs/sphinx/toolchains.md b/docs/sphinx/toolchains.md
new file mode 100644
index 0000000..6e0203a
--- /dev/null
+++ b/docs/sphinx/toolchains.md
@@ -0,0 +1,223 @@
+# Configuring Python toolchains and runtimes
+
+This documents how to configure the Python toolchain and runtimes for different
+use cases.
+
+## Bzlmod MODULE configuration
+
+How to configure `rules_python` in your MODULE.bazel file depends on how and why
+you're using Python. There are 4 basic use cases:
+
+1. A root module that always uses Python. For example, you're building a
+ Python application.
+2. A library module with dev-only uses of Python. For example, a Java project
+ that only uses Python as part of testing itself.
+3. A library module without version constraints. For example, a rule set with
+ Python build tools, but defers to the user as to what Python version is used
+ for the tools.
+4. A library module with version constraints. For example, a rule set with
+ Python build tools, and the module requires a specific version of Python
+ be used with its tools.
+
+### Root modules
+
+Root modules are always the top-most module. These are special in two ways:
+
+1. Some `rules_python` bzlmod APIs are only respected by the root module.
+2. The root module can force module overrides and specific module dependency
+ ordering.
+
+When configuring `rules_python` for a root module, you typically want to
+explicitly specify the Python version you want to use. This ensures that
+dependencies don't change the Python version out from under you. Remember that
+`rules_python` will set a version by default, but it will change regularly as
+it tracks a recent Python version.
+
+NOTE: If your root module only uses Python for development of the module itself,
+you should read the dev-only library module section.
+
+```
+bazel_dep(name="rules_python", version=...)
+python = use_extension("@rules_python//extensions:python.bzl", "python")
+
+python.toolchain(python_version = "3.12", is_default = True)
+```
+
+### Library modules
+
+A library module is a module that can show up in arbitrary locations in the
+bzlmod module graph -- it's unknown where in the breadth-first search order the
+module will be relative to other modules. For example, `rules_python` is a
+library module.
+
+#### Library modules with dev-only Python usage
+
+A library module with dev-only Python usage is usually one where Python is only
+used as part of its tests. For example, a module for Java rules might run some
+Python program to generate test data, but real usage of the rules don't need
+Python to work. To configure this, follow the root-module setup, but remember to
+specify `dev_dependency = True` to the bzlmod APIs:
+
+```
+# MODULE.bazel
+bazel_dep(name = "rules_python", version=..., dev_dependency = True)
+
+python = use_extension(
+ "@rules_python//extensions:python.bzl",
+ "python",
+ dev_dependency = True
+)
+
+python.toolchain(python_version = "3.12", is_default=True)
+```
+
+#### Library modules without version constraints
+
+A library module without version constraints is one where the version of Python
+used for the Python programs it runs isn't chosen by the module itself. Instead,
+it's up to the root module to pick an appropriate version of Python.
+
+For this case, configuration is simple: just depend on `rules_python` and use
+the normal `//python:py_binary.bzl` et al rules. There is no need to call
+`python.toolchain` -- rules_python ensures _some_ Python version is available,
+but more often the root module will specify some version.
+
+```
+# MODULE.bazel
+bazel_dep(name = "rules_python", version=...)
+```
+
+#### Library modules with version constraints
+
+A library module with version constraints is one where the module requires a
+specific Python version be used with its tools. This has some pros/cons:
+
+* It allows the library's tools to use a different version of Python than
+ the rest of the build. For example, a user's program could use Python 3.12,
+ while the library module's tools use Python 3.10.
+* It reduces the support burden for the library module because the library only needs
+ to test for the particular Python version they intend to run as.
+* It raises the support burden for the library module because the version of
+ Python being used needs to be regularly incremented.
+* It has higher build overhead because additional runtimes and libraries need
+ to be downloaded, and Bazel has to keep additional configuration state.
+
+To configure this, request the Python versions needed in MODULE.bazel and use
+the version-aware rules for `py_binary`.
+
+```
+# MODULE.bazel
+bazel_dep(name = "rules_python", version=...)
+
+python = use_extension("@rules_python//extensions:python.bzl", "python")
+python.toolchain(python_version = "3.12")
+
+# BUILD.bazel
+load("@python_versions//3.12:defs.bzl", "py_binary")
+
+py_binary(...)
+```
+
+### Pinning to a Python version
+
+Pinning to a version allows targets to force that a specific Python version is
+used, even if the root module configures a different version as a default. This
+is most useful for two cases:
+
+1. For submodules to ensure they run with the appropriate Python version
+2. To allow incremental, per-target, upgrading to newer Python versions,
+ typically in a mono-repo situation.
+
+To configure a submodule with the version-aware rules, request the particular
+version you need, then use the `@python_versions` repo to use the rules that
+force specific versions:
+
+```starlark
+python = use_extension("@rules_python//python/extensions:python.bzl", "python")
+
+python.toolchain(
+ python_version = "3.11",
+)
+use_repo(python, "python_versions")
+```
+
+Then use e.g. `load("@python_versions//3.11:defs.bzl", "py_binary")` to use
+the rules that force that particular version. Multiple versions can be specified
+and use within a single build.
+
+For more documentation, see the bzlmod examples under the {gh-path}`examples`
+folder. Look for the examples that contain a `MODULE.bazel` file.
+
+### Other toolchain details
+
+The `python.toolchain()` call makes its contents available under a repo named
+`python_X_Y`, where X and Y are the major and minor versions. For example,
+`python.toolchain(python_version="3.11")` creates the repo `@python_3_11`.
+Remember to call `use_repo()` to make repos visible to your module:
+`use_repo(python, "python_3_11")`
+
+#### Toolchain usage in other rules
+
+Python toolchains can be utilized in other bazel rules, such as `genrule()`, by adding the `toolchains=["@rules_python//python:current_py_toolchain"]` attribute. You can obtain the path to the Python interpreter using the `$(PYTHON2)` and `$(PYTHON3)` ["Make" Variables](https://bazel.build/reference/be/make-variables). See the
+{gh-path}`test_current_py_toolchain <tests/load_from_macro/BUILD.bazel>` target for an example.
+
+
+## Workspace configuration
+
+To import rules_python in your project, you first need to add it to your
+`WORKSPACE` file, using the snippet provided in the
+[release you choose](https://github.com/bazelbuild/rules_python/releases)
+
+To depend on a particular unreleased version, you can do the following:
+
+```starlark
+load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
+
+
+# Update the SHA and VERSION to the lastest version available here:
+# https://github.com/bazelbuild/rules_python/releases.
+
+SHA="84aec9e21cc56fbc7f1335035a71c850d1b9b5cc6ff497306f84cced9a769841"
+
+VERSION="0.23.1"
+
+http_archive(
+ name = "rules_python",
+ sha256 = SHA,
+ strip_prefix = "rules_python-{}".format(VERSION),
+ url = "https://github.com/bazelbuild/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(VERSION,VERSION),
+)
+
+load("@rules_python//python:repositories.bzl", "py_repositories")
+
+py_repositories()
+```
+
+#### Workspace toolchain registration
+
+To register a hermetic Python toolchain rather than rely on a system-installed interpreter for runtime execution, you can add to the `WORKSPACE` file:
+
+```starlark
+load("@rules_python//python:repositories.bzl", "python_register_toolchains")
+
+python_register_toolchains(
+ name = "python_3_11",
+ # Available versions are listed in @rules_python//python:versions.bzl.
+ # We recommend using the same version your team is already standardized on.
+ python_version = "3.11",
+)
+
+load("@python_3_11//:defs.bzl", "interpreter")
+
+load("@rules_python//python:pip.bzl", "pip_parse")
+
+pip_parse(
+ ...
+ python_interpreter_target = interpreter,
+ ...
+)
+```
+
+After registration, your Python targets will use the toolchain's interpreter during execution, but a system-installed interpreter
+is still used to 'bootstrap' Python targets (see https://github.com/bazelbuild/rules_python/issues/691).
+You may also find some quirks while using this toolchain. Please refer to [python-build-standalone documentation's _Quirks_ section](https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html).
