feat: add //python:none as public target to disable exec_interpreter (#2226)
When writing the toolchain docs, I realized there wasn't a public target
to use for
disabling the exec_interpreter.
Fixed by adding an alias to the internal target.
Along the way:
* Add the exec tools and cc toolchains to the doc gen
* A few improvements to the cc/exec tools docs
* Add public bzl file for py_exec_tools_toolchain and PyExecToolsInfo
* Fix a bug in sphinx_bzl where local names were hiding global names
even if the
requested type didn't match (e.g. a macro foo referring to rule foo)
* Fix xrefs in the python/cc/index.md; it wasn't setting the default
domain to bzl
* Fix object type definition for attributes: the object type name was
"attribute",
but everything else was using "attr"; switched to "attr"
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c4c9920..33aecf0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -73,7 +73,8 @@
`TOOL_VERSIONS` for registering patched toolchains please consider setting
the `patch_strip` explicitly to `1` if you depend on this value - in the
future the value may change to default to `0`.
-
+* (toolchains) Added `//python:none`, a special target for use with
+ {obj}`py_exec_tools_toolchain.exec_interpreter` to treat the value as `None`.
### Removed
* (toolchains): Removed accidentally exposed `http_archive` symbol from
diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel
index 2b407db..149e2c5 100644
--- a/docs/BUILD.bazel
+++ b/docs/BUILD.bazel
@@ -72,7 +72,6 @@
deps = [
":bzl_api_docs",
":py_api_srcs",
- ":py_cc_toolchain",
":py_runtime_pair",
"//sphinxdocs/docs:docs_lib",
],
@@ -86,16 +85,18 @@
"//python:pip_bzl",
"//python:py_binary_bzl",
"//python:py_cc_link_params_info_bzl",
+ "//python:py_exec_tools_info_bzl",
+ "//python:py_exec_tools_toolchain_bzl",
"//python:py_executable_info_bzl",
"//python:py_library_bzl",
"//python:py_runtime_bzl",
"//python:py_runtime_info_bzl",
"//python:py_test_bzl",
"//python:repositories_bzl",
+ "//python/cc:py_cc_toolchain_bzl",
"//python/cc:py_cc_toolchain_info_bzl",
"//python/entry_points:py_console_script_binary_bzl",
- "//python/private:py_exec_tools_info_bzl",
- "//python/private:py_exec_tools_toolchain_bzl",
+ "//python/private:py_cc_toolchain_rule_bzl",
"//python/private/common:py_binary_rule_bazel_bzl",
"//python/private/common:py_library_rule_bazel_bzl",
"//python/private/common:py_runtime_rule_bzl",
@@ -113,16 +114,6 @@
)
sphinx_stardoc(
- name = "py_cc_toolchain",
- src = "//python/private:py_cc_toolchain_rule.bzl",
- prefix = "api/rules_python/",
- public_load_path = "//python/cc:py_cc_toolchain.bzl",
- tags = ["docs"],
- target_compatible_with = _TARGET_COMPATIBLE_WITH,
- deps = ["//python/cc:py_cc_toolchain_bzl"],
-)
-
-sphinx_stardoc(
name = "py_runtime_pair",
src = "//python/private:py_runtime_pair_rule_bzl",
prefix = "api/rules_python/",
diff --git a/docs/api/rules_python/python/cc/index.md b/docs/api/rules_python/python/cc/index.md
index 233b130..82c5934 100644
--- a/docs/api/rules_python/python/cc/index.md
+++ b/docs/api/rules_python/python/cc/index.md
@@ -1,3 +1,5 @@
+:::{default-domain} bzl
+:::
:::{bzl:currentfile} //python/cc:BUILD.bazel
:::
# //python/cc
@@ -31,4 +33,9 @@
Toolchain type identifier for the Python C toolchain.
This toolchain type is typically implemented by {obj}`py_cc_toolchain`.
+
+::::{seealso}
+{any}`Custom Toolchains` for how to define custom toolchains
+::::
+
:::
diff --git a/docs/api/rules_python/python/index.md b/docs/api/rules_python/python/index.md
index 6ce5e7c..bc5a731 100644
--- a/docs/api/rules_python/python/index.md
+++ b/docs/api/rules_python/python/index.md
@@ -10,7 +10,7 @@
Identifier for the toolchain type for the target platform.
This toolchain type gives information about the runtime for the target platform.
-It is typically implemented by the {obj}`py_runtime` rule
+It is typically implemented by the {obj}`py_runtime` rule.
::::{seealso}
{any}`Custom Toolchains` for how to define custom toolchains
@@ -21,6 +21,14 @@
:::{bzl:target} exec_tools_toolchain_type
Identifier for the toolchain type for exec tools used to build Python targets.
+
+This toolchain type gives information about tools needed to build Python targets
+at build time. It is typically implemented by the {obj}`py_exec_tools_toolchain`
+rule.
+
+::::{seealso}
+{any}`Custom Toolchains` for how to define custom toolchains
+::::
:::
:::{bzl:target} current_py_toolchain
@@ -28,7 +36,7 @@
Helper target to resolve to the consumer's current Python toolchain. This target
provides:
-* `PyRuntimeInfo`: The consuming target's target toolchain information
+* {obj}`PyRuntimeInfo`: The consuming target's target toolchain information
:::
@@ -42,3 +50,16 @@
:::
::::
+:::{target} none
+A special target so that label attributes with default values can be set to
+`None`.
+
+Bazel interprets `None` to mean "use the default value", which
+makes it impossible to have a label attribute with a default value that is
+optional. To work around this, a target with a special provider is used;
+internally rules check for this, then treat the value as `None`.
+
+::::{versionadded} 0.36.0
+::::
+
+:::
diff --git a/docs/toolchains.md b/docs/toolchains.md
index b5f664f..2ac0099 100644
--- a/docs/toolchains.md
+++ b/docs/toolchains.md
@@ -395,7 +395,7 @@
py_exec_tools_toolchain(
name = "exec_tools_toolchain_impl",
- exec_interpreter = "@rules_python/python:null_target",
+ exec_interpreter = "@rules_python/python:none",
precompiler = "precompiler-cpython-3.12"
)
diff --git a/python/BUILD.bazel b/python/BUILD.bazel
index 40880a1..6fcde38 100644
--- a/python/BUILD.bazel
+++ b/python/BUILD.bazel
@@ -140,6 +140,18 @@
)
bzl_library(
+ name = "py_exec_tools_info_bzl",
+ srcs = ["py_exec_tools_info.bzl"],
+ deps = ["//python/private:py_exec_tools_info_bzl"],
+)
+
+bzl_library(
+ name = "py_exec_tools_toolchain_bzl",
+ srcs = ["py_exec_tools_toolchain.bzl"],
+ deps = ["//python/private:py_exec_tools_toolchain_bzl"],
+)
+
+bzl_library(
name = "py_executable_info_bzl",
srcs = ["py_executable_info.bzl"],
deps = ["//python/private:py_executable_info_bzl"],
@@ -308,6 +320,12 @@
visibility = ["//visibility:public"],
)
+# Special target to indicate `None` for label attributes a default value.
+alias(
+ name = "none",
+ actual = "//python/private:sentinel",
+)
+
# Definitions for a Python toolchain that, at execution time, attempts to detect
# a platform runtime having the appropriate major Python version. Consider this
# a toolchain of last resort.
diff --git a/python/cc/BUILD.bazel b/python/cc/BUILD.bazel
index d384d05..f4e4aeb 100644
--- a/python/cc/BUILD.bazel
+++ b/python/cc/BUILD.bazel
@@ -40,7 +40,7 @@
name = "py_cc_toolchain_bzl",
srcs = ["py_cc_toolchain.bzl"],
visibility = ["//visibility:public"],
- deps = ["//python/private:py_cc_toolchain_bzl"],
+ deps = ["//python/private:py_cc_toolchain_macro_bzl"],
)
bzl_library(
diff --git a/python/cc/py_cc_toolchain_info.bzl b/python/cc/py_cc_toolchain_info.bzl
index 9ea394a..3164f89 100644
--- a/python/cc/py_cc_toolchain_info.bzl
+++ b/python/cc/py_cc_toolchain_info.bzl
@@ -1,4 +1,4 @@
-# Copyright 2023 The Bazel Authors. All rights reserved.
+# Copyright 2024 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -11,11 +11,12 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
+"""Provider for C/C++ information from the toolchain.
-"""Provider for C/C++ information about the Python runtime.
-
-NOTE: This is a beta-quality feature. APIs subject to change until
-https://github.com/bazelbuild/rules_python/issues/824 is considered done.
+:::{seealso}
+* {any}`Custom toolchains` for how to define custom toolchains.
+* {obj}`py_cc_toolchain` rule for defining the toolchain.
+:::
"""
load("//python/private:py_cc_toolchain_info.bzl", _PyCcToolchainInfo = "PyCcToolchainInfo")
diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel
index a35e2f7..e0de7d3 100644
--- a/python/private/BUILD.bazel
+++ b/python/private/BUILD.bazel
@@ -168,15 +168,16 @@
)
bzl_library(
- name = "py_cc_toolchain_bzl",
- srcs = [
- "py_cc_toolchain_macro.bzl",
- "py_cc_toolchain_rule.bzl",
+ name = "py_cc_toolchain_macro_bzl",
+ srcs = ["py_cc_toolchain_macro.bzl"],
+ deps = [
+ ":py_cc_toolchain_rule_bzl",
],
- visibility = [
- "//docs:__subpackages__",
- "//python/cc:__pkg__",
- ],
+)
+
+bzl_library(
+ name = "py_cc_toolchain_rule_bzl",
+ srcs = ["py_cc_toolchain_rule.bzl"],
deps = [
":py_cc_toolchain_info_bzl",
":rules_cc_srcs_bzl",
@@ -188,7 +189,6 @@
bzl_library(
name = "py_cc_toolchain_info_bzl",
srcs = ["py_cc_toolchain_info.bzl"],
- visibility = ["//python/cc:__pkg__"],
)
bzl_library(
@@ -370,7 +370,6 @@
[
"coverage.patch",
"repack_whl.py",
- "py_cc_toolchain_rule.bzl",
"py_package.bzl",
"py_wheel.bzl",
"py_wheel_normalize_pep440.bzl",
diff --git a/python/private/py_cc_toolchain_macro.bzl b/python/private/py_cc_toolchain_macro.bzl
index 35276f7..416caac 100644
--- a/python/private/py_cc_toolchain_macro.bzl
+++ b/python/private/py_cc_toolchain_macro.bzl
@@ -22,8 +22,10 @@
def py_cc_toolchain(**kwargs):
"""Creates a py_cc_toolchain target.
+ This is a macro around the {rule}`py_cc_toolchain` rule.
+
Args:
- **kwargs: Keyword args to pass onto underlying rule.
+ **kwargs: Keyword args to pass onto underlying {rule}`py_cc_toolchain` rule.
"""
# This tag is added to easily identify usages through other macros.
diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl
index 2c52a2e..279f86c 100644
--- a/python/private/py_cc_toolchain_rule.bzl
+++ b/python/private/py_cc_toolchain_rule.bzl
@@ -78,5 +78,11 @@
This rule carries information about the C/C++ side of a Python runtime, e.g.
headers, shared libraries, etc.
+
+This provides `ToolchainInfo` with the following attributes:
+* `py_cc_toolchain`: {type}`PyCcToolchainInfo`
+* `toolchain_label`: {type}`Label` _only present when `--visibile_for_testing=True`
+ for internal testing_. The rule's label; this allows identifying what toolchain
+ implmentation was selected for testing purposes.
""",
)
diff --git a/python/private/py_exec_tools_toolchain.bzl b/python/private/py_exec_tools_toolchain.bzl
index 26c09ca..957448f 100644
--- a/python/private/py_exec_tools_toolchain.bzl
+++ b/python/private/py_exec_tools_toolchain.bzl
@@ -39,20 +39,42 @@
py_exec_tools_toolchain = rule(
implementation = _py_exec_tools_toolchain_impl,
+ doc = """
+Provides a toolchain for build time tools.
+
+This provides `ToolchainInfo` with the following attributes:
+* `exec_tools`: {type}`PyExecToolsInfo`
+* `toolchain_label`: {type}`Label` _only present when `--visibile_for_testing=True`
+ for internal testing_. The rule's label; this allows identifying what toolchain
+ implmentation was selected for testing purposes.
+""",
attrs = {
"exec_interpreter": attr.label(
default = "//python/private:current_interpreter_executable",
cfg = "exec",
doc = """
-The interpreter to use in the exec config. To disable, specify the
-special target `//python/private:sentinel`. See PyExecToolsInfo.exec_interpreter
-for further docs.
+An interpreter that is directly usable in the exec configuration
+
+If not specified, the interpreter from {obj}`//python:toolchain_type` will
+be used.
+
+To disable, specify the special target {obj}`//python:none`; the raw value `None`
+will use the default.
+
+:::{note}
+This is only useful for `ctx.actions.run` calls that _directly_ invoke the
+interpreter, which is fairly uncommon and low level. It is better to use a
+`cfg="exec"` attribute that points to a `py_binary` rule instead, which will
+handle all the necessary transitions and runtime setup to invoke a program.
+:::
+
+See {obj}`PyExecToolsInfo.exec_interpreter` for further docs.
""",
),
"precompiler": attr.label(
allow_files = True,
cfg = "exec",
- doc = "See PyExecToolsInfo.precompiler",
+ doc = "See {obj}`PyExecToolsInfo.precompiler`",
),
"_visible_for_testing": attr.label(
default = "//python/private:visible_for_testing",
diff --git a/python/py_exec_tools_info.bzl b/python/py_exec_tools_info.bzl
new file mode 100644
index 0000000..4384123
--- /dev/null
+++ b/python/py_exec_tools_info.bzl
@@ -0,0 +1,24 @@
+# Copyright 2024 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Provider for the exec tools toolchain.
+
+:::{seealso}
+* {any}`Custom toolchains` for how to define custom toolchains.
+* {obj}`py_cc_toolchain` rule for defining the toolchain.
+:::
+"""
+
+load("//python/private:py_exec_tools_info.bzl", _PyExecToolsInfo = "PyExecToolsInfo")
+
+PyExecToolsInfo = _PyExecToolsInfo
diff --git a/python/py_exec_tools_toolchain.bzl b/python/py_exec_tools_toolchain.bzl
new file mode 100644
index 0000000..6e0a663
--- /dev/null
+++ b/python/py_exec_tools_toolchain.bzl
@@ -0,0 +1,18 @@
+# Copyright 2024 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Toolchain for build-time tools."""
+
+load("//python/private:py_exec_tools_toolchain.bzl", _py_exec_tools_toolchain = "py_exec_tools_toolchain")
+
+py_exec_tools_toolchain = _py_exec_tools_toolchain
diff --git a/sphinxdocs/inventories/bazel_inventory.txt b/sphinxdocs/inventories/bazel_inventory.txt
index fd2bac7..969c772 100644
--- a/sphinxdocs/inventories/bazel_inventory.txt
+++ b/sphinxdocs/inventories/bazel_inventory.txt
@@ -59,7 +59,7 @@
ctx.workspace_name bzl:obj 1 rules/lib/builtins/ctx#workspace_name -
depset bzl:type 1 rules/lib/depset -
dict bzl:type 1 rules/lib/dict -
-exec_compatible_with bzl:attribute 1 reference/be/common-definitions#common.exec_compatible_with -
+exec_compatible_with bzl:attr 1 reference/be/common-definitions#common.exec_compatible_with -
int bzl:type 1 rules/lib/int -
label bzl:type 1 concepts/labels -
list bzl:type 1 rules/lib/list -
@@ -133,13 +133,13 @@
runfiles.symlinks bzl:type 1 rules/lib/builtins/runfiles#symlinks -
str bzl:type 1 rules/lib/string -
struct bzl:type 1 rules/lib/builtins/struct -
-target_compatible_with bzl:attribute 1 reference/be/common-definitions#common.target_compatible_with -
+target_compatible_with bzl:attr 1 reference/be/common-definitions#common.target_compatible_with -
testing bzl:obj 1 rules/lib/toplevel/testing -
testing.ExecutionInfo bzl:function 1 rules/lib/toplevel/testing#ExecutionInfo -
testing.TestEnvironment bzl:function 1 rules/lib/toplevel/testing#TestEnvironment -
testing.analysis_test bzl:rule 1 rules/lib/toplevel/testing#analysis_test -
toolchain bzl:rule 1 reference/be/platforms-and-toolchains#toolchain -
toolchain.exec_compatible_with bzl:rule 1 reference/be/platforms-and-toolchains#toolchain.exec_compatible_with -
-toolchain.target_settings bzl:attribute 1 reference/be/platforms-and-toolchains#toolchain.target_settings -
-toolchain.target_compatible_with bzl:attribute 1 reference/be/platforms-and-toolchains#toolchain.target_compatible_with -
+toolchain.target_settings bzl:attr 1 reference/be/platforms-and-toolchains#toolchain.target_settings -
+toolchain.target_compatible_with bzl:attr 1 reference/be/platforms-and-toolchains#toolchain.target_compatible_with -
toolchain_type bzl:type 1 rules/lib/builtins/toolchain_type.html -
diff --git a/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/src/sphinx_bzl/bzl.py
index cbd35a9..54b1285 100644
--- a/sphinxdocs/src/sphinx_bzl/bzl.py
+++ b/sphinxdocs/src/sphinx_bzl/bzl.py
@@ -1439,9 +1439,7 @@
object_types = {
"arg": domains.ObjType("arg", "arg", "obj"), # macro/function arg
"aspect": domains.ObjType("aspect", "aspect", "obj"),
- "attribute": domains.ObjType(
- "attribute", "attribute", "attr", "obj"
- ), # rule attribute
+ "attr": domains.ObjType("attr", "attr", "obj"), # rule attribute
"function": domains.ObjType("function", "func", "obj"),
"method": domains.ObjType("method", "method", "obj"),
"module-extension": domains.ObjType(
@@ -1460,6 +1458,7 @@
# types are objects that have a constructor and methods/attrs
"type": domains.ObjType("type", "type", "obj"),
}
+
# This controls:
# * What is recognized when parsing, e.g. ":bzl:ref:`foo`" requires
# "ref" to be in the role dict below.
@@ -1508,7 +1507,7 @@
# dict[str, dict[str, _ObjectEntry]]
"doc_names": {},
# Objects by a shorter or alternative name
- # dict[str, _ObjectEntry]
+ # dict[str, dict[str id, _ObjectEntry]]
"alt_names": {},
}
@@ -1588,8 +1587,14 @@
# Note that the flag value could contain `=`
if "=" in target:
target = target[: target.find("=")]
+
if target in self.data["doc_names"].get(fromdocname, {}):
- return self.data["doc_names"][fromdocname][target]
+ entry = self.data["doc_names"][fromdocname][target]
+ # Prevent a local doc name masking a global alt name when its of
+ # a different type. e.g. when the macro `foo` refers to the
+ # rule `foo` in another doc.
+ if object_type in self.object_types[entry.object_type].roles:
+ return entry
if object_type == "obj":
search_space = self.data["objects"]
@@ -1600,7 +1605,15 @@
_log_debug("find_entry: alt_names=%s", sorted(self.data["alt_names"].keys()))
if target in self.data["alt_names"]:
- return self.data["alt_names"][target]
+ # Give preference to shorter object ids. This is a work around
+ # to allow e.g. `FooInfo` to refer to the FooInfo type rather than
+ # the `FooInfo` constructor.
+ entries = sorted(
+ self.data["alt_names"][target].items(), key=lambda item: len(item[0])
+ )
+ for _, entry in entries:
+ if object_type in self.object_types[entry.object_type].roles:
+ return entry
return None
@@ -1633,17 +1646,8 @@
alt_names.append(label + (f"%{symbol}" if symbol else ""))
for alt_name in sorted(set(alt_names)):
- if alt_name in self.data["alt_names"]:
- existing = self.data["alt_names"][alt_name]
- # This situation usually occurs for the constructor function
- # of a provider, but could occur for e.g. an exported struct
- # with an attribute the same name as the struct. For lack
- # of a better option, take the shorter entry, on the assumption
- # it refers to some container of the longer entry.
- if len(entry.full_id) < len(existing.full_id):
- self.data["alt_names"][alt_name] = entry
- else:
- self.data["alt_names"][alt_name] = entry
+ self.data["alt_names"].setdefault(alt_name, {})
+ self.data["alt_names"][alt_name][entry.full_id] = entry
docname = entry.index_entry.docname
self.data["doc_names"].setdefault(docname, {})
@@ -1653,11 +1657,11 @@
self, docnames: list[str], otherdata: dict[str, typing.Any]
) -> None:
# Merge in simple dict[key, value] data
- for top_key in ("objects", "alt_names"):
+ for top_key in ("objects",):
self.data[top_key].update(otherdata.get(top_key, {}))
# Merge in two-level dict[top_key, dict[sub_key, value]] data
- for top_key in ("objects_by_type", "doc_names"):
+ for top_key in ("objects_by_type", "doc_names", "alt_names"):
existing_top_map = self.data[top_key]
for sub_key, sub_values in otherdata.get(top_key, {}).items():
if sub_key not in existing_top_map:
