feat(toolchains): ABI3 Python headers target (#3274)
Until now, we silently link extensions with both stable and unstable ABI
libs,
with the latter taking precedence in symbol resolution, because it
appears first
in the linker command AND, crucially, contains all CPython symbols
present in
the stable ABI library, thus overriding them. This has the effect that
stable
ABI extensions on Windows are usable only with the Python distribution
that they
were built on.
To fix, a separate ABI3 header target is introduced, and should be used
for C++
extensions on Windows if stable ABI builds are requested.
Idea as formulated by `@dgrunwald-qt` in
https://github.com/nicholasjng/nanobind-bazel/issues/72#issuecomment-3249959583.
This is motivated by
https://github.com/nicholasjng/nanobind-bazel/issues/72.
This change shifts stable ABI selection on Windows to the extension
developer, where
it has arguably always been (they had to set the `Py_LIMITED_API`
macro).
An upside of this approach is that with a separate target, the question
"stable ABI
or not" can be decided on an extension-by-extension basis, giving
maximum flexibility
to developers. This should not influence the wheel platform target,
because a wheel
is marked ABI3 if and only if all of its extensions are marked as ABI3.
---------
Co-authored-by: Richard Levasseur <rlevasseur@google.com>
Co-authored-by: Richard Levasseur <richardlev@gmail.com>
diff --git a/AGENTS.md b/AGENTS.md
index 3d5219c..671b85c 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -19,6 +19,14 @@
When tasks complete successfully, quote Monty Python, but work it naturally
into the sentence, not verbatim.
+When adding `{versionadded}` or `{versionchanged}` sections, add them add the
+end of the documentation text.
+
+### Starlark style
+
+For doc strings, using triple quoted strings when the doc string is more than
+three lines. Do not use a trailing backslack (`\`) for the opening triple-quote.
+
### bzl_library targets for bzl source files
* A `bzl_library` target should be defined for every `.bzl` file outside
@@ -78,7 +86,17 @@
* Act as an expert in tech writing, Sphinx, MyST, and markdown.
* Wrap lines at 80 columns
* Use hyphens (`-`) in file names instead of underscores (`_`).
+ * In Sphinx MyST markup, outer directives must have more colons than inner
+ directives. For example:
+ ```
+ ::::{outerdirective}
+ outer text
+ :::{innertdirective}
+ inner text
+ :::
+ ::::
+ ```
Generated API references can be found by:
* Running `bazel build //docs:docs` and inspecting the generated files
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7e5598a..02f1df6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -96,6 +96,15 @@
`WORKSPACE` files. See the
{ref}`common-deps-with-multiple-pypi-versions` guide on using common
dependencies with multiple PyPI versions` for an example.
+* (toolchains) Stable ABI headers support added. To use, depend on
+ {obj}`//python/cc:current_py_cc_headers_abi3`. This allows Windows builds
+ a way to depend on headers without the potentially Python unstable ABI
+ objects from the regular {obj}`//python/cc:current_py_cc_headers` target
+ being included.
+ * Adds {obj}`//python/cc:current_py_cc_headers_abi3`,
+ {obj}`py_cc_toolchain.headers_abi3`, and {obj}`PyCcToolchainInfo.headers_abi3`.
+ * {obj}`//python:features.bzl%features.headers_abi3` can be used to
+ feature-detect the presense of the above.
{#v1-6-2}
## [1.6.2] - 2025-09-21
diff --git a/docs/api/rules_python/python/cc/index.md b/docs/api/rules_python/python/cc/index.md
index 82c5934..2f4e3ae 100644
--- a/docs/api/rules_python/python/cc/index.md
+++ b/docs/api/rules_python/python/cc/index.md
@@ -4,7 +4,7 @@
:::
# //python/cc
-:::{bzl:target} current_py_cc_headers
+::::{bzl:target} current_py_cc_headers
A convenience target that provides the Python headers. It uses toolchain
resolution to find the headers for the Python runtime matching the interpreter
@@ -14,8 +14,33 @@
This target provides:
* `CcInfo`: The C++ information about the Python headers.
+
+:::{seealso}
+
+The {obj}`:current_py_cc_headers_abi3` target for explicitly using the
+stable ABI.
:::
+::::
+
+::::{bzl:target} current_py_cc_headers_abi3
+
+A convenience target that provides the Python ABI3 headers (stable ABI headers).
+It uses toolchain resolution to find the headers for the Python runtime matching
+the interpreter that will be used. This basically forwards the underlying
+`cc_library(name="python_headers_abi3")` target defined in the `@python_X_Y`
+repo.
+
+This target provides:
+
+* `CcInfo`: The C++ information about the Python ABI3 headers.
+
+:::{versionadded} VERSION_NEXT_FEATURE
+The {obj}`features.headers_abi3` attribute can be used to detect if this target
+is available or not.
+:::
+::::
+
:::{bzl:target} current_py_cc_libs
A convenience target that provides the Python libraries. It uses toolchain
diff --git a/docs/howto/python-headers.md b/docs/howto/python-headers.md
index f830feb..fa8c2ce 100644
--- a/docs/howto/python-headers.md
+++ b/docs/howto/python-headers.md
@@ -8,9 +8,10 @@
[toolchain](toolchains).
The recommended way to get the headers is to depend on the
-`@rules_python//python/cc:current_py_cc_headers` target. This is a helper
-target that uses toolchain resolution to find the correct headers for the
-target platform.
+{obj}`@rules_python//python/cc:current_py_cc_headers` or
+{obj}`@rules_python//python/cc:current_py_cc_headers_abi3`
+targets. These are convenience targets that use toolchain resolution to find
+the correct headers for the target platform.
## Using the headers
@@ -27,4 +28,27 @@
```
This setup ensures that your C extension code can find and use the Python
-headers during compilation.
\ No newline at end of file
+headers during compilation.
+
+:::{note}
+The `:current_py_cc_headers` target provides all the Python headers. This _may_
+include ABI-specific information.
+:::
+
+## Using the stable ABI headers
+
+If you're building for the [Python stable ABI](https://docs.python.org/3/c-api/stable.html),
+then depend on {obj}`@rules_python//python/cc:current_py_cc_headers_abi3`. This
+target contains only objects relevant to the Python stable ABI. Remember to
+define
+[`Py_LIMITED_API`](https://docs.python.org/3/c-api/stable.html#c.Py_LIMITED_API)
+when building such extensions.
+
+```bazel
+# BUILD.bazel
+cc_library(
+ name = "my_stable_abi_extension",
+ srcs = ["my_stable_abi_extension.c"],
+ deps = ["@rules_python//python/cc:current_py_cc_headers_abi3"],
+)
+```
diff --git a/docs/pyproject.toml b/docs/pyproject.toml
index 2bcb31b..9a089df 100644
--- a/docs/pyproject.toml
+++ b/docs/pyproject.toml
@@ -12,5 +12,6 @@
"readthedocs-sphinx-ext",
"absl-py",
"typing-extensions",
- "sphinx-reredirects"
+ "sphinx-reredirects",
+ "pefile"
]
diff --git a/docs/requirements.txt b/docs/requirements.txt
index cda477c..5929e73 100644
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -111,9 +111,9 @@
--hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \
--hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6
# via sphinx
-docutils==0.22 \
- --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \
- --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f
+docutils==0.21.2 \
+ --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \
+ --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2
# via
# myst-parser
# sphinx
@@ -232,6 +232,10 @@
# via
# readthedocs-sphinx-ext
# sphinx
+pefile==2024.8.26 \
+ --hash=sha256:3ff6c5d8b43e8c37bb6e6dd5085658d658a7a0bdcd20b6a07b1fcfc1c4e9d632 \
+ --hash=sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f
+ # via rules-python-docs (docs/pyproject.toml)
pygments==2.19.2 \
--hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \
--hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b
diff --git a/python/cc/BUILD.bazel b/python/cc/BUILD.bazel
index f4e4aeb..f7686c4 100644
--- a/python/cc/BUILD.bazel
+++ b/python/cc/BUILD.bazel
@@ -2,7 +2,7 @@
load("@bazel_skylib//:bzl_library.bzl", "bzl_library")
load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED")
-load("//python/private:current_py_cc_headers.bzl", "current_py_cc_headers")
+load("//python/private:current_py_cc_headers.bzl", "current_py_cc_headers", "current_py_cc_headers_abi3")
load("//python/private:current_py_cc_libs.bzl", "current_py_cc_libs")
package(
@@ -20,6 +20,17 @@
visibility = ["//visibility:public"],
)
+# This target provides the C ABI3 headers for whatever the current toolchain is
+# for the consuming rule. It basically acts like a cc_library by forwarding
+# on the providers for the underlying cc_library that the toolchain is using.
+current_py_cc_headers_abi3(
+ name = "current_py_cc_headers_abi3",
+ # Building this directly will fail unless a py cc toolchain is registered,
+ # and it's only under bzlmod that one is registered by default.
+ tags = [] if BZLMOD_ENABLED else ["manual"],
+ visibility = ["//visibility:public"],
+)
+
# This target provides the C libraries for whatever the current toolchain is for
# the consuming rule. It basically acts like a cc_library by forwarding on the
# providers for the underlying cc_library that the toolchain is using.
diff --git a/python/features.bzl b/python/features.bzl
index e3d1ffd..21ff588 100644
--- a/python/features.bzl
+++ b/python/features.bzl
@@ -22,6 +22,16 @@
def _features_typedef():
"""Information about features rules_python has implemented.
+ ::::{field} headers_abi3
+ :type: bool
+
+ True if the {obj}`@rules_python//python/cc:current_py_cc_headers_abi3`
+ target is available.
+
+ :::{versionadded} VERSION_NEXT_FEATURE
+ :::
+ ::::
+
::::{field} precompile
:type: bool
@@ -60,6 +70,7 @@
features = struct(
TYPEDEF = _features_typedef,
# keep sorted
+ headers_abi3 = True,
precompile = True,
py_info_venv_symlinks = True,
uses_builtin_rules = not config.enable_pystar,
diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel
index 5e2043c..0c8ccde 100644
--- a/python/private/BUILD.bazel
+++ b/python/private/BUILD.bazel
@@ -31,6 +31,7 @@
name = "distribution",
srcs = glob(["**"]) + [
"//python/private/api:distribution",
+ "//python/private/cc:distribution",
"//python/private/pypi:distribution",
"//python/private/whl_filegroup:distribution",
"//tools/build_defs/python/private:distribution",
@@ -360,6 +361,7 @@
":common_labels.bzl",
":py_cc_toolchain_info_bzl",
":rules_cc_srcs_bzl",
+ ":sentinel_bzl",
":util_bzl",
"@bazel_skylib//rules:common_settings",
],
diff --git a/python/private/cc/BUILD.bazel b/python/private/cc/BUILD.bazel
new file mode 100644
index 0000000..8f4fb46
--- /dev/null
+++ b/python/private/cc/BUILD.bazel
@@ -0,0 +1,20 @@
+load("@rules_cc//cc:cc_library.bzl", "cc_library")
+load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC")
+
+package(
+ default_visibility = ["//:__subpackages__"],
+)
+
+licenses(["notice"])
+
+filegroup(
+ name = "distribution",
+ srcs = glob(["**"]),
+)
+
+# An empty cc target for use when a cc target is needed to satisfy
+# Bazel, but its contents don't matter.
+cc_library(
+ name = "empty",
+ visibility = NOT_ACTUALLY_PUBLIC,
+)
diff --git a/python/private/current_py_cc_headers.bzl b/python/private/current_py_cc_headers.bzl
index 217904c..ef64631 100644
--- a/python/private/current_py_cc_headers.bzl
+++ b/python/private/current_py_cc_headers.bzl
@@ -12,19 +12,21 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-"""Implementation of current_py_cc_headers rule."""
+"""Implementation of current_py_cc_headers and current_py_cc_headers_abi3 rules.
+"""
load("@rules_cc//cc/common:cc_info.bzl", "CcInfo")
+load("//python/private:toolchain_types.bzl", "PY_CC_TOOLCHAIN_TYPE")
def _current_py_cc_headers_impl(ctx):
- py_cc_toolchain = ctx.toolchains["//python/cc:toolchain_type"].py_cc_toolchain
+ py_cc_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE].py_cc_toolchain
return py_cc_toolchain.headers.providers_map.values()
current_py_cc_headers = rule(
implementation = _current_py_cc_headers_impl,
- toolchains = ["//python/cc:toolchain_type"],
+ toolchains = [PY_CC_TOOLCHAIN_TYPE],
provides = [CcInfo],
- doc = """\
+ doc = """
Provides the currently active Python toolchain's C headers.
This is a wrapper around the underlying `cc_library()` for the
@@ -41,3 +43,40 @@
```
""",
)
+
+def _current_py_cc_headers_abi3_impl(ctx):
+ py_cc_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE].py_cc_toolchain
+ if not py_cc_toolchain.headers_abi3:
+ fail((
+ "The resolved {} toolchain does not provide abi3 headers. " +
+ "Verify the toolchain sets `.headers_abi3`, or use the " +
+ "`:current_py_cc_headers` target."
+ ).format(
+ PY_CC_TOOLCHAIN_TYPE,
+ ))
+ return py_cc_toolchain.headers_abi3.providers_map.values()
+
+current_py_cc_headers_abi3 = rule(
+ implementation = _current_py_cc_headers_abi3_impl,
+ toolchains = [PY_CC_TOOLCHAIN_TYPE],
+ provides = [CcInfo],
+ doc = """
+Provides the currently active Python toolchain's C ABI3 headers.
+
+This is a wrapper around the underlying `cc_library()` for the
+C ABI3 headers for the consuming target's currently active Python toolchain.
+
+To use, simply depend on this target where you would have wanted the
+toolchain's underlying `:python_headers_abi3` target:
+
+```starlark
+cc_library(
+ name = "foo",
+ deps = ["@rules_python//python/cc:current_py_cc_headers_abi3"]
+)
+```
+
+:::{versionadded} VERSION_NEXT_FEATURE
+:::
+""",
+)
diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl
index a35cd6b..a35ce8a 100644
--- a/python/private/hermetic_runtime_repo_setup.bzl
+++ b/python/private/hermetic_runtime_repo_setup.bzl
@@ -107,9 +107,9 @@
srcs = native.glob(["include/**/*.h"]),
)
cc_library(
- name = "python_headers",
+ name = "python_headers_abi3",
deps = select({
- "@bazel_tools//src/conditions:windows": [":interface", ":abi3_interface"],
+ "@bazel_tools//src/conditions:windows": [":abi3_interface"],
"//conditions:default": None,
}),
hdrs = [":includes"],
@@ -125,6 +125,13 @@
],
}),
)
+ cc_library(
+ name = "python_headers",
+ deps = [":python_headers_abi3"] + select({
+ "@bazel_tools//src/conditions:windows": [":interface"],
+ "//conditions:default": [],
+ }),
+ )
native.config_setting(
name = "is_freethreaded_linux",
flag_values = {
@@ -239,6 +246,7 @@
py_cc_toolchain(
name = "py_cc_toolchain",
headers = ":python_headers",
+ headers_abi3 = ":python_headers_abi3",
# TODO #3155: add libctl, libtk
libs = ":libpython",
python_version = python_version,
diff --git a/python/private/local_runtime_repo_setup.bzl b/python/private/local_runtime_repo_setup.bzl
index 5d3a781..6cff1ae 100644
--- a/python/private/local_runtime_repo_setup.bzl
+++ b/python/private/local_runtime_repo_setup.bzl
@@ -67,26 +67,34 @@
# See https://docs.python.org/3/extending/windows.html
# However not all python installations (such as manylinux) include shared or static libraries,
# so only create the import library when interface_library is set.
- import_deps = []
+ full_abi_deps = []
+ abi3_deps = []
if interface_library:
cc_import(
name = "_python_interface_library",
interface_library = interface_library,
system_provided = 1,
)
- import_deps = [":_python_interface_library"]
+ if interface_library.endswith("{}.lib".format(major)):
+ abi3_deps = [":_python_interface_library"]
+ else:
+ full_abi_deps = [":_python_interface_library"]
cc_library(
- name = "_python_headers",
+ name = "_python_headers_abi3",
# NOTE: Keep in sync with watch_tree() called in local_runtime_repo
srcs = native.glob(
include = ["include/**/*.h"],
exclude = ["include/numpy/**"], # numpy headers are handled separately
allow_empty = True, # A Python install may not have C headers
),
- deps = import_deps,
+ deps = abi3_deps,
includes = ["include"],
)
+ cc_library(
+ name = "_python_headers",
+ deps = [":_python_headers_abi3"] + full_abi_deps,
+ )
cc_library(
name = "_libpython",
@@ -123,6 +131,7 @@
py_cc_toolchain(
name = "py_cc_toolchain",
headers = ":_python_headers",
+ headers_abi3 = ":_python_headers_abi3",
libs = ":_libpython",
python_version = major_minor_micro,
visibility = ["//visibility:public"],
diff --git a/python/private/py_cc_toolchain_info.bzl b/python/private/py_cc_toolchain_info.bzl
index c5cdbd9..8cb3680 100644
--- a/python/private/py_cc_toolchain_info.bzl
+++ b/python/private/py_cc_toolchain_info.bzl
@@ -40,7 +40,36 @@
e.g. `:current_py_cc_headers` to act as the underlying headers target it
represents).
""",
- "libs": """\
+ "headers_abi3": """
+:type: struct | None
+
+If available, information about ABI3 (stable ABI) header files, struct with
+fields:
+ * providers_map: a dict of string to provider instances. The key should be
+ a fully qualified name (e.g. `@rules_foo//bar:baz.bzl#MyInfo`) of the
+ provider to uniquely identify its type.
+
+ The following keys are always present:
+ * CcInfo: the CcInfo provider instance for the headers.
+ * DefaultInfo: the DefaultInfo provider instance for the headers.
+
+ A map is used to allow additional providers from the originating headers
+ target (typically a `cc_library`) to be propagated to consumers (directly
+ exposing a Target object can cause memory issues and is an anti-pattern).
+
+ When consuming this map, it's suggested to use `providers_map.values()` to
+ return all providers; or copy the map and filter out or replace keys as
+ appropriate. Note that any keys beginning with `_` (underscore) are
+ considered private and should be forward along as-is (this better allows
+ e.g. `:current_py_cc_headers` to act as the underlying headers target it
+ represents).
+
+:::{versionadded} VERSION_NEXT_FEATURE
+The {obj}`features.headers_abi3` attribute can be used to detect if this
+attribute is available or not.
+:::
+""",
+ "libs": """
:type: struct | None
If available, information about C libraries, struct with fields:
diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl
index 8adf73c..b5c997e 100644
--- a/python/private/py_cc_toolchain_rule.bzl
+++ b/python/private/py_cc_toolchain_rule.bzl
@@ -22,6 +22,7 @@
load("@rules_cc//cc/common:cc_info.bzl", "CcInfo")
load(":common_labels.bzl", "labels")
load(":py_cc_toolchain_info.bzl", "PyCcToolchainInfo")
+load(":sentinel.bzl", "SentinelInfo")
def _py_cc_toolchain_impl(ctx):
if ctx.attr.libs:
@@ -34,6 +35,16 @@
else:
libs = None
+ if ctx.attr.headers_abi3 and SentinelInfo not in ctx.attr.headers_abi3:
+ headers_abi3 = struct(
+ providers_map = {
+ "CcInfo": ctx.attr.headers_abi3[CcInfo],
+ "DefaultInfo": ctx.attr.headers_abi3[DefaultInfo],
+ },
+ )
+ else:
+ headers_abi3 = None
+
py_cc_toolchain = PyCcToolchainInfo(
headers = struct(
providers_map = {
@@ -41,6 +52,7 @@
"DefaultInfo": ctx.attr.headers[DefaultInfo],
},
),
+ headers_abi3 = headers_abi3,
libs = libs,
python_version = ctx.attr.python_version,
)
@@ -61,6 +73,20 @@
providers = [CcInfo],
mandatory = True,
),
+ "headers_abi3": attr.label(
+ doc = """
+Target that provides the Python ABI3 (stable abi) headers.
+
+Typically this is a cc_library target.
+
+:::{versionadded} VERSION_NEXT_FEATURE
+The {obj}`features.headers_abi3` attribute can be used to detect if this
+attribute is available or not.
+:::
+""",
+ default = "//python:none",
+ providers = [[SentinelInfo], [CcInfo]],
+ ),
"libs": attr.label(
doc = ("Target that provides the Python runtime libraries for linking. " +
"Typically this is a cc_library target of `.so` files."),
@@ -74,7 +100,7 @@
default = labels.VISIBLE_FOR_TESTING,
),
},
- doc = """\
+ doc = """
A toolchain for a Python runtime's C/C++ information (e.g. headers)
This rule carries information about the C/C++ side of a Python runtime, e.g.
diff --git a/python/private/runtime_env_toolchain.bzl b/python/private/runtime_env_toolchain.bzl
index 1956ad5..de74900 100644
--- a/python/private/runtime_env_toolchain.bzl
+++ b/python/private/runtime_env_toolchain.bzl
@@ -107,8 +107,9 @@
)
py_cc_toolchain(
name = "_runtime_env_py_cc_toolchain_impl",
- headers = ":_empty_cc_lib",
- libs = ":_empty_cc_lib",
+ headers = "//python/private/cc:empty",
+ headers_abi3 = "//python/private/cc:empty",
+ libs = "//python/private/cc:empty",
python_version = "0.0",
tags = ["manual"],
)
diff --git a/python/private/visibility.bzl b/python/private/visibility.bzl
new file mode 100644
index 0000000..3883e23
--- /dev/null
+++ b/python/private/visibility.bzl
@@ -0,0 +1,7 @@
+"""Shared code for use with visibility specs."""
+
+# Use when a target isn't actually public, but needs public
+# visibility to keep Bazel happy.
+# Such cases are typically for defaults of rule attributes or macro args that
+# get used outside of rules_python itself.
+NOT_ACTUALLY_PUBLIC = ["//visibility:public"]
diff --git a/tests/cc/current_py_cc_headers/BUILD.bazel b/tests/cc/current_py_cc_headers/BUILD.bazel
index e2d6a1b..21723b5 100644
--- a/tests/cc/current_py_cc_headers/BUILD.bazel
+++ b/tests/cc/current_py_cc_headers/BUILD.bazel
@@ -12,6 +12,30 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+load("@rules_cc//cc:cc_binary.bzl", "cc_binary")
+load("//python:py_test.bzl", "py_test")
load(":current_py_cc_headers_tests.bzl", "current_py_cc_headers_test_suite")
current_py_cc_headers_test_suite(name = "current_py_cc_headers_tests")
+
+cc_binary(
+ name = "bin_abi3",
+ srcs = ["bin_abi3.cc"],
+ defines = ["Py_LIMITED_API=0x030A0000"],
+ linkshared = True,
+ deps = [
+ "//python/cc:current_py_cc_headers_abi3",
+ ],
+)
+
+py_test(
+ name = "abi3_headers_linkage_test_py",
+ srcs = ["abi3_headers_linkage_test.py"],
+ data = [":bin_abi3"],
+ main = "abi3_headers_linkage_test.py",
+ target_compatible_with = ["@platforms//os:windows"],
+ deps = [
+ "//python/runfiles",
+ "@dev_pip//pefile",
+ ],
+)
diff --git a/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py b/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py
new file mode 100644
index 0000000..6c33765
--- /dev/null
+++ b/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py
@@ -0,0 +1,28 @@
+import os.path
+import pathlib
+import sys
+import unittest
+
+import pefile
+
+from python.runfiles import runfiles
+
+
+class CheckLinkageTest(unittest.TestCase):
+ @unittest.skipUnless(sys.platform.startswith("win"), "requires windows")
+ def test_linkage_windows(self):
+ rf = runfiles.Create()
+ dll_path = rf.Rlocation("rules_python/tests/cc/current_py_cc_headers/bin_abi3.dll")
+ pe = pefile.PE(dll_path)
+ if not hasattr(pe, "DIRECTORY_ENTRY_IMPORT"):
+ self.fail("No import directory found.")
+
+ imported_dlls = [
+ entry.dll.decode("utf-8").lower() for entry in pe.DIRECTORY_ENTRY_IMPORT
+ ]
+ python_dlls = [dll for dll in imported_dlls if dll.startswith("python3")]
+ self.assertEqual(python_dlls, ["python3.dll"])
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/tests/cc/current_py_cc_headers/bin_abi3.cc b/tests/cc/current_py_cc_headers/bin_abi3.cc
new file mode 100644
index 0000000..0211dbb
--- /dev/null
+++ b/tests/cc/current_py_cc_headers/bin_abi3.cc
@@ -0,0 +1,17 @@
+#include <Python.h>
+
+int SomeFunction() {
+ // Early return to prevent the broken code below from running.
+ if (true) {
+ return 0;
+ }
+
+ // The below code won't actually run. We just reference some Python
+ // symbols so the compiler and linker do some work to verify they are
+ // able to resolve the symbols.
+ // To make it actually run, more custom initialization is necessary.
+ // See https://docs.python.org/3/c-api/intro.html#embedding-python
+ Py_Initialize();
+ Py_Finalize();
+ return 0;
+}
diff --git a/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl b/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl
index 818f7ff..f52d93e 100644
--- a/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl
+++ b/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl
@@ -22,25 +22,7 @@
_tests = []
-def _test_current_toolchain_headers(name):
- analysis_test(
- name = name,
- impl = _test_current_toolchain_headers_impl,
- target = "//python/cc:current_py_cc_headers",
- config_settings = {
- "//command_line_option:extra_toolchains": [CC_TOOLCHAIN],
- },
- attrs = {
- "header_files": attr.label_list(
- default = [
- "//tests/support/cc_toolchains:py_header_files",
- ],
- allow_files = True,
- ),
- },
- )
-
-def _test_current_toolchain_headers_impl(env, target):
+def _verify_headers_target(env, target):
# Check that the forwarded CcInfo looks vaguely correct.
compilation_context = env.expect.that_target(target).provider(
CcInfo,
@@ -70,6 +52,27 @@
matching.str_matches("*/cc_toolchains/data.txt"),
)
+def _test_current_toolchain_headers(name):
+ analysis_test(
+ name = name,
+ impl = _test_current_toolchain_headers_impl,
+ target = "//python/cc:current_py_cc_headers",
+ config_settings = {
+ "//command_line_option:extra_toolchains": [CC_TOOLCHAIN],
+ },
+ attrs = {
+ "header_files": attr.label_list(
+ default = [
+ "//tests/support/cc_toolchains:py_headers_files",
+ ],
+ allow_files = True,
+ ),
+ },
+ )
+
+def _test_current_toolchain_headers_impl(env, target):
+ _verify_headers_target(env, target)
+
_tests.append(_test_current_toolchain_headers)
def _test_toolchain_is_registered_by_default(name):
@@ -84,6 +87,29 @@
_tests.append(_test_toolchain_is_registered_by_default)
+def _test_current_toolchain_headers_abi3(name):
+ analysis_test(
+ name = name,
+ impl = _test_current_toolchain_headers_abi3_impl,
+ target = "//python/cc:current_py_cc_headers_abi3",
+ config_settings = {
+ "//command_line_option:extra_toolchains": [CC_TOOLCHAIN],
+ },
+ attrs = {
+ "header_files": attr.label_list(
+ default = [
+ "//tests/support/cc_toolchains:py_headers_abi3_files",
+ ],
+ allow_files = True,
+ ),
+ },
+ )
+
+def _test_current_toolchain_headers_abi3_impl(env, target):
+ _verify_headers_target(env, target)
+
+_tests.append(_test_current_toolchain_headers_abi3)
+
def current_py_cc_headers_test_suite(name):
test_suite(
name = name,
diff --git a/tests/cc/current_py_cc_libs/BUILD.bazel b/tests/cc/current_py_cc_libs/BUILD.bazel
index 9269553..6b4e80f 100644
--- a/tests/cc/current_py_cc_libs/BUILD.bazel
+++ b/tests/cc/current_py_cc_libs/BUILD.bazel
@@ -12,11 +12,11 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+load("@rules_cc//cc:cc_test.bzl", "cc_test")
load(":current_py_cc_libs_tests.bzl", "current_py_cc_libs_test_suite")
current_py_cc_libs_test_suite(name = "current_py_cc_libs_tests")
-# buildifier: disable=native-cc
cc_test(
name = "python_libs_linking_test",
srcs = ["python_libs_linking_test.cc"],
@@ -31,14 +31,12 @@
# the expected Windows libraries are all present in the expected location.
# Since we define the Py_LIMITED_API macro, we expect the linker to go search
# for libs/python3.lib.
-# buildifier: disable=native-cc
cc_test(
- name = "python_abi3_libs_linking_windows_test",
+ name = "python_abi3_libs_linking_test",
srcs = ["python_libs_linking_test.cc"],
defines = ["Py_LIMITED_API=0x030A0000"],
- target_compatible_with = ["@platforms//os:windows"],
deps = [
- "@rules_python//python/cc:current_py_cc_headers",
+ "@rules_python//python/cc:current_py_cc_headers_abi3",
"@rules_python//python/cc:current_py_cc_libs",
],
)
diff --git a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl
index ba8e089..975f0d5 100644
--- a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl
+++ b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl
@@ -28,8 +28,12 @@
impl = _test_py_cc_toolchain_impl,
target = "//tests/support/cc_toolchains:fake_py_cc_toolchain_impl",
attrs = {
+ "header_abi3_files": attr.label_list(
+ default = ["//tests/support/cc_toolchains:py_headers_abi3_files"],
+ allow_files = True,
+ ),
"header_files": attr.label_list(
- default = ["//tests/support/cc_toolchains:py_header_files"],
+ default = ["//tests/support/cc_toolchains:py_headers_files"],
allow_files = True,
),
},
@@ -44,6 +48,7 @@
)
toolchain.python_version().equals("3.999")
+ # ===== Verify headers info =====
headers_providers = toolchain.headers().providers_map()
headers_providers.keys().contains_exactly(["CcInfo", "DefaultInfo"])
@@ -57,9 +62,15 @@
env.ctx.files.header_files,
)
+ # NOTE: Bazel 8 and lower put cc_library.includes into `.system_includes`,
+ # while Bazel 9 put it in `.includes`. Both result in the includes being
+ # added as system includes, so either is acceptable for the expected
+ # `#include <Python.h>` to work.
+ includes = compilation_context.actual.includes.to_list() + compilation_context.actual.system_includes.to_list()
+
# NOTE: The include dir gets added twice, once for the source path,
- # and once for the config-specific path, but we don't care about that.
- compilation_context.system_includes().contains_at_least_predicates([
+ # and once for the config-specific path.
+ env.expect.that_collection(includes).contains_at_least_predicates([
matching.str_matches("*/py_include"),
])
@@ -68,6 +79,32 @@
matching.str_matches("*/cc_toolchains/data.txt"),
)
+ # ===== Verify headers_abi3 info =====
+ headers_abi3_providers = toolchain.headers_abi3().providers_map()
+ headers_abi3_providers.keys().contains_exactly(["CcInfo", "DefaultInfo"])
+
+ cc_info = headers_abi3_providers.get("CcInfo", factory = cc_info_subject)
+
+ compilation_context = cc_info.compilation_context()
+ compilation_context.direct_headers().contains_exactly(
+ env.ctx.files.header_abi3_files,
+ )
+ compilation_context.direct_public_headers().contains_exactly(
+ env.ctx.files.header_abi3_files,
+ )
+
+ # NOTE: Bazel 8 and lower put cc_library.includes into `.system_includes`,
+ # while Bazel 9 put it in `.includes`. Both result in the includes being
+ # added as system includes, so either is acceptable for the expected
+ # `#include <Python.h>` to work.
+ includes = compilation_context.actual.includes.to_list() + compilation_context.actual.system_includes.to_list()
+
+ default_info = headers_abi3_providers.get("DefaultInfo", factory = subjects.default_info)
+ default_info.runfiles().contains_predicate(
+ matching.str_matches("*/cc_toolchains/data.txt"),
+ )
+
+ # ===== Verify libs info =====
libs_providers = toolchain.libs().providers_map()
libs_providers.keys().contains_exactly(["CcInfo", "DefaultInfo"])
diff --git a/tests/support/cc_toolchains/BUILD.bazel b/tests/support/cc_toolchains/BUILD.bazel
index afa88dc..1c1a714 100644
--- a/tests/support/cc_toolchains/BUILD.bazel
+++ b/tests/support/cc_toolchains/BUILD.bazel
@@ -23,9 +23,18 @@
# Factored out for testing
filegroup(
- name = "py_header_files",
+ name = "py_headers_files",
srcs = [
"py_header.h",
+ ":py_headers_abi3_files",
+ ],
+)
+
+# Factored out for testing
+filegroup(
+ name = "py_headers_abi3_files",
+ srcs = [
+ "py_abi3_header.h",
"py_include/py_include.h",
],
)
@@ -46,20 +55,28 @@
py_cc_toolchain(
name = "fake_py_cc_toolchain_impl",
headers = ":py_headers",
+ headers_abi3 = ":py_headers_abi3",
libs = ":fake_libs",
python_version = "3.999",
tags = PREVENT_IMPLICIT_BUILDING_TAGS,
)
cc_library(
- name = "py_headers",
- hdrs = [":py_header_files"],
+ name = "py_headers_abi3",
+ hdrs = [":py_headers_abi3_files"],
data = ["data.txt"],
includes = ["py_include"],
tags = PREVENT_IMPLICIT_BUILDING_TAGS,
)
cc_library(
+ name = "py_headers",
+ hdrs = [":py_headers_files"],
+ tags = PREVENT_IMPLICIT_BUILDING_TAGS,
+ deps = [":py_headers_abi3"],
+)
+
+cc_library(
name = "fake_libs",
srcs = ["libpython3.so"],
data = ["libdata.txt"],
diff --git a/tests/support/cc_toolchains/py_abi3_header.h b/tests/support/cc_toolchains/py_abi3_header.h
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/tests/support/cc_toolchains/py_abi3_header.h
diff --git a/tests/support/py_cc_toolchain_info_subject.bzl b/tests/support/py_cc_toolchain_info_subject.bzl
index 4d3647c..3820e04 100644
--- a/tests/support/py_cc_toolchain_info_subject.bzl
+++ b/tests/support/py_cc_toolchain_info_subject.bzl
@@ -19,6 +19,7 @@
# buildifier: disable=uninitialized
public = struct(
headers = lambda *a, **k: _py_cc_toolchain_info_subject_headers(self, *a, **k),
+ headers_abi3 = lambda *a, **k: _py_cc_toolchain_info_subject_headers_abi3(self, *a, **k),
libs = lambda *a, **k: _py_cc_toolchain_info_subject_libs(self, *a, **k),
python_version = lambda *a, **k: _py_cc_toolchain_info_subject_python_version(self, *a, **k),
actual = info,
@@ -35,6 +36,15 @@
),
)
+def _py_cc_toolchain_info_subject_headers_abi3(self):
+ return subjects.struct(
+ self.actual.headers_abi3,
+ meta = self.meta.derive("headers_abi3()"),
+ attrs = dict(
+ providers_map = subjects.dict,
+ ),
+ )
+
def _py_cc_toolchain_info_subject_libs(self):
return subjects.struct(
self.actual.libs,
