pw_build: Support building Python wheels
Implement the .wheel subtarget for pw_python_package. This builds a
wheel for that Python package. Python wheels can be collected in a
directory using pw_mirror_tree's path_data_keys option for the "wheels"
key.
Fixed: 239
Change-Id: I43d756ce9714ba834b4590de702efeafc666b9ee
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/36762
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Keir Mierle <keir@google.com>
Reviewed-by: Joe Ethier <jethier@google.com>
diff --git a/docs/python_build.rst b/docs/python_build.rst
index 4bc4fcd..b89c788 100644
--- a/docs/python_build.rst
+++ b/docs/python_build.rst
@@ -278,20 +278,14 @@
Python build supports creating wheels for individual packages and groups of
packages. Building the ``.wheel`` subtarget creates a ``.whl`` file for the
package using the PyPA's `build <https://pypa-build.readthedocs.io/en/stable/>`_
-tool. The location of this file is recorded with `GN metadata
+tool.
+
+The ``.wheel`` subtarget of ``pw_python_package`` records the location of
+the generated wheel with `GN metadata
<https://gn.googlesource.com/gn/+/master/docs/reference.md#var_metadata>`_.
-
-The ``pw_python_wheels`` template creates a collection of wheels from a list of
-``pw_python_package`` targets and their dependencies. It uses GN metadata to
-locate the wheels for all transitive dependencies. This collection can be used
-to deploy packages to different Python environments without requiring the
-original source repository.
-
-.. admonition:: Under construction
-
- Pigweed's wheel building is not yet fully implemented. ``pw_python_wheels``
- currently only supports listing individual ``setup.py`` files. This will be
- updated to automatically collect wheels for all transitive dependencies.
+Wheels for a Python package and its transitive dependencies can be collected
+from the ``pw_python_package_wheels`` key. See
+:ref:`module-pw_build-python-wheels`.
Protocol buffers
^^^^^^^^^^^^^^^^
diff --git a/pw_build/BUILD.gn b/pw_build/BUILD.gn
index c9399fb..7c4be4a 100644
--- a/pw_build/BUILD.gn
+++ b/pw_build/BUILD.gn
@@ -132,6 +132,7 @@
# Requirements for the pw_python_package lint targets.
pw_python_requirements("python_lint") {
requirements = [
+ "build",
"mypy==0.800",
"pylint==2.6.0",
]
diff --git a/pw_build/python.gni b/pw_build/python.gni
index d04d027..2b1ce22 100644
--- a/pw_build/python.gni
+++ b/pw_build/python.gni
@@ -1,4 +1,4 @@
-# Copyright 2020 The Pigweed Authors
+# Copyright 2021 The Pigweed Authors
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
# use this file except in compliance with the License. You may obtain a copy of
@@ -38,13 +38,11 @@
# - $name.lint.pylint - Runs pylint (if enabled).
# - $name.tests - Runs all tests for this package.
# - $name.install - Installs the package in a venv.
-# - $name.wheel - Builds a Python wheel for the package. (Not implemented.)
+# - $name.wheel - Builds a Python wheel for the package.
#
# All Python packages are instantiated with the default toolchain, regardless of
# the current toolchain.
#
-# TODO(pwbug/239): Implement wheel building.
-#
# Args:
# setup: List of setup file paths (setup.py or pyproject.toml & setup.cfg),
# which must all be in the same directory.
@@ -68,6 +66,7 @@
# provided, mypy's default configuration file search is used. mypy is
# executed from the package's setup directory, so mypy.ini files in that
# directory will take precedence over others.
+#
template("pw_python_package") {
# The Python targets are always instantiated in the default toolchain. Use
# fully qualified labels so that the toolchain is not lost.
@@ -340,18 +339,27 @@
}
}
- # TODO(pwbug/239): Add support for building groups of wheels. The code below
- # is incomplete and untested.
+ # Builds a Python wheel for this package. Records the output directory
+ # in the pw_python_package_wheels metadata key.
pw_python_action("$target_name.wheel") {
- script = "$dir_pw_build/py/pw_build/python_wheels.py"
+ metadata = {
+ pw_python_package_wheels = [ "$target_out_dir/$target_name" ]
+ }
+
+ module = "build"
args = [
- "--out_dir",
- rebase_path(target_out_dir),
- ]
- args += rebase_path(_all_py_files)
+ rebase_path(_setup_dir),
+ "--wheel",
+ "--no-isolation",
+ "--outdir",
+ ] + rebase_path(metadata.pw_python_package_wheels)
- deps = [ ":${invoker.target_name}.install" ]
+ deps = [ ":${invoker.target_name}" ]
+ foreach(dep, _python_deps) {
+ deps += [ string_replace(dep, "(", ".wheel(") ]
+ }
+
stamp = true
}
} else {
@@ -467,6 +475,9 @@
inputs = [ invoker.mypy_ini ]
}
}
+
+ # Generated packages with linting disabled never need the whole file list.
+ not_needed([ "_all_py_files" ])
}
} else {
# Create groups with the public target names ($target_name, $target_name.lint,
diff --git a/pw_build/python.rst b/pw_build/python.rst
index 7157746..520b333 100644
--- a/pw_build/python.rst
+++ b/pw_build/python.rst
@@ -75,6 +75,30 @@
pylintrc = "$dir_pigweed/.pylintrc"
}
+
+.. _module-pw_build-python-wheels:
+
+Collecting Python wheels for distribution
+-----------------------------------------
+The ``.wheel`` subtarget generates a wheel (``.whl``) for the Python package.
+Wheels for a package and its transitive dependencies can be collected by
+traversing the ``pw_python_package_wheels`` `GN metadata
+<https://gn.googlesource.com/gn/+/master/docs/reference.md#var_metadata>`_ key,
+which lists the output directory for each wheel.
+
+The ``pw_mirror_tree`` template can be used to collect wheels in an output
+directory:
+
+.. code-block::
+
+ import("$dir_pw_build/mirror_tree.gni")
+
+ pw_mirror_tree("my_wheels") {
+ path_data_keys = [ "pw_python_package_wheels" ]
+ deps = [ ":python_packages" ]
+ directory = "$root_out_dir/the_wheels"
+ }
+
pw_python_script
================
A ``pw_python_script`` represents a set of standalone Python scripts and/or
@@ -88,13 +112,6 @@
These targets do not add any files. Their subtargets simply forward to those of
their dependencies.
-pw_python_wheels
-================
-Builds and collects Python wheels for one or more ``pw_python_package`` targets.
-A package's ``.wheel`` subtarget builds the wheel for just that package.
-``pw_python_package`` collects wheels from all of its transitive dependencies
-and collects them in a specified directory.
-
pw_python_requirements
======================
Represents a set of local and PyPI requirements, with no associated source
