pw_build_info/docs.rst

.. _module-pw_build_info:

=============
pw_build_info
=============
.. pigweed-module::
   :name: pw_build_info

pw_build_info provides tooling, build integration, and libraries for generating,
embedding, and parsing build-related information that is embedded into
binaries. Simple numeric version numbering doesn't typically express things
like where the binary originated, what devices it's compatible with, whether
local changes were present when the binary was built, and more. pw_build_info
simplifies the process of integrating rich version metadata to answer more
complex questions about compiled binaries.

.. _module-pw_build_info-gnu-build-ids:

-------------
GNU build IDs
-------------
This module provides C++ and python libraries for reading GNU build IDs
generated by the link step of a C++ executable. These build IDs are essentially
hashes of the final linked binary, meaning two identical binaries will have
identical build IDs. This can be used to accurately identify matching
binaries.

Linux executables that depend on the ``build_id`` GN target will automatically
generate GNU build IDs. Windows and macOS binaries cannot use this target as
the implementation of GNU build IDs depends on the ELF file format.

A separate GN target ``build_id_or_noop`` is available which provides an empty
build ID on platforms where GNU build ID is not available while still providing
a real GNU build ID where supported.

Getting started
===============
To generate GNU build IDs as part of your firmware image, you'll need to update
your embedded target's linker script.

Updating your linker script
---------------------------
If your project has a custom linker script, you'll need to update it to include
a section to contain the generated build ID. This section should be placed
alongside the ``.text`` and ``.rodata`` sections, and named
``.note.gnu.build-id``.

.. code-block:: none

   /* Main executable code. */
   .code : ALIGN(4)
   {
     . = ALIGN(4);
     /* Application code. */
     *(.text)
     *(.text*)
     KEEP(*(.init))
     KEEP(*(.fini))
     ...
   } >FLASH

   /* GNU build ID section. */
   .note.gnu.build-id :
   {
     . = ALIGN(4);
     gnu_build_id_begin = .;
     *(.note.gnu.build-id);
   } >FLASH

   /* Explicitly initialized global and static data. (.data) */
   .static_init_ram : ALIGN(4)
   {
     *(.data)
     *(.data*)
     ...
   } >RAM AT> FLASH


Alternatively, you can copy the following linker snippet into a pre-existing
section. This makes reading the build ID slower, so whenever possible prefer
creating a dedicated section for the build ID.

.. literalinclude:: build_id_linker_snippet.ld

An example of directly inserting a build ID into an existing section is
provided below:

.. code-block:: none

   /* Main executable code. */
   .code : ALIGN(4)
   {
     . = ALIGN(4);
     /* Application code. */
     *(.text)
     *(.text*)
     KEEP(*(.init))
     KEEP(*(.fini))

     . = ALIGN(4);
     gnu_build_id_begin = .;
     *(.note.gnu.build-id);

     ...
   } >FLASH

If your linker script is auto-generated, you may be able to use the
``INSERT AFTER`` linker script directive to append the build ID as seen in the
Linux host support for pw_build_info's build ID integration:

.. literalinclude:: add_build_id_to_default_linker_script.ld

Generating the build ID
-----------------------
When you depend on ``"$dir_pw_build_info:build_id``, a GNU build ID will be
generated at the final link step of any binaries that depend on that library
(whether directly or transitively). Those binaries will be able to read the
build ID by calling ``pw::build_info::BuildId()``. Note that the build ID
is not a string, but raw binary data, so to print it you'll need to convert
it to hex or base64. It is possible to call ``pw::build_info::LogBuildId()``
function to print it (as hexadecimal).

Python API reference
====================

.. py:function:: read_build_id_from_section(elf_file: BinaryIO) -> \
                     bytes | None

  Reads a GNU build ID from an ELF binary by searching for a
  ``.note.gnu.build-id`` section.

.. py:function:: read_build_id_from_symbol(elf_file: BinaryIO) -> \
                     bytes | None

  Reads a GNU build ID from an ELF binary by searching for a
  ``gnu_build_id_begin`` symbol. This can be a rather slow operation.

.. py:function:: read_build_id(elf_file: BinaryIO) -> bytes | None

  Reads a GNU build ID from an ELF binary, first checking for a GNU build ID
  section and then falling back to search for a ``gnu_build_id_begin`` symbol.

.. py:function:: find_matching_elf(uuid: bytes, search_dir: Path) -> \
                     Path | None

  Recursively searches a directory for an ELF file with a matching UUID.

  Warning: This can take on the order of several seconds.

Python utility
==============
GNU build IDs can be parsed out of ELF files using the ``build_id`` python tool.
Simply point the tool to a binary with a GNU build ID and the build ID will be
printed out if it is found.

.. code-block:: sh

   $ python -m pw_build_info.build_id my_device_image.elf
   d43cce74f18522052f77a1fa3fb7a25fe33f40dd

------------------------------
Bazel Workspace Status Command
------------------------------
Bazel supports running a single command to inspect the current workspace status
each build. This is configured via the ``--workspace_status_command`` flag,
typically in ``.bazelrc``, e.g.:

.. code-block::

   build --workspace_status_command=path/to/pigweed/pw_build_info/git_workspace_status_command.sh

More info about `bazel workspace status commands <https://bazel.build/docs/user-manual#workspace-status-command>`__.

pw_build_info provides a script and bazel tool for substituting variables from
the workspace_status_command into source file templates. See
``substitute_workspace_status.bzl``.

Auto-generated headers in this module leverage this facility to expose
workspace status information to C++.

Downstream projects may wish to provide their own workspace status script which
invokes Pigweed-provided workspace status scripts, e.g.:

.. code-block:: sh

   #!/bin/bash
   pigweed/pw_build_info/git_workspace_status_command.sh || exit $?
   echo "STABLE_CUSTOM_ATTRIBUTE pizza" || exit $?


------------
Build Origin
------------
You can use the ``//pw_build_info:build_origin`` header to embed a "build
origin" into your firmware.

The build origin is a string identifying the unique "build" from which this
firmware originates. This will typically refer to a CI/CD build or job
identifier.

The recommended format is ``<build_system>/<unique_id>`` but the exact format
of this string is up to the user. Examples:

* GitHub Actions: ``github/${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}``
* GitLab CI/CD: ``gitlab/${CI_PIPELINE_ID}``
* Jenkins: ``jenkins/${BUILD_NUMBER}``
* LUCI: ``luci/<build-id>``

Local builds can leave this field empty or specify ``local``.

Projects using Pigweed's LUCI build system should use the LUCI format specified
above.

This is only supported in Bazel.

Downstream projects must use a Bazel workspace_status_command to set the
``STABLE_PW_BUILD_INFO_BUILD_ORIGIN`` key.

This is intended for use in the ``pw.snapshot.Metadata.build_origin`` proto field.

---------------
Git Commit Info
---------------
If your project uses ``git`` and builds with ``bazel``, you can use the
auto-generated ``//pw_build_info:git_build_info`` header to embed which git
commit your binary was built from. This requires a bit of setup to use.

Bazel Workspace Status Command
==============================
If you are using no other ``workspace_status_command``, simply add the
following to your ``.bazelrc,`` replacing the ``path/to/pigweed`` portion with
where you have pigweed checked out.

.. code-block::

   build --workspace_status_command=path/to/pigweed/pw_build_info/git_workspace_status_command.sh

Otherwise, *invoke* this script from your own.

Use ``pw_build_info/git_build_info.h`` Header
=============================================
Add a dependency on ``//pw_build_info:git_build_info``:

.. code-block:: python

   cc_binary(
     name = "main",
     srcs = ["main.cc"],
     deps = [
        "@pigweed//pw_build_info:git_build_info",
        "//pw_log",
      ],
   )

Include the header. The following constants are available:

* ``pw::InlineBasicString pw::build_info::kGitCommit``: The git commit this
  binary was built from. Includes ``-dirty`` suffix if ``kGitTreeDirty`` is
  true.
* ``bool pw::build_info::kGitTreeDirty``: True if there were any uncommitted
  changes.

.. code-block:: cpp

   #include "pw_build_info/git_build_info.h"
   #include "pw_log/log.h"
   #include "pw_string/string.h"

   int main() {
     PW_LOG_INFO("kGitCommit %s", pw::build_info::kGitCommit.c_str());
     PW_LOG_INFO("kGitTreeDirty %d", pw::build_info::kGitTreeDirty);
     return 0;
   }