Design: App-Specific Devicetree (DTS) Support

Background

Current Devicetree Operation in zephyr-bazel

In the current architecture of zephyr-bazel, Devicetree (DTS) processing is static and bound to the hardware platform. The build system operates as follows:

  1. Static Board Definition: Every board package (e.g., boards/nordic/nrf52833dk) statically defines a dts_library and a dts_cc_library target in its BUILD.bazel file (e.g., @zephyr//boards/nordic/nrf52833dk:devicetree_generated).
  2. Single DTS Preprocessing: This static target preprocesses a single, board-specific DTS file (e.g., nrf52833dk_nrf52833.dts) and generates a static devicetree_generated.h header.
  3. Platform Redirection: The user-facing Bazel platform definition for the board uses the flags attribute to redirect the global @zephyr//:dts_cc_library label_flag to this static board-level target.
  4. Static Compilation: When zephyr_cc_library targets (like the kernel, drivers, or the application itself) are compiled, they depend on @zephyr//:dts_cc_library, thus receiving the static, board-level devicetree_generated.h.

While this model is simple and highly performant (as DTS is only compiled once per board), it treats Devicetree as a static hardware property that cannot be modified by the application.

Conflict with Zephyr CMake Behavior

In the native Zephyr build system (CMake/West), Devicetree is not static. It is highly contextual and resolved dynamically for every (application, board) combination.

When building an application, Zephyr's CMake system starts with the base board Devicetree (.dts or .yaml in HWM v2) and eagerly applies a series of overlays in a specific order:

  1. Shield Overlays: Overlays introduced by selected shields (e.g., Arduino headers, sensor shields).
  2. Board-Specific App Overlays: Overlays in the application's boards/ directory matching the target board (e.g., app/boards/<board>.overlay).
  3. App-Specific Overlays: The default application overlay (e.g., app/app.overlay or custom files specified via DTC_OVERLAY_FILE).

The final devicetree_generated.h compiled into the application is the result of merging all these layers.

In zephyr-bazel's current static design, all these overlays (app-specific, board-specific app overlays, and shields) are completely ignored during the compilation of the code.

Why We Need to Upgrade

To achieve parity with the native Zephyr build system and support realistic production workloads, we must upgrade zephyr-bazel to support app-specific DTS. This is critical for two main reasons:

1. Application-Level Hardware Customization (DTS Overlays)

Real-world embedded applications rarely run on stock evaluation boards without modifications. Applications frequently need to:

  • Enable/disable specific on-board peripherals (e.g., enabling an extra SPI bus).
  • Configure external sensors connected to expansion headers (e.g., adding an I2C sensor node).
  • Define custom GPIO aliases (e.g., led0 = &red_led) or partition tables.

Without app-specific overlays, developers cannot customize the hardware configuration for their application, forcing them to create fake “OOT boards” for every minor hardware variance, which leads to repository bloat.

2. Bootloader Partitioning (e.g., MCUboot compatibility)

The lack of dynamic DTS support is a blocker for supporting applications that run alongside a bootloader like MCUboot.

When building an application with a bootloader:

  • The flash memory must be partitioned. The bootloader occupies the start of flash, and the application must be shifted to a higher address (e.g., slot0_partition).
  • In Zephyr, this partitioning is defined via Devicetree overlays.
  • The main application Kconfig symbols (like CONFIG_FLASH_LOAD_OFFSET) and code depend on these devicetree partitions.

If DTS remains static at the board level, we cannot apply the partition overlays dynamically. The application would compile assuming it starts at address 0x0 (overwriting the bootloader), making bootloader-compatible configurations impossible to support in Bazel.

Furthermore, under Zephyr Hardware Model v2 (HWM v2), Kconfig and Devicetree are tightly coupled. Kconfig generation (handled by kconfig_gen_values.py in zephyr-bazel) relies on a pre-processed devicetree (edt.pickle) to resolve DT-driven Kconfig symbols. If the devicetree used during Kconfig generation (currently board-only) does not match the devicetree used during compilation, it will result in silent, hard-to-debug configuration mismatches.

Alternatives Considered

We evaluated several approaches to introduce app-specific Devicetree support into zephyr-bazel, balancing Bazel loading-phase performance against build correctness.

Alternative A: Analysis-Phase Merging via Custom Bazel Actions

Under this approach, we would keep the dts_library and dts_cc_library rules at the board level but modify them to accept app-specific overlays as attributes. Bazel would execute pcpp and the Zephyr DTS generator (gen_defines.py) as standard actions during the Analysis Phase.

  • Pros:
    • Excellent Loading-Phase Performance: No extra repositories are declared in the loading phase, keeping the Bzlmod lockfile small.
    • Incremental Builds: Bazel can cache DTS compilation actions efficiently.
  • Cons:
    • Breaks Kconfig Integration (Not Acceptable): In Zephyr (especially Hardware Model v2), Kconfig and Devicetree are tightly coupled. The Kconfig generation script (kconfig_gen_values.py) runs during the loading phase (within repository rules) and requires the preprocessed devicetree structure (edt.pickle) to resolve DT-driven Kconfig symbols. If DTS is compiled in the analysis phase, Kconfig generation cannot access the merged devicetree. Breaking this sync is not acceptable as it leads to silent, broken builds where drivers are misconfigured.

Alternative B: Move DTS Compilation to Contextual @zc_ Repositories

In this approach, we merge DTS processing into the existing contextual @zc_<hash>_<board> repository generation rule (gen_zephyr_config). The repository rule, which already executes kconfig_gen_values.py, is updated to also preprocess the DTS (including all app and board overlays) and execute gen_defines.py to produce the final devicetree_generated.h and edt.pickle.

  • Pros:
    • Full Correctness: Kconfig generation has access to the exact merged devicetree (edt.pickle), ensuring perfect synchronization between Kconfig symbols and Devicetree nodes.
    • Zero-Boilerplate Redirection: The @zc_ repository simply exports the generated header and redirects the @zephyr//:dts_cc_library label_flag to its own local target.
  • Cons:
    • Slightly Slower Fetch Phase: Running the DTS pipeline inside the repository rule adds overhead to the fetch phase. However, due to Bazel's lazy repository execution, this overhead is only paid for the combinations actually being built.

Alternative C: Separate DTS-Specific @zcd_ Repositories

We considered separating the concerns by creating distinct repositories for DTS configurations (e.g., @zcd_<hash>_<board>) separate from the Kconfig repositories (@zc_<hash>_<board>).

  • Pros:
    • Isolation: Separates Kconfig and DTS generation into independent hermetic repository targets.
  • Cons:
    • Lockfile Explosion: Doubles the number of repository declarations in the Bzlmod lockfile (from $N$ to $2N$ per combination), violating the scalability goals of the board discovery design.
    • Circular Dependency Risk: Kconfig requires the DTS output (edt.pickle) to resolve symbols. If they are in separate repositories, managing the dependency order and invalidation becomes extremely complex and fragile.

Proposed Solution: Unified Contextual Configuration (@zc_ Repositories)

We propose adopting Alternative B: consolidating both Kconfig and Devicetree (DTS) generation into the loading-phase contextual @zc_<hash>_<board> repositories.

Instead of treating Devicetree as a static property of the board package, the DTS compilation is treated as a contextual property of the specific (application, board) combination.

graph TD
    subgraph Main Workspace
        App[Application: apps/blinky]
        Board[Board: boards/nrf52840dk]
    end

    subgraph Loading Phase: Module Extension
        Ext[zephyr_setup_apps] -->|Declares| Repo[@zc_a7f2_nrf52840dk]
    end

    subgraph Fetch Phase: Repository Rule
        Repo -->|1. Merges DTS + Overlays| DTS[zephyr.dts]
        DTS -->|2. Runs gen_edt| EDT[edt.pickle]
        EDT -->|3. Runs gen_defines| HDR[devicetree_generated.h]
        EDT -->|4. Runs kconfig.py| Conf[.config]
        Conf -->|5. Generates BUILD| BUILD[BUILD.bazel]
    end

    subgraph Analysis Phase: Transition
        UserBuild[bazel build //apps/blinky] -->|Transition| Platform[@zc_a7f2_nrf52840dk//:platform]
    end

    style Repo fill:#f9f,stroke:#333,stroke-width:2px

How It Works

1. Unified Repository Fetch Phase

When Bazel triggers the fetch of a @zc_<hash>_<board> repository, the gen_zephyr_config repository rule executes a unified configuration pipeline using a hermetic Python environment:

  1. Overlay Discovery: The repository rule scans the application's directory and identifies all applicable DTS overlays:
    • The default application overlay (e.g., app.overlay).
    • Board-specific application overlays (e.g., boards/<board>.overlay).
  2. DTS Preprocessing: The rule preprocesses the base board .dts file together with all discovered overlays using the preprocessor from the hermetic compiler toolchain (or a fallback Python preprocessor like pcpp), producing a unified zephyr.dts.
  3. EDT Generation (edt.pickle): Zephyr's native gen_edt.py script is executed on the preprocessed zephyr.dts to generate the Devicetree structure database (edt.pickle).
  4. Header Generation (devicetree_generated.h): Zephyr's gen_defines.py is executed using edt.pickle to generate the final devicetree_generated.h containing all node macros.
  5. Synchronized Kconfig Execution: The Kconfig generation tool (kconfig_gen_values.py) is executed in the same sequence. Because edt.pickle was generated in step 3 with the correct app-specific overlays, the Kconfig parser correctly resolves all DT-driven Kconfig symbols.

2. Transparent Target Redirection

The @zc_ repository generates a BUILD.bazel file that exports the generated headers and defines local targets:

  • devicetree_generated Library: A local cc_library target is defined, wrapping the generated devicetree_generated.h:
    cc_library(
    name = "devicetree_generated",
    hdrs = ["zephyr/devicetree_generated.h"],
    includes = ["."],
    visibility = ["//visibility:public"],
)
  • Platform Binding: The generated platform target in the @zc_ repository uses its flags attribute to bind the global @zephyr//:dts_cc_library label_flag to this local target:
    platform(
    name = "platform",
    parents = ["//boards/nordic/nrf52840dk:nrf52840dk"],
    flags = [
        # Autoconf redirection
        "--@zephyr//:autoconf_file=@zc_a7f2_nrf52840dk//:zephyr/autoconf.h",
        # DTS Redirection
        "--@zephyr//:dts_cc_library=@zc_a7f2_nrf52840dk//:devicetree_generated",
    ],
)

When compilation of the kernel or application libraries occurs, Bazel transparently routes the dependency to the @zc_ repository's version of the devicetree header, achieving 100% correctness.

Detailed Design

This section details the specific changes required in zephyr-bazel Starlark and Python files to support app-specific Devicetree generation.

1. Repository Rule Watches and Configuration (gen_zephyr_config)

The existing gen_zephyr_config repository rule in bazel/private/zephyr_kconfig_gen_values.bzl is used to generate the contextual configuration.

In _gen_zephyr_config_impl:

  1. Watch Directories: We watch:

    • The application's directory recursively (rctx.watch_tree(app_dir_path)). Watching the entire application directory is critical to catch the creation of the boards/ directory or new overlays.
    • The board's directory (using rctx.watch_tree(board_dir)). Watching the board directory is critical to support Out-of-Tree (OOT) board development, ensuring changes to the base board DTS are detected.

    Implementation Details for Starlark:

    if app_dir:
    rctx.watch_tree(rctx.path(app_dir))
if board_dir:
    rctx.watch_tree(rctx.path(board_dir))

  2. Pass Arguments: We pass the resolved paths to get_kconfig_args (in kconfig_args.bzl), which forwards them to kconfig_gen_values.py.

2. Concatenated DTS Preprocessing in kconfig_gen_values.py

To ensure 100% compatibility with Zephyr's CMake merging behavior, kconfig_gen_values.py will dynamically generate a “stub” DTS file that includes the board DTS and all overlays in the correct topological order.

Discovery of Overlays (Python)

In kconfig_gen_values.py, we update the discovery logic to collect overlays in the following order:

  1. Base Board DTS: Located in board_dir (e.g., <board>.dts).
  2. Board Overlays in App: boards/<board_name>.overlay or boards/<board_name>_<qualifiers>.overlay under args.app_dir.
  3. Default App Overlay: app.overlay in args.app_dir.

Note: Shield Overlays are currently out of scope for this design as shield support is not yet implemented in zephyr-bazel (shields are currently stubbed out).

Implementation Details for Overlay Discovery: Use os.environ.get("BOARD") and os.environ.get("BOARD_QUALIFIERS") to resolve the board name and qualifiers. Qualifiers should have their leading slash stripped and internal slashes replaced with underscores for filenames (similar to get_conf_files logic).

def discover_overlays(app_dir, board_dir):
    board_name = os.environ.get("BOARD")
    board_qualifiers = os.environ.get("BOARD_QUALIFIERS", "").lstrip("/")

    overlays = []

    # 1. Board-specific overlays in app
    if board_qualifiers:
        safe_q = board_qualifiers.replace("/", "_")
        soc_board_overlay = os.path.join(
            app_dir, "boards", f"{board_name}_{safe_q}.overlay"
        )
        if os.path.exists(soc_board_overlay):
            overlays.append(soc_board_overlay)

    board_overlay = os.path.join(app_dir, "boards", f"{board_name}.overlay")
    if os.path.exists(board_overlay):
        overlays.append(board_overlay)

    # 2. Default app overlay
    app_overlay = os.path.join(app_dir, "app.overlay")
    if os.path.exists(app_overlay):
        overlays.append(app_overlay)

    return overlays

Dynamic Preprocessor Stub

Instead of running pcpp directly on the board's .dts file, we dynamically generate a temporary dts_input.c file in the output directory containing only the #include statements for the files that actually exist. This avoids the need to pass complex preprocessor defines (like -DHAS_APP_OVERLAY) to pcpp.

/* Generated by zephyr-bazel - do not edit! */
#include "/absolute/path/to/board.dts"
#include "/absolute/path/to/board_app_overlay.overlay"  /* Only if exists */
#include "/absolute/path/to/app.overlay"                /* Only if exists */

We then execute the C preprocessor (pcpp) on this dts_input.c file, passing the appropriate -I include paths for Zephyr core, modules, and OOT roots. This outputs zephyr.dts.

3. Invoking Zephyr's gen_defines.py

Once edt.pickle is generated by gen_edt.py (which already happens today), we invoke Zephyr's gen_defines.py to produce devicetree_generated.h.

def generate_dts_headers(zephyr_base, output_dir, edt_pickle_path):
    gen_defines_py = os.path.join(zephyr_base, "scripts", "dts", "gen_defines.py")
    output_header = os.path.join(output_dir, "zephyr", "devicetree_generated.h")
    os.makedirs(os.path.dirname(output_header), exist_ok=True)

    cmd = [
        sys.executable,
        gen_defines_py,
        "--edt-pickle", edt_pickle_path,
        "--header-out", output_header,
    ]

    # Ensure edtlib is in PYTHONPATH (portable using os.pathsep)
    env = os.environ.copy()
    env["PYTHONPATH"] = os.pathsep.join([
        os.path.join(zephyr_base, "scripts", "dts"),
        os.path.join(zephyr_base, "scripts", "dts", "python-devicetree", "src"),
        env.get("PYTHONPATH", ""),
    ])

    subprocess.run(cmd, env=env, check=True)

4. Local Target Definitions in the Generated @zc_ Repo

We update generate_bazel_build in kconfig_gen_values.py to write the local DTS library targets into the generated BUILD.bazel file.

[!NOTE] Repository Name Placeholder (@zc_target): The Python script does not know the dynamic canonical name of the repository. It must write the literal placeholder @zc_target in the generated BUILD.bazel file. The Starlark repository rule in zephyr_kconfig_gen_values.bzl will post-process the file and replace @zc_target with the actual canonical name (e.g., "@@" + rctx.name).

def generate_bazel_build(args, kconf, output_dir, kconfiglib):
    # ... existing logic ...

    with open(os.path.join(output_dir, "BUILD.bazel"), "w") as f:
        f.write('load("@rules_cc//cc:defs.bzl", "cc_library")\n')
        f.write('package(default_visibility = ["//visibility:public"])\n\n')

        # Export headers
        f.write('exports_files(["zephyr/autoconf.h", "zephyr/devicetree_generated.h", ".config"])\n\n')

        # Local DTS Target
        f.write('cc_library(\n')
        f.write('    name = "devicetree_generated",\n')
        f.write('    hdrs = ["zephyr/devicetree_generated.h"],\n')
        f.write('    includes = ["."],\n')
        f.write(')\n\n')

        # Autoconf Target
        f.write('cc_library(\n')
        f.write('    name = "autoconf_library",\n')
        f.write('    hdrs = ["zephyr/autoconf.h"],\n')
        f.write('    includes = ["."],\n')
        f.write('    deps = [":devicetree_generated"],\n') # Depends on local DTS
        f.write(')\n\n')

        # ... autoconf_symbols ...

        # Platform definition
        f.write('platform(\n')
        f.write('    name = "%s",\n' % target_name)
        f.write('    parents = ["%s"],\n' % args.parent_platform)
        f.write('    flags = [\n')
        # ... Kconfig flags ...
        f.write('        "--@zephyr//:autoconf_file=@zc_target//:zephyr/autoconf.h",\n')
        f.write('        "--@zephyr//:autoconf_library=@zc_target//:autoconf_library",\n')
        f.write('        "--@zephyr//:autoconf_symbols_to_link=@zc_target//:autoconf_symbols",\n')
        # Redirect global DTS cc_library to local target
        f.write('        "--@zephyr//:dts_cc_library=@zc_target//:devicetree_generated",\n')
        f.write('    ],\n')
        f.write(')\n')

5. Invalidation and Correctness Guarantees

To guarantee Bazel correctness and prevent stale configuration headers:

  1. watch_tree Invalidation: gen_zephyr_config watches:
    • The entire application directory (rctx.watch_tree(app_dir)).
    • The board directory (rctx.watch_tree(board_dir)). This is critical for Out-of-Tree (OOT) boards located in the workspace, ensuring that any change to the base board .dts or board.yml correctly invalidates the repository and triggers re-generation. Any change to watched files immediately invalidates the repository, forcing a re-fetch and re-generation of devicetree_generated.h and autoconf.h.
  2. Action Invalidation: Since zephyr_cc_library targets depend on @zephyr//:dts_cc_library (which is bound to @zc_...//:devicetree_generated), any modification to the generated header forces Bazel to recompile all dependent C/C++ source files during the next build invocation.

Implementation & Verification Plan

This section outlines the conceptual steps to implement the app-specific DTS support and details the testing strategy, using existing example applications to verify correctness.

Phase 1: Python DTS Preprocessing & Header Generation

Tasks

  1. Update kconfig_gen_values.py to implement the concatenated preprocessor stub generation.
  2. Integrate gen_defines.py subprocess execution.

Unit Testing Strategy (Python)

We will add unit tests to the existing Python test file kconfig_gen_values_test.py to test the DTS processing functions in isolation. Leverage the existing TestKconfigGenValues class and its setUp() method which provides a mock environment.

Run the tests using:

bazelisk test //scripts/build:kconfig_gen_values_test
  • Test 1: Overlay Discovery Resolution: Mock the filesystem and verify that the script correctly resolves the hierarchy of overlays (board DTS -> board app overlay -> default app overlay) and returns the correct paths.
  • Test 2: Preprocessor Stub Correctness: Verify that the generated C preprocessor stub (dts_input.c) contains the correct #include statements in the exact topological order.
  • Test 3: Error Propagation: Mock a failing preprocessor (pcpp) or gen_edt.py invocation and verify that the script captures stderr and exits with a non-zero code, outputting a clear error message.

Edge Cases to Verify

  • No Overlays Present: If the application has no app.overlay and no boards/ folder, verify that the preprocessor stub is still valid and successfully processes only the base board DTS.
  • Special Characters in Paths: Verify that paths containing spaces, dashes, or underscores (e.g., an OOT app in my-projects/blinky app/) are correctly escaped in the #include directives to prevent compiler errors.
  • Missing Base Board DTS: If the base board DTS cannot be located, the script must fail immediately with a descriptive FileNotFoundError.

Phase 2: Starlark Integration and Platform Redirection

Tasks

  1. Update gen_zephyr_config to declare the local devicetree_generated cc_library and bind @zephyr//:dts_cc_library in the generated BUILD.bazel.
  2. Remove static dts_library rules from board overlay BUILD files.

Integration Testing Strategy (Bazel using hello_bazel)

We will use the existing examples/hello_bazel application to verify the DTS overlay integration end-to-end.

  • Test 1: Overlay Application and Code Verification:
    1. Create examples/hello_bazel/app.overlay containing a test node:
      / {
    bazel_test_node: bazel-test-node {
        compatible = "bazel-test-node";
        status = "okay";
    };
};
    2. Modify examples/hello_bazel/hello_bazel.cc to assert the node's existence at compile time:
      #include <zephyr/devicetree.h>
#if !DT_NODE_EXISTS(DT_NODELABEL(bazel_test_node))
#error "App overlay was not applied!"
#endif
    3. Build the application:
      bazel build //examples/hello_bazel --platforms=@zephyr//boards/nordic/nrf52840dk:nrf52840dk
      Verify that the build succeeds, proving the overlay was compiled in.
    4. Delete examples/hello_bazel/app.overlay and rebuild. Verify that the build fails with the expected #error message.
  • Test 2: Target Redirection Resolution: Execute a bazel query to verify the dependency graph:
    bazel query "deps(//examples/hello_bazel:hello_bazel)" --output=graph
    Verify that the dependency path for devicetree flows through the local @zc_...//:devicetree_generated target and not the static board package.
  • Test 3: Invalidation Check (Overlay Modification):
    1. Build the application with the overlay.
    2. Modify app.overlay (e.g., add a comment or change a property).
    3. Re-run the build. Verify that Bazel detects the change, re-fetches the @zc_ repository, and recompiles hello_bazel.cc.

Edge Cases to Verify

  • Parallel Context Resolution: Build two different applications (e.g., //examples/hello_bazel and another example) for the same board in a single invocation. Verify that both apps build successfully and receive their respective, isolated configurations without collision.
  • Missing/Empty Bindings Directory: If a module is missing its dts/bindings directory, verify that gen_defines.py still runs successfully by ignoring the missing path rather than crashing.