Rename ci_select to change_impact (#4123) The selector now answers both build and HIL impact, so it is named for what it does. CI hand-off files become selection.json; the obsolete selector design docs are removed, and the rule table lives only in the module docstring.
diff --git a/.circleci/config.yml b/.circleci/config.yml index bc0b0a0..3d53856 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml
@@ -23,14 +23,14 @@ # The selection lands in a FILE and never travels as an argv: a mass-sweep # diff selects hundreds of KB, and E2BIG would fail the step before the # `||` fallback could fire while EXAMPLE_MAP below stayed scoped. - SELECT_FILE=ci_select_out.json + SELECT_FILE=selection.json rm -f "$SELECT_FILE" if [ -n "${CIRCLE_PULL_REQUEST:-}" ]; then git fetch --no-tags origin master || true - # the suites gate the selector: test_ci_select.py owns the rules, + # the suites gate the selector: test_change_impact.py owns the rules, # test_build_select.py the check_build contract, and test_ci_metrics.py the # config2 sentinel contract this job rewrites - if python3 test/hil/test/test_ci_select.py >/dev/null 2>&1 && + if python3 test/hil/test/test_change_impact.py >/dev/null 2>&1 && python3 .claude/test/test_build_select.py >/dev/null 2>&1 && python3 test/hil/test/test_ci_metrics.py >/dev/null 2>&1; then python3 .claude/skills/build/scripts/check_build.py --select-only --base origin/master > "$SELECT_FILE" ||
diff --git a/.claude/skills/build/SKILL.md b/.claude/skills/build/SKILL.md index a3d57cb..b5145c7 100644 --- a/.claude/skills/build/SKILL.md +++ b/.claude/skills/build/SKILL.md
@@ -23,7 +23,7 @@ The last stdout line is JSON: `pass`, per-board `status` (`ok`, `failed`, `skipped`, `error`), `built` (elf files this run wrote), `firstError`, `buildDir`, how the boards were resolved, and the scope paths nothing builds, split into `nothingToBuild` and `uncovered`. Exit 0 pass, 1 a board failed, 2 usage or resolution error with the message in `error` (a missing dependency names its remedy there) or a refused receipt with its reason in `receipt.error`, 3 an uncovered path. `-v` streams the build to stderr. -`--select-only` is the selection contract: CI and any agent that needs "what does this change affect" read its manifest rather than calling `tools/ci_select.py` or parsing its log. It prints one JSON line and stops before any dependency check or build, under an isolated stdlib-only interpreter: +`--select-only` is the selection contract: CI and any agent that needs "what does this change affect" read its manifest rather than calling `tools/change_impact.py` or parsing its log. It prints one JSON line and stops before any dependency check or build, under an isolated stdlib-only interpreter: - `version` 1. `input`: `mode` (`paths`, `base`, `worktree`, `endpoints`), the ref as given, and the SHAs it resolved to (`base_sha`, `merge_base`, `head`, `deps_base`). Committed modes classify the checked-out tree, so `--endpoints A..B` refuses a `B` that is not HEAD. - `build`: `full` and `needed` (booleans); `families` maps each selected family to `examples` (`"all"` or a non-empty list), and is empty when `full`; `required_boards` (changed `hw/bsp/<f>/boards/<b>` still in the tree); `required_targets` (`examples-membrowse-upload` for `tools/membrowse_cli.py`); `dropped` (families pruned away, each with its line); `paths`, one record per changed path: `rule`, `effect` (`none` nothing to verify, `gap` firmware no build compiles, `select`, `full`), the `families` and `examples` the rule names (`null` means every one), `port`, `targets`, and the human `reason`. `effect` is what the path contributes to the selection, not a coverage verdict. @@ -34,8 +34,8 @@ ## Judgment - **Full example set by default.** `-e` narrows to named examples for a quick check; a change in `src/` or a class driver is verified by the full set on every resolved board before it is called green. -- **Scope, not guesswork.** `--scope` takes changed paths, expanding a directory into its tracked files first, and runs `tools/ci_select.py`: one board per affected family; a board whose own `hw/bsp/<family>/boards/<board>/` files changed is built itself, else a rig-roster board of the family, else the first in the family. A full-matrix answer (core or unclassified paths) builds `stm32f407disco` and `raspberry_pi_pico`, plus any board whose own files changed, plus one board per family a port, bsp or mcu path in the scope names: the pair stands in for the matrix on core code, not on a port it does not contain. A changed driver under `src/portable/` adds, per family it maps to, a board whose `hw/bsp/family.json` row lists the driver and, where the host preprocessor can say, selects the USB IPs its guard names: `stm32l412nucleo` for fsdev next to a DWC2 pick on stm32l4, `feather_rp2040_max3421` for `hcd_max3421.c`. A special target the changed path needs (the manifest's `required_targets`: `examples-membrowse-upload` for `tools/membrowse_cli.py`) is `-T`; a default sweep is no evidence for it. `tools/code_size.py` is validated by CI's code-size step, which no local build runs, so it always stays `uncovered` here. -- **A path nothing builds** is one of two kinds, in `ci_select`'s words, and the script sorts them so that a mixed change gets the stricter answer: +- **Scope, not guesswork.** `--scope` takes changed paths, expanding a directory into its tracked files first, and runs `tools/change_impact.py`: one board per affected family; a board whose own `hw/bsp/<family>/boards/<board>/` files changed is built itself, else a rig-roster board of the family, else the first in the family. A full-matrix answer (core or unclassified paths) builds `stm32f407disco` and `raspberry_pi_pico`, plus any board whose own files changed, plus one board per family a port, bsp or mcu path in the scope names: the pair stands in for the matrix on core code, not on a port it does not contain. A changed driver under `src/portable/` adds, per family it maps to, a board whose `hw/bsp/family.json` row lists the driver and, where the host preprocessor can say, selects the USB IPs its guard names: `stm32l412nucleo` for fsdev next to a DWC2 pick on stm32l4, `feather_rp2040_max3421` for `hcd_max3421.c`. A special target the changed path needs (the manifest's `required_targets`: `examples-membrowse-upload` for `tools/membrowse_cli.py`) is `-T`; a default sweep is no evidence for it. `tools/code_size.py` is validated by CI's code-size step, which no local build runs, so it always stays `uncovered` here. +- **A path nothing builds** is one of two kinds, in `change_impact`'s words, and the script sorts them so that a mixed change gets the stricter answer: - `nothingToBuild` (docs, `.claude/`, `*.md`, unit tests, HIL harness): there is nothing a build could verify, which is not a failure. A scope of only such paths is `pass` true with no boards, exit 0; quote the reason in your notes so nobody mistakes it for a build that ran. - `uncovered` (a class or typec no example config enables, a lib nothing builds, a port mapping to no board family, a family the selector pruned because nothing in it builds, an example or class no built board wrote an elf for, a port driver no built board of its families kept a line of after preprocessing): firmware this run did not compile, judged against the boards and elfs it actually produced. The port test is the driver's own compile command run with `-E` on every instance in an example this run built, so a board whose configuration guards the driver away (`dcd_nrf5x.c` on NRF54, `hcd_rp2040.c` under MAX3421) is not coverage whatever else went green; the reason names each board's verdict. `-e` or `-T` hands coverage to you: the script then does not report what you chose to leave out, and a `-T` target that writes no elf is not a verified build of anything. Exit 3 and `pass` false, even when every other path in the scope built green — a class driver plus the core file that registers it is the common shape, and the green boards say nothing about the driver. The verdict is false and the `uncovered` reason, quoted verbatim, is the finding: a human adds coverage (the next bullet) or accepts the gap. - **Private dirs for parallel agents.** Builds land in `cmake-build/cmake-build-agent-<pid>-<board>`; remove yours when done. `--shared` writes `cmake-build/cmake-build-<board>`, the dir `hil_test.py` flashes from by default, and needs exclusive ownership of that board: never while a HIL run or another agent is on it. `--variants <hil config>` builds instead each variant that roster gives a named board, into the `cmake-build-<variant>` `hil_test.py` runs it from, with your own defines and flags followed by the variant's, so a repeated `-D` takes the variant's value; a `--cflag` macro the variant's flags also set is a `-Werror` redefinition, not an override (a board without a variant list as itself); it refuses a board the roster lacks.
diff --git a/.claude/skills/build/scripts/check_build.py b/.claude/skills/build/scripts/check_build.py index d95ef08..78dd368 100755 --- a/.claude/skills/build/scripts/check_build.py +++ b/.claude/skills/build/scripts/check_build.py
@@ -5,10 +5,10 @@ [-e role/name]... [-T target]... [-D SYMBOL]... [--cflag FLAG]... [--shared [--variants CONFIG [--receipt FILE]]] check_build.py --select-only (--scope PATH... | --base REF [--worktree] | --endpoints A..B) [--config ROSTER]... ---select-only prints ci_select's selection manifest v1 (SKILL.md) and stops: the selection +--select-only prints change_impact's selection manifest v1 (SKILL.md) and stops: the selection contract CI and agents read. -Scope resolution goes through tools/ci_select.py: one board per affected family +Scope resolution goes through tools/change_impact.py: one board per affected family (a rig-roster board of that family first, else the first in hw/bsp/<family>/boards, preferring one that builds an example the change affects), plus one more board per changed driver none of them compiles by its hw/bsp/family.json row, or the @@ -31,7 +31,7 @@ "status", "built" and "okExamples" (elfs this run wrote), "firstError", "familyJson" (tools/build.py's line on the board's catalog row, when the configure was the default one)}], "resolution", "nothingToBuild", "uncovered", "familyJsonChanged" (a build -rewrote hw/bsp/family.json; commit it with the change)}. A scope path nothing builds is one of two kinds, in ci_select's words: +rewrote hw/bsp/family.json; commit it with the change)}. A scope path nothing builds is one of two kinds, in change_impact's words: "nothingToBuild" is nothing to verify (docs, .claude/, unit tests, HIL harness); "uncovered" is firmware no board's build compiled (a class no example enables, a lib nothing builds, a port no built board kept a line of after preprocessing - wrong @@ -59,7 +59,7 @@ ROW = re.compile(r'^\|\s*(\S+)\s*\|\s*(.+?)\s*\|\s*\x1b\[\d+m(OK|Failed|Skipped)\x1b\[0m', re.M) sys.path.insert(0, str(ROOT / 'tools')) import build as tools_build # noqa: E402 tools/build.py, first on the path above -import ci_select # noqa: E402 the same classifier run below, here for its option knowledge +import change_impact # noqa: E402 the same classifier run below, here for its option knowledge import family_json # noqa: E402 hw/bsp/family.json: what each board's default configure compiles sys.path.insert(0, str(ROOT / 'test' / 'hil' / 'helper')) import hil_report # noqa: E402 stdlib-only; board_variants() reads a roster board's builds @@ -78,7 +78,7 @@ def expand_scope(scope): - """The files of every directory in the scope, new ones included. ci_select and the + """The files of every directory in the scope, new ones included. change_impact and the changed-board rule both classify file paths: a bare directory matches neither, so a board directory would resolve to its family's sample instead of the board itself. Untracked-but-not-ignored files count, because the work being verified is usually @@ -97,7 +97,7 @@ elif (ROOT / p).exists() or tracked(p): out.append(p) else: - # ci_select classifies an unknown path as the full matrix, whose pair then + # change_impact classifies an unknown path as the full matrix, whose pair then # covers it: a typo would come back green for a path no change touched fail(f'{p} does not exist and is not a tracked file; only a path present in the ' f'tree or deleted from it can be in the scope') @@ -112,12 +112,12 @@ def select(scope=None, base=None, configs=(HIL_CONFIG,), endpoints=None, worktree=False): - """ci_select's selection manifest v1 for a path list, the branch diff against base + """change_impact's selection manifest v1 for a path list, the branch diff against base (with worktree, the uncommitted tree too) or a push's endpoints A..B. A scope that carries a get_deps.py edit resolves its dep entries against HEAD (--deps-base). Isolated (-I -S) as on a bare CI runner: the selector is stdlib-only, so an import that needs site-packages fails here first.""" - selector = [sys.executable, '-I', '-S', str(ROOT / 'tools' / 'ci_select.py'), '--manifest'] + selector = [sys.executable, '-I', '-S', str(ROOT / 'tools' / 'change_impact.py'), '--manifest'] with tempfile.TemporaryDirectory() as tmp: # the path list goes away with it, fail() included if base: cmd = selector + ['--base', base] + (['--worktree'] if worktree else []) @@ -129,7 +129,7 @@ cmd = selector + ['--diff-file', str(listing), '--deps-base', 'HEAD'] r = subprocess.run(cmd + [str(c) for c in configs], capture_output=True, text=True, cwd=ROOT) if r.returncode != 0: - fail(f'ci_select failed:\n{r.stderr.strip()}') + fail(f'change_impact failed:\n{r.stderr.strip()}') return json.loads(r.stdout.splitlines()[-1]) @@ -145,7 +145,7 @@ not a claim that guarded drivers have no variant-specific behaviour. Add a plain first pick when that leaves nothing. A candidate that compiles one of the affected examples is preferred, the criterion dropped when it leaves no - candidate: ci_select keeps a family when ANY of its boards builds the selection under + candidate: change_impact keeps a family when ANY of its boards builds the selection under EITHER build system, so the first candidate can be one skipped for every affected example (samd11's cynthion_d11 is skip.txt'd out of device/mtp) and verify none of the change. @@ -492,7 +492,7 @@ def coverage(records, results, chosen=False, targets=(), dropped=()): - """ci_select's per-path build records as (nothing to verify, uncovered firmware), + """change_impact's per-path build records as (nothing to verify, uncovered firmware), judged against what was actually built, each entry the record's reason. Effect 'none' is nothing to verify (non-code, a get_deps.py edit that changes no entry); 'gap' is firmware no build compiles (a class, typec or lib no example enables, a @@ -808,7 +808,7 @@ p.add_argument('--worktree', action='store_true', help='with --base: include uncommitted and untracked changes in the scope') p.add_argument('--select-only', action='store_true', - help="print ci_select's selection manifest for the scope and stop: no dependency check, no build") + help="print change_impact's selection manifest for the scope and stop: no dependency check, no build") p.add_argument('-e', '--example', action='append', default=[], help='only these examples (role/name)') p.add_argument('-T', '--target', action='append', default=[], help='build target (default all)') p.add_argument('-D', '--define', action='append', default=[], @@ -826,7 +826,7 @@ p.add_argument('--receipt', metavar='FILE', help='with --variants and every example: write the HIL build receipt after a passing build') p.add_argument('--config', action='append', default=None, - help='rig roster for ci_select (repeatable; default: tinyusb.json)') + help='rig roster for change_impact (repeatable; default: tinyusb.json)') p.add_argument('-v', '--verbose', action='store_true', help='stream build output to stderr') a = p.parse_args(argv) os.chdir(ROOT) # tools/build.py's example listing reads examples/ relative to the root
diff --git a/.claude/skills/build/scripts/coverage_audit.py b/.claude/skills/build/scripts/coverage_audit.py index d584c80..8aac5b4 100755 --- a/.claude/skills/build/scripts/coverage_audit.py +++ b/.claude/skills/build/scripts/coverage_audit.py
@@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Audit tools/ci_select.py's build selection against the real build graph. Audit only: +"""Audit tools/change_impact.py's build selection against the real build graph. Audit only: nothing in selection reads its output. coverage_audit.py graph --root CHECKOUT --out DIR (--board B... | --boards-file F) [--force] @@ -21,7 +21,7 @@ `replay` walks first-parent commits of REF (default HEAD) and, for each, compares the (board, example) pairs whose recorded files the commit changed against what the current -ci_select's build view selects for the same paths. A required pair the selection does +change_impact's build view selects for the same paths. A required pair the selection does not cover is an under-selection candidate, to be confirmed by reading the source. Paths the graph never saw are listed per commit; a commit that deletes a file needs its historical tree, since the current graph cannot know what built the deleted file. @@ -430,7 +430,7 @@ def cmd_replay(a): sys.path.insert(0, str(ROOT / 'tools')) - import ci_select # noqa: E402 the rules this branch ships + import change_impact # noqa: E402 the rules this branch ships index, family_of, status = load_index(a.graph) commits = a.commits or run_ok(['git', 'rev-list', '--first-parent', f'--since={a.since}', a.ref], ROOT).split() @@ -441,11 +441,11 @@ files = [p for _, p in rows] deleted = [p for s, p in rows if s.startswith('D')] gd = None - if ci_select.GET_DEPS_PATH in files: - gd = ci_select._deps_families( - lambda: ci_select.git_show(f'{c}^1:{ci_select.GET_DEPS_PATH}', str(ROOT)), - lambda: ci_select.git_show(f'{c}:{ci_select.GET_DEPS_PATH}', str(ROOT)), str(ROOT)) - view = ci_select.classify_build(files, str(ROOT), gd) + if change_impact.GET_DEPS_PATH in files: + gd = change_impact._deps_families( + lambda: change_impact.git_show(f'{c}^1:{change_impact.GET_DEPS_PATH}', str(ROOT)), + lambda: change_impact.git_show(f'{c}:{change_impact.GET_DEPS_PATH}', str(ROOT)), str(ROOT)) + view = change_impact.classify_build(files, str(ROOT), gd) v = judge(files, deleted, index, family_of, view) v.update(commit=c[:9], subject=run_ok(['git', 'log', '-1', '--format=%s', c], ROOT).strip(), full=view['full'], families=len(view['families']))
diff --git a/.claude/skills/hil/SKILL.md b/.claude/skills/hil/SKILL.md index fad52fd..67ebc14 100644 --- a/.claude/skills/hil/SKILL.md +++ b/.claude/skills/hil/SKILL.md
@@ -48,11 +48,11 @@ ## PR-scoped selection -`tools/ci_select.py` maps a diff to affected boards/tests (used by CI on PRs; fail-open +`tools/change_impact.py` maps a diff to affected boards/tests (used by CI on PRs; fail-open to the full matrix). Manual use: ```bash -SEL=$(python3 tools/ci_select.py --base master test/hil/tinyusb.json) +SEL=$(python3 tools/change_impact.py --base master test/hil/tinyusb.json) FULL=$(printf '%s' "$SEL" | python3 -c "import json,sys; print(json.load(sys.stdin)['full'])") ARGS=$(printf '%s' "$SEL" | python3 -c "import json,sys; print(json.load(sys.stdin)['args']['tinyusb.json'])") if [ "$FULL" = "True" ] || [ -n "$ARGS" ]; then @@ -66,8 +66,8 @@ unclassified change) and "nothing selected" (skip). Skip only when `full` is false AND `args` is empty. Unit suites (no hardware) live in `test/hil/test/test_*.py`; the `hil-test` pre-commit hook -runs every `test_hil*.py`, `ci-select-test` the two `test_ci_*` suites plus -`test_hil_util.BottomLayer`. `test_ci_select.py` covers only selection, `test_ci_metrics.py` +runs every `test_hil*.py`, `change-impact-test` `test_change_impact.py`, `test_ci_metrics.py` and +`test_hil_util.BottomLayer`. `test_change_impact.py` covers only selection, `test_ci_metrics.py` only the code-size plumbing; the bounded reads and the build and pool guards live in `test_hil_bounded.py` and `test_hil_util.py`; `test_hil_report.py` covers the report document, `test_hil_rtt.py` the RTT console and `test_hil_pool_check.py` the pool check's verdicts and output. Run them all when changing `test/hil`:
diff --git a/.claude/skills/usbtest/scripts/run_case.py b/.claude/skills/usbtest/scripts/run_case.py index 24971ad..dd37c50 100644 --- a/.claude/skills/usbtest/scripts/run_case.py +++ b/.claude/skills/usbtest/scripts/run_case.py
@@ -42,7 +42,7 @@ HIL_DIR = ROOT / 'test' / 'hil' sys.path.insert(0, str(HIL_DIR)) sys.path.insert(0, str(ROOT / 'tools')) -import ci_select # noqa: E402 +import change_impact # noqa: E402 import hil_flash # noqa: E402 import usbtest # noqa: E402 from helper import hil_lock, hil_report, hil_util # noqa: E402 @@ -109,7 +109,7 @@ raise Refused(f'{board_name} is in boards-skip of {config}') raise Refused(f'{board_name} is not a board in {config}') board = boards[board_name] - if 'device/usbtest' not in ci_select.board_tests(board): + if 'device/usbtest' not in change_impact.board_tests(board): raise Refused(f'{board_name} does not run device/usbtest in {config} (its "tests" entry); ' f'its device port may not reach this host') flasher = board.get('flasher')
diff --git a/.claude/test/test_build.py b/.claude/test/test_build.py index 74c3308..9d12ecc 100644 --- a/.claude/test/test_build.py +++ b/.claude/test/test_build.py
@@ -1,4 +1,4 @@ -"""Tests for the build skill's check_build.py: scope resolution through ci_select, the +"""Tests for the build skill's check_build.py: scope resolution through change_impact, the dependency preflight, and the verdict it derives from tools/build.py's rows. The build itself is stubbed; a real board build is verified by running the script.""" import importlib.util @@ -22,8 +22,8 @@ def rec(reason, rule, effect='select', families=None, examples=None, port=None, targets=()): - """ci_select's build record for a reason line, its path the text before ': '.""" - return build.ci_select.build_record(reason.split(': ', 1)[0], rule, effect, reason, + """change_impact's build record for a reason line, its path the text before ': '.""" + return build.change_impact.build_record(reason.split(': ', 1)[0], rule, effect, reason, families, examples, port, targets) @@ -41,8 +41,8 @@ boards, how = build.boards_for(manifest(full=True, required=['stm32f411blackpill'])) self.assertEqual(boards, build.FULL_MATRIX_BOARDS + ['stm32f411blackpill']) - def test_base_mode_selects_through_ci_select_base_and_keeps_changed_boards(self): - # ci_select --base sees dependency revision changes that a path list cannot; + def test_base_mode_selects_through_change_impact_base_and_keeps_changed_boards(self): + # change_impact --base sees dependency revision changes that a path list cannot; # the manifest's required boards keep the board the diff changed m = manifest(families=['stm32f4'], required=['stm32f411blackpill']) with mock.patch.object(build, 'select', return_value=m) as sel, \ @@ -51,7 +51,7 @@ self.assertEqual(sel.call_args[0][:2], (None, 'master')) self.assertEqual(b1.call_args[0][0], 'stm32f411blackpill') - def test_select_forwards_each_mode_to_ci_select_manifest(self): + def test_select_forwards_each_mode_to_change_impact_manifest(self): cases = ((dict(base='master'), ['--base', 'master']), (dict(base='master', worktree=True), ['--base', 'master', '--worktree']), (dict(endpoints='a..b'), ['--endpoints', 'a..b'])) @@ -74,7 +74,7 @@ def _main_scope(self, scope, built=None, args=()): built = built or {'board': 'b', 'family': 'f', 'buildDir': 'd', 'status': 'ok', 'firstError': '', 'okExamples': ['cdc_msc', 'cdc_msc_hid']} - # the scopes below name paths that exist nowhere, to drive ci_select's records: + # the scopes below name paths that exist nowhere, to drive change_impact's records: # let them through the existence check as tracked deletions would be with mock.patch.object(build, 'build_one', return_value=built) as b1, mock.patch('sys.stdout') as out, \ mock.patch.object(build, 'tracked', return_value=True): @@ -89,7 +89,7 @@ def test_firmware_no_build_compiles_is_exit_3_with_the_reason_per_path(self): # a class no example enables, a lib nothing builds and a port mapping to no - # family are unverified firmware, in ci_select's own words + # family are unverified firmware, in change_impact's own words for scope, marker in ( (['src/class/bth/bth_device.c'], 'enabled by no example'), (['lib/SEGGER_RTT/RTT/SEGGER_RTT.c'], 'built by no example'), @@ -191,7 +191,7 @@ def test_a_membrowse_script_change_needs_its_own_target_not_a_default_sweep(self): # examples-membrowse-upload is a plain add_custom_target: `all` never runs # tools/membrowse_cli.py, so a green sweep is no evidence for it - r = build.ci_select.classify_build(['tools/membrowse_cli.py'], str(build.ROOT))['paths'][0] + r = build.change_impact.classify_build(['tools/membrowse_cli.py'], str(build.ROOT))['paths'][0] built = [{'family': 'stm32f4', 'okExamples': ['cdc_msc']}] gap = (f"{r['reason']} (the default sweep builds `all`, which does not run " f'examples-membrowse-upload: rerun with -T all -T examples-membrowse-upload)') @@ -204,7 +204,7 @@ def test_the_size_script_is_never_verified_by_a_local_build(self): # code_size.py runs in CI's code-size step only; no target or example stands in - r = build.ci_select.classify_build(['tools/code_size.py'], str(build.ROOT))['paths'][0] + r = build.change_impact.classify_build(['tools/code_size.py'], str(build.ROOT))['paths'][0] built = [{'family': 'stm32f4', 'okExamples': ['cdc_msc']}] for kw in ({}, {'chosen': True, 'targets': ('all', 'examples-membrowse-upload')}): gaps = build.coverage([r], built, **kw)[1] @@ -212,7 +212,7 @@ self.assertIn("CI's code-size step", gaps[0]) def test_a_core_stack_path_needs_a_built_example_of_its_role(self): - records = build.ci_select.classify_build(['src/host/usbh.c', 'src/device/usbd.c'], str(build.ROOT))['paths'] + records = build.change_impact.classify_build(['src/host/usbh.c', 'src/device/usbd.c'], str(build.ROOT))['paths'] scope = [r['path'] for r in records] device_only = [{'family': 'stm32f4', 'okExamples': ['cdc_msc']}] self.assertEqual(build.coverage(records, device_only)[1], [records[0]['reason']]) @@ -222,7 +222,7 @@ self.assertEqual(build.coverage(records, dual)[1], []) def test_get_deps_edit_changing_no_entry_is_nothing_to_build(self): - r = build.ci_select.classify_build(['tools/get_deps.py'], str(build.ROOT), set())['paths'][0] + r = build.change_impact.classify_build(['tools/get_deps.py'], str(build.ROOT), set())['paths'][0] self.assertEqual(build.coverage([r], []), ([r['reason']], [])) def test_non_code_paths_beside_code_do_not_fail_the_scope(self): @@ -274,11 +274,11 @@ self.assertEqual(build.representatives(pool, None, ['raspberrypi/rp2040/dcd_rp2040.c']), [pool[0]]) def test_a_directory_scope_expands_to_its_tracked_files(self): - # a bare board directory matches neither ci_select's rules nor the changed-board + # a bare board directory matches neither change_impact's rules nor the changed-board # rule, and would resolve to the family's sample instead of the board edited files = build.expand_scope(['hw/bsp/stm32f4/boards/stm32f411blackpill']) self.assertIn('hw/bsp/stm32f4/boards/stm32f411blackpill/board.h', files) - m = build.ci_select.manifest(files, str(build.ROOT), [], None, {})[0] + m = build.change_impact.manifest(files, str(build.ROOT), [], None, {})[0] self.assertEqual(m['build']['required_boards'], ['stm32f411blackpill']) self.assertEqual(build.boards_for(m)[0], ['stm32f411blackpill'])
diff --git a/.claude/test/test_build_select.py b/.claude/test/test_build_select.py index 8575b9d..5bc7653 100644 --- a/.claude/test/test_build_select.py +++ b/.claude/test/test_build_select.py
@@ -1,6 +1,6 @@ -"""The selection contract: `check_build.py --select-only` prints tools/ci_select.py's +"""The selection contract: `check_build.py --select-only` prints tools/change_impact.py's manifest v1 and stops before any dependency check or build, under the bare-runner -interpreter CI gives it; ci_select's input modes resolve their revisions once and +interpreter CI gives it; change_impact's input modes resolve their revisions once and refuse what they cannot answer.""" import json import os @@ -13,7 +13,7 @@ ROOT = Path(__file__).resolve().parents[2] sys.path.insert(0, str(ROOT / 'tools')) -import ci_select # noqa: E402 +import change_impact # noqa: E402 CHECK_BUILD = ROOT / '.claude' / 'skills' / 'build' / 'scripts' / 'check_build.py' # runs inside an isolated interpreter: every path to a build raises there @@ -78,7 +78,7 @@ def setUp(self): # a pre-commit hook exports GIT_DIR/GIT_INDEX_FILE for the outer repo: git in the - # scratch repo (ours and ci_select's) must not inherit them + # scratch repo (ours and change_impact's) must not inherit them env = mock.patch.dict(os.environ, {k: v for k, v in os.environ.items() if not k.startswith('GIT_')}, clear=True) env.start() @@ -109,7 +109,7 @@ p.write_text(text) def test_base_diffs_merge_base_to_head_and_records_the_shas(self): - files, gd, inp = ci_select.select_input(self.repo, base='main') + files, gd, inp = change_impact.select_input(self.repo, base='main') self.assertEqual(files, ['b.c']) self.assertIsNone(gd) self.assertEqual(inp, {'mode': 'base', 'base': 'main', 'base_sha': self.base, 'merge_base': self.base, @@ -118,12 +118,12 @@ def test_worktree_adds_uncommitted_and_untracked_files(self): self.write('a.c', 'changed\n') self.write('new.c', 'n\n') - files, _, inp = ci_select.select_input(self.repo, base='main', worktree=True) + files, _, inp = change_impact.select_input(self.repo, base='main', worktree=True) self.assertEqual(sorted(files), ['a.c', 'b.c', 'new.c']) self.assertEqual(inp['mode'], 'worktree') def test_endpoints_diff_a_to_b_without_a_merge_base(self): - files, _, inp = ci_select.select_input(self.repo, endpoints=f'{self.base}..HEAD') + files, _, inp = change_impact.select_input(self.repo, endpoints=f'{self.base}..HEAD') self.assertEqual(files, ['b.c']) self.assertEqual((inp['mode'], inp['base_sha'], inp['merge_base'], inp['head']), ('endpoints', self.base, None, self.head)) @@ -136,9 +136,9 @@ git(self.repo, 'commit', '-qm', 'main moves') main = git(self.repo, 'rev-parse', 'HEAD') git(self.repo, 'checkout', '-q', 'topic') - files, _, inp = ci_select.select_input(self.repo, endpoints=f'{main}..HEAD') + files, _, inp = change_impact.select_input(self.repo, endpoints=f'{main}..HEAD') self.assertEqual(sorted(files), ['b.c', 'm.c']) - self.assertEqual(ci_select.select_input(self.repo, base=main)[0], ['b.c']) + self.assertEqual(change_impact.select_input(self.repo, base=main)[0], ['b.c']) def test_a_stacked_branch_selects_against_base_not_its_parent(self): # CircleCI has no base-branch var and passes master (.circleci/config.yml): for a @@ -151,42 +151,42 @@ self.write('c.c', 'c\n') git(self.repo, 'add', '.') git(self.repo, 'commit', '-qm', 'child reverts a.c') - self.assertEqual(sorted(ci_select.select_input(self.repo, base='main')[0]), ['b.c', 'c.c']) - self.assertEqual(sorted(ci_select.select_input(self.repo, base='topic')[0]), ['a.c', 'c.c']) + self.assertEqual(sorted(change_impact.select_input(self.repo, base='main')[0]), ['b.c', 'c.c']) + self.assertEqual(sorted(change_impact.select_input(self.repo, base='topic')[0]), ['a.c', 'c.c']) def test_a_get_deps_edit_resolves_its_families_in_every_mode(self): # never the None that would mean "unresolvable, full matrix" self.write('tools/get_deps.py', "deps_optional = {'x': ['u', 'abc', 'fam1']}\n") listing = Path(self.repo, 'scope.txt') listing.write_text('tools/get_deps.py\n') - self.assertEqual(ci_select.select_input(self.repo, diff_file=str(listing), deps_base='HEAD')[1], {'fam1'}) - self.assertEqual(ci_select.select_input(self.repo, base='main', worktree=True)[1], {'fam1'}) + self.assertEqual(change_impact.select_input(self.repo, diff_file=str(listing), deps_base='HEAD')[1], {'fam1'}) + self.assertEqual(change_impact.select_input(self.repo, base='main', worktree=True)[1], {'fam1'}) git(self.repo, 'commit', '-qam', 'bump') - self.assertEqual(ci_select.select_input(self.repo, base='main')[1], {'fam1'}) - self.assertEqual(ci_select.select_input(self.repo, endpoints=f'{self.head}..HEAD')[1], {'fam1'}) + self.assertEqual(change_impact.select_input(self.repo, base='main')[1], {'fam1'}) + self.assertEqual(change_impact.select_input(self.repo, endpoints=f'{self.head}..HEAD')[1], {'fam1'}) def test_endpoints_refuse_what_they_cannot_answer(self): for bad in (f'{self.base}..{self.base}', '0000000000000000000000000000000000000000..HEAD', 'HEAD', '..HEAD', 'HEAD...main', 'nosuchref..HEAD'): - with self.assertRaises(ci_select.SelectError, msg=bad): - ci_select.select_input(self.repo, endpoints=bad) + with self.assertRaises(change_impact.SelectError, msg=bad): + change_impact.select_input(self.repo, endpoints=bad) def test_a_ref_that_names_no_commit_is_refused(self): - with self.assertRaises(ci_select.SelectError): - ci_select.select_input(self.repo, base='nosuchref') + with self.assertRaises(change_impact.SelectError): + change_impact.select_input(self.repo, base='nosuchref') def test_paths_with_deps_base_refuse_an_unchanged_get_deps(self): listing = Path(self.repo, 'scope.txt') listing.write_text('tools/get_deps.py\n') - with self.assertRaises(ci_select.SelectError): - ci_select.select_input(self.repo, diff_file=str(listing), deps_base='HEAD') - files, gd, inp = ci_select.select_input(self.repo, diff_file=str(listing)) + with self.assertRaises(change_impact.SelectError): + change_impact.select_input(self.repo, diff_file=str(listing), deps_base='HEAD') + files, gd, inp = change_impact.select_input(self.repo, diff_file=str(listing)) self.assertEqual((files, gd, inp['mode'], inp['deps_base']), (['tools/get_deps.py'], None, 'paths', None)) class ManifestTest(unittest.TestCase): def test_paths_carry_null_for_every_family_never_an_empty_list(self): - m, _, _ = ci_select.manifest(['src/class/cdc/cdc_device.c', 'hw/bsp/stm32f4/family.c', + m, _, _ = change_impact.manifest(['src/class/cdc/cdc_device.c', 'hw/bsp/stm32f4/family.c', 'src/portable/no_vendor/no_driver/dcd_bogus.c'], str(ROOT), [], None, {}) recs = {p['path']: p for p in m['build']['paths']} self.assertIsNone(recs['src/class/cdc/cdc_device.c']['families']) @@ -196,7 +196,7 @@ self.assertEqual(m['build']['families']['stm32f4']['examples'], 'all') def test_full_lists_no_families_and_targets_come_from_records(self): - m, _, _ = ci_select.manifest(['tools/membrowse_cli.py'], str(ROOT), [], None, {}) + m, _, _ = change_impact.manifest(['tools/membrowse_cli.py'], str(ROOT), [], None, {}) self.assertEqual((m['build']['full'], m['build']['families'], m['build']['required_targets']), (True, {}, ['examples-membrowse-upload'])) self.assertIs(m['hil']['needed'], False)
diff --git a/.github/scripts/ci_set_matrix.py b/.github/scripts/ci_set_matrix.py index e4aaf26..ee3de69 100755 --- a/.github/scripts/ci_set_matrix.py +++ b/.github/scripts/ci_set_matrix.py
@@ -140,8 +140,8 @@ # imported here, under the fall-open: a broken selector must still leave the # unscoped matrix sys.path.insert(0, os.path.join(REPO, 'tools')) - import ci_select - return ci_select.check_manifest(select) + import change_impact + return change_impact.check_manifest(select) except Exception as e: print(f'ci_set_matrix: UNSCOPED - selection unusable ({e}), emitting the full matrix', file=sys.stderr) return None
diff --git a/.github/scripts/hil_ci_set_matrix.py b/.github/scripts/hil_ci_set_matrix.py index 0aaca42..02a5fcb 100644 --- a/.github/scripts/hil_ci_set_matrix.py +++ b/.github/scripts/hil_ci_set_matrix.py
@@ -43,8 +43,8 @@ # imported here, under the fall-open: a broken selector must still leave the # full roster sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', 'tools')) - import ci_select - hil = ci_select.check_manifest(json.loads(raw))['hil'] + import change_impact + hil = change_impact.check_manifest(json.loads(raw))['hil'] except Exception as e: # fail-open: an unusable selection must never red the job # ALL of it is unusable, hil_examples included: keeping the -e lists would build # a few examples per board while the rig, unfiltered, runs its whole test list
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 1d9b95c..186a973 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml
@@ -47,15 +47,15 @@ fi CODE=true SUITES_OK=false - if [ -n "$SEL" ] && python3 test/hil/test/test_ci_select.py >/dev/null 2>&1 && + if [ -n "$SEL" ] && python3 test/hil/test/test_change_impact.py >/dev/null 2>&1 && python3 .claude/test/test_build_select.py >/dev/null 2>&1; then SUITES_OK=true CODE=$(python3 .claude/skills/build/scripts/check_build.py --select-only $SEL \ --config test/hil/tinyusb.json --config test/hil/hfp.json | python3 -c ' import json, sys sys.path.insert(0, "tools") - import ci_select - print(str(ci_select.build_gate(json.load(sys.stdin))).lower())') || CODE=true + import change_impact + print(str(change_impact.build_gate(json.load(sys.stdin))).lower())') || CODE=true fi [ "$CODE" = false ] || CODE=true echo "code=$CODE suites_ok=$SUITES_OK" @@ -102,7 +102,7 @@ # advisory workflow that nothing here can `needs:`. Test-failing selector => # full matrix, same as a crashing one. SELECT_JSON='' - if ! python3 test/hil/test/test_ci_select.py || ! python3 .claude/test/test_build_select.py; then + if ! python3 test/hil/test/test_change_impact.py || ! python3 .claude/test/test_build_select.py; then echo "::warning::selector unit suites failed - falling back to the full matrix" elif ! SELECT_JSON=$(python3 .claude/skills/build/scripts/check_build.py --select-only --base "origin/$BASE_REF" \ --config test/hil/tinyusb.json --config test/hil/hfp.json); then @@ -115,24 +115,24 @@ # KB, and an env var that big makes the consuming exec fail with E2BIG BEFORE # any fallback in it can run. Written here, ahead of its first reader. # No file (non-PR event, or any fallback) = full matrix. - rm -f ci_select_out.json + rm -f selection.json if [ -n "$SELECT_JSON" ]; then - printf '%s' "$SELECT_JSON" > ci_select_out.json + printf '%s' "$SELECT_JSON" > selection.json fi # One args/run pair per rig job, split by flasher: a job whose own subset is # empty skips explicitly instead of running a board filter that matches zero # boards ("No tests were run." exits 0 and would read as a green HIL run). OUT='' - if [ -s ci_select_out.json ]; then + if [ -s selection.json ]; then OUT=$(python3 -c ' import json, re, sys sys.path.insert(0, "tools") - import ci_select + import change_impact # the reading hil_ci_set_matrix.py applies too: both must agree - one falling # open to the whole roster while the other computes run=false buys a full # 37-leg build and still zero hardware coverage - s = ci_select.check_manifest(json.load(open("ci_select_out.json")))["hil"] + s = change_impact.check_manifest(json.load(open("selection.json")))["hil"] # a missing rig entry must not read as "nothing selected": run_hfp=false skips # hil-hfp-iar before it can run its own selection tin = s["args_flasher"].get("tinyusb.json") @@ -146,16 +146,16 @@ # roster board names reach $GITHUB_OUTPUT as bare NAME=VALUE lines; a # newline in one would inject extra run_* lines and flip which rig jobs run. # ":" and "," are part of the normal shape - a partial filter is - # `-bt <board>:<test>,<test>` (ci_select._board_args) + # `-bt <board>:<test>,<test>` (change_impact._board_args) if not re.fullmatch(r"[-A-Za-z0-9_/ .=+:,]*", a): sys.exit("unexpected characters in the " + key + " board filter") print("args_" + key + "=" + a) print("run_" + key + "=" + ("true" if (s["full"] or a) else "false")) ') || OUT='' if [ -z "$OUT" ]; then - echo "::warning::ci_select output unusable - falling back to the full HIL matrix" + echo "::warning::change_impact output unusable - falling back to the full HIL matrix" # the same unusable selection must not stay behind for the build axis - rm -f ci_select_out.json + rm -f selection.json fi fi if [ -z "$OUT" ]; then @@ -173,7 +173,7 @@ # Build matrix, scoped by the PR selection when one exists. Best-effort: # ci_set_matrix falls back to the full matrix itself on unusable JSON, # and a missing file (non-PR event, selector fallback) means no flags. - SELECT_FILE=ci_select_out.json + SELECT_FILE=selection.json [ -s "$SELECT_FILE" ] || SELECT_FILE='' BUILD_SELECT_FILE="$SELECT_FILE" PINNED_JSON='' @@ -705,20 +705,20 @@ SUITES_OK: ${{ needs.check-paths.outputs.selector_suites_ok }} run: | # Best-effort: a selector failure here falls back to the full hfp matrix (no - # ci_select.json, no SEL_* vars), never fails the job. check-paths ran the + # selection.json, no SEL_* vars), never fails the job. check-paths ran the # selector's suites on this commit; this is the rig's runner, so no rerun here. if [ "$SUITES_OK" != true ]; then echo "::warning::selector unit suites did not pass in check-paths - running the full hfp matrix" - rm -f ci_select.json + rm -f selection.json exit 0 fi if ! python3 .claude/skills/build/scripts/check_build.py --select-only --base "origin/$BASE_REF" \ - --config test/hil/hfp.json > ci_select.json; then + --config test/hil/hfp.json > selection.json; then echo "::warning::selector failed - running the full hfp matrix" - rm -f ci_select.json + rm -f selection.json exit 0 fi - # ci_select.json is passed to hil_ci_set_matrix.py --select below to scope the + # selection.json is passed to hil_ci_set_matrix.py --select below to scope the # build; it already honours hil.full by ignoring the board list. # The hil_test.py args go to a file, never to $GITHUB_ENV: they are derived # from roster board names, which a PR can edit. Only SEL_RUN (a literal @@ -726,14 +726,14 @@ if ! SEL_RUN=$(python3 -c ' import json, sys sys.path.insert(0, "tools") - import ci_select - s = ci_select.check_manifest(json.load(open("ci_select.json")))["hil"] + import change_impact + s = change_impact.check_manifest(json.load(open("selection.json")))["hil"] a = s["args"]["hfp.json"] open("hil_sel_args.txt", "w").write(a) print("true" if (s["full"] or a) else "false") '); then - echo "::warning::ci_select output unusable - running the full hfp matrix" - rm -f ci_select.json hil_sel_args.txt + echo "::warning::change_impact output unusable - running the full hfp matrix" + rm -f selection.json hil_sel_args.txt exit 0 fi echo "SEL_RUN=$SEL_RUN" @@ -746,8 +746,8 @@ # can exceed MAX_ARG_STRLEN on a big diff, and this job's design is to fall # back to the full hfp matrix on any selector trouble, not to fail the step. MATRIX_JSON='' - if [ -f ci_select.json ]; then - MATRIX_JSON=$(python .github/scripts/hil_ci_set_matrix.py --select-file ci_select.json test/hil/hfp.json) || MATRIX_JSON='' + if [ -f selection.json ]; then + MATRIX_JSON=$(python .github/scripts/hil_ci_set_matrix.py --select-file selection.json test/hil/hfp.json) || MATRIX_JSON='' if [ -z "$MATRIX_JSON" ]; then echo "::warning::scoped hfp matrix failed - building the full hfp matrix" fi
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index fc9931c..fdff4f8 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml
@@ -60,10 +60,10 @@ # file patterns overlap where both suites care. hil-test runs test_hil*.py only # (~45s: deliberate hang and timeout simulations) and is scoped to the rig harness # that owns them. The one part of it the selector depends on - the BottomLayer - # stdlib-closure AST guard over tools/ci_select.py and its imports - is named - # explicitly by ci-select-test instead, so a tools/ or workflow edit costs 4s + # stdlib-closure AST guard over tools/change_impact.py and its imports - is named + # explicitly by change-impact-test instead, so a tools/ or workflow edit costs 4s # rather than 80s of hang simulations that have nothing to say about it. - # ci-select-test runs the two selector-adjacent suites (~4s together) that read + # change-impact-test runs the two selector-adjacent suites (~4s together) that read # hw/bsp (board.cmake, FAMILY_MCUS), src (portable dirs + class include graph), # examples (tusb_config.h, skip/only.txt), hw/mcu, the rig rosters under test/hil # (a roster edit changes what the selector emits), .circleci (the sentinel contract @@ -86,10 +86,10 @@ always_run: true pass_filenames: false language: system - - id: ci-select-test - name: ci-select-test - files: ^(hw/bsp/|hw/mcu/|src/|examples/|test/hil/|tools/(ci_select|build|build_utils|family_json|get_deps|rtt)\.py$|\.github/(scripts|workflows)/|\.circleci/) - entry: sh -c "python3 test/hil/test/test_ci_select.py && python3 test/hil/test/test_ci_metrics.py && cd test/hil/test && python3 -m unittest -q test_hil_util.BottomLayer" + - id: change-impact-test + name: change-impact-test + files: ^(hw/bsp/|hw/mcu/|src/|examples/|test/hil/|tools/(change_impact|build|build_utils|family_json|get_deps|rtt)\.py$|\.github/(scripts|workflows)/|\.circleci/) + entry: sh -c "python3 test/hil/test/test_change_impact.py && python3 test/hil/test/test_ci_metrics.py && cd test/hil/test && python3 -m unittest -q test_hil_util.BottomLayer" pass_filenames: false language: system - id: setup-toolchain-test @@ -112,7 +112,7 @@ - id: drivers-coverage name: drivers-coverage - files: ^(\.github/ci-pinned-boards\.json$|\.github/scripts/ci_set_matrix\.py$|tools/drivers_coverage_check\.py$|tools/ci_select\.py$|tools/build_utils\.py$|hw/bsp/|src/portable/|examples/|test/hil/test/test_drivers_coverage\.py$|tools/build\.py$|test/hil/test/test_ci_boards\.py$|tools/code_size\.py$|test/hil/test/test_code_size\.py$|\.github/scripts/code_size_ci\.py$|test/hil/test/test_code_size_ci\.py$|tools/membrowse_cli\.py$|test/hil/test/test_membrowse_cli\.py$|test/hil/tinyusb\.json$|test/hil/hfp\.json$) + files: ^(\.github/ci-pinned-boards\.json$|\.github/scripts/ci_set_matrix\.py$|tools/drivers_coverage_check\.py$|tools/change_impact\.py$|tools/build_utils\.py$|hw/bsp/|src/portable/|examples/|test/hil/test/test_drivers_coverage\.py$|tools/build\.py$|test/hil/test/test_ci_boards\.py$|tools/code_size\.py$|test/hil/test/test_code_size\.py$|\.github/scripts/code_size_ci\.py$|test/hil/test/test_code_size_ci\.py$|tools/membrowse_cli\.py$|test/hil/test/test_membrowse_cli\.py$|test/hil/tinyusb\.json$|test/hil/hfp\.json$) entry: sh -c "cd test/hil/test && python3 -m unittest -q test_drivers_coverage test_ci_boards test_code_size test_code_size_ci test_membrowse_cli" pass_filenames: false language: python
diff --git a/docs/reference/hardware-in-the-loop.md b/docs/reference/hardware-in-the-loop.md index 7cdcaa5..293b8b2 100644 --- a/docs/reference/hardware-in-the-loop.md +++ b/docs/reference/hardware-in-the-loop.md
@@ -281,7 +281,7 @@ `test/hil/hil_test.py`, which flashes each board and runs its tests. Espressif boards run in `hil-tinyusb-esp`, gated on the slower ESP-IDF build, and `hil-hfp-iar` builds with IAR inside the job. -3. On pull requests, `tools/ci_select.py` narrows the run to the boards a diff can +3. On pull requests, `tools/change_impact.py` narrows the run to the boards a diff can affect — and each board's build to the examples its tests need — falling open to the full matrix when it cannot tell. The same pass scopes the build matrix. 4. Each board is arbitrated by a kernel flock in `/tmp/tinyusb-hil-locks/`, so interactive
diff --git a/docs/superpowers/followup/hil-farm.md b/docs/superpowers/followup/hil-farm.md index a8d39c3..d50cced 100644 --- a/docs/superpowers/followup/hil-farm.md +++ b/docs/superpowers/followup/hil-farm.md
@@ -26,7 +26,7 @@ Project layer: the ~30 `test_*` cases and `_tests_for` map in `hil_test.py` (~1500 lines), the example list in `hil_util.py:31-52`, `find_firmware` and the `examples/cmake-build-<board>[-variant]/<example>` layout, `build_board`, -`tools/ci_select.py`, DUT identity (VID `cafe`), "missing binary = skip" and +`tools/change_impact.py`, DUT identity (VID `cafe`), "missing binary = skip" and "metric string = pass" policies. Seams where TinyUSB leaks into generic files (Codex, verified by path):
diff --git a/docs/superpowers/followup/rtt-full-move.md b/docs/superpowers/followup/rtt-full-move.md index c0459f8..e44e7c9 100644 --- a/docs/superpowers/followup/rtt-full-move.md +++ b/docs/superpowers/followup/rtt-full-move.md
@@ -14,7 +14,7 @@ re-exports `JlinkRtt`, `OpenocdRtt`, `RttError`, `strip_banner`, `RTT_BANNER_RE`; `hil_test.py` opens `JlinkRtt` as the console of a board with `"logger": "rtt"` (only `ea4088_quickstart` in `tinyusb.json`). -- `tools/ci_select.py` imports `hil_util` on GitHub-hosted ubuntu-latest runners +- `tools/change_impact.py` imports `hil_util` on GitHub-hosted ubuntu-latest runners (build.yml path-selection jobs), so the file must be present there today. - `hil_remote.py` (hil skill) stages `tools/rtt.py` next to `test/hil`; build.yml's path filter and the pre-commit `hil-test` pattern name the file; `test_hil_rtt.py` asserts @@ -28,7 +28,7 @@ `strip_banner`, from `HIL_RTT_PY` (default `~/.claude/skills/rtt/scripts/rtt.py`), registered in `sys.modules` so `RttError` still pickles across the fork pool; the ImportError names the - resolved path, the variable and the install command. `ci_select` then never + resolved path, the variable and the install command. `change_impact` then never touches it. 2. Delete `tools/rtt.py`; drop its `HARNESS_FILES` entry in `hil_remote.py`, the build.yml path filter entry, the pre-commit pattern and the staging assertion.
diff --git a/docs/superpowers/plans/2026-07-29-hil-select.md b/docs/superpowers/plans/2026-07-29-hil-select.md deleted file mode 100644 index a8abf98..0000000 --- a/docs/superpowers/plans/2026-07-29-hil-select.md +++ /dev/null
@@ -1,856 +0,0 @@ -# PR-Scoped HIL Selection Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** A diff→(boards, tests) selector (`test/hil/hil_select.py`) that scopes CI's HIL build+test jobs on pull requests and is reusable locally, per `docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md`. - -**Architecture:** Pure-stdlib classification engine (changed files → per-board test selection, fail-open to full) + thin CLI emitting JSON with per-rig `hil_test.py` arg strings; consumed by `hil_ci_set_matrix.py --select` (prunes hil-build) and shell steps in the three HIL jobs (prunes rig runs). Test lists shared via new `hil_examples.py`. - -**Tech Stack:** Python 3.11 stdlib only (`re`, `json`, `glob`, `subprocess` for git), `unittest` for tests, GitHub Actions YAML. - -## Global Constraints - -- Work in worktree `.claude/worktrees/hil-select` (branch `claude/hil-select`); never touch the primary checkout. -- `hil_select.py`, `hil_examples.py`, `test_hil_select.py` import NOTHING outside the stdlib and each other — in particular never `hil_test`/`hil_flash`/`hil_lock` (GitHub's bare runner has no pyserial/pymtp). -- Fail-open: any changed file matching no classification rule ⇒ `full: true`. Scoping applies to `pull_request` events only; push/scheduled runs stay full. -- Behavior-preserving for existing tools: `hil_test.py` runtime behavior unchanged (only its test-list constants move to `hil_examples.py`); `hil_ci_set_matrix.py` without `--select` emits byte-identical output to today. -- The selector only ever emits board names present in the given roster (`config['boards']`); `boards-skip` is invisible to it. -- Commit messages: imperative, scoped, NO Co-Authored-By/Claude-Session trailers. -- Every commit: `python3 -m py_compile` clean on touched python files, `python3 test/hil/test_hil_select.py` green (once it exists), `pre-commit run --files <touched>` clean. - ---- - -### Task 1: hil_examples.py + selection engine with unit tests - -**Files:** -- Create: `test/hil/hil_examples.py` -- Create: `test/hil/hil_select.py` (engine only; CLI comes in Task 2) -- Create: `test/hil/test_hil_select.py` -- Modify: `test/hil/hil_test.py` (import test lists from hil_examples) -- Modify: `test/hil/hil_ci.sh` (scp list gains `hil_examples.py`) - -**Interfaces:** -- Produces `hil_examples.py`: `device_tests: list[str]`, `dual_tests: list[str]`, `host_test: list[str]` — the three lists moved VERBATIM (incl. comments) from `hil_test.py`. -- Produces `hil_select.py` engine API used by Task 2: - - `classify(changed_files: list[str], repo_root: str, rosters: list[tuple[str, list[dict]]]) -> dict` - returning `{'full': bool, 'boards': {board_name: 'all' | sorted list[str]}, 'reasons': list[str]}` - where `rosters` = `[(config_path, config['boards']), ...]`. - - `board_roles(board: dict) -> set[str]` — subset of `{'device', 'host'}` from the roster - entry's `tests` flags (`device`/`host`/`dual` booleans; an `only` list contributes the - roles of its entries' path prefixes; `dual` implies both roles). - - `board_family(board_name: str, repo_root: str) -> str | None` — the `<family>` for which - `hw/bsp/<family>/boards/<board_name>` exists. - - `port_families(port_dir: str, repo_root: str) -> set[str]` — directories of - `hw/bsp/*/family.cmake` and `hw/bsp/*/family.mk` whose text contains `port_dir` - (e.g. `raspberrypi/rp2040`). - - `class_examples(class_dir: str, role: str, repo_root: str) -> set[str]` — tests from - `hil_examples` lists whose example `tusb_config.h` enables the class for that role (regex - `#define\s+CFG_TUD_<C>\s+\(?\s*0*[1-9]` / `CFG_TUH_<C>`; exceptions per spec: - `dfu_rt_device.*`→`CFG_TUD_DFU_RUNTIME`, `dfu_device.*`→`CFG_TUD_DFU`, class dir `net` - → `CFG_TUD_ECM_RNDIS|CFG_TUD_NCM`). Test path `device/x` ⇒ config at - `examples/device/x/src/tusb_config.h`; same pattern for `host/` and `dual/`. - -- [ ] **Step 1: Move the test lists into `hil_examples.py`** - -Create `test/hil/hil_examples.py`: - -```python -#!/usr/bin/env python3 -# SPDX-License-Identifier: MIT -# HIL example test lists, shared by hil_test.py (runner) and hil_select.py -# (PR-diff selector). Stdlib-only: hil_select runs on bare CI runners. -``` - -then MOVE the `device_tests`, `dual_tests`, `host_test` list definitions (and their preceding -comment block "The per-board run order is shuffled...") VERBATIM from `hil_test.py` into it. -In `hil_test.py`, add `from hil_examples import device_tests, dual_tests, host_test` where the -lists were (a `from`-import of data constants is fine here — they are read-only lists used by -name throughout `test_board`). Add `"$ROOT_DIR/test/hil/hil_examples.py" \` to the -`hil_ci.sh` scp list after the `hil_lock.py` line. - -- [ ] **Step 2: Verify the move broke nothing** - -Run: `cd /home/hathach/code/tinyusb/.claude/worktrees/hil-select && python3 -m py_compile test/hil/hil_examples.py test/hil/hil_test.py && python3 test/hil/hil_test.py --help >/dev/null && echo ok` -Expected: `ok` - -- [ ] **Step 3: Write the failing unit tests (spec acceptance cases)** - -Create `test/hil/test_hil_select.py`. ROSTER is a trimmed but real-shaped fixture; tests call -the engine API directly (no git, no CLI): - -```python -#!/usr/bin/env python3 -# SPDX-License-Identifier: MIT -# Unit tests for hil_select.py — pure logic, no hardware, no git. Run directly: -# python3 test/hil/test_hil_select.py -import os -import sys -import unittest - -sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) -import hil_select -from hil_examples import device_tests, dual_tests, host_test - -REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - -ROSTER = [ - # device-only, rp2040 family - {'name': 'raspberry_pi_pico', 'uid': 'u1', - 'tests': {'device': True, 'host': True, 'dual': True}}, - # device-only, stm32f4 family - {'name': 'stm32f407disco', 'uid': 'u2', - 'tests': {'device': True, 'host': False, 'dual': False}}, - # host-only board - {'name': 'raspberry_pi_pico2', 'uid': 'u3', - 'tests': {'device': False, 'host': True, 'dual': False}}, - # only-list board (espressif-style) - {'name': 'espressif_s3_devkitm', 'uid': 'u4', - 'tests': {'only': ['device/cdc_msc_freertos', 'host/device_info']}}, -] -ROSTERS = [('test/hil/tinyusb.json', ROSTER)] - - -def sel(files): - return hil_select.classify(files, REPO, ROSTERS) - - -class TestPortRule(unittest.TestCase): - def test_dcd_rp2040_selects_pico_family_only(self): - s = sel(['src/portable/raspberrypi/rp2040/dcd_rp2040.c']) - self.assertFalse(s['full']) - self.assertIn('raspberry_pi_pico', s['boards']) - self.assertNotIn('stm32f407disco', s['boards']) - self.assertNotIn('espressif_s3_devkitm', s['boards']) - # device role: no host tests in pico's list - self.assertTrue(all(not t.startswith('host/') for t in s['boards']['raspberry_pi_pico'])) - # host-only boards drop out entirely on a device-role change - self.assertNotIn('raspberry_pi_pico2', s['boards']) - - def test_shared_port_file_is_both_roles(self): - s = sel(['src/portable/synopsys/dwc2/dwc2_common.c']) - self.assertFalse(s['full']) - self.assertNotIn('raspberry_pi_pico', s['boards']) # rp2040 is not a dwc2 family - self.assertIn('stm32f407disco', s['boards']) # stm32f4 is - - -class TestCoreRoleRule(unittest.TestCase): - def test_usbd_selects_all_device_tests_everywhere(self): - s = sel(['src/device/usbd.c']) - self.assertFalse(s['full']) - self.assertNotIn('raspberry_pi_pico2', s['boards']) # host-only board dropped - pico = s['boards']['raspberry_pi_pico'] - self.assertTrue(set(device_tests).issubset(set(pico))) - self.assertTrue(set(dual_tests).issubset(set(pico))) # dual survives device role - self.assertTrue(all(not t.startswith('host/') for t in pico)) - # only-list board: selection intersects its only-list - esp = s['boards']['espressif_s3_devkitm'] - self.assertEqual(esp, ['device/cdc_msc_freertos']) - - def test_host_change_drops_device(self): - s = sel(['src/host/usbh.c']) - self.assertFalse(s['full']) - self.assertIn('raspberry_pi_pico2', s['boards']) - self.assertNotIn('stm32f407disco', s['boards']) # device-only board dropped - - -class TestClassRule(unittest.TestCase): - def test_cdc_device_selects_cdc_examples_only(self): - s = sel(['src/class/cdc/cdc_device.c']) - self.assertFalse(s['full']) - pico = s['boards']['raspberry_pi_pico'] - self.assertIn('device/cdc_msc', pico) - self.assertIn('device/cdc_dual_ports', pico) - self.assertNotIn('device/msc_dual_lun', pico) # CFG_TUD_CDC 0 there - self.assertNotIn('device/usbtest', pico) # CFG_TUD_CDC 0 there - self.assertTrue(all(not t.startswith('host/') for t in pico)) - - def test_msc_host_selects_host_side(self): - s = sel(['src/class/msc/msc_host.c']) - self.assertFalse(s['full']) - self.assertNotIn('stm32f407disco', s['boards']) # device-only board - pico2 = s['boards']['raspberry_pi_pico2'] - self.assertIn('host/msc_file_explorer', pico2) - self.assertTrue(all(not t.startswith('device/') for t in pico2)) - - -class TestFallbackRules(unittest.TestCase): - def test_unknown_tool_is_full(self): - s = sel(['tools/random_new_script.py']) - self.assertTrue(s['full']) - - def test_docs_only_is_empty_not_full(self): - s = sel(['docs/info/contributing.rst', 'README.rst']) - self.assertFalse(s['full']) - self.assertEqual(s['boards'], {}) - - def test_bsp_family_selects_family_boards(self): - s = sel(['hw/bsp/rp2040/family.cmake']) - self.assertFalse(s['full']) - self.assertIn('raspberry_pi_pico', s['boards']) - self.assertEqual(s['boards']['raspberry_pi_pico'], 'all') - self.assertNotIn('stm32f407disco', s['boards']) - - def test_bsp_board_narrows_to_board(self): - s = sel(['hw/bsp/rp2040/boards/raspberry_pi_pico/board.h']) - self.assertFalse(s['full']) - self.assertEqual(list(s['boards'].keys()), ['raspberry_pi_pico']) - - def test_example_change_selects_that_example(self): - s = sel(['examples/device/cdc_msc/src/main.c']) - self.assertFalse(s['full']) - self.assertEqual(s['boards']['raspberry_pi_pico'], ['device/cdc_msc']) - - def test_core_common_is_full(self): - for f in ['src/tusb.c', 'src/common/tusb_fifo.c', 'src/osal/osal_freertos.h']: - self.assertTrue(sel([f])['full'], f) - - def test_harness_is_full(self): - for f in ['test/hil/hil_test.py', '.github/workflows/build.yml', 'hw/mcu/nxp/x.c', 'lib/foo/x.c']: - self.assertTrue(sel([f])['full'], f) - - def test_mixed_roles_no_pruning(self): - s = sel(['src/device/usbd.c', 'src/host/usbh.c']) - self.assertFalse(s['full']) - self.assertIn('raspberry_pi_pico2', s['boards']) - self.assertIn('stm32f407disco', s['boards']) - - -if __name__ == '__main__': - unittest.main(verbosity=1) -``` - -- [ ] **Step 4: Run tests to verify they fail** - -Run: `python3 test/hil/test_hil_select.py 2>&1 | tail -2` -Expected: `ModuleNotFoundError: No module named 'hil_select'` (or import error). - -- [ ] **Step 5: Implement the engine** - -Create `test/hil/hil_select.py`: - -```python -#!/usr/bin/env python3 -# SPDX-License-Identifier: MIT -"""PR-diff -> HIL selection: which rig boards and which tests a change can affect. - -Stdlib-only (runs on bare CI runners; never imports hil_test/hil_flash/hil_lock). -Fail-open: any file no rule classifies forces the full matrix. See -docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md. -""" -import argparse -import glob -import json -import os -import re -import subprocess -import sys - -from hil_examples import device_tests, dual_tests, host_test - -ALL_TESTS = {'device': device_tests, 'dual': dual_tests, 'host': host_test} - -# class dir -> config macro suffix exceptions (rule 3); dfu is per-file, handled inline -NET_MACROS = ('ECM_RNDIS', 'NCM') - -_NONCODE_RE = re.compile( - r'^(docs/|\.claude/|.*\.(md|rst|txt)$|LICENSE)') -_FULL_RE = re.compile( - r'^(src/common/|src/osal/|src/tusb\.c$|src/tusb\.h$|src/tusb_option\.h$|' - r'test/hil/|\.github/workflows/build.*\.yml$|\.github/actions/|' - r'tools/build\.py$|tools/get_deps\.py$|tools/cmake/|hw/mcu/|lib/|' - r'hw/bsp/(family_support\.cmake|board_api\.h|board\.c|ansi_escape\.h)$|' - r'examples/build_system/|examples/CMakeLists\.txt$)') - - -def test_role(test: str) -> str: - return test.split('/', 1)[0] # 'device' | 'dual' | 'host' - - -def board_roles(board: dict) -> set: - t = board.get('tests', {}) - roles = set() - if t.get('device'): - roles.add('device') - if t.get('host'): - roles.add('host') - if t.get('dual'): - roles.update(('device', 'host')) - for only in t.get('only', []): - r = test_role(only) - roles.update(('device', 'host') if r == 'dual' else (r,)) - return roles - - -def board_tests(board: dict) -> list: - """Every test this board would run today (mirrors hil_test.test_board's default).""" - t = board.get('tests', {}) - if 'only' in t: - run = list(t['only']) - else: - run = [] - if t.get('device'): - run += device_tests - if t.get('dual'): - run += dual_tests - if t.get('host'): - run += host_test - return [x for x in run if x not in t.get('skip', [])] - - -def board_family(board_name: str, repo_root: str): - hits = glob.glob(os.path.join(repo_root, 'hw/bsp/*/boards', board_name)) - return os.path.basename(os.path.dirname(os.path.dirname(hits[0]))) if hits else None - - -def port_families(port_dir: str, repo_root: str) -> set: - fams = set() - for f in glob.glob(os.path.join(repo_root, 'hw/bsp/*/family.cmake')) + \ - glob.glob(os.path.join(repo_root, 'hw/bsp/*/family.mk')): - try: - if port_dir in open(f).read(): - fams.add(os.path.basename(os.path.dirname(f))) - except OSError: - pass - return fams - - -def _config_enables(cfg_path: str, macros) -> bool: - try: - text = open(cfg_path).read() - except OSError: - return False - return any(re.search(rf'#define\s+{m}\s+\(?\s*0*[1-9]', text) for m in macros) - - -def class_examples(macros, role: str, repo_root: str) -> set: - """Tests (from role's + dual lists) whose example config enables any macro.""" - pools = {'device': device_tests + dual_tests, 'host': host_test + dual_tests} - out = set() - for test in pools[role]: - cfg = os.path.join(repo_root, 'examples', test, 'src', 'tusb_config.h') - if _config_enables(cfg, macros): - out.add(test) - return out - - -class _Sel: - """Accumulates contributions. board->set(tests) plus 'all-board' markers.""" - def __init__(self): - self.full = False - self.by_board = {} # name -> set of tests, or 'all' - self.roles = set() # roles touched by any contribution - self.reasons = [] - - def add(self, boards, tests, reason): - """tests: 'all' or iterable of test paths.""" - self.reasons.append(reason) - for b in boards: - cur = self.by_board.get(b) - if tests == 'all' or cur == 'all': - self.by_board[b] = 'all' - else: - self.by_board[b] = (cur or set()) | set(tests) - - def force_full(self, reason): - self.full = True - self.reasons.append(reason) - - -def _classify_one(path, repo_root, roster_boards, s: _Sel): - base = os.path.basename(path) - if _NONCODE_RE.match(path): - s.reasons.append(f'{path}: non-code, no contribution') - return - if _FULL_RE.match(path): - s.force_full(f'{path}: core/infra -> full matrix') - return - - m = re.match(r'src/portable/((?:[^/]+/)?[^/]+)/', path) - if m: - port = m.group(1) - if re.match(r'(dcd_|.*_device)', base): - roles = {'device'} - elif re.match(r'(hcd_|.*_host)', base): - roles = {'host'} - else: - roles = {'device', 'host'} - fams = port_families(port, repo_root) - boards = [b['name'] for b in roster_boards - if board_family(b['name'], repo_root) in fams and (board_roles(b) & roles)] - tests = [t for r in roles for t in ALL_TESTS[r]] + dual_tests - s.roles.update(roles) - s.add(boards, tests, f'{path}: port {port} -> families {sorted(fams)} -> boards {boards} ({"/".join(sorted(roles))})') - return - - m = re.match(r'src/class/([^/]+)/', path) - if m: - cls = m.group(1) - if re.search(r'_device\.[ch]$', base): - roles = {'device'} - elif re.search(r'_host\.[ch]$', base): - roles = {'host'} - else: - roles = {'device', 'host'} - # macro names per role - def macros(prefix): - if cls == 'net': - return [f'CFG_{prefix}_{m2}' for m2 in NET_MACROS] - if cls == 'dfu': - if base.startswith('dfu_rt'): - return [f'CFG_{prefix}_DFU_RUNTIME'] - if base.startswith('dfu_device') or base.startswith('dfu_host'): - return [f'CFG_{prefix}_DFU'] - return [f'CFG_{prefix}_DFU', f'CFG_{prefix}_DFU_RUNTIME'] - return [f'CFG_{prefix}_{cls.upper()}'] - tests = set() - if 'device' in roles: - tests |= class_examples(macros('TUD'), 'device', repo_root) - if 'host' in roles: - tests |= class_examples(macros('TUH'), 'host', repo_root) - boards = [b['name'] for b in roster_boards if board_roles(b) & roles] - s.roles.update(roles) - s.add(boards, tests, f'{path}: class {cls} -> {sorted(tests)} ({"/".join(sorted(roles))})') - return - - m = re.match(r'src/(device|host)/', path) - if m: - role = m.group(1) - boards = [b['name'] for b in roster_boards if role in board_roles(b)] - s.roles.add(role) - s.add(boards, ALL_TESTS[role] + dual_tests, f'{path}: core {role} stack -> all {role} tests') - return - - m = re.match(r'hw/bsp/([^/]+)/(?:boards/([^/]+)/)?', path) - if m: - fam, brd = m.group(1), m.group(2) - if brd: - boards = [b['name'] for b in roster_boards if b['name'] == brd] - why = f'{path}: bsp board {brd}' - else: - boards = [b['name'] for b in roster_boards - if board_family(b['name'], repo_root) == fam] - why = f'{path}: bsp family {fam}' - s.roles.update(('device', 'host')) - s.add(boards, 'all', f'{why} -> boards {boards}') - return - - m = re.match(r'examples/(device|host|dual)/([^/]+)/', path) - if m: - test = f'{m.group(1)}/{m.group(2)}' - known = any(test in pool for pool in ALL_TESTS.values()) - if known: - boards = [b['name'] for b in roster_boards] - role = test_role(test) - s.roles.update(('device', 'host') if role == 'dual' else (role,)) - s.add(boards, [test], f'{path}: example -> {test} on all boards') - else: - s.reasons.append(f'{path}: example not in HIL lists, no contribution') - return - - s.force_full(f'{path}: unclassified -> full matrix') - - -def classify(changed_files, repo_root, rosters): - all_boards = [] - seen = set() - for _, boards in rosters: - for b in boards: - if b['name'] not in seen: - seen.add(b['name']) - all_boards.append(b) - - s = _Sel() - for path in changed_files: - _classify_one(path, repo_root, all_boards, s) - if s.full: - break - - if s.full: - return {'full': True, 'boards': {b['name']: 'all' for b in all_boards}, - 'reasons': s.reasons} - - # role pruning: single-role selections drop the other role's tests and boards - by_name = {b['name']: b for b in all_boards} - out = {} - for name, tests in s.by_board.items(): - allowed = board_tests(by_name[name]) - if tests == 'all': - kept = list(allowed) - else: - kept = [t for t in allowed if t in tests] - if s.roles and s.roles != {'device', 'host'}: - role = next(iter(s.roles)) - kept = [t for t in kept if test_role(t) in (role, 'dual')] - if kept: - out[name] = 'all' if set(kept) == set(allowed) else sorted(kept) - return {'full': False, 'boards': out, 'reasons': s.reasons} -``` - -- [ ] **Step 6: Run tests to verify they pass** - -Run: `python3 test/hil/test_hil_select.py` -Expected: all tests PASS (OK line). Iterate on the engine (not the tests) until green; if a -test premise contradicts the repo (e.g. a family name), verify against the tree and fix the -test only with evidence noted in your report. - -- [ ] **Step 7: Commit** - -```bash -git add test/hil/hil_examples.py test/hil/hil_select.py test/hil/test_hil_select.py test/hil/hil_test.py test/hil/hil_ci.sh -git commit -m "hil: add PR-diff selection engine (hil_select) with shared example lists" -``` - ---- - -### Task 2: CLI + args emission - -**Files:** -- Modify: `test/hil/hil_select.py` (add `selection_args`, `main`) -- Modify: `test/hil/test_hil_select.py` (add CLI/args tests) - -**Interfaces:** -- Consumes: Task 1's `classify` and roster shapes. -- Produces: - - `selection_args(sel: dict, rosters) -> dict` mapping each config path's basename to the - `hil_test.py` argument string for that rig: for each selected board ON that roster, - `-b <name>`, plus `-bt <name>:<t1>,<t2>` when the board's entry is a list (not 'all'). - Empty string when no selected board is on that roster. When `sel['full']`, every roster - board gets bare `-b`? NO — full means "today's behavior": `selection_args` returns `''` - for every config (no filtering args at all). - - CLI: `python3 test/hil/hil_select.py [--base REF | --diff-file PATH] CONFIG...` printing - the JSON `{'full', 'boards', 'args', 'reasons'}` to stdout, reasons also to stderr - (one line each, prefixed `hil_select: `). Non-zero exit only on operational errors - (bad ref, unreadable config) — never on an empty selection. - -- [ ] **Step 1: Add failing CLI/args tests to `test_hil_select.py`** - -```python -class TestArgsEmission(unittest.TestCase): - def test_args_for_scoped_selection(self): - s = sel(['src/portable/raspberrypi/rp2040/dcd_rp2040.c']) - args = hil_select.selection_args(s, ROSTERS) - a = args['tinyusb.json'] - self.assertIn('-b raspberry_pi_pico', a) - self.assertNotIn('stm32f407disco', a) - self.assertIn('-bt raspberry_pi_pico:', a) # device-only subset of a device+host board - - def test_args_full_is_empty(self): - s = sel(['tools/random_new_script.py']) - self.assertEqual(hil_select.selection_args(s, ROSTERS), {'tinyusb.json': ''}) - - def test_args_all_board_gets_bare_b(self): - s = sel(['hw/bsp/rp2040/boards/raspberry_pi_pico/board.h']) - a = hil_select.selection_args(s, ROSTERS)['tinyusb.json'] - self.assertIn('-b raspberry_pi_pico', a) - self.assertNotIn('-bt', a) - - def test_cli_diff_file(self): - import subprocess, tempfile, json as j - with tempfile.NamedTemporaryFile('w', suffix='.txt', delete=False) as f: - f.write('src/class/cdc/cdc_device.c\n') - path = f.name - r = subprocess.run([sys.executable, os.path.join(REPO, 'test/hil/hil_select.py'), - '--diff-file', path, os.path.join(REPO, 'test/hil/tinyusb.json')], - capture_output=True, text=True) - self.assertEqual(r.returncode, 0, r.stderr) - out = j.loads(r.stdout) - self.assertFalse(out['full']) - self.assertIn('tinyusb.json', out['args']) - self.assertTrue(any('cdc_device' in line for line in out['reasons'])) - os.unlink(path) -``` - -- [ ] **Step 2: Run to verify the new tests fail** - -Run: `python3 test/hil/test_hil_select.py 2>&1 | tail -3` -Expected: failures/errors mentioning `selection_args`. - -- [ ] **Step 3: Implement `selection_args` and `main`** - -Append to `hil_select.py`: - -```python -def selection_args(sel, rosters): - args = {} - for cfg_path, boards in rosters: - key = os.path.basename(cfg_path) - if sel['full']: - args[key] = '' - continue - parts = [] - for b in boards: - chosen = sel['boards'].get(b['name']) - if chosen is None: - continue - parts.append(f'-b {b["name"]}') - if chosen != 'all': - parts.append(f'-bt {b["name"]}:{",".join(chosen)}') - args[key] = ' '.join(parts) - return args - - -def changed_files_from_git(base, repo_root): - mb = subprocess.run(['git', 'merge-base', 'HEAD', base], cwd=repo_root, - capture_output=True, text=True, check=True).stdout.strip() - diff = subprocess.run(['git', 'diff', '--name-only', f'{mb}..HEAD'], cwd=repo_root, - capture_output=True, text=True, check=True).stdout - return [l for l in diff.splitlines() if l.strip()] - - -def main(): - ap = argparse.ArgumentParser(description=__doc__) - g = ap.add_mutually_exclusive_group(required=True) - g.add_argument('--base', help='git ref to diff against (merge-base..HEAD)') - g.add_argument('--diff-file', help='newline-separated changed-file list') - ap.add_argument('configs', nargs='+', help='rig roster JSON file(s)') - a = ap.parse_args() - - repo_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - rosters = [] - for c in a.configs: - with open(c) as f: - rosters.append((c, json.load(f)['boards'])) - - files = (open(a.diff_file).read().splitlines() if a.diff_file - else changed_files_from_git(a.base, repo_root)) - files = [f for f in files if f.strip()] - - s = classify(files, repo_root, rosters) - s['args'] = selection_args(s, rosters) - for r in s['reasons']: - print(f'hil_select: {r}', file=sys.stderr) - print(json.dumps(s)) - - -if __name__ == '__main__': - main() -``` - -(The `parts.append f'...'` line above is pseudo-highlighted; write valid Python: -`parts.append(f'-bt {b["name"]}:{",".join(chosen)}')`.) - -- [ ] **Step 4: Run the full suite** - -Run: `python3 test/hil/test_hil_select.py && chmod +x test/hil/hil_select.py` -Expected: OK. - -- [ ] **Step 5: Smoke against the real repo state** - -Run: `python3 test/hil/hil_select.py --base HEAD test/hil/tinyusb.json test/hil/hfp.json` -Expected: empty diff ⇒ `{"full": false, "boards": {}, "args": {"tinyusb.json": "", "hfp.json": ""}, ...}` exit 0. -Then: `printf 'src/portable/wch/dcd_ch32_usbfs.c\n' > /tmp/d.txt && python3 test/hil/hil_select.py --diff-file /tmp/d.txt test/hil/tinyusb.json | python3 -m json.tool | head -20` -Expected: only WCH-family boards (nanoch32v203, ch32v103r_r1_1v0, ch32v307v_r1_1v0 — whichever reference that port) with device tests. - -- [ ] **Step 6: Commit** - -```bash -git add test/hil/hil_select.py test/hil/test_hil_select.py -git commit -m "hil: hil_select CLI with per-rig hil_test argument emission" -``` - ---- - -### Task 3: hil_ci_set_matrix --select + build.yml wiring - -**Files:** -- Modify: `test/hil/hil_ci_set_matrix.py` -- Modify: `.github/workflows/build.yml` (set-matrix job; hil-build consumers unchanged; hil-tinyusb + hil-tinyusb-esp steps) - -**Interfaces:** -- Consumes: Task 2's CLI JSON (`full`, `boards`, `args`). -- Produces: - - `hil_ci_set_matrix.py [--select JSON_STRING] CONFIG...`: with `--select` and - `full == false`, boards not in `select['boards']` are skipped when building the toolchain - buckets; otherwise identical behavior. Buckets stay present (possibly `[]`) so - `fromJSON(...)[toolchain]` keeps resolving. - - set-matrix outputs: `hil_select_json` (compact selection), `hil_args_tinyusb`, - `hil_args_hfp`, `hil_run_tinyusb`, `hil_run_hfp` (string 'true'/'false'). - -- [ ] **Step 1: Add `--select` to `hil_ci_set_matrix.py`** - -In `main()` add: - -```python - parser.add_argument('--select', help='hil_select.py JSON; scopes boards when full=false') -``` - -and after parsing: - -```python - selected = None - sel = json.loads(args.select) if args.select else None - if sel and not sel.get('full'): - selected = set(sel.get('boards', {})) -``` - -then inside the per-board loop, first line: - -```python - if selected is not None and board['name'] not in selected: - continue -``` - -- [ ] **Step 2: Verify byte-identical without --select and scoped with it** - -Run: `python3 test/hil/hil_ci_set_matrix.py test/hil/tinyusb.json test/hil/hfp.json > /tmp/m1.json && git stash -q && python3 test/hil/hil_ci_set_matrix.py test/hil/tinyusb.json test/hil/hfp.json > /tmp/m0.json && git stash pop -q && diff /tmp/m0.json /tmp/m1.json && echo identical` -Expected: `identical`. -Then: `python3 test/hil/hil_ci_set_matrix.py --select '{"full": false, "boards": {"raspberry_pi_pico": "all"}}' test/hil/tinyusb.json test/hil/hfp.json` -Expected: JSON whose `arm-gcc` list contains only the raspberry_pi_pico entry, `riscv-gcc`/`esp-idf` = []. - -- [ ] **Step 3: Wire set-matrix in `.github/workflows/build.yml`** - -In the `set-matrix` job: give the checkout full history and add the selection step between -checkout and matrix generation; make the HIL matrix use it: - -```yaml - - name: Checkout TinyUSB - uses: actions/checkout@v6 - with: - fetch-depth: 0 - - - name: HIL selection (PR only) - id: hil-select - if: github.event_name == 'pull_request' - run: | - python3 test/hil/test_hil_select.py - SELECT_JSON=$(python3 test/hil/hil_select.py --base "origin/${{ github.base_ref }}" test/hil/tinyusb.json test/hil/hfp.json) - echo "select=$SELECT_JSON" >> $GITHUB_OUTPUT - python3 - "$SELECT_JSON" >> $GITHUB_OUTPUT <<'EOF' - import json, sys - s = json.loads(sys.argv[1]) - args = s.get('args', {}) - for cfg, key in (('tinyusb.json', 'tinyusb'), ('hfp.json', 'hfp')): - a = args.get(cfg, '') - run = 'true' if (s['full'] or a) else 'false' - print(f'args_{key}={a}') - print(f'run_{key}={run}') - EOF -``` - -and in the existing "Generate matrix json" step, change the HIL line to: - -```yaml - # HIL matrix (merged from tinyusb + hifiphile configs), scoped on PRs - SELECT='${{ steps.hil-select.outputs.select }}' - HIL_MATRIX_JSON=$(python test/hil/hil_ci_set_matrix.py ${SELECT:+--select "$SELECT"} test/hil/tinyusb.json test/hil/hfp.json) -``` - -Add to the job's `outputs:` block: - -```yaml - hil_args_tinyusb: ${{ steps.hil-select.outputs.args_tinyusb }} - hil_args_hfp: ${{ steps.hil-select.outputs.args_hfp }} - hil_run_tinyusb: ${{ steps.hil-select.outputs.run_tinyusb }} - hil_run_hfp: ${{ steps.hil-select.outputs.run_hfp }} -``` - -(On non-PR events the step is skipped: outputs are empty strings — the consumers below treat -empty `run_*` as 'true' and empty args as no filtering, i.e. today's behavior.) - -- [ ] **Step 4: Wire the rig jobs** - -In the `hil-tinyusb` job (the matrixed one covering both rigs), find the step that runs -`hil_test.py --retry 1 ${{ matrix.test_args }} ${{ env.HIL_JSON }} $RERUN_ARGS` (~line 360) -and change the step's `run:` to select per-rig args and honor the skip flag: - -```yaml - run: | - case "$HIL_JSON" in - *tinyusb.json) SEL_ARGS='${{ needs.set-matrix.outputs.hil_args_tinyusb }}'; SEL_RUN='${{ needs.set-matrix.outputs.hil_run_tinyusb }}' ;; - *hfp.json) SEL_ARGS='${{ needs.set-matrix.outputs.hil_args_hfp }}'; SEL_RUN='${{ needs.set-matrix.outputs.hil_run_hfp }}' ;; - esac - if [ "$SEL_RUN" = "false" ]; then echo "HIL skipped by PR selection (no affected boards on this rig)"; exit 0; fi - python3 test/hil/hil_test.py --retry 1 ${{ matrix.test_args }} $SEL_ARGS ${{ env.HIL_JSON }} $RERUN_ARGS -``` - -Apply the same pattern to the second `hil_test.py` invocation at ~line 423 (`hil-tinyusb-esp`, -which is tinyusb-rig only: use the `hil_args_tinyusb`/`hil_run_tinyusb` outputs directly, no -case needed) and to the hfp job's direct `python3 test/hil/hil_test.py hfp.json` call at -~line 487 (use `hil_args_hfp`/`hil_run_hfp`). Preserve each step's existing surrounding lines -(report-dir env, RERUN_ARGS logic) — only inject the SEL_ARGS/SEL_RUN mechanics. - -- [ ] **Step 5: Validate the YAML and the exact shell locally** - -Run: `pre-commit run check-yaml --files .github/workflows/build.yml && python3 -c "import yaml; yaml.safe_load(open('.github/workflows/build.yml')); print('yaml ok')"` -Expected: `yaml ok` (pyyaml is available; if not, `pip install --user pyyaml` first). -Also simulate the selection step's python inline script: -`SELECT_JSON=$(python3 test/hil/hil_select.py --diff-file /tmp/d.txt test/hil/tinyusb.json test/hil/hfp.json) && python3 -c "import json,sys; s=json.loads(sys.argv[1]); print(s['args'])" "$SELECT_JSON"` -Expected: the args dict prints. - -- [ ] **Step 6: Commit** - -```bash -git add test/hil/hil_ci_set_matrix.py .github/workflows/build.yml -git commit -m "ci: scope HIL build+test matrix by PR diff via hil_select" -``` - ---- - -### Task 4: pre-pr + hil skill docs, final validation - -**Files:** -- Modify: `.claude/skills/pre-pr/SKILL.md` (mapping section delegates to the selector) -- Modify: `.claude/skills/hil/SKILL.md` (document the selector for manual runs) - -**Interfaces:** -- Consumes: Task 2's CLI. - -- [ ] **Step 1: Rewrite pre-pr's "2. Map changes to boards" section** - -Replace the section's grep heuristics (keep its numbered-section structure and the roster/cap -policy) with: - -```markdown -## 2. Map changes to boards - -- `python3 test/hil/hil_select.py --base $BASE test/hil/tinyusb.json` → JSON with the affected - rig boards (`boards`) and per-file `reasons`. `full: true` means a broad/infra change. -- Build-board sampling: from the selection's boards (or, when `full`, the representative set - `stm32f407disco` + `raspberry_pi_pico`), pick ONE board per family, preferring rig-roster - boards; cap at 4 and tell the user which families the cap dropped. The boards list must - NEVER end up empty — final fallback is `[stm32f407disco]`. -- A `full: true` selection or an empty one (docs-only) keeps today's behavior: minimal - software-only gate for docs-only, representative set otherwise. -``` - -- [ ] **Step 2: Add a short "PR-scoped selection" note to the hil skill** - -Append to `.claude/skills/hil/SKILL.md` after the pool-check section: - -```markdown -## PR-scoped selection - -`test/hil/hil_select.py` maps a diff to affected boards/tests (used by CI on PRs; fail-open -to the full matrix). Manual use: - -```bash -ARGS=$(python3 test/hil/hil_select.py --base master test/hil/tinyusb.json | python3 -c "import json,sys; print(json.load(sys.stdin)['args']['tinyusb.json'])") -python3 test/hil/hil_test.py -B examples $ARGS test/hil/tinyusb.json -``` - -Unit suite: `python3 test/hil/test_hil_select.py` (no hardware). -``` - -- [ ] **Step 3: Full validation sweep** - -Run: `python3 test/hil/test_hil_select.py && python3 -m py_compile test/hil/hil_select.py test/hil/hil_examples.py test/hil/hil_ci_set_matrix.py test/hil/hil_test.py && python3 test/hil/hil_test.py --help >/dev/null && pre-commit run --files $(git diff --name-only claude/hil-pool-check..HEAD) && echo ALL-GREEN` -Expected: `ALL-GREEN`. - -- [ ] **Step 4: Real-diff spot checks (acceptance)** - -Run each and eyeball the JSON (record outputs in your report): -```bash -for f in 'src/portable/raspberrypi/rp2040/dcd_rp2040.c' 'src/device/usbd.c' 'src/class/cdc/cdc_device.c' 'src/host/usbh.c'; do - printf '%s\n' "$f" > /tmp/d.txt - echo "=== $f"; python3 test/hil/hil_select.py --diff-file /tmp/d.txt test/hil/tinyusb.json test/hil/hfp.json 2>/dev/null | python3 -m json.tool | sed -n '1,25p' -done -``` -Expected: matches the spec's acceptance examples (pico-family only / all-device / CDC examples -only / host side only, with hfp.json args populated only where hfp boards qualify). - -- [ ] **Step 5: Commit** - -```bash -git add .claude/skills/pre-pr/SKILL.md .claude/skills/hil/SKILL.md -git commit -m "docs: pre-pr and hil skill use hil_select for PR-scoped boards" -```
diff --git a/docs/superpowers/plans/2026-08-19-ci-build-family-filter.md b/docs/superpowers/plans/2026-08-19-ci-build-family-filter.md deleted file mode 100644 index 0d8b9cf..0000000 --- a/docs/superpowers/plans/2026-08-19-ci-build-family-filter.md +++ /dev/null
@@ -1,1804 +0,0 @@ -# PR-Scoped CI Selection Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Promote `test/hil/helper/hil_select.py` to a repo-wide `tools/ci_select.py` whose one classification of a PR diff narrows three CI axes — build families, per-family example targets, and per-board HIL examples — wired into both GitHub Actions and CircleCI. - -**Architecture:** The selector gains an independent build classifier beside the untouched HIL one (17-rule table in the spec). `ci_set_matrix.py` filters the family matrix from the selector JSON; the per-family example map travels as a side channel (GHA job output / CircleCI pipeline parameter), resolved to `-e` flags per build job by a new `tools/build.py --example` filter. `hil_ci_set_matrix.py` appends `-e` per rig board. Code metrics gain per-example artifacts and a (family, example)-intersection compare. - -**Tech Stack:** Python 3 stdlib (selector must run on bare CI runners), GitHub Actions YAML, CircleCI dynamic config (continuation orb), jq, CMake/Ninja. - -**Spec:** `docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md` — read it first; every rule number below refers to its rule table. - -## Global Constraints - -- Commit messages: imperative mood, **no** `Co-Authored-By:` or `Claude-Session:` trailers (hathach is sole author — this overrides harness defaults). -- Never stage or touch `.idea/`. Always `git add` explicit paths, never `-A`. -- Bare-runner Python modules (`tools/ci_select.py`, `tools/build.py`, `tools/build_utils.py`, everything under `test/hil/helper/`) stay stdlib-only at module level — `test_hil_util.BottomLayer` enforces this; extend its lists, never work around them. -- `ci_select.py` stdout is machine-read JSON; every diagnostic goes to stderr. -- The family reference scan is **CMake-only** (`family.cmake` + espressif component `CMakeLists.txt`, never `family.mk`): CMake is the first-class build system, Make follows it. -- Fail-open everywhere: a selector/matrix-script failure must yield the full matrix, never a red job or a silently-empty one. -- Python style: match the existing modules (4-space indent in tools/ and test/hil/, terse targeted comments explaining *why*). -- YAML: 2-space indent, match surrounding style in `.github/workflows/` and `.circleci/`. -- Run suites from the repo root. Selector suite: `python3 test/hil/test/test_ci_select.py` (after Task 1). Full HIL-side suite: `python3 -m unittest discover -s test/hil/test`. - ---- - -### Task 1: Move the selector to `tools/ci_select.py` (mechanical, no behavior change) - -**Files:** -- Move: `test/hil/helper/hil_select.py` → `tools/ci_select.py` (git mv) -- Move: `test/hil/test/test_hil_select.py` → `test/hil/test/test_ci_select.py` (git mv) -- Modify: `test/hil/test/test_hil_util.py` (BottomLayer lists), `test/hil/hil_ci.sh` (scp list), `.pre-commit-config.yaml` (both hooks), `.github/workflows/build.yml` (4 path refs), `.claude/skills/pre-pr/SKILL.md`, `test/hil/helper/hil_util.py:21` (comment), `test/hil/hil_flash.py:297` (comment) - -**Interfaces:** -- Produces: module `tools/ci_select.py` importable as `ci_select` with `tools/` on `sys.path`; module attribute `_REPO_ROOT` (absolute repo root); CLI `python3 tools/ci_select.py --base REF|--diff-file F CONFIG.json...` — output JSON byte-compatible with today's `hil_select.py`. -- Consumes: `test/hil/helper/hil_util.py` rosters (unchanged). - -- [ ] **Step 1: git mv both files** - -```bash -git mv test/hil/helper/hil_select.py tools/ci_select.py -git mv test/hil/test/test_hil_select.py test/hil/test/test_ci_select.py -``` - -- [ ] **Step 2: Fix `tools/ci_select.py` imports and repo root** - -Replace the current path setup (line 24, `sys.path.insert(0, os.path.dirname(os.path.dirname(...)))` and its comment) with: - -```python -# tools/ -> repo root is ONE level up. Guarded by TestModuleMove.test_repo_root_guard: -# a wrong parent count here silently re-points every repo-relative glob (it happened -# at the helper/ move). -_REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) -sys.path.insert(0, os.path.join(_REPO_ROOT, 'test', 'hil')) # for `from helper...` -from helper.hil_util import device_tests, dual_tests, host_test -``` - -In `main()`, replace the 4-level `repo_root` derivation (lines 503-505) with `repo_root = _REPO_ROOT`. Change the stderr prefix at line 519 from `hil_select:` to `ci_select:`. Update the module docstring: it now lives in `tools/`, serves HIL and (from Task 3) build selection; keep the fail-open sentence and the spec pointer, adding this spec's path. - -- [ ] **Step 3: Fix `test/hil/test/test_ci_select.py` imports** - -Replace the header import block (`from helper import hil_select`) so `REPO` is computed first, then: - -```python -REPO = os.path.dirname(os.path.dirname(os.path.dirname( - os.path.dirname(os.path.abspath(__file__))))) -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # test/hil, for hil_flash/helper -sys.path.insert(0, os.path.join(REPO, 'tools')) -import hil_flash -import ci_select -from helper.hil_util import device_tests, dual_tests -``` - -Then `sed -i 's/\bhil_select\b/ci_select/g' test/hil/test/test_ci_select.py` and fix the header comment (file names, run command). Add the guard test: - -```python -class TestModuleMove(unittest.TestCase): - def test_repo_root_guard(self): - # __file__-derived root: moving the module without re-deriving the parent - # count re-points every scan at the wrong tree (it happened once already) - self.assertTrue(os.path.isdir(os.path.join(ci_select._REPO_ROOT, 'src'))) - self.assertTrue(os.path.isdir(os.path.join(ci_select._REPO_ROOT, 'hw', 'bsp'))) - self.assertEqual(os.path.realpath(ci_select._REPO_ROOT), os.path.realpath(REPO)) -``` - -- [ ] **Step 4: Update every reference** - -- `test/hil/test/test_hil_util.py` BottomLayer: in `test_bare_runner_modules_stay_stdlib_only`, replace `'hil_select'` with `'ci_select'` in the `local` set and replace `'helper/hil_select'` with `'../../tools/ci_select'` in the module-path tuple (the loop builds `hil_dir / f'{mod}.py'`, so a relative path out of test/hil works). Update the docstring sentence naming hil_select. -- `test/hil/hil_ci.sh`: delete the `"$ROOT_DIR/test/hil/helper/hil_select.py" \` scp line (nothing on the rig imports it). -- `.pre-commit-config.yaml`: rename hook `hil-select-test` → `ci-select-test`; `entry: python3 test/hil/test/test_ci_select.py`; `files: ^(hw/bsp/|src/|examples/|tools/ci_select\.py$)`. In the `hil-test` hook comment, s/test_hil_select/test_ci_select/. -- `.github/workflows/build.yml`: four call sites — lines ~82/84 (set-matrix) and ~632/637 (hil-hfp-iar): `test/hil/test/test_hil_select.py` → `test/hil/test/test_ci_select.py`, `test/hil/helper/hil_select.py` → `tools/ci_select.py`; s/hil_select/ci_select/ in the adjacent `::warning::` strings and comments (keep `hil_select.json` file names as `ci_select.json` for consistency — update both writers and both readers in the hfp-iar job). -- `.claude/skills/pre-pr/SKILL.md`: `python3 test/hil/helper/hil_select.py` → `python3 tools/ci_select.py`. -- Comments only: `test/hil/helper/hil_util.py:21` (hil_select → ci_select), `test/hil/hil_flash.py:297` (test_hil_select → test_ci_select). - -- [ ] **Step 5: Verify** - -```bash -python3 test/hil/test/test_ci_select.py # all pass -python3 -m unittest discover -s test/hil/test # all pass (~55 s) -python3 tools/ci_select.py --diff-file /dev/null test/hil/tinyusb.json | python3 -m json.tool >/dev/null -grep -rn "hil_select" --include='*.py' --include='*.yml' --include='*.yaml' --include='*.sh' --include='*.md' . | grep -v docs/superpowers | grep -v '\.worktrees' -``` - -Expected: suites green; last grep returns nothing (historical spec docs are the only allowed hits). - -- [ ] **Step 6: Commit** - -```bash -git add tools/ci_select.py test/hil/test/test_ci_select.py test/hil/test/test_hil_util.py \ - test/hil/hil_ci.sh .pre-commit-config.yaml .github/workflows/build.yml \ - .claude/skills/pre-pr/SKILL.md test/hil/helper/hil_util.py test/hil/hil_flash.py -git commit -m "tools: promote hil_select.py to tools/ci_select.py" -``` - ---- - -### Task 2: Generalize the family scan and re-rule `hw/mcu/**` (HIL side) - -**Files:** -- Modify: `tools/ci_select.py` (`port_families` → `path_families` + `mcu_families`, `_FULL_RE`, `_classify_one`) -- Test: `test/hil/test/test_ci_select.py` - -**Interfaces:** -- Produces: `path_families(rel_dir: str, repo_root: str) -> set[str]` — families whose `family.cmake`/espressif component CMakeLists reference `rel_dir` at a directory boundary; `mcu_families(path: str, repo_root: str) -> set[str]` — longest-resolving-prefix lookup for a changed `hw/mcu/...` path; `port_families(port_dir, repo_root)` kept as a thin wrapper (existing callers/tests unchanged). -- HIL JSON change: `hw/mcu/**` no longer forces `full: true`; it selects the resolved families' boards, all their tests (spec rule 7). - -- [ ] **Step 1: Write the failing tests** (append to `test_ci_select.py`) - -```python -class TestPathFamilies(unittest.TestCase): - def test_port_wrapper_unchanged(self): - self.assertEqual(ci_select.port_families('raspberrypi/rp2040', REPO), {'rp2040'}) - self.assertIn('stm32f4', ci_select.port_families('synopsys/dwc2', REPO)) - - def test_boundary_without_trailing_slash(self): - # hw/bsp/nrf/family.cmake writes `${TOP}/hw/mcu/nordic/nrfx` — no trailing - # slash; the match must accept a directory-boundary end-of-token - self.assertEqual(ci_select.path_families('hw/mcu/nordic/nrfx', REPO), {'nrf'}) - - def test_boundary_rejects_prefix_sibling(self): - # 'microchip/pic' must not inherit pic32mz's references (and pic32mz itself - # is family.mk-only, which the CMake-only scan never reads) - self.assertEqual(ci_select.port_families('microchip/pic', REPO), set()) - self.assertEqual(ci_select.port_families('microchip/pic32mz', REPO), set()) - - def test_mcu_families_prefix_walk(self): - self.assertEqual(ci_select.mcu_families('hw/mcu/nordic/nrf5x/nrf_clock.h', REPO), {'nrf'}) - self.assertEqual(ci_select.mcu_families('hw/mcu/dialog/da1469x/x.h', REPO), {'da1469x'}) - self.assertEqual(ci_select.mcu_families('hw/mcu/no_such_vendor/x.c', REPO), set()) - - -class TestMcuHilRule(unittest.TestCase): - def test_mcu_no_longer_forces_full(self): - s = ci_select.classify(['hw/mcu/nordic/nrf5x/nrf_clock.h'], REPO, ROSTERS) - self.assertFalse(s['full']) - self.assertIn('nrf', s['families']) # recorded even with no nrf rig board - - def test_mcu_selects_family_boards(self): - got = on_roster(self, 'feather_nrf52840_express', 'pca10056', 'pca10095') - s = ci_select.classify(['hw/mcu/nordic/nrf5x/nrf_clock.h'], REPO, real_rosters()) - self.assertFalse(s['full']) - for b in got: - self.assertIn(b, s['boards']) - - -class TestOrphanInvariant(unittest.TestCase): - ALLOW = {'microchip/pic', 'microchip/pic32mz'} # spec: known orphans, CMake builds neither - - def test_every_port_resolves_to_a_family(self): - for d in sorted(glob.glob(os.path.join(REPO, 'src/portable/*/*'))): - if not os.path.isdir(d): - continue - port = os.path.relpath(d, os.path.join(REPO, 'src/portable')).replace(os.sep, '/') - fams = ci_select.port_families(port, REPO) - if port in self.ALLOW: - self.assertEqual(fams, set(), f'{port}: no longer an orphan - drop it from ALLOW') - else: - self.assertTrue(fams, f'{port}: no family.cmake references it - wire it up or allowlist it') - - def test_tracked_mcu_vendors_resolve(self): - import subprocess as sp - r = sp.run(['git', 'ls-files', 'hw/mcu'], cwd=REPO, capture_output=True, text=True) - if r.returncode != 0: - self.skipTest('not a git checkout') - vendors = sorted({'/'.join(p.split('/')[:3]) for p in r.stdout.split()}) - for v in vendors: - self.assertTrue(ci_select.mcu_families(v + '/x.c', REPO), f'{v}: resolves to no family') -``` - -- [ ] **Step 2: Run to verify failure** - -Run: `python3 test/hil/test/test_ci_select.py TestPathFamilies -v` -Expected: FAIL/ERROR — `path_families`/`mcu_families` not defined. - -- [ ] **Step 3: Implement** - -In `tools/ci_select.py`, replace `port_families` with: - -```python -@functools.lru_cache(maxsize=None) -def path_families(rel_dir: str, repo_root: str) -> set: - """Board families whose family.cmake (or espressif component CMakeLists) - references rel_dir at a directory boundary. CMake only, on every axis: CMake - is the first-class build system and Make follows it, so family.mk is never - read - a port wired up in family.mk alone (microchip/pic32mz) is built by no - CI job and resolves to nothing. Boundary = '/', whitespace, quote, paren, - brace or end: `${TOP}/hw/mcu/nordic/nrfx` has no trailing slash, while bare - 'microchip/pic' must not match '.../microchip/pic32mz/...'.""" - fams = set() - bsp_root = os.path.join(repo_root, 'hw/bsp') - pat = re.compile(re.escape(rel_dir) + r'(?=[/\s"\')}]|$)', re.M) - for f in glob.glob(os.path.join(bsp_root, '*/family.cmake')) + \ - glob.glob(os.path.join(bsp_root, '*/components/*/CMakeLists.txt')): - try: - if pat.search(open(f).read()): - fams.add(os.path.relpath(f, bsp_root).split(os.sep, 1)[0]) - except OSError: - pass - return fams - - -def port_families(port_dir: str, repo_root: str) -> set: - return path_families('src/portable/' + port_dir, repo_root) - - -def mcu_families(path: str, repo_root: str) -> set: - """Families referencing a changed hw/mcu path: longest resolving dir prefix, - hw/mcu/<vendor>/<sub>/... down to hw/mcu/<vendor>.""" - parts = path.split('/') - for n in range(len(parts) - 1, 2, -1): - fams = path_families('/'.join(parts[:n]), repo_root) - if fams: - return fams - return set() -``` - -Keep the old docstring's CMake-only rationale for HIL (folded into the new one). Remove `hw/mcu/|` from `_FULL_RE`. In `_classify_one`, insert after the `hw/bsp/` block, before the `examples/` block: - -```python - if re.match(r'hw/mcu/', path): - fams = mcu_families(path, repo_root) - s.families.update(fams) - boards = [b['name'] for b in roster_boards - if board_family(b['name'], repo_root) in fams] - s.roles.update(('device', 'host')) - s.add(boards, 'all', f'{path}: mcu dir -> families {sorted(fams)} -> boards {boards}') - return -``` - -- [ ] **Step 4: Run tests** - -Run: `python3 test/hil/test/test_ci_select.py -v 2>&1 | tail -5` -Expected: all pass (the pre-existing port tests exercise the wrapper). - -- [ ] **Step 5: Commit** - -```bash -git add tools/ci_select.py test/hil/test/test_ci_select.py -git commit -m "ci_select: generalize family scan to hw/mcu, drop hw/mcu from HIL full-matrix rule" -``` - ---- - -### Task 3: Build classifier — rules 1-17, raw two-axis selection - -**Files:** -- Modify: `tools/ci_select.py` -- Test: `test/hil/test/test_ci_select.py` - -**Interfaces:** -- Produces: `classify_build(changed_files, repo_root) -> dict` with keys `full: bool`, `families: [str]` (sorted bsp-dir names), `family_examples: {family: [example]}` (key absent ⇒ that family builds all examples; examples as `role/name`), `reasons: [str]`. Also `all_examples(repo_root) -> tuple[str]`, `role_examples(repo_root, roles) -> set[str]`, `all_bsp_families(repo_root) -> list[str]`. Buildability pruning is Task 4 — this task emits the raw rule output. -- Consumes: `path_families`, `mcu_families`, `class_macros`, `class_include_edges`, `_config_enables`, `_NONCODE_RE` (all existing). - -- [ ] **Step 1: Write the failing tests** - -```python -class TestBuildClassifier(unittest.TestCase): - def b(self, files): - return ci_select.classify_build(files, REPO) - - def test_noncode_and_test_hil_contribute_nothing(self): # rules 1, 2 - s = self.b(['docs/info/index.rst', 'README.rst', 'test/hil/hil_test.py', '.claude/skills/hil/SKILL.md']) - self.assertFalse(s['full']) - self.assertEqual(s['families'], []) - self.assertEqual(s['family_examples'], {}) - - def test_port_device_rule(self): # rule 3 - s = self.b(['src/portable/raspberrypi/rp2040/dcd_rp2040.c']) - self.assertFalse(s['full']) - self.assertEqual(s['families'], ['rp2040']) - exs = s['family_examples']['rp2040'] - self.assertIn('device/cdc_msc', exs) - self.assertFalse(any(e.startswith(('host/', 'typec/')) for e in exs)) - # dual inclusion asserted on the pure role helper: whether a dual example - # survives Task 4's buildability pruning depends on the environment-gated - # CI board pick, so the classifier-output assertion must not rely on it - self.assertIn('dual/host_info_to_device_cdc', - ci_select.role_examples(REPO, ('device', 'dual'))) - self.assertNotIn('host/bare_api', ci_select.role_examples(REPO, ('device', 'dual'))) - - def test_port_host_rule(self): # rule 4 - s = self.b(['src/portable/analog/max3421/hcd_max3421.c']) - self.assertFalse(s['full']) - # max3421 is referenced only by the espressif component CMakeLists — and - # espressif is in no provider's family list, so this may prune to nothing - self.assertLessEqual(set(s['families']), {'espressif'}) - for exs in s['family_examples'].values(): - self.assertFalse(any(e.startswith(('device/', 'typec/')) for e in exs)) - - def test_port_shared_file_selects_all_examples(self): # rule 5 - s = self.b(['src/portable/synopsys/dwc2/dwc2_common.c']) - self.assertFalse(s['full']) - self.assertIn('stm32f4', s['families']) - self.assertNotIn('rp2040', s['families']) - self.assertNotIn('stm32f4', s['family_examples']) # 'all' => no map key - - def test_bsp_family_rule(self): # rule 6 - s = self.b(['hw/bsp/stm32f4/boards/stm32f407disco/board.h']) - self.assertEqual(s['families'], ['stm32f4']) - self.assertNotIn('stm32f4', s['family_examples']) - - def test_bsp_top_level_file_is_full(self): # rule 16 - self.assertTrue(self.b(['hw/bsp/board.c'])['full']) - self.assertTrue(self.b(['hw/bsp/family_support.cmake'])['full']) - - def test_mcu_rule(self): # rule 7 - s = self.b(['hw/mcu/nordic/nrf5x/nrf_clock.h']) - self.assertEqual(s['families'], ['nrf']) - s = self.b(['hw/mcu/no_such_vendor/x.c']) # empty means empty - self.assertFalse(s['full']) - self.assertEqual(s['families'], []) - - def test_class_device_rule(self): # rule 8 - s = self.b(['src/class/cdc/cdc_device.c']) - self.assertFalse(s['full']) - # near-all families (Task 4's pruning may drop a few); never equality - # against all_bsp_families — that's a tuple, and pruning shrinks the list - self.assertIn('stm32f4', s['families']) - self.assertGreater(len(s['families']), 50) - exs = s['family_examples']['stm32f4'] - self.assertIn('device/cdc_msc', exs) - self.assertNotIn('device/hid_composite', exs) - self.assertNotIn('host/cdc_msc_hid', exs) # TUH_CDC examples are rule 9's - - def test_class_host_rule(self): # rule 9 - s = self.b(['src/class/msc/msc_host.c']) - exs = s['family_examples']['stm32f4'] - self.assertIn('host/msc_file_explorer', exs) - self.assertNotIn('device/cdc_msc', exs) - - def test_class_shared_header_and_include_edge(self): # rule 10 - s = self.b(['src/class/audio/audio.h']) - exs = s['family_examples']['stm32f4'] - self.assertIn('device/audio_test', exs) - self.assertIn('device/midi_test', exs) # midi headers include audio.h - - def test_core_device_rule(self): # rule 11 - s = self.b(['src/device/usbd.c']) - exs = s['family_examples']['stm32f4'] - self.assertIn('device/cdc_msc', exs) - # no dual In-assertion: dual examples are only.txt-gated to max3421/pio-usb - # boards, so pruning legitimately drops them on a plain stm32f4 board - self.assertFalse(any(e.startswith(('host/', 'typec/')) for e in exs)) - - def test_core_host_rule(self): # rule 12 - s = self.b(['src/host/usbh.c']) - exs = s['family_examples']['stm32f4'] - self.assertFalse(any(e.startswith(('device/', 'typec/')) for e in exs)) - - def test_example_rule(self): # rules 13, 14 - s = self.b(['examples/device/cdc_msc/src/main.c']) - self.assertEqual(s['family_examples']['stm32f4'], ['device/cdc_msc']) - s = self.b(['examples/device/board_test/src/main.c']) - self.assertEqual(s['family_examples']['stm32f4'], ['device/board_test']) - s = self.b(['examples/device/no_such_example/src/main.c']) # deleted example: nothing - self.assertFalse(s['full']) - self.assertEqual(s['families'], []) - - def test_full_paths(self): # rules 15-17 - for p in ('src/common/tusb_fifo.c', 'src/osal/osal.h', 'src/tusb.c', - 'src/tusb_option.h', 'lib/SEGGER_RTT/RTT/SEGGER_RTT.c', - 'tools/build.py', 'tools/get_deps.py', 'tools/cmake/cpu/cortex-m4.cmake', - 'examples/CMakeLists.txt', 'examples/device/CMakeLists.txt', - 'examples/build_system/cmake/cpu.cmake', '.github/workflows/build.yml', - 'sonar-project.properties', 'some/unknown/path.c'): - self.assertTrue(self.b([p])['full'], p) - - def test_mixed_diff_unions_per_family(self): - s = self.b(['src/portable/raspberrypi/rp2040/dcd_rp2040.c', 'src/class/cdc/cdc_device.c']) - self.assertFalse(s['full']) - self.assertIn('stm32f4', s['families']) - self.assertGreater(len(s['families']), 50) - self.assertIn('device/hid_composite', s['family_examples']['rp2040']) # from the dcd rule - self.assertNotIn('device/hid_composite', s['family_examples']['stm32f4']) # cdc-only there - - def test_example_names_are_real_dirs(self): - for ex in ci_select.all_examples(REPO): - role, name = ex.split('/') - self.assertTrue(os.path.isdir(os.path.join(REPO, 'examples', role, name)), ex) - self.assertRegex(ex, r'^(device|dual|host|typec)/[A-Za-z0-9_]+$') -``` - -Note for `test_mixed_diff_unions_per_family`: it encodes the per-family union — rp2040 gets DEV+DUAL ∪ cdc-set, every other family only the cdc-set (spec §Two axes). Buildability pruning may later remove entries; these Task-3 tests use families/examples that survive pruning (stm32f4 and rp2040 build all the named examples), so they stay valid after Task 4. - -- [ ] **Step 2: Run to verify failure** - -Run: `python3 test/hil/test/test_ci_select.py TestBuildClassifier -v 2>&1 | tail -3` -Expected: ERROR — `classify_build` not defined. - -- [ ] **Step 3: Implement** (append to `tools/ci_select.py`, after the HIL classifier) - -```python -# ------------------------------------------------------------- -# Build-axis classifier (spec rule table, docs/superpowers/specs/ -# 2026-08-19-ci-build-family-filter-design.md). Independent of the HIL -# classifier: same diff, second walk, its own fail-open. -# ------------------------------------------------------------- -_EX_ROLES = ('device', 'dual', 'host', 'typec') - - -@functools.lru_cache(maxsize=None) -def all_examples(repo_root: str) -> tuple: - """Every examples/<role>/<name> with a CMakeLists.txt, as 'role/name'.""" - out = [] - for role in _EX_ROLES: - for d in sorted(glob.glob(os.path.join(repo_root, 'examples', role, '*/'))): - if os.path.isfile(os.path.join(d, 'CMakeLists.txt')): - out.append(f'{role}/{os.path.basename(d.rstrip(os.sep))}') - return tuple(out) - - -def role_examples(repo_root: str, roles) -> set: - want = set(roles) - return {e for e in all_examples(repo_root) if e.split('/', 1)[0] in want} - - -@functools.lru_cache(maxsize=None) -def all_bsp_families(repo_root: str) -> tuple: - return tuple(sorted(d for d in os.listdir(os.path.join(repo_root, 'hw/bsp')) - if os.path.isdir(os.path.join(repo_root, 'hw/bsp', d)))) - - -def _build_class_examples(cls: str, base: str, roles: set, repo_root: str) -> set: - """Examples (all 46, not the HIL lists) whose tusb_config.h enables the class's - macros for the given roles, plus classes that #include the changed header.""" - via = sorted(class_include_edges(repo_root).get(f'{cls}/{base}', ())) - out = set() - for prefix, role in (('TUD', 'device'), ('TUH', 'host')): - if role not in roles: - continue - macros = class_macros(cls, base, prefix) + \ - [m for c in via for m in class_macros(c, '', prefix)] - for ex in all_examples(repo_root): - cfg = os.path.join(repo_root, 'examples', ex, 'src', 'tusb_config.h') - if _config_enables(cfg, macros): - out.add(ex) - return out - - -class _BSel: - """family -> set(examples) | 'all', unioned per family.""" - def __init__(self): - self.full = False - self.fam_ex = {} - self.reasons = [] - - def add(self, fams, examples, reason): - self.reasons.append(reason) - for f in fams: - cur = self.fam_ex.get(f) - if examples == 'all' or cur == 'all': - self.fam_ex[f] = 'all' - else: - self.fam_ex[f] = (cur or set()) | set(examples) - - def force_full(self, reason): - self.full = True - self.reasons.append(reason) - - -def _classify_build_one(path, repo_root, s: _BSel): - base = os.path.basename(path) - if _NONCODE_RE.match(path): # rule 1 - return - if re.match(r'test/hil/', path): # rule 2 - s.reasons.append(f'{path}: HIL harness, no build contribution') - return - m = re.match(r'src/portable/((?:[^/]+/)?[^/]+)/', path) - if m: # rules 3-5 - port = m.group(1) - fams = port_families(port, repo_root) - if re.match(r'(dcd_|.*_device)', base): - exs = role_examples(repo_root, ('device', 'dual')) - elif re.match(r'(hcd_|.*_host)', base): - exs = role_examples(repo_root, ('host', 'dual')) - else: - exs = 'all' - s.add(fams, exs, f'{path}: port {port} -> families {sorted(fams)}') - return - if re.match(r'hw/bsp/[^/]+/', path): # rule 6 - fam = path.split('/')[2] - s.add({fam}, 'all', f'{path}: bsp family {fam}') - return - if re.match(r'hw/mcu/', path): # rule 7 - fams = mcu_families(path, repo_root) - s.add(fams, 'all', f'{path}: mcu -> families {sorted(fams)}') - return - m = re.match(r'src/class/([^/]+)/', path) - if m: # rules 8-10 - cls = m.group(1) - if re.search(r'_device\.[ch]$', base): - roles = {'device'} - elif re.search(r'_host\.[ch]$', base): - roles = {'host'} - else: - roles = {'device', 'host'} - exs = _build_class_examples(cls, base, roles, repo_root) - s.add(all_bsp_families(repo_root), exs, - f'{path}: class {cls} -> {sorted(exs)}') - return - m = re.match(r'src/(device|host)/', path) - if m: # rules 11-12 - role = m.group(1) - s.add(all_bsp_families(repo_root), role_examples(repo_root, (role, 'dual')), - f'{path}: core {role} stack') - return - m = re.match(r'examples/(device|dual|host|typec)/([^/]+)/', path) - if m: # rules 13-14 - ex = f'{m.group(1)}/{m.group(2)}' - if ex in all_examples(repo_root): - s.add(all_bsp_families(repo_root), {ex}, f'{path}: example {ex}') - else: - # a deleted example builds nothing; removing it from the role - # CMakeLists (rule 15) is what forces the full matrix - s.reasons.append(f'{path}: not an example dir, no build contribution') - return - s.force_full(f'{path}: unclassified -> full build matrix') # rules 15-17 - - -def classify_build(changed_files, repo_root): - s = _BSel() - for p in changed_files: - _classify_build_one(p, repo_root, s) - if s.full: - return {'full': True, 'families': list(all_bsp_families(repo_root)), - 'family_examples': {}, 'reasons': s.reasons} - fams, fam_ex = [], {} - for fam, exs in sorted(s.fam_ex.items()): - fams.append(fam) - if exs != 'all': - fam_ex[fam] = sorted(exs) - return {'full': False, 'families': fams, 'family_examples': fam_ex, - 'reasons': s.reasons} -``` - -Note: `examples/<role>/CMakeLists.txt` has no trailing slash after the second component, so the example regex misses it and it correctly falls through to `force_full` (rule 15) — `test_full_paths` pins this. - -- [ ] **Step 4: Run tests** - -Run: `python3 test/hil/test/test_ci_select.py TestBuildClassifier -v` -Expected: all pass. Then the full file: `python3 test/hil/test/test_ci_select.py 2>&1 | tail -3` — all pass. - -- [ ] **Step 5: Commit** - -```bash -git add tools/ci_select.py test/hil/test/test_ci_select.py -git commit -m "ci_select: add build-axis classifier (families x example targets)" -``` - ---- - -### Task 4: Buildability post-filter, `build` + `hil_examples` output keys - -**Files:** -- Modify: `tools/ci_select.py` (imports, post-filter, `main()`), `test/hil/test/test_hil_util.py` (BottomLayer lists) -- Test: `test/hil/test/test_ci_select.py` - -**Interfaces:** -- Produces: `classify_build` result is now pruned: each family's list intersected with what that family's CI board can build (`build_utils.skip_example`); family dropped when nothing survives; map key omitted when the kept set equals everything the board can build. `hil_examples(sel, rosters) -> {board: [example]}` — the board's selected tests (`sel['boards'][name]` when narrowed, else `board_tests`) plus always `device/board_test`. CLI JSON gains top-level `"build": {...}` (always) and `"hil_examples": {...}` (when rosters given; emitted even when `full` is true). -- Consumes: `tools/build_utils.skip_example(example, board)`; `tools/build.py:get_family_boards(family, one_random, one_first)` (module import — no behavior change to build.py yet). - -- [ ] **Step 1: Write the failing tests** - -```python -class TestBuildPostFilter(unittest.TestCase): - def test_kept_examples_are_buildable(self): - import build_utils, build as build_py - s = ci_select.classify_build(['src/class/msc/msc_host.c'], REPO) - self.assertFalse(s['full']) - # families that cannot build a single TUH_MSC example drop out entirely - self.assertNotIn('msp430', s['families']) - old = os.getcwd() - os.chdir(REPO) - try: - for fam, exs in s['family_examples'].items(): - board = build_py.get_family_boards(fam, False, True)[0] - for e in exs: - self.assertFalse(build_utils.skip_example(e, board), f'{fam}: {e}') - finally: - os.chdir(old) - - def test_unfiltered_family_has_no_map_key(self): - s = ci_select.classify_build(['hw/bsp/stm32f4/family.c'], REPO) - self.assertEqual(s['families'], ['stm32f4']) - self.assertEqual(s['family_examples'], {}) - - def test_no_stdout_pollution(self): - # get_family_boards prints on odd families; the selector's stdout is JSON - import io, contextlib - buf = io.StringIO() - with contextlib.redirect_stdout(buf): - ci_select.classify_build(['src/class/msc/msc_host.c'], REPO) - self.assertEqual(buf.getvalue(), '') - - -class TestHilExamples(unittest.TestCase): - def test_board_test_always_present_and_full_emits(self): - s = ci_select.classify(['src/common/tusb_fifo.c'], REPO, ROSTERS) # full - he = ci_select.hil_examples(s, ROSTERS) - self.assertEqual(set(he), {b['name'] for b in ROSTER}) - for name, exs in he.items(): - self.assertIn('device/board_test', exs) - - def test_narrowed_board_gets_chosen_tests_only(self): - s = ci_select.classify(['examples/device/cdc_msc/src/main.c'], REPO, ROSTERS) - he = ci_select.hil_examples(s, ROSTERS) - self.assertEqual(he['stm32f407disco'], ['device/board_test', 'device/cdc_msc']) - - def test_full_board_gets_its_whole_test_list(self): - s = ci_select.classify(['hw/bsp/stm32f4/boards/stm32f407disco/board.h'], REPO, ROSTERS) - he = ci_select.hil_examples(s, ROSTERS) - want = set(ci_select.board_tests(ROSTER[1])) | {'device/board_test'} - self.assertEqual(set(he['stm32f407disco']), want) - self.assertNotIn('raspberry_pi_pico', he) # deselected board: no firmware needed - - -class TestCliJson(unittest.TestCase): - def test_build_key_without_rosters(self): - r = subprocess.run([sys.executable, os.path.join(REPO, 'tools/ci_select.py'), - '--diff-file', '/dev/null'], capture_output=True, text=True) - self.assertEqual(r.returncode, 0, r.stderr) - j = json.loads(r.stdout) - self.assertIn('build', j) - self.assertNotIn('hil_examples', j) # rosters not given - - def test_build_and_hil_keys_with_rosters(self): - import tempfile - with tempfile.NamedTemporaryFile('w', suffix='.txt', delete=False) as f: - f.write('src/portable/raspberrypi/rp2040/dcd_rp2040.c\n') - df = f.name - r = subprocess.run([sys.executable, os.path.join(REPO, 'tools/ci_select.py'), - '--diff-file', df, os.path.join(REPO, 'test/hil/tinyusb.json')], - capture_output=True, text=True) - os.unlink(df) - j = json.loads(r.stdout) - self.assertEqual(j['build']['families'], ['rp2040']) - self.assertIn('hil_examples', j) - for exs in j['hil_examples'].values(): - self.assertIn('device/board_test', exs) -``` - -(`subprocess`, `sys` are already imported in the test file.) - -- [ ] **Step 2: Run to verify failure** - -Run: `python3 test/hil/test/test_ci_select.py TestBuildPostFilter TestHilExamples TestCliJson -v 2>&1 | tail -3` -Expected: FAIL — no pruning, no `hil_examples`, no `build` key. - -- [ ] **Step 3: Implement** - -In `tools/ci_select.py` module header, after the existing `helper` import, add: - -```python -import contextlib -import io - -sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) # tools/, for build helpers -import build_utils -import build as build_py -``` - -(`contextlib`/`io` go into the stdlib import block at the top.) Add the pruning helpers and rewrite the tail of `classify_build`: - -```python -@contextlib.contextmanager -def _in_repo(repo_root): - """build_utils/build.py use repo-relative paths; scope a chdir around them. - get_family_boards also prints on an empty family - swallow stdout so the - selector's machine-read JSON stays clean (diagnostics belong on stderr).""" - old = os.getcwd() - os.chdir(repo_root) - try: - with contextlib.redirect_stdout(io.StringIO()): - yield - finally: - os.chdir(old) - - -def _prune_buildable(fams, fam_ex, repo_root): - """Intersect each family's selection with what its CI board can build - (build_utils.skip_example - the same skip.txt/only.txt data CMake's - family_filter reads). get_family_boards mirrors the build jobs' one-first - pick, CI preferred/skip lists included.""" - out_fams, out_ex = [], {} - allex = list(all_examples(repo_root)) - with _in_repo(repo_root): - for fam in fams: - boards = build_py.get_family_boards(fam, False, True) - if not boards: - out_fams.append(fam) # unknown layout: keep unfiltered - continue - board = boards[0] - buildable = [e for e in allex if not build_utils.skip_example(e, board)] - want = fam_ex.get(fam) - kept = buildable if want is None else [e for e in want if e in set(buildable)] - if not kept: - continue # this diff builds nothing for this family - out_fams.append(fam) - if set(kept) != set(buildable): - out_ex[fam] = kept - return out_fams, out_ex -``` - -Replace `classify_build`'s non-full return with: - -```python - fams = sorted(s.fam_ex) - fam_ex = {f: sorted(e) for f, e in s.fam_ex.items() if e != 'all'} - fams, fam_ex = _prune_buildable(fams, fam_ex, repo_root) - return {'full': False, 'families': fams, 'family_examples': fam_ex, - 'reasons': s.reasons} -``` - -Add `hil_examples` beside `selection_args`: - -```python -def hil_examples(sel, rosters): - """{board: examples hil-build must produce}: the board's selected tests plus - device/board_test, which hil_test.py flashes to park at every variant - boundary and at end-of-board teardown. Emitted for full selections too - the - HIL example universe is a fraction of the tree regardless of the diff.""" - by_name = {} - for _, boards in rosters: - for b in boards: - by_name.setdefault(b['name'], b) - if sel['full']: - chosen = {n: 'all' for n in by_name} - else: - chosen = sel['boards'] - out = {} - for name, tests in chosen.items(): - run = board_tests(by_name[name]) if tests == 'all' else list(tests) - out[name] = sorted(set(run) | {'device/board_test'}) - return out -``` - -In `main()`: change the configs argument to optional — `ap.add_argument('configs', nargs='*', help='rig roster JSON file(s); omit for the build view alone')` — so CircleCI (which never touches HIL) can run without rosters; with no configs, `rosters` is `[]`, the HIL keys degrade to empty, and `hil_examples` is omitted. Then after the `args_flasher` line: - -```python - if rosters: - s['hil_examples'] = hil_examples(s, rosters) - s['build'] = classify_build(files, repo_root) - for r in s['build']['reasons']: - print(f'ci_select[build]: {r}', file=sys.stderr) -``` - -Update `test/hil/test/test_hil_util.py` BottomLayer: add `'build'`, `'build_utils'` to the `local` allowed set and `'../../tools/build'`, `'../../tools/build_utils'` to the module-path tuple (ci_select now imports both on the bare runner). - -- [ ] **Step 4: Run tests + timing check** - -```bash -python3 test/hil/test/test_ci_select.py 2>&1 | tail -3 -python3 -m unittest discover -s test/hil/test 2>&1 | tail -3 -time python3 tools/ci_select.py --diff-file <(echo src/class/cdc/cdc_device.c) test/hil/tinyusb.json >/dev/null -``` - -Expected: suites pass; the timed run stays under ~5 s (skip_example over 75 families × 46 examples re-reads small files — if it exceeds that, memoize `skip_example` results per (example, board) inside `_prune_buildable`). - -- [ ] **Step 5: Commit** - -```bash -git add tools/ci_select.py test/hil/test/test_ci_select.py test/hil/test/test_hil_util.py -git commit -m "ci_select: prune build selection by example buildability, emit build + hil_examples keys" -``` - ---- - -### Task 5: `ci_set_matrix.py --select / --base` - -**Files:** -- Modify: `.github/scripts/ci_set_matrix.py` -- Test: `test/hil/test/test_ci_select.py` - -**Interfaces:** -- Produces: CLI `python .github/scripts/ci_set_matrix.py [--select JSON | --base REF]`. No flags → byte-identical to today's output. `--select`: families intersected with `select.build.families` unless `build.full`; unusable JSON → full matrix + stderr warning. `--base REF`: runs `tools/ci_select.py --base REF` itself and proceeds as `--select`. Output shape `{toolchain: [family]}` unchanged. - -- [ ] **Step 1: Write the failing tests** - -```python -SET_MATRIX = os.path.join(REPO, '.github/scripts/ci_set_matrix.py') - -class TestCiSetMatrix(unittest.TestCase): - def run_matrix(self, *args): - return subprocess.run([sys.executable, SET_MATRIX, *args], - capture_output=True, text=True) - - def test_no_flags_is_todays_output(self): - r = self.run_matrix() - self.assertEqual(r.returncode, 0, r.stderr) - self.baseline = json.loads(r.stdout) - self.assertIn('stm32f4', self.baseline['arm-gcc']) - - def test_select_full_is_identical(self): - base = json.loads(self.run_matrix().stdout) - sel = json.dumps({'build': {'full': True, 'families': [], 'family_examples': {}}}) - self.assertEqual(json.loads(self.run_matrix('--select', sel).stdout), base) - - def test_select_narrow_is_a_subset(self): - sel = json.dumps({'build': {'full': False, 'families': ['rp2040', 'stm32f4'], - 'family_examples': {}}}) - m = json.loads(self.run_matrix('--select', sel).stdout) - self.assertEqual(m['arm-gcc'], ['rp2040', 'stm32f4']) - self.assertEqual(m['riscv-gcc'], []) - self.assertEqual(set(m), set(json.loads(self.run_matrix().stdout))) # all keys kept - - def test_malformed_select_falls_open(self): - base = json.loads(self.run_matrix().stdout) - r = self.run_matrix('--select', 'not json {') - self.assertEqual(r.returncode, 0) - self.assertEqual(json.loads(r.stdout), base) - self.assertIn('full matrix', r.stderr) -``` - -- [ ] **Step 2: Run to verify failure** - -Run: `python3 test/hil/test/test_ci_select.py TestCiSetMatrix -v 2>&1 | tail -3` -Expected: FAIL — argparse rejects `--select`. - -- [ ] **Step 3: Implement** - -In `.github/scripts/ci_set_matrix.py`, add imports `argparse, os, subprocess, sys` and replace `set_matrix_json` + the main guard: - -```python -def set_matrix_json(select=None): - sel_fams = None - if select: - b = select.get('build') or {} - if b.get('full') is False: - sel_fams = set(b.get('families') or []) - matrix = {} - for toolchain in toolchain_list: - fams = [family for family, tc in family_list.items() if toolchain in tc] - if sel_fams is not None: - fams = [f for f in fams if f in sel_fams] - matrix[toolchain] = fams - print(json.dumps(matrix)) - - -def main(): - parser = argparse.ArgumentParser() - group = parser.add_mutually_exclusive_group() - group.add_argument('--select', help='tools/ci_select.py JSON; scopes families when build.full is false') - group.add_argument('--base', help='git ref: run tools/ci_select.py --base REF and scope from it') - args = parser.parse_args() - - select = None - try: - if args.select: - select = json.loads(args.select) - elif args.base: - root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - r = subprocess.run([sys.executable, os.path.join(root, 'tools', 'ci_select.py'), - '--base', args.base], - capture_output=True, text=True, cwd=root, check=True) - select = json.loads(r.stdout) - except Exception as e: # fail-open: an unusable selection must never turn into a red job - print(f'ci_set_matrix: selection unusable ({e}) - full matrix', file=sys.stderr) - select = None - set_matrix_json(select) - - -if __name__ == '__main__': - main() -``` - -- [ ] **Step 4: Run tests** - -Run: `python3 test/hil/test/test_ci_select.py TestCiSetMatrix -v` — all pass. -Also: `python3 .github/scripts/ci_set_matrix.py | diff - <(git show HEAD:.github/scripts/ci_set_matrix.py | python3 -)` → no diff (byte-identical default output). - -- [ ] **Step 5: Extend the pre-commit hook scope and commit** - -In `.pre-commit-config.yaml`, `ci-select-test` hook: `files: ^(hw/bsp/|src/|examples/|tools/(ci_select|build|build_utils)\.py$|\.github/scripts/)`. - -```bash -git add .github/scripts/ci_set_matrix.py test/hil/test/test_ci_select.py .pre-commit-config.yaml -git commit -m "ci_set_matrix: scope the family matrix from a ci_select selection" -``` - ---- - -### Task 6: `hil_ci_set_matrix.py` emits `-e` per board - -**Files:** -- Modify: `.github/scripts/hil_ci_set_matrix.py` -- Test: `test/hil/test/test_ci_select.py` - -**Interfaces:** -- Produces: each build entry for board `B` gains ` -e <ex>` for every entry of `select.hil_examples[B]` (before variant expansion, so all of a board's variants carry the same list). No `hil_examples` key (hand runs, old selectors) → output byte-identical to today. -- Consumed by: `hil-build` / `hil-build-esp` (via `build_util.yml` → `tools/build.py`), `hil-hfp-iar`'s inline build loop — all funnel into `tools/build.py`, which learns `-e` in Task 7. - -- [ ] **Step 1: Write the failing tests** - -```python -HIL_SET_MATRIX = os.path.join(REPO, '.github/scripts/hil_ci_set_matrix.py') - -class TestHilCiSetMatrixExamples(unittest.TestCase): - def run_matrix(self, *args): - r = subprocess.run([sys.executable, HIL_SET_MATRIX, *args, - os.path.join(REPO, 'test/hil/tinyusb.json')], - capture_output=True, text=True) - self.assertEqual(r.returncode, 0, r.stderr) - return r.stdout - - def test_no_hil_examples_is_byte_identical(self): - plain = self.run_matrix() - sel = json.dumps({'full': True, 'boards': {}}) - self.assertEqual(self.run_matrix('--select', sel), plain) - - def test_examples_appended_per_board(self): - board = on_roster(self, 'stm32f407disco')[0] - sel = json.dumps({'full': False, 'boards': {board: 'all'}, - 'hil_examples': {board: ['device/board_test', 'device/cdc_msc']}}) - m = json.loads(self.run_matrix('--select', sel)) - entries = [e for entries in m.values() for e in entries] - self.assertTrue(entries) - for e in entries: - self.assertIn(f'-b {board}', e) - self.assertIn('-e device/board_test', e) - self.assertIn('-e device/cdc_msc', e) -``` - -- [ ] **Step 2: Run to verify failure** - -Run: `python3 test/hil/test/test_ci_select.py TestHilCiSetMatrixExamples -v` -Expected: `test_examples_appended_per_board` FAILS (no `-e` in entries). - -- [ ] **Step 3: Implement** - -In `hil_ci_set_matrix.py` `main()`, after the `selected` computation add `ex_map = (sel or {}).get('hil_examples', {})`, and in the board loop, after the `build.args` append (line ~72): - -```python - # PR selection: build only the examples this board will run (its test - # list plus device/board_test, the parking firmware) - tools/build.py -e. - # Absent key (hand runs, full non-PR builds) keeps --target all. - for ex in ex_map.get(name, []): - build_board += f' -e {ex}' -``` - -- [ ] **Step 4: Run tests** - -Run: `python3 test/hil/test/test_ci_select.py -v 2>&1 | tail -3` — all pass. - -- [ ] **Step 5: Commit** - -```bash -git add .github/scripts/hil_ci_set_matrix.py test/hil/test/test_ci_select.py -git commit -m "hil_ci_set_matrix: append per-board -e example filters from the selection" -``` - ---- - -### Task 7: `tools/build.py --example` - -**Files:** -- Modify: `tools/build.py` -- Test: `test/hil/test/test_ci_select.py` - -**Interfaces:** -- Produces: repeatable `-e/--example role/name`. Without it, behavior is exactly today's (`--target all`). With it: cmake builds one `--target <name>` per requested example the board can build (`build_utils.skip_example`), mapping `all` → example names and `examples-membrowse-upload` → `<name>-membrowse-upload` (the aggregate target `DEPENDS` every example — `hw/bsp/family_support.cmake:346-360` — and would rebuild the excluded ones); `tinyusb_metrics` and other targets pass through, order preserved. A board whose intersection is empty reports **skipped**. Make and espressif paths filter their example lists the same way. New helper `resolve_example_targets(build_targets, examples, board) -> list | None` (None = nothing buildable). - -- [ ] **Step 1: Write the failing tests** - -```python -class TestBuildPyExampleFilter(unittest.TestCase): - def setUp(self): - import build as build_py - self.build = build_py - self.old = os.getcwd() - os.chdir(REPO) # skip_example uses repo-relative paths - - def tearDown(self): - os.chdir(self.old) - - def test_all_maps_to_example_names(self): - t = self.build.resolve_example_targets(['all'], ['device/cdc_msc', 'device/dfu'], - 'stm32f407disco') - self.assertEqual(t, ['cdc_msc', 'dfu']) - - def test_membrowse_maps_per_example(self): - t = self.build.resolve_example_targets(['all', 'examples-membrowse-upload'], - ['device/cdc_msc'], 'stm32f407disco') - self.assertEqual(t, ['cdc_msc', 'cdc_msc-membrowse-upload']) - - def test_other_targets_pass_through_in_order(self): - t = self.build.resolve_example_targets(['all', 'tinyusb_metrics'], - ['device/cdc_msc'], 'stm32f407disco') - self.assertEqual(t, ['cdc_msc', 'tinyusb_metrics']) - - def test_unbuildable_examples_drop_and_empty_is_none(self): - # typec/power_delivery only builds on stm32g4-class parts, never on f4 - t = self.build.resolve_example_targets(['all'], - ['typec/power_delivery', 'device/cdc_msc'], - 'stm32f407disco') - self.assertEqual(t, ['cdc_msc']) - self.assertIsNone(self.build.resolve_example_targets(['all'], - ['typec/power_delivery'], - 'stm32f407disco')) -``` - -- [ ] **Step 2: Run to verify failure** - -Run: `python3 test/hil/test/test_ci_select.py TestBuildPyExampleFilter -v` -Expected: ERROR — `resolve_example_targets` not defined. - -- [ ] **Step 3: Implement** - -In `tools/build.py` add near `get_examples`: - -```python -def resolve_example_targets(build_targets, examples, board): - """Map generic targets onto per-example targets for a filtered build (-e). - 'all' -> the example executables; 'examples-membrowse-upload' -> per-example - upload targets (the aggregate DEPENDS on every example and would rebuild the - excluded ones); anything else (e.g. tinyusb_metrics) passes through. - Returns None when no requested example is buildable on this board.""" - buildable = [e for e in examples if not build_utils.skip_example(e, board)] - if not buildable: - return None - names = [e.split('/', 1)[1] for e in buildable] - out = [] - for t in build_targets: - if t == 'all': - out += names - elif t == 'examples-membrowse-upload': - out += [f'{n}-membrowse-upload' for n in names] - else: - out.append(t) - return list(dict.fromkeys(out)) -``` - -Thread `examples` (a list or `None`) through `main()` → `build_boards_list` → `cmake_board`/`make_board`: - -- `main()`: `parser.add_argument('-e', '--example', action='append', default=[], help='Only build these examples (role/name, repeatable). Default: all examples')`; pass `args.example or None` as a new final parameter of `build_boards_list`. -- `build_boards_list(..., examples=None)`: forward to both branches. -- `cmake_board(..., examples=None)`: in the espressif branch, after `all_examples = get_examples(family)` insert: - -```python - if examples is not None: - all_examples = [e for e in all_examples if e in examples] -``` - - In the generic branch, replace the target loop: - -```python - if rcmd.returncode == 0: - targets = build_targets - if examples is not None: - targets = resolve_example_targets(build_targets, examples, board) - if targets is None: - print_build_result(board, 'examples (PR filter)', 2, '-') - return [0, 0, 1] - cmd = ["cmake", "--build", build_dir, '--parallel', str(parallel_jobs)] - for target in targets: - rcmd = run_cmd(cmd + ['--target', target]) - if rcmd.returncode != 0: - break -``` - -- `make_board(..., examples=None)`: after `all_examples = get_examples(family)`: - -```python - if examples is not None: - all_examples = [e for e in all_examples if e in examples] - if not all_examples: - print_build_result(board, 'examples (PR filter)', 2, '-') - return [0, 0, 1] -``` - -- [ ] **Step 4: Run tests + a real filtered build** - -```bash -python3 test/hil/test/test_ci_select.py TestBuildPyExampleFilter -v -python3 tools/build.py -e device/cdc_msc -e device/cdc_dual_ports -b stm32f407disco -ls cmake-build/cmake-build-stm32f407disco/device/cdc_msc/cdc_msc.elf \ - cmake-build/cmake-build-stm32f407disco/device/cdc_dual_ports/cdc_dual_ports.elf -python3 tools/build.py -e typec/power_delivery -b stm32f407disco # expect: Skipped row, exit 0 -``` - -Expected: tests pass; both elfs exist; the typec run prints a Skipped result and exits 0. - -- [ ] **Step 5: Commit** - -```bash -git add tools/build.py test/hil/test/test_ci_select.py -git commit -m "build.py: add -e/--example filter with per-example target mapping" -``` - ---- - -### Task 8: `metrics.py --by-example` + by-example expansion + CMake wiring - -**Files:** -- Modify: `tools/metrics.py`, `examples/CMakeLists.txt`, `.pre-commit-config.yaml` -- Create + Test: `test/hil/test/test_ci_metrics.py` - -**Interfaces:** -- Produces: `metrics.py combine --by-example` additionally writes `<out>_by_example.json` = `{"<role>/<example>": {"files": [...]}}`, the example id taken from the map.json's two parent dirs (`<build>/<role>/<example>/*.map.json`). `combine` also accepts a by-example JSON as *input*, expanding each example to one data entry, with `--only-examples a,b` filtering which. `combine_files(input_files, filters=None, only_examples=None)`. Existing outputs byte-identical when the new flags are absent. -- Consumed by: `examples/CMakeLists.txt` `tinyusb_metrics` target (adds the flag), Task 9's pair-compare, Task 10's artifact upload. - -- [ ] **Step 1: Write the failing tests** (new file `test/hil/test/test_ci_metrics.py`) - -```python -#!/usr/bin/env python3 -# SPDX-License-Identifier: MIT -# Unit tests for the by-example half of tools/metrics.py and the (family, example) -# pair-compare script. Stdlib only; synthetic map.json fixtures, no builds. -# python3 test/hil/test/test_ci_metrics.py -import json -import os -import subprocess -import sys -import tempfile -import unittest - -REPO = os.path.dirname(os.path.dirname(os.path.dirname( - os.path.dirname(os.path.abspath(__file__))))) -METRICS = os.path.join(REPO, 'tools', 'metrics.py') - - -def fake_map(path, files): - os.makedirs(os.path.dirname(path), exist_ok=True) - with open(path, 'w') as f: - json.dump({'files': files}, f) - - -def entry(name, size, path_prefix='tinyusb/src'): - return {'file': name, 'path': f'{path_prefix}/{name}', 'size': size, - 'symbols': [{'name': f'{name}_fn', 'size': size}], 'sections': {'.text': size}} - - -class TestByExample(unittest.TestCase): - def build_tree(self, td): - fake_map(os.path.join(td, 'device', 'cdc_msc', 'cdc_msc.map.json'), - [entry('usbd.c', 100), entry('cdc_device.c', 50)]) - fake_map(os.path.join(td, 'host', 'bare_api', 'bare_api.map.json'), - [entry('usbh.c', 200)]) - - def test_by_example_output(self): - with tempfile.TemporaryDirectory() as td: - self.build_tree(td) - out = os.path.join(td, 'metrics') - r = subprocess.run([sys.executable, METRICS, 'combine', '-q', '-j', - '--by-example', '-o', out, - os.path.join(td, '*', '*', '*.map.json')], - capture_output=True, text=True) - self.assertEqual(r.returncode, 0, r.stderr) - by_ex = json.load(open(out + '_by_example.json')) - self.assertEqual(set(by_ex), {'device/cdc_msc', 'host/bare_api'}) - self.assertEqual({f['file'] for f in by_ex['device/cdc_msc']['files']}, - {'usbd.c', 'cdc_device.c'}) - # the plain averaged output is unchanged by the extra flag - avg = json.load(open(out + '.json')) - self.assertIn('files', avg) - - def test_by_example_json_roundtrips_as_combine_input(self): - with tempfile.TemporaryDirectory() as td: - self.build_tree(td) - out = os.path.join(td, 'metrics') - subprocess.run([sys.executable, METRICS, 'combine', '-q', '-j', '--by-example', - '-o', out, os.path.join(td, '*', '*', '*.map.json')], check=True) - out2 = os.path.join(td, 'sub') - r = subprocess.run([sys.executable, METRICS, 'combine', '-q', '-j', - '--only-examples', 'device/cdc_msc', - '-o', out2, out + '_by_example.json'], - capture_output=True, text=True) - self.assertEqual(r.returncode, 0, r.stderr) - sub = json.load(open(out2 + '.json')) - names = {f['file'] for f in sub['files']} - self.assertEqual(names, {'usbd.c', 'cdc_device.c'}) # bare_api filtered out - - -if __name__ == '__main__': - unittest.main() -``` - -- [ ] **Step 2: Run to verify failure** - -Run: `python3 test/hil/test/test_ci_metrics.py -v` -Expected: FAIL — argparse rejects `--by-example`. - -- [ ] **Step 3: Implement in `tools/metrics.py`** - -`combine_files` signature → `combine_files(input_files, filters=None, only_examples=None)`. Inside the `.json` branch, after `json.load`, insert the by-example expansion before the filter logic: - -```python - if 'files' not in json_data and json_data and \ - all(isinstance(v, dict) and 'files' in v for v in json_data.values()): - # a metrics_by_example.json: one data entry per example - for ex in sorted(json_data): - if only_examples and ex not in only_examples: - continue - sub = {'files': list(json_data[ex]['files'])} - if filters: - sub['files'] = [f for f in sub['files'] - if f.get('path') and any(x in f['path'] for x in filters)] - all_json_data['file_list'].append(f'{fin}:{ex}') - all_json_data['data'].append(sub) - continue -``` - -Add a writer near `write_json_output`: - -```python -def write_by_example(input_files, filters, path): - """{<role>/<example>: {files: [...]}} from map.json inputs laid out as - <build>/<role>/<example>/<name>.map.json (examples/CMakeLists.txt's pattern).""" - out = {} - for fin in input_files: - d = os.path.dirname(os.path.abspath(fin)) - ex = f'{os.path.basename(os.path.dirname(d))}/{os.path.basename(d)}' - data = combine_files([fin], filters) - if data['data']: - out.setdefault(ex, {'files': []})['files'] += data['data'][0].get('files', []) - with open(path, 'w', encoding='utf-8') as f: - json.dump(out, f) -``` - -`cmd_combine`: pass `only_examples=set(args.only_examples.split(',')) if args.only_examples else None` into `combine_files`, and after the existing outputs: - -```python - if args.by_example: - write_by_example(input_files, args.filters, args.out + '_by_example.json') -``` - -Argparse additions on the combine subparser: - -```python - combine_parser.add_argument('--by-example', dest='by_example', action='store_true', - help='Also write <out>_by_example.json: per-example file lists keyed by role/example') - combine_parser.add_argument('--only-examples', dest='only_examples', default='', - help='Comma-separated role/example ids to keep when reading by-example JSON inputs') -``` - -- [ ] **Step 4: Wire CMake + hooks** - -`examples/CMakeLists.txt` `tinyusb_metrics` target: change the command to -`combine -f tinyusb/src -j --by-example -o ${CMAKE_BINARY_DIR}/metrics` (one added flag). -`.pre-commit-config.yaml` `hil-test` hook: `files: ^(test/hil/|examples/device/mtp/src/|tools/metrics\.py$|\.github/scripts/metrics_pair_compare\.py$)`. - -- [ ] **Step 5: Run tests** - -```bash -python3 test/hil/test/test_ci_metrics.py -v # pass -python3 -m unittest discover -s test/hil/test 2>&1 | tail -3 # discovery picks the new file up -``` - -- [ ] **Step 6: Commit** - -```bash -git add tools/metrics.py examples/CMakeLists.txt test/hil/test/test_ci_metrics.py .pre-commit-config.yaml -git commit -m "metrics: emit and consume per-example size data (--by-example, --only-examples)" -``` - ---- - -### Task 9: `(family, example)`-intersection compare script - -**Files:** -- Create: `.github/scripts/metrics_pair_compare.py` -- Test: `test/hil/test/test_ci_metrics.py` - -**Interfaces:** -- Produces: CLI `metrics_pair_compare.py --base-dir D1 --new-dir D2 [--out metrics_compare]`. Each dir is searched recursively for `cmake-build-<board>/metrics_by_example.json`; board → family via `hw/bsp/*/boards/<board>`. Writes `<out>.md`: the standard compare table over the intersection of `(family, example)` pairs, then a scope footer naming the compared families and any pairs missing on one side. Empty intersection → an explanatory one-line `.md`, exit 0. -- Consumes: `tools/metrics.py` internals `combine_files`/`compute_avg`-backed `compare_files` and `write_compare_markdown` (via `sys.path` import). - -- [ ] **Step 1: Write the failing tests** (append to `test_ci_metrics.py`) - -```python -PAIR_COMPARE = os.path.join(REPO, '.github/scripts/metrics_pair_compare.py') - - -def fake_by_example(root, board, data): - d = os.path.join(root, f'cmake-build-{board}') - os.makedirs(d, exist_ok=True) - with open(os.path.join(d, 'metrics_by_example.json'), 'w') as f: - json.dump(data, f) - - -class TestPairCompare(unittest.TestCase): - def test_intersection_compare(self): - with tempfile.TemporaryDirectory() as td: - base, new = os.path.join(td, 'base'), os.path.join(td, 'new') - # real board names so board->family resolution works against hw/bsp - fake_by_example(base, 'raspberry_pi_pico', - {'device/cdc_msc': {'files': [entry('usbd.c', 100)]}, - 'device/dfu': {'files': [entry('dfu_device.c', 10)]}}) - fake_by_example(new, 'raspberry_pi_pico', - {'device/cdc_msc': {'files': [entry('usbd.c', 120)]}}) - out = os.path.join(td, 'cmp') - r = subprocess.run([sys.executable, PAIR_COMPARE, '--base-dir', base, - '--new-dir', new, '--out', out], - capture_output=True, text=True) - self.assertEqual(r.returncode, 0, r.stderr) - md = open(out + '.md').read() - self.assertIn('usbd.c', md) - self.assertNotIn('dfu_device.c', md) # not on both sides - self.assertIn('rp2040', md) # scope footer - self.assertIn('device/dfu', md) # named as dropped - - def test_empty_intersection_writes_note(self): - with tempfile.TemporaryDirectory() as td: - base, new = os.path.join(td, 'base'), os.path.join(td, 'new') - fake_by_example(base, 'raspberry_pi_pico', {'device/dfu': {'files': [entry('a.c', 1)]}}) - fake_by_example(new, 'stm32f407disco', {'device/cdc_msc': {'files': [entry('b.c', 1)]}}) - out = os.path.join(td, 'cmp') - r = subprocess.run([sys.executable, PAIR_COMPARE, '--base-dir', base, - '--new-dir', new, '--out', out], - capture_output=True, text=True) - self.assertEqual(r.returncode, 0, r.stderr) - self.assertIn('skipped', open(out + '.md').read()) -``` - -- [ ] **Step 2: Run to verify failure** - -Run: `python3 test/hil/test/test_ci_metrics.py TestPairCompare -v` -Expected: FAIL — script does not exist. - -- [ ] **Step 3: Implement `.github/scripts/metrics_pair_compare.py`** - -```python -#!/usr/bin/env python3 -"""Family+example-matched code-size compare for PR-scoped builds. - -The averaged metrics baseline (metrics-tinyusb) spans every family and example; -a scoped PR builds a subset, so comparing against it is apples-to-oranges. This -compares the intersection of (family, example) pairs present on BOTH sides, -averaged over exactly those pairs, and names what was dropped. See -docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md #code-metrics. -""" -import argparse -import glob -import json -import os -import sys -import tempfile - -sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', '..', 'tools')) -import metrics - - -def board_family(board, repo_root): - hits = glob.glob(os.path.join(repo_root, 'hw/bsp/*/boards', board)) - return os.path.basename(os.path.dirname(os.path.dirname(hits[0]))) if hits else None - - -def collect(root, repo_root): - """{(family, 'role/example'): [file entries]} from every - **/cmake-build-<board>/metrics_by_example.json under root.""" - pairs = {} - pat = os.path.join(root, '**', 'metrics_by_example.json') - for f in sorted(glob.glob(pat, recursive=True)): - board = os.path.basename(os.path.dirname(f)) - if not board.startswith('cmake-build-'): - continue - fam = board_family(board[len('cmake-build-'):], repo_root) - if not fam: - print(f'pair_compare: no family for {board}, skipping', file=sys.stderr) - continue - try: - data = json.load(open(f)) - except (OSError, ValueError) as e: - print(f'pair_compare: unreadable {f} ({e}), skipping', file=sys.stderr) - continue - for ex, ent in data.items(): - pairs.setdefault((fam, ex), []).extend(ent.get('files', [])) - return pairs - - -def main(): - ap = argparse.ArgumentParser(description=__doc__) - ap.add_argument('--base-dir', required=True) - ap.add_argument('--new-dir', required=True) - ap.add_argument('--out', default='metrics_compare') - a = ap.parse_args() - repo_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - - base = collect(a.base_dir, repo_root) - new = collect(a.new_dir, repo_root) - common = sorted(set(base) & set(new)) - dropped = sorted(set(base) ^ set(new)) - - if not common: - with open(a.out + '.md', 'w') as f: - f.write('_Code-size comparison skipped: no (family, example) pair was built ' - 'on both the base branch and this PR._\n') - return - - def synth(pairs, path): - with open(path, 'w') as f: - json.dump({'files': [e for k in common for e in pairs[k]]}, f) - - with tempfile.TemporaryDirectory() as td: - b, n = os.path.join(td, 'base.json'), os.path.join(td, 'new.json') - synth(base, b) - synth(new, n) - comparison = metrics.compare_files(b, n, ['tinyusb/src']) - if comparison is None: - with open(a.out + '.md', 'w') as f: - f.write('_Code-size comparison failed to produce data._\n') - return - metrics.write_compare_markdown(comparison, a.out + '.md', 'name+') - - with open(a.out + '.md', 'a') as f: - fams = sorted({k[0] for k in common}) - f.write(f'\n_Scoped compare: {len(common)} (family, example) pairs across ' - f'{", ".join(fams)}._\n') - if dropped: - f.write('_Not compared (missing on one side): ' - + ', '.join(f'{fam}:{ex}' for fam, ex in dropped) + '._\n') - - -if __name__ == '__main__': - main() -``` - -- [ ] **Step 4: Run tests** - -Run: `python3 test/hil/test/test_ci_metrics.py -v` — all pass. - -- [ ] **Step 5: Commit** - -```bash -git add .github/scripts/metrics_pair_compare.py test/hil/test/test_ci_metrics.py -git commit -m "ci: add (family, example)-intersection code-size compare for scoped PRs" -``` - ---- - -### Task 10: GitHub Actions wiring (`build.yml` + `build_util.yml`) - -**Files:** -- Modify: `.github/workflows/build.yml`, `.github/workflows/build_util.yml` - -**Interfaces:** -- `set-matrix` new outputs: `example_map` (JSON `{family: [example]}`), `build_filtered` (`'true'`/`'false'`), `build_families_regex` (`fam1|fam2`, only when filtered). -- `build_util.yml` new input `example-map` (string, default `''`); when set, each leg resolves `-e` flags for its `matrix.arg` family and appends them (via env `$EX_ARGS`) to the Build and Membrowse invocations; metrics upload also grabs `metrics_by_example.json`. -- `code-metrics` gains `needs: set-matrix` and a scoped-baseline path. - -- [ ] **Step 1: Rename + thread the selection in `set-matrix`** - -Rename the step `HIL selection (PR only)` → `CI selection (PR only)` (id stays `hil-select`; renaming the id would touch every `steps.hil-select` reference — leave it). In the **Generate matrix json** step, replace the first three lines of the script (`MATRIX_JSON=$(python .github/scripts/ci_set_matrix.py)` and the two echo lines) with: - -```bash - # Build matrix, scoped by the PR selection when one exists. Best-effort: - # ci_set_matrix falls back to the full matrix itself on unusable JSON, - # and an empty $SELECT (non-PR event, selector fallback) means no flags. - if [ -n "$SELECT" ]; then - MATRIX_JSON=$(python .github/scripts/ci_set_matrix.py --select "$SELECT") || MATRIX_JSON='' - else - MATRIX_JSON='' - fi - [ -z "$MATRIX_JSON" ] && MATRIX_JSON=$(python .github/scripts/ci_set_matrix.py) - echo "matrix=$MATRIX_JSON" - echo "matrix=$MATRIX_JSON" >> $GITHUB_OUTPUT - - # Build-axis extras: the per-family example map rides as a side channel - # (a value inside matrix entries would break CircleCI's family parameter - # and multiply GHA matrix legs). NOTE jq's // treats false like null, so - # .build.full is compared explicitly. - EXAMPLE_MAP=$(printf '%s' "${SELECT:-null}" | jq -c '.build.family_examples // {}') || EXAMPLE_MAP='{}' - BUILD_FILTERED=$(printf '%s' "${SELECT:-null}" | jq -r 'if (.build? | type) == "object" and .build.full == false then "true" else "false" end') || BUILD_FILTERED='false' - FAM_REGEX='' - if [ "$BUILD_FILTERED" = "true" ]; then - FAM_REGEX=$(printf '%s' "$SELECT" | jq -r '.build.families | join("|")') || FAM_REGEX='' - [ -z "$FAM_REGEX" ] && BUILD_FILTERED='false' - fi - echo "example_map=$EXAMPLE_MAP" >> $GITHUB_OUTPUT - echo "build_filtered=$BUILD_FILTERED" >> $GITHUB_OUTPUT - echo "build_families_regex=$FAM_REGEX" >> $GITHUB_OUTPUT -``` - -Add to the `set-matrix` job `outputs:` block: - -```yaml - example_map: ${{ steps.set-matrix-json.outputs.example_map }} - build_filtered: ${{ steps.set-matrix-json.outputs.build_filtered }} - build_families_regex: ${{ steps.set-matrix-json.outputs.build_families_regex }} -``` - -- [ ] **Step 2: `build_util.yml` — example-map input** - -Add the input: - -```yaml - example-map: - required: false - default: '' - type: string -``` - -Insert between **Get Dependencies** and **Build**: - -```yaml - - name: Resolve PR example filter - if: inputs.example-map != '' && inputs.example-map != '{}' - env: - # values are PR-derived - keep them out of ${{ }} script interpolation - # (env expansion word-splits but never re-parses shell metacharacters) - EXAMPLE_MAP: ${{ inputs.example-map }} - FAMILY: ${{ matrix.arg }} - run: | - # -e flags for this family; a family absent from the map builds everything - EX_ARGS=$(printf '%s' "$EXAMPLE_MAP" | jq -r --arg fam "$FAMILY" '(.[$fam] // []) | map("-e " + .) | join(" ")') || EX_ARGS='' - echo "EX_ARGS=$EX_ARGS" - echo "EX_ARGS=$EX_ARGS" >> $GITHUB_ENV -``` - -Append `$EX_ARGS` to all three `tools/build.py` invocations (the esp-idf docker line, the generic Build line, and the Membrowse line — build.py maps `examples-membrowse-upload` per example when `-e` is active, because the aggregate target rebuilds everything). Extend the metrics upload: - -```yaml - path: | - cmake-build/cmake-build-*/metrics.json - cmake-build/cmake-build-*/metrics_by_example.json -``` - -- [ ] **Step 3: `cmake` job passes the map** - -In the `cmake` job's `with:` block add `example-map: ${{ needs.set-matrix.outputs.example_map }}`. Do **not** add it to `hil-build`/`hil-build-esp`/`build-os` — hil legs carry `-e` inside their matrix entries; build-os keeps the full example set. - -- [ ] **Step 4: `code-metrics` scoped baseline** - -Verify the download action supports regexp names: -`curl -fsSL https://raw.githubusercontent.com/dawidd6/action-download-artifact/v11/action.yml | grep -n name_is_regexp` — expect a hit. (Fallback if absent: replace the download step below with a `gh run download`-based loop over `build_families_regex` split on `|`, using `gh api` to find the newest master run per artifact; keep the same directory layout.) - -Change `needs: [ check-paths, cmake ]` → `needs: [ check-paths, cmake, set-matrix ]`. Guard the two unscoped steps with the filtered flag: on **Download Base Branch Metrics** change the `if:` to - -```yaml - if: (github.event_name == 'pull_request' || github.event_name == 'workflow_dispatch') && needs.set-matrix.outputs.build_filtered != 'true' -``` - -and on **Compare with Base Branch** change `if: github.event_name != 'push'` to - -```yaml - if: github.event_name != 'push' && needs.set-matrix.outputs.build_filtered != 'true' -``` - -Insert after **Download Base Branch Metrics**: - -```yaml - - name: Download base per-family metrics (scoped PR) - if: github.event_name == 'pull_request' && needs.set-matrix.outputs.build_filtered == 'true' - uses: dawidd6/action-download-artifact@v11 - with: - workflow: build.yml - workflow_conclusion: '' - search_artifacts: true # a docs-only master push uploads no per-family artifacts - branch: ${{ github.base_ref }} - name: ^metrics-(${{ needs.set-matrix.outputs.build_families_regex }})$ - name_is_regexp: true - path: base-family-metrics - continue-on-error: true - - - name: Compare with Base Branch (scoped) - if: github.event_name == 'pull_request' && needs.set-matrix.outputs.build_filtered == 'true' - run: | - # never fall back to the averaged metrics-tinyusb here: a scoped PR vs the - # 64-family/46-example average is exactly the mismatch this path prevents - python .github/scripts/metrics_pair_compare.py \ - --base-dir base-family-metrics --new-dir cmake-build --out metrics_compare - cat metrics_compare.md -``` - -(The PR-side `cmake-build/` dir already holds this run's `metrics_by_example.json` files from the artifact download at the top of the job.) - -- [ ] **Step 5: Validate and commit** - -```bash -python3 -c "import yaml,sys; yaml.safe_load(open('.github/workflows/build.yml')); yaml.safe_load(open('.github/workflows/build_util.yml')); print('yaml ok')" -command -v actionlint >/dev/null && actionlint .github/workflows/build.yml .github/workflows/build_util.yml || true -git add .github/workflows/build.yml .github/workflows/build_util.yml -git commit -m "ci: scope the GHA build matrix and code-metrics baseline by PR selection" -``` - ---- - -### Task 11: CircleCI wiring - -**Files:** -- Modify: `.circleci/config.yml`, `.circleci/config2.yml` - -**Interfaces:** -- `config.yml` set-matrix: on PRs, runs the selector (gated on its own unit suite), scopes `MATRIX_JSON` via `--select`, skips empty toolchains, and forwards `example-map` + `build-filtered` to the continued workflow as pipeline parameters. -- `config2.yml`: declares those parameters; the `build` command resolves `-e` flags per family; `code-metrics` compare is bypassed with a note when filtered; a `no-op` job keeps the workflow valid when nothing is selected. - -- [ ] **Step 1: Verify the continuation orb accepts parameters** - -`curl -fsSL "https://circleci.com/developer/orbs/orb/circleci/continuation" | grep -io 'parameters' | head -1` — the `continuation/continue` command takes a `parameters` input (inline JSON or a file path). If the page is unreachable, proceed — the orb has carried this input since 0.2; the fallback is `parameters: '{"example-map": ...}'` inline via an env-composed string. - -- [ ] **Step 2: `config.yml` — selector + scoping + parameters** - -In the `Set matrix` run command, replace the first two lines (`MATRIX_JSON=$(python .github/scripts/ci_set_matrix.py)` and its echo) with: - -```bash - # PR-scoped selection (best-effort: any failure falls back to the full - # matrix). CircleCI has no base-branch var; tinyusb PRs target master. - SELECT_JSON='' - if [ -n "${CIRCLE_PULL_REQUEST:-}" ]; then - git fetch --no-tags origin master || true - if python3 test/hil/test/test_ci_select.py >/dev/null 2>&1; then - SELECT_JSON=$(python3 tools/ci_select.py --base origin/master) || SELECT_JSON='' - else - echo "ci_select unit suite failed - using the full matrix" - fi - fi - MATRIX_JSON='' - if [ -n "$SELECT_JSON" ]; then - MATRIX_JSON=$(python .github/scripts/ci_set_matrix.py --select "$SELECT_JSON") || MATRIX_JSON='' - fi - [ -z "$MATRIX_JSON" ] && MATRIX_JSON=$(python .github/scripts/ci_set_matrix.py) - echo "MATRIX_JSON=$MATRIX_JSON" - - EXAMPLE_MAP=$(printf '%s' "${SELECT_JSON:-null}" | jq -c '.build.family_examples // {}') || EXAMPLE_MAP='{}' - BUILD_FILTERED=$(printf '%s' "${SELECT_JSON:-null}" | jq -r 'if (.build? | type) == "object" and .build.full == false then "true" else "false" end') || BUILD_FILTERED='false' - jq -n --arg map "$EXAMPLE_MAP" --arg filt "$BUILD_FILTERED" \ - '{"example-map": $map, "build-filtered": $filt}' > /tmp/continue_params.json -``` - -In the toolchain loop, after `FAMILY=$(echo $MATRIX_JSON | jq -r ".\"$toolchain\"")` add: - -```bash - if [ "$(echo "$FAMILY" | jq 'length')" = "0" ]; then - # an empty matrix parameter is a hard CircleCI config error, not a skip - echo "skip build-${build_system}-${toolchain}: no families selected" - continue - fi -``` - -(the `continue` also keeps the alias out of `BUILD_ALIASES`, so `code-metrics` never requires a job that was not generated). Guard the code-metrics emission and keep the workflow non-empty: - -```bash - if [ ${#BUILD_ALIASES[@]} -gt 0 ]; then - echo " - code-metrics:" >> .circleci/config2.yml - echo " requires:" >> .circleci/config2.yml - for alias in "${BUILD_ALIASES[@]}"; do - echo " - $alias" >> .circleci/config2.yml - done - else - # a workflow with zero jobs is invalid config - echo " - no-op" >> .circleci/config2.yml - fi -``` - -(replacing the current unconditional code-metrics block). Change the continuation call to: - -```yaml - - continuation/continue: - configuration_path: .circleci/config2.yml - parameters: /tmp/continue_params.json -``` - -- [ ] **Step 3: `config2.yml` — parameters, `-e` resolution, scoped-compare note, no-op job** - -At the top, after `version: 2.1`: - -```yaml -parameters: - example-map: - type: string - default: "{}" - build-filtered: - type: string - default: "false" -``` - -In the `build` command's **Build** step, before the toolchain if/else, insert: - -```bash - # PR example filter for this family ('{}' or a missing key = build all). - # The parameter is a JSON string composed by set-matrix from ci_select. - EX_ARGS=$(printf '%s' '<< pipeline.parameters.example-map >>' | jq -r --arg fam "<< parameters.family >>" '(.[$fam] // []) | map("-e " + .) | join(" ")' 2>/dev/null) || EX_ARGS='' -``` - -and append `$EX_ARGS` to both `tools/build.py` invocations (docker esp-idf and the generic one). In `code-metrics`, wrap the existing compare `when:` condition with the filter guard and add the note branch: - -```yaml - - when: - condition: - and: - - not: - equal: [ master, << pipeline.git.branch >> ] - - equal: [ "false", << pipeline.parameters.build-filtered >> ] - steps: - # ... the existing Download Base Branch Metrics + Compare + store_artifacts steps, unchanged ... - - when: - condition: - and: - - not: - equal: [ master, << pipeline.git.branch >> ] - - equal: [ "true", << pipeline.parameters.build-filtered >> ] - steps: - - run: - name: Scoped build - comparison unavailable - command: | - # CircleCI stores only the averaged metrics.json; the per-example - # baseline lives on GHA. See the GHA code-metrics PR comment. - echo "_Code-size comparison skipped on CircleCI: this PR built a scoped example set._" > metrics_compare.md - - store_artifacts: - path: metrics_compare.md - destination: metrics_compare.md -``` - -Add the no-op job beside the other job definitions: - -```yaml - no-op: - docker: - - image: cimg/base:current - resource_class: small - steps: - - run: - name: No families selected - command: echo "PR selection - no families to build on CircleCI" -``` - -- [ ] **Step 4: Validate and commit** - -```bash -python3 -c "import yaml; yaml.safe_load(open('.circleci/config.yml')); yaml.safe_load(open('.circleci/config2.yml')); print('yaml ok')" -command -v circleci >/dev/null && circleci config validate .circleci/config.yml || true -git add .circleci/config.yml .circleci/config2.yml -git commit -m "ci: scope the CircleCI build matrix and example set by PR selection" -``` - ---- - -### Task 12: End-to-end validation, review, hand-off - -**Files:** none new — verification only (fix-ups amend the relevant earlier area). - -- [ ] **Step 1: Full hooks + suites** - -```bash -pre-commit run --all-files # ~55 s; HIL hooks exercise real timeouts deliberately -``` - -Expected: all hooks pass (`ci-select-test` and `hil-test` among them). - -- [ ] **Step 2: Selector scenario table** - -```bash -for f in src/portable/raspberrypi/rp2040/dcd_rp2040.c src/class/cdc/cdc_device.c \ - src/host/usbh.c examples/device/cdc_msc/src/main.c test/hil/hil_test.py \ - src/common/tusb_fifo.c hw/mcu/nordic/nrf5x/x.h; do - echo "== $f" - python3 tools/ci_select.py --diff-file <(echo "$f") test/hil/tinyusb.json 2>/dev/null | \ - python3 -c "import json,sys; s=json.load(sys.stdin); b=s['build']; print('hil_full:', s['full'], ' build_full:', b['full'], ' fams:', len(b['families']), ' mapped:', len(b['family_examples']))" -done -``` - -Expected (spot-check against the spec's measured table): rp2040 → 1 family; cdc_device → all families, mapped lists; usbh → ~25 families; example → all families, 1-example lists; test/hil → 0 families, hil_full true; common → build_full true; hw/mcu → 1 family (`nrf`). - -- [ ] **Step 3: Matrix + build smoke** - -```bash -SEL=$(python3 tools/ci_select.py --diff-file <(echo src/portable/raspberrypi/rp2040/dcd_rp2040.c) test/hil/tinyusb.json 2>/dev/null) -python3 .github/scripts/ci_set_matrix.py --select "$SEL" | python3 -m json.tool | head -python3 .github/scripts/hil_ci_set_matrix.py --select "$SEL" test/hil/tinyusb.json | python3 -m json.tool | head -python3 tools/build.py -e device/cdc_msc -b stm32f407disco --target all --target tinyusb_metrics -python3 -c "import json; d=json.load(open('cmake-build/cmake-build-stm32f407disco/metrics_by_example.json')); print(sorted(d))" -``` - -Expected: matrix shows only rp2040 under arm-gcc; hil matrix entries carry `-e ... -e device/board_test`; the by-example JSON lists exactly `['device/cdc_msc']`. - -- [ ] **Step 4: Full example set for one board** (repo validation rule after tool changes) - -```bash -cd examples && cmake -B cmake-build-stm32f407disco -DBOARD=stm32f407disco -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel . \ - && cmake --build cmake-build-stm32f407disco && cd .. -``` - -Expected: builds green (objcopy warnings non-critical per CLAUDE.md). - -- [ ] **Step 5: Local review, then stop** - -Run the `/code-review` skill on the branch diff (user policy: every push carrying local changes gets a local review pass first) and fix what holds up, amending into the appropriate task commits. Then **stop and hand back to the user** — pushing `build-filter` and opening the PR is their call; note for the PR description that the workflow changes only fully prove out on a real PR run (first PR after merge-to-branch should be watched with `gh pr checks --watch`, and the `hil-select` step's warnings checked for silent fallbacks). - ---- - -## Self-Review Notes - -- Spec coverage: rule table (Tasks 2-4), CMake-only scan (Task 2), orphan invariant (Task 2), build/hil_examples JSON contract (Task 4), `ci_set_matrix` flags (Task 5), `hil_ci_set_matrix -e` (Task 6), `build.py -e` incl. membrowse aggregate-dependency workaround (Task 7), metrics by-example + intersection compare + never-fall-back rule (Tasks 8-10), GHA side channel + injection-safe env passing (Task 10), CircleCI empty-toolchain/alias/no-op fixes + parameters (Task 11), move fallout table (Task 1). -- Known deviation from the spec text, both directions justified inline: `hil_examples` uses the *narrowed* chosen test list when a board is narrowed (the spec's JSON example implies this; its prose says `board_tests` — the narrowed form is a strict subset and matches what the rig runs, and re-run specs are subsets of it). -- Spec's measured "hcd_max3421.c → 1 leg" is really 1 *bsp* family (`espressif`) that neither provider's family list builds → 0 CI legs; Task 3's rule-4 test therefore asserts shape, not that specific count.
diff --git a/docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md b/docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md deleted file mode 100644 index 898b3c8..0000000 --- a/docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md +++ /dev/null
@@ -1,179 +0,0 @@ -# PR-scoped HIL selection: helper/hil_select.py - -**Date:** 2026-07-29 -**Branch:** `claude/hil-select` (based on `claude/hil-pool-check`, which carries the -hil_lock/hil_flash split and the current rig rosters) - -## Motivation - -Every PR currently builds and runs the full HIL matrix (both rigs, every roster board, every -test). Most PRs touch one port or one class: a `dcd_rp2040` change cannot affect an STM32 board, -a `cdc_device.c` change cannot affect an MSC-only example, and a device-stack change cannot -affect host tests. Scoping HIL to the affected boards/tests cuts CI wall time and rig wear -without losing relevant coverage. - -## Goal / non-goals - -**Goal:** a shared selector that maps a PR diff to (boards, per-board test lists), wired into -CI's `set-matrix` on `pull_request` events (pruning both `hil-build` and the rig jobs) and -callable locally (pre-pr, manual runs). Scoping may only shrink coverage when the mapping is -confident; every uncertainty widens to the full matrix. - -**Non-goals:** -- Variant-level selection (all variants of a selected board run). -- Scoping the non-HIL build jobs (cmake/CircleCI one-per-family builds are independent build - coverage and stay untouched). -- Scoping push/master/scheduled runs (always full). -- Changing hil_test.py behavior (the selector only *composes* existing `-b`/`-bt` args). - -## Component: `test/hil/helper/hil_select.py` - -Stdlib-only, importable and CLI. Lives beside the harness so `hil_ci.sh` copies are unaffected -(it runs on the GitHub runner / dev PC, not on the rig). It must NOT import `hil_test.py` -(which drags pyserial/pymtp onto the bare GitHub runner): the three test lists -(`device_tests`, `dual_tests`, `host_test`) move verbatim into the stdlib-only -`test/hil/helper/hil_util.py` that both `hil_test.py` and `hil_select.py` import (behavior -preserving; `hil_ci.sh` copies the whole `helper/` directory). - -``` -python3 test/hil/helper/hil_select.py --base <ref> [--diff-file <path>] CONFIG.json [CONFIG.json...] -``` - -- `--base REF`: changed files = `git diff --name-only $(git merge-base HEAD REF)..HEAD` - (mirrors pre-pr). `--diff-file`: newline-separated file list instead of git (unit tests, CI - reuse of a precomputed diff). -- Output (stdout, JSON): - -```json -{ - "full": false, - "boards": {"raspberry_pi_pico": "all", "stm32f407disco": ["device/cdc_msc", "device/cdc_dual_ports"]}, - "args": {"tinyusb.json": "-b raspberry_pi_pico -b stm32f407disco -bt stm32f407disco:device/cdc_msc,device/cdc_dual_ports", - "hfp.json": ""}, - "reasons": ["src/portable/raspberrypi/rp2040/dcd_rp2040.c: port rp2040 -> family rp2040 -> boards [raspberry_pi_pico, ...] (device role)"] -} -``` - -- `full: true` ⇒ `boards`/`args` cover the entire rosters (identical to today's behavior). -- `args` maps each input config file to the hil_test.py argument string for that rig: `-b` per - selected board on that roster, plus `-bt BOARD:t1,t2` for boards with a restricted test list - ("all" boards get bare `-b`). An empty string means: nothing on this rig is affected — the - rig job is skipped for this PR. -- Per-file reasoning lines (`file → rule → contribution`) go in `reasons` and to stderr, so the - CI log answers "why did/didn't HIL run X" without archaeology. - -## Classification rules - -Each changed file yields a contribution; the selection is the union. Any file matching no rule -sets `full: true` (fail-open). Rules, first match wins: - -1. **Non-code:** `docs/**`, `.claude/**` (except the workflows below via rule 8), `*.md`, - `*.rst`, `LICENSE*` → contributes nothing. -2. **Port:** `src/portable/<vendor>/<ip>/**` (or single-level `src/portable/<name>/**`). - Role from basename: `dcd_*`/`*_device*` → device; `hcd_*`/`*_host*` → host; anything else - (shared port files, e.g. `dwc2/dwc2_common.c`) → both. Families = directories of - `hw/bsp/*/family.cmake|family.mk` whose text references `<vendor>/<ip>` (pre-pr's grep), - boards = those families' entries on the input rosters. Tests = all tests of that role - (device_tests / host_test from hil_test.py's lists; dual_tests count as both roles). -3. **Class:** `src/class/<c>/*_device.*` → all device-capable roster boards; tests = the - device/dual examples in hil_test.py's lists whose `examples/<role>/<ex>/src/tusb_config.h` - defines `CFG_TUD_<C>` with a nonzero value (derived at runtime; `<C>` = upper-cased class - dir, with the map `musb→n/a`-style exceptions NOT needed — class dirs and config macros - share names: cdc, msc, hid, midi, audio, video, vendor, usbtmc, mtp, printer. Two - exceptions: in class dir `dfu`, `dfu_rt_device.*` maps to CFG_TUD_DFU_RUNTIME and - `dfu_device.*` to CFG_TUD_DFU; class dir `net` maps to CFG_TUD_ECM_RNDIS|CFG_TUD_NCM.) `*_host.*` analogously via `CFG_TUH_<C>`. Shared class files (e.g. `cdc.h`) → - both roles' matching examples. A class with zero matching examples contributes nothing - (known path, does not force full). -4. **Core role:** `src/device/**` → all device-capable boards, all device tests (+dual); - `src/host/**` → all host-capable boards, all host tests (+dual). -5. **Core common:** `src/common/**`, `src/osal/**`, `src/tusb.c`, `src/tusb.h`, - `src/tusb_option.h` → full. -6. **BSP:** `hw/bsp/<family>/**` → that family's roster boards, all their tests; - `hw/bsp/<family>/boards/<board>/**` narrows to that board if it is on a roster, and - contributes nothing when it is not (an off-rig board cannot be HIL-tested; known path, - does not force full). - Family-agnostic BSP files (`hw/bsp/board_api.h`, `hw/bsp/board.c`, ansi_escape.h) → full. -7. **Example:** `examples/<role>/<ex>/**` → all roster boards, tests = that example if present - in hil_test.py's lists, else contributes nothing. `examples/build_system/**`, top-level - `examples/CMakeLists.txt` → full. `examples/device/board_test/**` → full: it is the park - firmware hil_test.py flashes on every board (variant boundary + teardown), not a test. -8. **Harness/infra:** `test/hil/**`, `.github/workflows/build*.yml`, - `.github/actions/**`, `tools/build.py`, `tools/get_deps.py`, `tools/cmake/**`, - `hw/mcu/**`, `lib/**` → full. -9. **Everything else** (`test/unit-test/**`, `tools/**` not above, unknown paths) → full. - (Unit-test-only changes could safely skip HIL, but per the fail-open stance anything not - explicitly classified widens; narrowing rule 9 is a later refinement.) - -**Role pruning:** after the union, if only device-role contributions exist, host-only boards -drop out and host tests are stripped from mixed boards (vice versa for host-only changes). -Dual tests survive either role. Board capability (device/host) comes from the roster entry's -`tests` flags/only-list, same logic hil_test.py uses. - -**No-rig-coverage case:** a cleanly classified change whose boards intersect a roster to the -empty set yields an empty `args` string for that rig and a stderr line saying so — the rig job -is skipped, not widened (running unrelated boards would test nothing relevant). - -**Roster source:** `config['boards']` only (boards-skip stays parked). - -## CI wiring (`.github/workflows/build.yml`) - -- `set-matrix` (PR events only): after generating today's matrices, run - `helper/hil_select.py --base origin/${{ github.base_ref }} test/hil/tinyusb.json test/hil/hfp.json` - (checkout with enough history to reach the merge base: `fetch-depth: 0` on this one job, or - an explicit `git fetch origin $BASE_REF`). New job outputs: `hil_select_full`, - `hil_args_tinyusb`, `hil_args_hfp`, plus the selected-board list consumed by the matrix - generator. Non-PR events: skip the selector, outputs default to full/empty-args-means-all. -- `hil_ci_set_matrix.py` gains `--select '<json>'`: when given and `full` is false, it emits - build entries only for selected boards (per config). Untouched otherwise. -- `hil-tinyusb` job (one matrixed job covering both rigs, selected by `matrix.hil_json`): a - step picks the rig's selector args in shell (`case "$HIL_JSON" in ...`) from the set-matrix - outputs and either appends them to the `hil_test.py` invocation or exits the step early with - a "HIL skipped by selection" log line when that rig has nothing to run (`run` flag output - false). The separate `hil-tinyusb-esp` job (esptool split) gets the same treatment with the - tinyusb args. Non-PR events: outputs default to run=true with empty args (today's behavior). -- The `--flasher`/`--exclude-flasher` split in the existing matrix `test_args` composes fine - with `-b` (hil_test.py applies both filters). - -## Local use - -- pre-pr's "Map changes to boards" step delegates to - `python3 test/hil/helper/hil_select.py --base $BASE test/hil/tinyusb.json` and derives its - one-board-per-family sample from the selector's board set (its capping/sampling policy is - unchanged — the selector provides the affected set, pre-pr samples it). -- Manual: `python3 test/hil/hil_test.py -B examples $(python3 test/hil/helper/hil_select.py --base master test/hil/tinyusb.json | jq -r '.args["tinyusb.json"]') test/hil/tinyusb.json` - — documented in the hil skill. - -## Testing - -`test/hil/test/test_hil_select.py` — stdlib `unittest`, no hardware, injected diffs via -`--diff-file`/API. Cases (the acceptance examples): -1. `src/portable/raspberrypi/rp2040/dcd_rp2040.c` → only rp2040-family roster boards, device - tests only, host-only boards absent, `full` false. -2. `src/device/usbd.c` → every device-capable board on both rosters, all device tests + dual, - no host-only board, no host tests. -3. `src/class/cdc/cdc_device.c` → only examples with CFG_TUD_CDC enabled (must include - device/cdc_msc and device/cdc_dual_ports; must exclude device/msc_dual_lun and all - host tests). -4. `src/class/msc/msc_host.c` → host-capable boards only, host examples with CFG_TUH_MSC. -5. `tools/random_new_script.py` → `full: true`. -6. `docs/foo.rst` alone → contributes nothing ⇒ empty selection, `full` false, all `args` - empty (CI additionally has check-paths gating; the selector's answer is still honest). -7. `hw/bsp/rp2040/family.cmake` → rp2040-family boards, all their tests. -8. Mixed device+host diff → no pruning (both roles present). -The suite runs in `set-matrix` before the selector is used, and locally via -`python3 test/hil/test/test_hil_select.py`. - -## Safety properties - -- Fail-open: unknown/infra paths ⇒ full matrix; selector crash in CI ⇒ job fails visibly - (never silently skips HIL). -- Only `pull_request` events are scoped. -- The selection JSON + per-file reasons are printed in the job log for audit. -- hil_test.py errors on `-b` names not in the config — the selector only emits roster names, - and the unit suite locks that invariant. - -## Sequencing - -Lands on `claude/hil-select` on top of the pool-check/split stack. Follow-ups it does not -include: narrowing rule 9 for unit-test-only changes; variant-level selection; pre-pr skill -text update ships in the same change (its mapping section shrinks to a selector call).
diff --git a/docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md b/docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md deleted file mode 100644 index 019910c..0000000 --- a/docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md +++ /dev/null
@@ -1,473 +0,0 @@ -# PR-scoped CI selection: promoting hil_select to tools/ci_select.py - -**Date:** 2026-08-19 -**Branch:** `build-filter` - -## Motivation - -Every PR builds every example on one board per family, on both CI providers: **74 legs / -2494 example-builds** on the GitHub Actions `cmake` job, and 129 family-legs per build system -on CircleCI (which runs cmake *and* make, plus clang/IAR). Most PRs touch one port, one class, -or one example, and a `hid_host.c` change cannot break an MSC device example on msp430. - -`hil-build` is worse in a different way: it builds **1702 example-builds** (37 board-builds × -46 examples, `--target all`) to run a test suite that needs at most **515**. The HIL example -universe is only 21 of the 46 examples in tree, and the median board needs 15 of them. - -`test/hil/helper/hil_select.py` already maps a PR diff to affected boards and per-board test -lists for HIL, and already owns both mappings the build matrix needs: port-to-family, and -class-macro-to-example -(`docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md`). That design listed -"scoping the non-HIL build jobs" as an explicit non-goal; this is that follow-up. - -## Goal / non-goals - -**Goal:** promote the selector to a repo-wide `tools/ci_select.py` whose single classification -of a diff drives **all three** CI axes from one rule table — which families to build, which -example targets to build on each, and which rig boards run which tests — wired into -`ci_set_matrix.py` and `hil_ci_set_matrix.py` so both providers and the rig filter from one -source. Scoping applies to `pull_request` events only; push, release and `workflow_dispatch` -keep the full matrix. - -**Non-goals:** -- Variant-level or board-level selection below one-board-per-family on the build axis (all - variants of a selected HIL board still build and run). -- Changing `hil_test.py` behaviour. The selector only *composes* existing `-b` / `-bt` args. -- Changing which tests HIL decides to run. The HIL board/test decision is preserved except for - the single rule-7 change called out below. - -## The rule table - -One classification, three outputs. Every rule yields build families, build examples, and HIL -boards/tests. Pairs are unioned **per family** (build) and **per board** (HIL), so a mixed diff -never inflates one axis with another's breadth. - -`DEV` = 33 `examples/device/*`, `HOST` = 9, `DUAL` = 3, `TYPEC` = 1, `ALL` = 46. -`FAM` = the families whose `family.cmake` references the changed path (CMake only — see below). -"roster boards" = boards on `test/hil/{tinyusb,hfp}.json`. - -| # | Changed path | Build families | Build examples | HIL boards → tests | -| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | ----------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | -| 1 | `docs/`, `.claude/`, `*.md`, `*.rst`, `LICENSE` | — | — | — | -| 1b | `.gitignore`, `.clang-format`, `.agents`, `.codex/**`, `.idea/**`, `test/{fuzz,unit-test}/**`, `test/hil/test/**`, non-build `.github/**`, packaging manifests | — | — | — | -| 2 | `test/hil/**` (not `test/hil/test/**`) | — | — | all boards → all tests | -| 2b | `tools/drivers_coverage_check.py` | — (local-only tooling, no CI build runs it) | — | — (nothing on the rig runs it) | -| 2c | `.github/ci-pinned-boards.json` | `ALL` | `ALL` | — (CI board data; no rig board's behaviour depends on it) | -| 2d | `tools/membrowse_cli.py`, `tools/code_size.py` | `ALL` | `ALL` | — (size scripts the pinned build legs run; no rig board runs them) | -| 2e | `.claude/skills/build/scripts/check_build.py` (ahead of rule 1) | `ALL` | `ALL` | all boards → all tests (the selection contract CI and agents call) | -| 3 | `src/portable/<port>/dcd_*`, `*_device.[ch]` | `FAM` | `DEV`+`DUAL` | `FAM`'s device-role boards → device+dual tests | -| 4 | `src/portable/<port>/hcd_*`, `*_host.[ch]` | `FAM` | `HOST`+`DUAL` | `FAM`'s host-role boards → host+dual tests | -| 5 | `src/portable/<port>/**` (anything else) | `FAM` | `ALL` | `FAM`'s boards → all their tests | -| 5b | `src/portable/<port>/**` where `FAM` is empty | — | — | — (empty resolves to nothing on BOTH axes) | -| 6 | `hw/bsp/<family>/**` | that family | `ALL` | that family's boards → all tests (a `boards/<board>/` path narrows to that board) | -| 7 | `hw/mcu/<vendor>/**` | `FAM` — empty resolves to nothing (maintainer ruling) | `ALL` | `FAM`'s boards → all tests; empty resolves to nothing (maintainer ruling) ⚠ *see below* | -| 8 | `src/class/<cls>/*_device.[ch]` | `ALL` | examples enabling `CFG_TUD_<CLS>` | device-role boards → HIL tests enabling `CFG_TUD_<CLS>` | -| 9 | `src/class/<cls>/*_host.[ch]` | `ALL` | examples enabling `CFG_TUH_<CLS>` | host-role boards → HIL tests enabling `CFG_TUH_<CLS>` | -| 10 | `src/class/<cls>/**` (shared header) | `ALL` | either, **plus include-edge classes and direct includers** | both roles → same, plus include-edge classes and direct includers | -| 11 | `src/device/**` | `ALL` | `DEV`+`DUAL` | device-role boards → device+dual tests | -| 12 | `src/host/**` | `ALL` | `HOST`+`DUAL` | host-role boards → host+dual tests | -| 12b | `src/typec/**` | `ALL` | examples enabling `CFG_TUC_ENABLED` | — (no rig board runs a typec test) | -| 13 | `examples/<role>/<name>/**` | `ALL` | just `<name>` | if `<name>` is a HIL test: all boards → that test; else nothing | -| 14 | `examples/device/board_test/**` | `ALL` | just `board_test` | all boards → all tests (HIL parking firmware) | -| 15 | `examples/build_system/**`, `examples/CMakeLists.txt`, `examples/<role>/CMakeLists.txt` | `ALL` | `ALL` | all boards → all tests | -| 16 | `src/common/`, `src/osal/`, `src/tusb.[ch]`, `src/tusb_option.h`, `tools/{build,build_utils,ci_select,family_json}.py`, `src/CMakeLists.txt`, `src/tinyusb.mk`, `hw/bsp/{family_support.{cmake,mk},family_rules.mk,zephyr_board_aliases.cmake,board.c,board_api.h,ansi_escape.h}`, `.github/**`, `.circleci/**` | `ALL` | `ALL` | all boards → all tests | -| 16a | `lib/<name>/**` | `ALL` | examples whose own `CMakeLists.txt`/`Makefile` names `lib/<name>` | those examples that are HIL tests, on all boards; empty resolves to nothing | -| 16b | `tools/get_deps.py` | families whose `deps_mandatory`/`deps_optional` entries changed | `ALL` | those families' boards → all tests; a logic change, an `'all'` entry, no base content or a changed token naming no family → full | -| 17 | anything unclassified (no tracked file reaches this — TestNoTrackedFileIsUnclassified) | `ALL` | `ALL` | all boards → all tests (fail-open) | - -**Rule 2 is deliberately asymmetric.** A `test/hil/**` change is invisible to the family matrix -but is exactly what the rig exercises, so it builds nothing and runs everything. - -`test/hil/test/**` is carved out to rule 1b: it holds the harness's own unit tests, which -nothing on the rig runs (pre-commit does, and `build.yml` runs `test_ci_select.py` as the -gate before trusting a selection). A bare `test/hil/` prefix was booking the full 27-board -rig for diffs that cannot reach it. The carve-out is a claim about that directory's -contents, so a test pins its file list: add anything the rig reads and it fails. - -**Rule 7 is the one HIL-side behaviour change in this design.** Today `hw/mcu/` sits in -`hil_select`'s `_FULL_RE` and forces the full HIL matrix. Since the build axis now resolves -those paths to a family through the same scan, forcing full on the rig is inconsistent. The -path fires rarely — 4 commits in 3 years — so this is low-risk either way; if you would rather -keep the HIL view untouched, rule 7's HIL column becomes "all boards → all tests" and nothing -else in this design changes. - -Rules 8–10 reuse machinery `hil_select` already has — `class_macros`, `_config_enables`, -`class_include_edges` — applied over all 46 examples' `src/tusb_config.h` for the build axis -and over the HIL test list for the HIL axis. The include edges are why an `audio.h` change also -selects the MIDI examples (`midi{,2}_{device,host}.h` include `class/audio/audio.h`) and a -`cdc.h` change the net one. - -### Buildability post-filter (build axis) - -After the pairs are unioned, every `(family, examples)` pair is pruned with -`build_utils.skip_example(example, <family's first board>)` — the same `skip.txt` / `only.txt` -data CMake's `family_filter` uses (40 `skip.txt`, 13 `only.txt` in tree). Examples the family -cannot build are dropped; a family left with none is dropped entirely. - -This is where most of the host-side saving comes from: only 23 of 75 CI families can build -`host/bare_api` at all, and 2 can build `typec/power_delivery`. - -### Measured effect - -GHA `cmake` job, baseline **74 legs / 2494 example-builds**; `hil-build`, baseline **1702 -example-builds** across 37 board-builds. - -| PR shape | Build legs | Build ex-builds | HIL boards | hil-build ex-builds | -| ---------------------------- | ---------: | --------------: | ---------: | ------------------: | -| `dcd_rp2040.c` | 1 | 35 | 2 | 32 | -| `hcd_max3421.c` | 1 | 10 | 7 | 36 | -| `hw/bsp/stm32f4/**` | 1 | 45 | 1 | 15 | -| `dcd_dwc2.c` | 20 | 646 | 10 | 184 | -| `hid_host.c` | 24 | 68 | — | — | -| `msc_host.c` | — | — | 9 | 40 | -| `usbh.c` | 25 | 217 | 10 | 57 | -| `msc_device.c` | 74 | 350 | — | — | -| `cdc_device.c` | 74 | 588 | 27 | 192 | -| `examples/device/cdc_msc/**` | 73 | 73 | 25 | 60 | -| `usbd.c` | 74 | 2297 | 27 | 472 | -| `src/common/**` (full) | 74 | 2494 | 30 | 515 | -| `test/hil/**` only | 0 | 0 | 30 | 515 | - -The full-matrix row is the headline for `hil-build`: even with **no** PR narrowing, per-board -example selection takes it from 1702 to 515. - -### Why "empty means empty" - -Both views answer an empty `FAM` the same way (rule 5b): nothing. The HIL view used to force -the full 30-board rig there, on the theory that an empty result might be a scan miss — but the -build view answered the identical condition with zero families for the same path, so the rig -ran every board to validate a file that nothing compiled. In the build view that theory costs -74 legs, and the -evidence does not support it: of the 28 `src/portable/*/*` directories, **26 resolve to at -least one family**. The two that do not are both real orphans as far as CI is concerned: -`microchip/pic` (only `dcd_pic.c` and a README, with no `hw/bsp/pic` family at all) and -`microchip/pic32mz` (`hw/bsp/pic32mz` has only a `family.mk`, and `pic32mz` is in neither -provider's family list, so no CI job builds it today). A file no CI job compiles cannot be -validated by building anything. - -The safety this gives up is recovered structurally: a unit test asserts every -`src/portable/*/*` and every tracked `hw/mcu/<vendor>` resolves to ≥1 family, with an explicit -allowlist of known orphans (`microchip/pic`, `microchip/pic32mz`). Adding a port without -wiring a family then fails -pre-commit instead of silently building nothing on every later PR. Same enforcement style as -the existing `test_hil_util.BottomLayer` structural tests. - -Fail-open survives where it belongs: an *unclassified* path or any exception widens to `ALL` on -every axis. - -### A class no example enables selects nothing - -`src/class/bth` is the live instance: no example's `tusb_config.h` sets `CFG_TUD_BTH`, so -rules 8-10 resolve to no examples and a bth-only PR builds nothing and runs nothing. That is -the empty-means-empty ruling applied to classes, and it is deliberate — nothing compiles the -file, so nothing can validate it, and the master-push build is the net. - -Worth stating plainly because the exposure changed: GHA used to rebuild everything for such -a PR by accident, through the empty-`families` bug in `build.yml`. With that fixed, both -providers now correctly build nothing, so `tud_bt_*` can be broken by a green PR. -`TestClassesWithNoEnablingExample` pins the set to `{bth}` so a second class cannot enter -this state unnoticed. - -### Why `hw/mcu/**` is rule 7 and not "full" - -`hw/mcu` is overwhelmingly dependency territory — `tools/get_deps.py` has 87 entries under it, -and those paths are gitignored, so they can never appear in a diff. Only 51 files survive -in-tree, touched 4 times in 3 years, and they resolve through the same scan the ports use: - -| Tracked directory | In `get_deps`? | Resolves to | -| ----------------------------- | ------------------------------------------- | ----------- | -| `hw/mcu/dialog/` (`da1469x`) | **no** — real in-repo MCU support, 21 files | `da1469x` | -| `hw/mcu/nordic/` (`nrf5x`) | beside the `nrfx` dep | `nrf` | -| `hw/mcu/sony/` (`cxd56`) | beside the `spresense-exported-sdk` dep | `cxd56` | -| `hw/mcu/bridgetek/` (`ft9xx`) | beside the `ft90x-sdk` dep | `ft9xx` | - -Rule 7 is therefore not a mechanism of its own — it is rules 3–5's scan pointed at a second -tree, because `src/portable/<port>` and `hw/mcu/<vendor>` ask the same question. - -Unlike the port rule, an `hw/mcu` path that resolves to no family contributes *nothing* on -either axis (maintainer ruling): if no family's build references it, no build compiles it. The -table above is kept honest by `test_tracked_mcu_vendors_resolve`, which fails pre-commit if a -tracked vendor directory stops resolving. - -### Why `lib/**` is rule 16a and scanned per example - -Its tracked contents (`SEGGER_RTT`, `networking`, `rt-thread`, `embedded-cli`, 22 commits in -3 years) are wired in at `examples/build_system` and per-example `CMakeLists.txt`, not per -family — so the family scan the ports use is the wrong instrument here: it would *wrongly* -narrow `SEGGER_RTT` to the three families that name the path in their `family.cmake`, while -the path is not compiled by any of them by default (it is reached only through `LOGGER=rtt`). -That scan stays applied to `src/portable/` and `hw/mcu/` only. - -Rule 16a asks the per-example question instead (maintainer ruling: only the examples that use -the lib need building): `lib_examples()` reads each example's own `CMakeLists.txt` and -`Makefile` and keeps the ones naming `lib/<name>` at a directory boundary. Every family stays -in play — any of them can build those examples — while the example list collapses: - -| Tracked lib | Examples that build it | HIL tests among them | -| -------------- | ----------------------------------------------------------- | -------------------- | -| `embedded-cli` | `host/msc_file_explorer`, `host/msc_file_explorer_freertos` | both | -| `networking` | `device/net_lwip_webserver` | none (test disabled) | -| `SEGGER_RTT` | — | — | -| `rt-thread` | — | — | - -`SEGGER_RTT` and `rt-thread` resolve to nothing, and "empty means empty" applies: no CI build -compiles them, so there is nothing to validate by building. - -### Why `tools/get_deps.py` is rule 16b - -`deps_mandatory` / `deps_optional` are data: `path -> [url, commit, 'fam1 fam2 ...']`. A commit -bump therefore affects exactly the families listed in that entry, and building the other 70+ is -pure waste. `get_deps_changed_families()` parses both sides of the file with `ast` (never -`exec` — this is PR content), diffs the two dict literals **separately**, and unions the family -tokens of every added, removed or edited entry, from **both** sides (a removed entry has only a -base side; an edited family list must cover the families that lose the dep as well as the ones -that gain it). Separately, because merging the dicts before diffing hides a *move* between -`deps_mandatory` and `deps_optional` — the value is untouched, but mandatory deps are fetched -for every family, so demoting one stops families fetching it. - -It falls open to the full matrix whenever the entries are not the whole answer: - -* anything outside the two dict assignments differs — a logic change to `get_deps` can change - what every family fetches (compared as `ast.dump(..., annotate_fields=False)` of the module - with those two assignments removed, so comments and reformatting alone are not a logic - change); -* an `'all'` entry (every mandatory dep) changed; -* the file will not parse; -* there is no base content: `--diff-file` mode has no git, so no merge-base blob; -* a changed entry carries a family token that names no `hw/bsp/<dir>` and is not one of the - known aliases. "Changed but unmappable" is not "nothing changed": reading it as the - latter empties the whole build matrix for a dep bump. - -The six known aliases (`sam3x`, `samd21`, `samd51`, `same5x`, `stm32l1`, `stm32l5`) are -pinned in `_DEPS_ALIAS_TOKENS` and select nothing. `get_deps` matches a token against a -requested family name verbatim (`f in entry[2].split()`), so these tokens match nothing -there either — four are pre-rename spellings listed beside the current name in the same -entry, and two name no family in the tree. (`fc100s` and `spresense` were on this list -until they were corrected in `get_deps.py`; those two were the only ones that left a -real dep unreachable for its own family.) A seventh appearing fails `TestOrphanInvariant`. - -## Component: `tools/ci_select.py` - -`git mv test/hil/helper/hil_select.py tools/ci_select.py` (history preserved). The HIL -classifier is unchanged apart from rule 7; a second, independent build classifier is added -beside it. One diff read, two classifiers, one unit suite. - -``` -python3 tools/ci_select.py --base <ref> [--diff-file <path>] [CONFIG.json ...] -``` - -`configs` becomes `nargs='*'`. With rosters it emits everything it emits today plus the new -keys; with none it emits only the build view, so CircleCI never needs to know HIL exists. - -```json -{ - "full": false, - "boards": {"raspberry_pi_pico": "all"}, - "families": ["rp2040"], - "args": {"tinyusb.json": "-b raspberry_pi_pico"}, - "args_flasher": {"tinyusb.json": {"openocd": "-b raspberry_pi_pico"}}, - "hil_examples": {"raspberry_pi_pico": ["device/cdc_msc", "device/board_test"]}, - "build": { - "full": false, - "families": ["rp2040"], - "family_examples": { - "rp2040": ["device/cdc_msc", "device/hid_composite", "dual/dynamic_switch"] - } - }, - "reasons": ["src/portable/raspberrypi/rp2040/dcd_rp2040.c: port rp2040 -> ..."] -} -``` - -`build.families` is the build family axis. `build.family_examples` maps a family to its example -list; **a family absent from the map builds all its examples**, so the common "narrow families, -all examples" case carries no payload. `build.full` true means no build narrowing at all. - -`hil_examples` is the new HIL build axis: per roster board, the examples `hil-build` must -produce. It is `board_tests(board)` — which the selector already computes — **plus -`device/board_test`**, which `hil_test.py` flashes to park every board at each variant boundary -and at end-of-board teardown (`hil_test.py:1798`, `:1866`). It is emitted even when -`full: true`, because the HIL example universe is 21 of 46 examples regardless of any diff. - -All pre-existing keys keep their exact meaning, so `.github/scripts/hil_ci_set_matrix.py`, the -HIL legs in `build.yml` and `.claude/skills/pre-pr/SKILL.md` need only a path update. - -### Shared helper change - -`port_families(port_dir, repo_root)` generalizes to a path-to-families reference scan over a -second tree (`hw/mcu/`). Its existing **CMake-only** behaviour is kept unchanged and is now the -rule for every axis: it scans `hw/bsp/*/family.cmake` plus the espressif component -`CMakeLists.txt`, and never `family.mk`. - -CMake is the first-class build system; Make follows whatever CMake decides. A family that -CMake does not wire up to a port is not a consumer of that port, and the Make legs on CircleCI -build the same families CMake does. Scanning `family.mk` as well would only ever *widen* the -selection to families CMake never builds, which is coverage nobody asked for — and it would -resolve `microchip/pic32mz` to a family that appears in no CI family list. - -One consequence to keep in view: because HIL and build now share one scan, there is no -per-caller flag, no second cache key, and no way for the two axes to disagree about which -families own a port. - -**Boundary matching.** A directory reference must match at a directory boundary — a trailing -`/` *or* end-of-token — not as a bare substring. Both traps are live: `hw/bsp/nrf/family.cmake` -writes `${TOP}/hw/mcu/nordic/nrf5x` with no trailing slash, while the existing port scan -requires a trailing `/` precisely to stop `microchip/pic` matching `microchip/pic32mz`. - -### Move fallout - -All mechanical, all one-line: - -| File | Change | -| ---------------------------------- | ----------------------------------------------------------------- | -| `tools/ci_select.py` | `sys.path` walk 4 levels → 2 | -| `test/hil/hil_ci.sh` | drop from the scp list (nothing on the rig imports it) | -| `test/hil/test/test_hil_select.py` | rename to `test_ci_select.py`, import path | -| `test/hil/test/test_hil_util.py` | `BottomLayer` stdlib-closure allowlist + module list | -| `.pre-commit-config.yaml` | both hooks (`hil-select-test` → `ci-select-test`, `files:` globs) | -| `.github/workflows/build.yml` | selector path, step name | -| `.claude/skills/pre-pr/SKILL.md` | selector path | - -The test file stays in `test/hil/test/` — it still consumes the rig rosters and `hil_util`. - -The selector gains one non-stdlib-but-local import: `tools/build_utils.skip_example` for the -buildability post-filter. `build_utils` imports only `subprocess`, `pathlib` and `re`, so the -stdlib closure the bare GitHub runner depends on is preserved; `BottomLayer` must be extended -to cover it. - -Because a wrong parents-count already broke this module once (there is a comment in the source -recording it), the moved module gets a guard test asserting its derived repo root contains -`src/` and `hw/bsp/`. - -## Component: `tools/build.py --example` - -`build.py` has no example filter today. `-T/--target` exists and maps to -`cmake --build --target <name>`, but it hard-fails on a target that does not exist, and absent -targets are routine (40 `skip.txt`, 13 `only.txt`). - -New repeatable `-e/--example <role>/<name>`: - -- Default (none given) keeps today's behaviour exactly: `--target all`. -- Given, each board's list is intersected with `build_utils.skip_example(example, board)`, then - passed as one `--target <name>` per example. Example target names are the directory names and - are unique across all four roles (verified: 46 examples, zero collisions). -- A board whose intersection is empty is reported **skipped**, not failed. -- The espressif path already builds per example via `get_examples` + `skip_example`; it takes - the same filter. - -Both the family matrix and `hil-build` use this one flag. - -## CI wiring - -### `.github/scripts/ci_set_matrix.py` - -Two mutually exclusive optional flags. **Output shape is unchanged** — `{toolchain: [family]}`, -just fewer families. With no flags the output is byte-for-byte today's, so push, release and -`workflow_dispatch` are untouched. - -| Flag | Caller | Behaviour | -| --------------- | -------- | -------------------------------------------------------- | -| `--select JSON` | GHA | consumes the selector JSON the workflow already computes | -| `--base REF` | CircleCI | runs `tools/ci_select.py` itself | - -`build.full` true, or any exception, prints the full matrix with a warning on stderr. - -### `.github/scripts/hil_ci_set_matrix.py` - -Already takes `--select` and already scopes boards. It additionally appends `-e <example>` per -board from `hil_examples`, so each `hil-build` entry builds only what its board will run plus -`board_test`. When `hil_examples` is absent (hand-runs), it falls back to today's `--target all`. - -### The example map is a side channel, not a matrix entry - -The build example list deliberately does **not** ride inside the family matrix entry string. On -CircleCI the `family` parameter is also passed to `python tools/get_deps.py -<< parameters.family >>` and tested with `if [ << parameters.family >> == "rp2040" ]` — a value -carrying `-e` flags breaks both — and CircleCI matrix parameters form a cartesian product, so a -parallel `example-args` parameter would multiply the jobs rather than zip with them. - -So `build.family_examples` travels as one JSON blob and each build job resolves its own entry: - -- **GHA:** `set-matrix` exposes it as an output; `build_util.yml` gains an optional - `example-map` input (default `''`); a step resolves `-e` flags for `matrix.arg` with `jq`. -- **CircleCI:** `set-matrix` writes `example_map.json` and `persist_to_workspace`s it; the - `build` job gains `attach_workspace` and resolves the same way. - -Consequence of keeping the matrix shape: CircleCI's generated `config2.yml` does not inflate -to one entry per family. `hil-build` -needs none of this — its matrix entries are already compound per-board strings from -`hil_ci_set_matrix.py`, so `-e` flags go straight in. - -### `.github/workflows/build.yml` - -The existing `HIL selection (PR only)` step in `set-matrix` is already gated on -`pull_request` — exactly the gate wanted. It is renamed, repointed at `tools/ci_select.py`, and -its `select` output is threaded into `ci_set_matrix.py --select`, so the filter costs zero extra -selector invocations. - -`build_util.yml`'s `if: inputs.build-args != '[]'` already skips a toolchain leg whose list is -empty, and a partially-skipped matrix aggregating to success is the documented pattern -`hil-build` already relies on. When every leg is empty (a `test/hil`-only PR), the `cmake` job -has nothing to build. Accepted: GitHub treats a skipped job as satisfying a required status -check, and HIL is unaffected because `hil-build` is a separate matrix. - -### `.circleci/config.yml` - -The `set-matrix` job passes `--base origin/master` when `CIRCLE_PULL_REQUEST` is set, after -`git fetch --no-tags origin master || true`; unfiltered otherwise. CircleCI does not expose the -PR base branch, so `master` is assumed — true for essentially every tinyusb PR, and any ref or -clone problem falls back to the full matrix. - -Two fixes the GHA side does not need: - -- `gen_build_entry` must **skip** a toolchain whose family list is `[]`. An empty matrix - parameter is a hard CircleCI config error, not a skipped job. - -## Testing - -Extended in `test/hil/test/test_ci_select.py` (stdlib-only, ~0.1 s, already a pre-commit hook -and already gating CI's selector step): - -- One case per rule 1–17, asserting all three outputs. -- Per-family union: a mixed diff (`dcd_rp2040.c` + `cdc_device.c`) gives `rp2040` the device - list and every other family the CDC list — not the cross product of both. -- Include edges: an `audio.h` change selects the MIDI examples; a `cdc.h` change the net one. -- `hil_examples` always contains `device/board_test` for every selected board, including when - `full: true`, and is otherwise exactly `board_tests(board)`. -- `hil_examples` never exceeds the 21-example HIL universe. -- The scan is CMake-only: a port referenced solely from a `family.mk` (`microchip/pic32mz`) - resolves to no family, and no `family.mk` is ever read. -- Boundary matching: `microchip/pic` does not inherit `microchip/pic32mz`'s families, and - `hw/mcu/nordic/nrf5x` resolves despite having no trailing slash at its reference site. -- Buildability post-filter: `typec/power_delivery` prunes to 2 families, `host/bare_api` to 23. -- Structural invariant: every `src/portable/*/*` and every tracked `hw/mcu/<vendor>` resolves - to ≥1 family, allowlist `{microchip/pic, microchip/pic32mz}`. -- Every name in `build.families` is a real `hw/bsp/<dir>`; every example name on either axis is - a real `examples/<role>/<name>` directory. -- Repo-root guard for the moved module. -- `ci_set_matrix.py`: no flags → byte-identical to today; `--select` with `build.full` → - identical; `--select` narrow → a subset; malformed `--select` → full plus a warning. -- `hil_ci_set_matrix.py`: no `hil_examples` → today's args byte-for-byte; with it → `-e` flags - appended per board, `board_test` always present. -- `build.py`: `-e` with an example the board skips builds nothing and reports skipped, not - failed; no `-e` still passes `--target all`. - -## Known gaps - -- **HIL re-run attempts.** A re-run spec is a subset of the original selection, so the - firmware `hil-build` produced already covers it. This holds only while re-run specs stay - subsets; a future "re-run with extra tests" feature would need `hil-build` re-run too. -- **Membrowse** receives rows for fewer families and fewer examples on filtered PRs. If that - service misbehaves, the escape hatch is keeping the membrowse upload leg unfiltered. -- **`typec/power_delivery`** is reached only through rules 5 and 13 (`src/portable/st/typec` - has neither a `dcd_` nor an `hcd_` prefix, so it selects `ALL` examples on its 5 families, - which the post-filter then prunes to 2). A dedicated typec rule is possible later; the - post-filter already makes it cheap. -- **A `test/hil`-only PR reports `cmake` as skipped** rather than passing. Accepted; - revertible with a one-family floor if branch protection turns out to disagree. -- **`microchip/pic32mz` builds nothing.** The scan is CMake-only and `hw/bsp/pic32mz` ships - only a `family.mk`, so a change there selects no family. That matches reality — `pic32mz` is - in neither provider's family list — but it means the port is unbuilt by CI whether or not - this design lands. Giving it a `family.cmake` is the fix, and is out of scope here. -- **Seven bsp families are in no CI toolchain today** (`espressif`, `efm32`, `same7x`, - `cxd56`, `f1c100s`, `pic32mz`, `py32f0`); the intersection drops them, matching current - behaviour. This change does not alter that.
diff --git a/hw/bsp/mcx/family.cmake b/hw/bsp/mcx/family.cmake index 00c8c4e..a723535 100644 --- a/hw/bsp/mcx/family.cmake +++ b/hw/bsp/mcx/family.cmake
@@ -95,7 +95,7 @@ endif() # PORT is set per board (board.cmake), so pick the driver at configure time. Spelled out - # rather than $<IF:${PORT},...> so the port path stays greppable: tools/ci_select.py + # rather than $<IF:${PORT},...> so the port path stays greppable: tools/change_impact.py # maps a portable-driver change to the families whose build file names that directory. if (PORT) set(PORT_SRC ${TOP}/src/portable/chipidea/ci_hs/dcd_ci_hs.c)
diff --git a/test/hil/helper/hil_util.py b/test/hil/helper/hil_util.py index fe8c347..d153b06 100644 --- a/test/hil/helper/hil_util.py +++ b/test/hil/helper/hil_util.py
@@ -20,7 +20,7 @@ # ------------------------------------------------------------- -# HIL example test lists, shared by hil_test.py (runner) and ci_select.py (PR-diff +# HIL example test lists, shared by hil_test.py (runner) and change_impact.py (PR-diff # selector). Run order is shuffled per board (see test_board); every example carries a # unique hardcoded idProduct (see its usb_descriptors.c). # -------------------------------------------------------------
diff --git a/test/hil/hil_flash.py b/test/hil/hil_flash.py index 7583d3a..7e9289e 100755 --- a/test/hil/hil_flash.py +++ b/test/hil/hil_flash.py
@@ -388,7 +388,7 @@ # The one place a flasher's firmware extension is decided. A flasher with no entry falls -# back to .elf-or-.bin and can be handed the wrong file — test_ci_select's +# back to .elf-or-.bin and can be handed the wrong file — test_change_impact's # TestRosterFlashersDispatch fails if a roster names one. FLASHER_SUFFIX = { 'esptool': '.bin',
diff --git a/test/hil/test/test_ci_select.py b/test/hil/test/test_change_impact.py similarity index 92% rename from test/hil/test/test_ci_select.py rename to test/hil/test/test_change_impact.py index 5bc9e2b..335386d 100644 --- a/test/hil/test/test_ci_select.py +++ b/test/hil/test/test_change_impact.py
@@ -1,9 +1,9 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: MIT -# Unit tests for ci_select.py — pure logic, no hardware, no git. Run directly: -# python3 test/hil/test/test_ci_select.py +# Unit tests for change_impact.py — pure logic, no hardware, no git. Run directly: +# python3 test/hil/test/test_change_impact.py # -# Imports stay stdlib + ci_select/hil_util/hil_flash ONLY: the pre-commit hil-test +# Imports stay stdlib + change_impact/hil_util/hil_flash ONLY: the pre-commit hil-test # hook runs this suite, on GitHub's bare runner in the pre-commit workflow as well as # locally, and that runner has no pyserial/pymtp. hil_flash is admissible because it # is stdlib + hil_util only (test_hil_util.BottomLayer enforces the stdlib closure of @@ -28,12 +28,12 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # test/hil, for hil_flash/helper sys.path.insert(0, os.path.join(REPO, 'tools')) import hil_flash -import ci_select +import change_impact from helper.hil_util import device_tests, dual_tests def _read(path): - # Fixed encoding, like ci_select._read: a non-UTF-8 locale would fail on tracked non-ASCII bytes. + # Fixed encoding, like change_impact._read: a non-UTF-8 locale would fail on tracked non-ASCII bytes. with open(path, encoding='utf-8', errors='replace') as f: return f.read() @@ -97,7 +97,7 @@ def sel(files): - return ci_select.classify(files, REPO, ROSTERS) + return change_impact.classify(files, REPO, ROSTERS) class TestPortRule(unittest.TestCase): @@ -167,20 +167,20 @@ device/audio_test_freertos. On boards that skip that example the per-board intersection emptied and an audio.h-only PR ran ZERO HIL on them.""" def test_edges_derived_from_includes(self): - edges = ci_select.class_include_edges(REPO) + edges = change_impact.class_include_edges(REPO) self.assertEqual({c for c, _ in edges.get('audio/audio.h')}, {'midi'}) self.assertIn(('midi', 'midi2_device.h'), edges.get('audio/audio.h')) self.assertEqual({c for c, _ in edges.get('cdc/cdc.h')}, {'net'}) def test_audio_header_selects_midi_example(self): - s = ci_select.classify(['src/class/audio/audio.h'], REPO, real_rosters()) + s = change_impact.classify(['src/class/audio/audio.h'], REPO, real_rosters()) self.assertFalse(s['full']) # every board that runs device/midi_test at all must run it here (boards with # a tests.only list, e.g. espressif, run the freertos examples instead) by_name = {b['name']: b for _, bs in real_rosters() for b in bs} checked = 0 for name, tests in s['boards'].items(): - if 'device/midi_test' in ci_select.board_tests(by_name[name]): + if 'device/midi_test' in change_impact.board_tests(by_name[name]): self.assertIn('device/midi_test', tests, name) checked += 1 self.assertTrue(checked) @@ -189,21 +189,21 @@ # both skip device/audio_test_freertos: without the midi edge their # intersection is empty and they drop out of the selection entirely boards = on_roster(self, 'metro_m4_express', 'nrf54lm20dk') - s = ci_select.classify(['src/class/audio/audio.h'], REPO, real_rosters()) + s = change_impact.classify(['src/class/audio/audio.h'], REPO, real_rosters()) for board in boards: self.assertEqual(s['boards'].get(board), ['device/midi_test'], board) def test_edge_keeps_the_including_files_own_macro(self): # mutant: map the edge to class_macros(c, '') again. midi2_{device,host}.h include # audio.h but compile under CFG_TUx_MIDI2, which only the midi2 examples enable - got = ci_select._build_class_examples('audio', 'audio.h', {'device', 'host'}, REPO) + got = change_impact._build_class_examples('audio', 'audio.h', {'device', 'host'}, REPO) self.assertIn('device/midi2_device', got) self.assertIn('host/midi2_host', got) def test_edge_is_per_header_not_per_class(self): # midi includes audio.h, not audio_device.h: an audio_device change must # not drag midi's examples in - s = ci_select.classify(['src/class/audio/audio_device.c'], REPO, real_rosters()) + s = change_impact.classify(['src/class/audio/audio_device.c'], REPO, real_rosters()) self.assertFalse(s['full']) for tests in s['boards'].values(): if tests != 'all': @@ -293,17 +293,17 @@ solo = [r for r in joint if r[0].endswith('hfp.json')] for files in self.DIFFS: with self.subTest(files=files): - a = ci_select.classify(list(files), REPO, joint) - b = ci_select.classify(list(files), REPO, solo) + a = change_impact.classify(list(files), REPO, joint) + b = change_impact.classify(list(files), REPO, solo) self.assertEqual(a['full'], b['full']) - self.assertEqual(ci_select.selection_args(a, joint)['hfp.json'], - ci_select.selection_args(b, solo)['hfp.json']) + self.assertEqual(change_impact.selection_args(a, joint)['hfp.json'], + change_impact.selection_args(b, solo)['hfp.json']) class TestArgsEmission(unittest.TestCase): def test_args_for_scoped_selection(self): s = sel(['src/portable/raspberrypi/rp2040/dcd_rp2040.c']) - args = ci_select.selection_args(s, ROSTERS) + args = change_impact.selection_args(s, ROSTERS) a = args['tinyusb.json'] self.assertIn('-b raspberry_pi_pico', a) self.assertNotIn('stm32f407disco', a) @@ -311,17 +311,17 @@ def test_args_full_is_empty(self): s = sel(['tools/random_new_script.py']) - self.assertEqual(ci_select.selection_args(s, ROSTERS), {'tinyusb.json': ''}) + self.assertEqual(change_impact.selection_args(s, ROSTERS), {'tinyusb.json': ''}) def test_args_all_board_gets_bare_b(self): s = sel(['hw/bsp/rp2040/boards/raspberry_pi_pico/board.h']) - a = ci_select.selection_args(s, ROSTERS)['tinyusb.json'] + a = change_impact.selection_args(s, ROSTERS)['tinyusb.json'] self.assertIn('-b raspberry_pi_pico', a) self.assertNotIn('-bt', a) def test_args_by_flasher_splits_esp_from_the_rest(self): s = sel(['src/device/usbd.c']) - per = ci_select.selection_args_by_flasher(s, ROSTERS)['tinyusb.json'] + per = change_impact.selection_args_by_flasher(s, ROSTERS)['tinyusb.json'] self.assertIn('espressif_s3_devkitm', per['esptool']) self.assertIn('raspberry_pi_pico', per['openocd']) self.assertNotIn('espressif_s3_devkitm', per.get('openocd', '') + per.get('jlink', '')) @@ -329,19 +329,19 @@ def test_args_by_flasher_omits_a_flasher_with_no_selected_board(self): # the esp CI leg must see no args at all here, not a filter matching zero boards s = sel(['hw/bsp/rp2040/boards/raspberry_pi_pico/board.h']) - per = ci_select.selection_args_by_flasher(s, ROSTERS)['tinyusb.json'] + per = change_impact.selection_args_by_flasher(s, ROSTERS)['tinyusb.json'] self.assertEqual(per, {'openocd': '-b raspberry_pi_pico'}) def test_args_by_flasher_full_is_empty(self): s = sel(['tools/random_new_script.py']) - self.assertEqual(ci_select.selection_args_by_flasher(s, ROSTERS), {'tinyusb.json': {}}) + self.assertEqual(change_impact.selection_args_by_flasher(s, ROSTERS), {'tinyusb.json': {}}) def test_cli_diff_file(self): import subprocess, tempfile, json as j with tempfile.NamedTemporaryFile('w', suffix='.txt', delete=False) as f: f.write('src/class/cdc/cdc_device.c\n') path = f.name - r = subprocess.run([sys.executable, os.path.join(REPO, 'tools/ci_select.py'), + r = subprocess.run([sys.executable, os.path.join(REPO, 'tools/change_impact.py'), '--diff-file', path, os.path.join(REPO, 'test/hil/tinyusb.json')], capture_output=True, text=True) self.assertEqual(r.returncode, 0, r.stderr) @@ -367,7 +367,7 @@ lives in a component CMakeLists.txt rather than family.cmake/family.mk.""" def test_dwc2_change_selects_espressif_boards(self): boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') - s = ci_select.classify(['src/portable/synopsys/dwc2/dcd_dwc2.c'], REPO, real_rosters()) + s = change_impact.classify(['src/portable/synopsys/dwc2/dcd_dwc2.c'], REPO, real_rosters()) self.assertFalse(s['full']) for board in boards: self.assertIn(board, s['boards']) @@ -391,26 +391,26 @@ def test_real_roster_max3421_selects_option_board(self): boards = on_roster(self, 'metro_m4_express') - s = ci_select.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, real_rosters()) + s = change_impact.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, real_rosters()) self.assertFalse(s['full']) for board in boards: self.assertIn(board, s['boards']) def test_option_selects_via_defines_and_flags(self): - s = ci_select.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, self.OPT_ROSTER) + s = change_impact.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, self.OPT_ROSTER) self.assertFalse(s['full']) self.assertIn('fake_dual_board', s['boards']) # variant defines self.assertIn('fake_host_board', s['boards']) # variant flags self.assertNotIn('fake_off_board', s['boards']) # variant defines, but =0 def test_device_role_port_does_not_pull_host_only_option_board(self): - s = ci_select.classify(['src/portable/analog/max3421/dcd_max3421.c'], REPO, self.OPT_ROSTER) + s = change_impact.classify(['src/portable/analog/max3421/dcd_max3421.c'], REPO, self.OPT_ROSTER) self.assertFalse(s['full']) self.assertNotIn('fake_host_board', s['boards']) # host-only board, device change self.assertIn('fake_dual_board', s['boards']) # device-capable option board def test_gates_parsed_from_family_support(self): - self.assertEqual(ci_select.port_option_gates(REPO).get('analog/max3421'), + self.assertEqual(change_impact.port_option_gates(REPO).get('analog/max3421'), {'MAX3421_HOST'}) def test_board_cmake_option_counts(self): @@ -418,12 +418,12 @@ (hw/bsp/espressif/boards/*/board.cmake -> set(MAX3421_HOST 1)); board_options() must see those too, or such a board joining the roster is silently dropped.""" self.assertIn('MAX3421_HOST', - ci_select.bsp_board_options('adafruit_feather_esp32s3', REPO)) + change_impact.bsp_board_options('adafruit_feather_esp32s3', REPO)) self.assertIn('CFG_TUH_RPI_PIO_USB', - ci_select.bsp_board_options('adafruit_fruit_jam', REPO)) + change_impact.bsp_board_options('adafruit_fruit_jam', REPO)) # commented-out `# set(MAX3421_HOST 1)` must not count self.assertNotIn('MAX3421_HOST', - ci_select.bsp_board_options('feather_nrf52840_express', REPO)) + change_impact.bsp_board_options('feather_nrf52840_express', REPO)) def test_board_cmake_option_selects_off_family_board(self): # adafruit_feather_esp32s3 is not on any rig roster; stand it in as one to @@ -431,7 +431,7 @@ roster = [('test/hil/opt.json', [ {'name': 'adafruit_feather_esp32s3', 'uid': 'o1', 'flasher': {'name': 'esptool'}, 'tests': {'device': False, 'host': True, 'dual': False}}])] - s = ci_select.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, roster) + s = change_impact.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, roster) self.assertFalse(s['full']) self.assertIn('adafruit_feather_esp32s3', s['boards']) @@ -441,7 +441,7 @@ roster = [('test/hil/opt.json', [ {'name': 'nrf5340dk', 'uid': 'o1', 'flasher': {'name': 'jlink'}, 'tests': {'device': False, 'host': True, 'dual': False}}])] - s = ci_select.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, roster) + s = change_impact.classify(['src/portable/analog/max3421/hcd_max3421.c'], REPO, roster) self.assertFalse(s['full']) self.assertEqual(s['boards'], {}) @@ -451,11 +451,11 @@ 'port_dir/' so a port dir is not a prefix of a sibling.""" def test_make_only_family_is_not_a_family(self): # hw/bsp/pic32mz has family.mk but no family.cmake - self.assertEqual(ci_select.port_families('microchip/pic32mz', REPO), set()) + self.assertEqual(change_impact.port_families('microchip/pic32mz', REPO), set()) def test_prefix_port_does_not_inherit_sibling_families(self): # bare-substring matching let 'microchip/pic' match '.../microchip/pic32mz/...' - self.assertEqual(ci_select.port_families('microchip/pic', REPO), set()) + self.assertEqual(change_impact.port_families('microchip/pic', REPO), set()) def test_make_only_port_contributes_nothing(self): s = sel(['src/portable/microchip/pic32mz/dcd_pic32mz.c']) @@ -464,8 +464,8 @@ self.assertTrue(any('no board family' in r for r in s['reasons']), s['reasons']) def test_cmake_families_still_found(self): - self.assertEqual(ci_select.port_families('raspberrypi/rp2040', REPO), {'rp2040'}) - self.assertIn('stm32f4', ci_select.port_families('synopsys/dwc2', REPO)) + self.assertEqual(change_impact.port_families('raspberrypi/rp2040', REPO), {'rp2040'}) + self.assertIn('stm32f4', change_impact.port_families('synopsys/dwc2', REPO)) class TestPortFamiliesCoverage(unittest.TestCase): @@ -511,7 +511,7 @@ for port in ports: if port in self.NO_FAMILY: continue - fams = ci_select.port_families(port, REPO) + fams = change_impact.port_families(port, REPO) self.assertTrue(fams, f'{port}: no family references this port ' f'(port_families() scan gap, or add to NO_FAMILY)') @@ -521,7 +521,7 @@ being invisible to the selector because it only knew the shared hil_util lists.""" def test_only_list_example_change_selects_it(self): boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') - s = ci_select.classify(['examples/device/hid_composite_freertos/src/main.c'], REPO, real_rosters()) + s = change_impact.classify(['examples/device/hid_composite_freertos/src/main.c'], REPO, real_rosters()) self.assertFalse(s['full']) for board in boards: self.assertEqual(s['boards'][board], ['device/hid_composite_freertos']) @@ -530,16 +530,16 @@ # cdc_msc_throughput is not a *_freertos example: espressif builds it only via # build.py's extra list, and selects it only via the roster only-list boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') - s = ci_select.classify(['examples/device/cdc_msc_throughput/src/main.c'], REPO, real_rosters()) + s = change_impact.classify(['examples/device/cdc_msc_throughput/src/main.c'], REPO, real_rosters()) self.assertFalse(s['full']) for board in boards: self.assertEqual(s['boards'][board], ['device/cdc_msc_throughput']) - self.assertIn('espressif', ci_select.classify_build( + self.assertIn('espressif', change_impact.classify_build( ['examples/device/cdc_msc_throughput/src/main.c'], REPO)['families']) def test_class_change_includes_only_list_boards(self): boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') - s = ci_select.classify(['src/class/hid/hid_device.c'], REPO, real_rosters()) + s = change_impact.classify(['src/class/hid/hid_device.c'], REPO, real_rosters()) self.assertFalse(s['full']) for board in boards: self.assertIn(board, s['boards']) @@ -552,7 +552,7 @@ (e.g. hid_composite_freertos) that aren't in the shared device_tests list.""" def test_dcd_change_includes_only_list_test(self): boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') - s = ci_select.classify(['src/portable/synopsys/dwc2/dcd_dwc2.c'], REPO, real_rosters()) + s = change_impact.classify(['src/portable/synopsys/dwc2/dcd_dwc2.c'], REPO, real_rosters()) self.assertFalse(s['full']) for board in boards: tests = s['boards'][board] @@ -565,7 +565,7 @@ def test_core_device_change_includes_only_list_test(self): boards = on_roster(self, 'espressif_s3_devkitm', 'espressif_p4_function_ev') - s = ci_select.classify(['src/device/usbd.c'], REPO, real_rosters()) + s = change_impact.classify(['src/device/usbd.c'], REPO, real_rosters()) self.assertFalse(s['full']) for board in boards: tests = s['boards'][board] @@ -577,7 +577,7 @@ self.assertIn('device/usbtest', tests) def test_host_change_does_not_leak_device_only_list_test(self): - s = ci_select.classify(['src/host/usbh.c'], REPO, real_rosters()) + s = change_impact.classify(['src/host/usbh.c'], REPO, real_rosters()) self.assertFalse(s['full']) for board, tests in s['boards'].items(): if tests == 'all': @@ -615,8 +615,8 @@ # full stays full: every roster board, and no args to narrow the run self.assertEqual(set(s['boards']), {b['name'] for b in ROSTER}) self.assertTrue(all(v == 'all' for v in s['boards'].values())) - self.assertEqual(ci_select.selection_args(s, ROSTERS), {'tinyusb.json': ''}) - self.assertEqual(ci_select.selection_args_by_flasher(s, ROSTERS), {'tinyusb.json': {}}) + self.assertEqual(change_impact.selection_args(s, ROSTERS), {'tinyusb.json': ''}) + self.assertEqual(change_impact.selection_args_by_flasher(s, ROSTERS), {'tinyusb.json': {}}) def test_family_order_does_not_matter(self): # same as above with the full-matrix file last (was the only order that worked) @@ -629,7 +629,7 @@ def test_diff_disables_rename_detection(self): """Without --no-renames git reports only a rename's destination, so moving an HIL-relevant file to a non-code path would be classified as non-code only.""" - self.assertIn('--no-renames', ci_select.GIT_DIFF_ARGV) + self.assertIn('--no-renames', change_impact.GIT_DIFF_ARGV) class TestPortWithoutFamilyContributesNothing(unittest.TestCase): @@ -638,13 +638,13 @@ nothing to run. Forcing the full 30-board rig here bought no coverage - the build walk answered the identical condition with zero families for the same path.""" def test_unreferenced_port_contributes_nothing(self): - orig = ci_select.port_families - ci_select.port_families = lambda port_dir, repo_root: set() + orig = change_impact.port_families + change_impact.port_families = lambda port_dir, repo_root: set() try: s = sel(['src/portable/vendor/newip/dcd_newip.c']) - b = ci_select.classify_build(['src/portable/vendor/newip/dcd_newip.c'], REPO) + b = change_impact.classify_build(['src/portable/vendor/newip/dcd_newip.c'], REPO) finally: - ci_select.port_families = orig + change_impact.port_families = orig self.assertFalse(s['full']) self.assertEqual(s['boards'], {}) self.assertTrue(any('no board family' in r for r in s['reasons']), s['reasons']) @@ -1016,42 +1016,42 @@ def test_repo_root_guard(self): # __file__-derived root: moving the module without re-deriving the parent # count re-points every scan at the wrong tree (it happened once already) - self.assertTrue(os.path.isdir(os.path.join(ci_select._REPO_ROOT, 'src'))) - self.assertTrue(os.path.isdir(os.path.join(ci_select._REPO_ROOT, 'hw', 'bsp'))) - self.assertEqual(os.path.realpath(ci_select._REPO_ROOT), os.path.realpath(REPO)) + self.assertTrue(os.path.isdir(os.path.join(change_impact._REPO_ROOT, 'src'))) + self.assertTrue(os.path.isdir(os.path.join(change_impact._REPO_ROOT, 'hw', 'bsp'))) + self.assertEqual(os.path.realpath(change_impact._REPO_ROOT), os.path.realpath(REPO)) class TestPathFamilies(unittest.TestCase): def test_port_wrapper_unchanged(self): - self.assertEqual(ci_select.port_families('raspberrypi/rp2040', REPO), {'rp2040'}) - self.assertIn('stm32f4', ci_select.port_families('synopsys/dwc2', REPO)) + self.assertEqual(change_impact.port_families('raspberrypi/rp2040', REPO), {'rp2040'}) + self.assertIn('stm32f4', change_impact.port_families('synopsys/dwc2', REPO)) def test_boundary_without_trailing_slash(self): # hw/bsp/nrf/family.cmake writes `${TOP}/hw/mcu/nordic/nrfx` — no trailing # slash; the match must accept a directory-boundary end-of-token - self.assertEqual(ci_select.path_families('hw/mcu/nordic/nrfx', REPO), {'nrf'}) + self.assertEqual(change_impact.path_families('hw/mcu/nordic/nrfx', REPO), {'nrf'}) def test_boundary_rejects_prefix_sibling(self): # 'microchip/pic' must not inherit pic32mz's references (and pic32mz itself # is family.mk-only, which the CMake-only scan never reads) - self.assertEqual(ci_select.port_families('microchip/pic', REPO), set()) - self.assertEqual(ci_select.port_families('microchip/pic32mz', REPO), set()) + self.assertEqual(change_impact.port_families('microchip/pic', REPO), set()) + self.assertEqual(change_impact.port_families('microchip/pic32mz', REPO), set()) def test_mcu_families_prefix_walk(self): - self.assertEqual(ci_select.mcu_families('hw/mcu/nordic/nrf5x/nrf_clock.h', REPO), {'nrf'}) - self.assertEqual(ci_select.mcu_families('hw/mcu/dialog/da1469x/x.h', REPO), {'da1469x'}) - self.assertEqual(ci_select.mcu_families('hw/mcu/no_such_vendor/x.c', REPO), set()) + self.assertEqual(change_impact.mcu_families('hw/mcu/nordic/nrf5x/nrf_clock.h', REPO), {'nrf'}) + self.assertEqual(change_impact.mcu_families('hw/mcu/dialog/da1469x/x.h', REPO), {'da1469x'}) + self.assertEqual(change_impact.mcu_families('hw/mcu/no_such_vendor/x.c', REPO), set()) class TestMcuHilRule(unittest.TestCase): def test_mcu_no_longer_forces_full(self): - s = ci_select.classify(['hw/mcu/nordic/nrf5x/nrf_clock.h'], REPO, ROSTERS) + s = change_impact.classify(['hw/mcu/nordic/nrf5x/nrf_clock.h'], REPO, ROSTERS) self.assertFalse(s['full']) self.assertIn('nrf', s['families']) # recorded even with no nrf rig board def test_mcu_selects_family_boards(self): got = on_roster(self, 'feather_nrf52840_express', 'pca10056', 'pca10095') - s = ci_select.classify(['hw/mcu/nordic/nrf5x/nrf_clock.h'], REPO, real_rosters()) + s = change_impact.classify(['hw/mcu/nordic/nrf5x/nrf_clock.h'], REPO, real_rosters()) self.assertFalse(s['full']) for b in got: self.assertIn(b, s['boards']) @@ -1060,7 +1060,7 @@ # empty means empty (maintainer ruling): if no family's build references the # path, no build consumes the change - there is nothing to compile or run. # test_tracked_mcu_vendors_resolve is the drift guard for a real vendor dir - s = ci_select.classify(['hw/mcu/no_such_vendor/x.c'], REPO, ROSTERS) + s = change_impact.classify(['hw/mcu/no_such_vendor/x.c'], REPO, ROSTERS) self.assertFalse(s['full']) self.assertEqual(s['boards'], {}) self.assertEqual(s['families'], []) @@ -1074,7 +1074,7 @@ if not os.path.isdir(d): continue port = os.path.relpath(d, os.path.join(REPO, 'src/portable')).replace(os.sep, '/') - fams = ci_select.port_families(port, REPO) + fams = change_impact.port_families(port, REPO) if port in self.ALLOW: self.assertEqual(fams, set(), f'{port}: no longer an orphan - drop it from ALLOW') else: @@ -1087,7 +1087,7 @@ self.skipTest('not a git checkout') vendors = sorted({'/'.join(p.split('/')[:3]) for p in r.stdout.split()}) for v in vendors: - self.assertTrue(ci_select.mcu_families(v + '/x.c', REPO), f'{v}: resolves to no family') + self.assertTrue(change_impact.mcu_families(v + '/x.c', REPO), f'{v}: resolves to no family') # hw/bsp families absent from ci_set_matrix's family_list, so a PR touching one # gets no compile coverage (ci_set_matrix treats a selection that intersects @@ -1099,7 +1099,7 @@ def test_every_bsp_family_is_in_the_ci_matrix(self): sys.path.insert(0, os.path.join(REPO, '.github/scripts')) import ci_set_matrix - fams = set(ci_select.all_bsp_families(REPO)) + fams = set(change_impact.all_bsp_families(REPO)) self.assertEqual(fams - set(ci_set_matrix.family_list), self.UNBUILT_FAMILIES, 'a hw/bsp family that no toolchain in ci_set_matrix.family_list ' 'builds: a PR touching only it now selects zero build legs. Wire ' @@ -1112,7 +1112,7 @@ appearing is a real bug in get_deps.py, not something to swallow.""" sys.path.insert(0, os.path.join(REPO, 'tools')) import get_deps - fams = set(ci_select.all_bsp_families(REPO)) + fams = set(change_impact.all_bsp_families(REPO)) stale = {} for name, d in (('deps_mandatory', get_deps.deps_mandatory), ('deps_optional', get_deps.deps_optional)): @@ -1123,9 +1123,9 @@ # subset, not equality: correcting a token in get_deps.py (fc100s -> f1c100s) # should be a one-file change, while a NEW unmappable token - which force-fulls # every get_deps edit that touches its entry - has to be a deliberate act - self.assertFalse(set(stale) - set(ci_select._DEPS_ALIAS_TOKENS), + self.assertFalse(set(stale) - set(change_impact._DEPS_ALIAS_TOKENS), f'get_deps family tokens naming no hw/bsp dir: ' - f'{ {k: v for k, v in stale.items() if k not in ci_select._DEPS_ALIAS_TOKENS} }') + f'{ {k: v for k, v in stale.items() if k not in change_impact._DEPS_ALIAS_TOKENS} }') class TestRostersDoNotOverlap(unittest.TestCase): @@ -1149,7 +1149,7 @@ and force-fulled 82 families and all 30 rig boards.""" def test_build_axis_selects_only_the_typec_examples(self): - s = ci_select.classify_build(['src/typec/usbc.c'], REPO) + s = change_impact.classify_build(['src/typec/usbc.c'], REPO) self.assertFalse(s['full']) self.assertTrue(s['families'], 'typec must be compiled somewhere') self.assertTrue(s['family_examples'], 'and the examples must be named') @@ -1161,7 +1161,7 @@ def test_every_typec_file_answers_the_same(self): for f in ('src/typec/usbc.c', 'src/typec/usbc.h', 'src/typec/tcd.h', 'src/typec/pd_types.h'): - s = ci_select.classify_build([f], REPO) + s = change_impact.classify_build([f], REPO) self.assertFalse(s['full'], f) self.assertTrue(s['families'], f) @@ -1174,17 +1174,17 @@ def test_it_tracks_the_enabling_config_rather_than_a_hardcoded_list(self): # the answer must come from CFG_TUC_ENABLED in the example configs, so it # follows a new typec example (or an old one switched off) on its own - want = ci_select.examples_enabling( - ci_select.role_examples(REPO, ('typec',)), ('CFG_TUC_ENABLED',), REPO) + want = change_impact.examples_enabling( + change_impact.role_examples(REPO, ('typec',)), ('CFG_TUC_ENABLED',), REPO) self.assertTrue(want, 'no example enables CFG_TUC_ENABLED - rule 12b is dead') got = set() - for exs in ci_select.classify_build(['src/typec/usbc.c'], REPO)['family_examples'].values(): + for exs in change_impact.classify_build(['src/typec/usbc.c'], REPO)['family_examples'].values(): got |= set(exs) self.assertEqual(got, want) class TestCachesAreKeyedOnTheTree(unittest.TestCase): - """build_utils caches on repo-RELATIVE paths while ci_select._in_repo() chdirs + """build_utils caches on repo-RELATIVE paths while change_impact._in_repo() chdirs between trees, so the cwd has to be part of every cache key. Without it a second tree gets the first tree's skip.txt/only.txt and FAMILY_MCUS.""" @@ -1227,8 +1227,8 @@ cls = os.path.basename(d) hit = False for base in sorted(os.path.basename(f) for f in _glob.glob(os.path.join(d, '*.[ch]'))): - roles = ci_select._class_roles(base) - if ci_select._build_class_examples(cls, base, roles, REPO): + roles = change_impact._class_roles(base) + if change_impact._build_class_examples(cls, base, roles, REPO): hit = True break if not hit: @@ -1244,19 +1244,19 @@ Rule 2 is a bare `test/hil/` prefix, so the harness's own unit tests were booking the full 27-board rig - ~11 minutes of exclusive hardware for a diff that cannot reach it. Nothing on the rig runs them: pre-commit does, and build.yml runs - test_ci_select.py as the gate before trusting a selection at all. + test_change_impact.py as the gate before trusting a selection at all. The carve-out is only safe while that directory holds nothing rig-affecting, which is what the second test pins.""" def test_the_harness_own_tests_select_nothing_on_either_axis(self): - for p in ('test/hil/test/test_ci_select.py', 'test/hil/test/test_ci_metrics.py', + for p in ('test/hil/test/test_change_impact.py', 'test/hil/test/test_ci_metrics.py', 'test/hil/test/test_hil_bounded.py', 'test/hil/test/stubs/pymtp.py', 'test/hil/test/stubs/hid.py', 'test/hil/test/usbtest_harness.py'): - s = ci_select.classify([p], REPO, ROSTERS) + s = change_impact.classify([p], REPO, ROSTERS) self.assertFalse(s['full'], p) self.assertFalse(s['boards'], p) - b = ci_select.classify_build([p], REPO) + b = change_impact.classify_build([p], REPO) self.assertFalse(b['full'], p) self.assertFalse(b['families'], p) @@ -1265,7 +1265,7 @@ # trusted to narrow their own blast radius for p in ('test/hil/hil_test.py', 'test/hil/tinyusb.json', 'test/hil/helper/hil_ci_set_matrix.py'): - s = ci_select.classify([p], REPO, ROSTERS) + s = change_impact.classify([p], REPO, ROSTERS) self.assertTrue(s['full'], f'{p} must still force the full rig') def test_nothing_rig_affecting_has_moved_into_the_carve_out(self): @@ -1278,9 +1278,9 @@ self.assertEqual(sorted(out.stdout.split()), [ 'test/hil/test/stubs/hid.py', 'test/hil/test/stubs/pymtp.py', + 'test/hil/test/test_change_impact.py', 'test/hil/test/test_ci_boards.py', 'test/hil/test/test_ci_metrics.py', - 'test/hil/test/test_ci_select.py', 'test/hil/test/test_code_size.py', 'test/hil/test/test_code_size_ci.py', 'test/hil/test/test_drivers_coverage.py', @@ -1319,7 +1319,7 @@ def test_a_device_only_port_diff_still_omits_families_it_cannot_narrow(self): # dcd_dwc2.c selects device+dual examples only, but a family whose host examples # are all unbuildable anyway ends up wanting its entire buildable set - b = ci_select.classify_build(['src/portable/synopsys/dwc2/dcd_dwc2.c'], REPO) + b = change_impact.classify_build(['src/portable/synopsys/dwc2/dcd_dwc2.c'], REPO) self.assertFalse(b['full']) self.assertTrue(b['families']) omitted = [f for f in b['families'] if f not in b['family_examples']] @@ -1331,7 +1331,7 @@ def test_a_family_that_can_build_more_than_the_diff_wants_keeps_its_list(self): # the other direction: one example selects itself and nothing else, so every # family it lands on must carry an explicit -e or CI builds all 46 - b = ci_select.classify_build(['examples/device/cdc_msc/src/main.c'], REPO) + b = change_impact.classify_build(['examples/device/cdc_msc/src/main.c'], REPO) self.assertFalse(b['full']) for fam in b['families']: self.assertEqual(b['family_examples'].get(fam), ['device/cdc_msc'], fam) @@ -1393,32 +1393,28 @@ # mutant: delete the _CLS_STEM_RE block. src/class/midi holds MIDI 1.0 AND 2.0; # examples/device/midi2_device is the only example enabling CFG_TUD_MIDI2 and the # only one that compiles midi2_device.c, but the directory macro alone misses it. - got = ci_select._build_class_examples('midi', 'midi2_device.c', {'device'}, REPO) + got = change_impact._build_class_examples('midi', 'midi2_device.c', {'device'}, REPO) self.assertIn('device/midi2_device', got, 'a midi2 change must select the example that compiles it') - host = ci_select._build_class_examples('midi', 'midi2_host.c', {'host'}, REPO) + host = change_impact._build_class_examples('midi', 'midi2_host.c', {'host'}, REPO) self.assertIn('host/midi2_host', host) # and the plain midi files must NOT drag midi2 in - plain = ci_select._build_class_examples('midi', 'midi_device.c', {'device'}, REPO) + plain = change_impact._build_class_examples('midi', 'midi_device.c', {'device'}, REPO) self.assertNotIn('device/midi2_device', plain) def test_a_port_change_selects_the_dual_examples(self): # mutant: drop `+ ('dual',)`. A dcd/hcd change must build the dual examples - # they exercise both stacks on one board, so a dwc2 break lands there first. - s = ci_select.classify_build(['src/portable/synopsys/dwc2/dcd_dwc2.c'], REPO) + s = change_impact.classify_build(['src/portable/synopsys/dwc2/dcd_dwc2.c'], REPO) duals = {e for exs in s['family_examples'].values() for e in exs if e.startswith('dual/')} self.assertTrue(duals, 'a dcd change selected no dual example') -class TestRuleTableIsCarbonOfTheSpec(unittest.TestCase): - """ci_select's module docstring carries the rule table so a reader landing in the - code does not have to open the spec to learn what rule 6 is. Both are maintained by - hand, so this pins them cell-for-cell: edit one without the other and this fails. - - It also pins the table against the CODE - every rule id the docstring claims must - appear as a `# rule N` marker on a branch of _classify_build_one, so a row cannot be - documented without a branch, or a branch renumbered without the table.""" +class TestRuleTableMatchesTheCode(unittest.TestCase): + """The rule table lives in change_impact's module docstring. Every rule id it + claims must appear as a `# rule N` marker on a branch of _classify_build_one, so a row + cannot be documented without a branch, or a branch renumbered without the table.""" @staticmethod def _rows(text): @@ -1432,18 +1428,9 @@ out.append(c) return out - def test_docstring_table_matches_the_spec(self): - spec = _read(os.path.join(REPO, 'docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md')) - doc, spec_rows = self._rows(ci_select.__doc__), self._rows(spec) - self.assertTrue(spec_rows, 'no rule table found in the spec') - self.assertEqual([r[0] for r in doc], [r[0] for r in spec_rows], - 'rule ids differ between ci_select.__doc__ and the spec') - for d, s in zip(doc, spec_rows): - self.assertEqual(d, s, f'rule {d[0]} differs between the docstring and the spec') - def test_every_documented_rule_has_a_branch(self): import re as _re - src = _read(os.path.join(REPO, 'tools/ci_select.py')) + src = _read(os.path.join(REPO, 'tools/change_impact.py')) marked = set() # handles `# rule 6`, `# rules 1, 1b` and `# rules 8-10` for m in _re.finditer(r'#\s*rules?\s+([0-9a-z, -]+)', src): @@ -1453,7 +1440,8 @@ marked.update(str(n) for n in range(int(rng.group(1)), int(rng.group(2)) + 1)) elif _re.fullmatch(r'\d+[a-z]?', tok.strip()): marked.add(tok.strip()) - documented = {r[0] for r in self._rows(ci_select.__doc__)} + documented = {r[0] for r in self._rows(change_impact.__doc__)} + self.assertTrue(documented, 'no rule table found in the module docstring') missing = sorted(documented - marked, key=lambda s: (int(_re.match(r'\d+', s).group()), s)) self.assertEqual(missing, [], f'documented rules with no `# rule N` branch marker: {missing}') @@ -1479,8 +1467,8 @@ self.assertGreater(len(files), 1000, 'suspiciously few tracked files') out = [] for f in files: - s = (ci_select.classify_build([f], REPO) if axis == 'build' - else ci_select.classify([f], REPO, real_rosters())) + s = (change_impact.classify_build([f], REPO) if axis == 'build' + else change_impact.classify([f], REPO, real_rosters())) if any('unclassified' in why for why in s['reasons']): out.append(f) return out @@ -1501,21 +1489,21 @@ """lib/** is not a full-matrix path: only the examples that build the lib need it.""" def b(self, files): - return ci_select.classify_build(files, REPO) + return change_impact.classify_build(files, REPO) def test_lib_examples_ground_truth(self): - self.assertEqual(ci_select.lib_examples('embedded-cli', REPO), + self.assertEqual(change_impact.lib_examples('embedded-cli', REPO), {'host/msc_file_explorer', 'host/msc_file_explorer_freertos'}) - self.assertEqual(ci_select.lib_examples('networking', REPO), + self.assertEqual(change_impact.lib_examples('networking', REPO), {'device/net_lwip_webserver'}) # only family_support.cmake's LOGGER=rtt plumbing names it, and no CI example # build turns that on - the scan is per-example on purpose - self.assertEqual(ci_select.lib_examples('SEGGER_RTT', REPO), set()) - self.assertEqual(ci_select.lib_examples('rt-thread', REPO), set()) + self.assertEqual(change_impact.lib_examples('SEGGER_RTT', REPO), set()) + self.assertEqual(change_impact.lib_examples('rt-thread', REPO), set()) def test_lib_examples_matches_at_a_directory_boundary(self): # 'lib/net' must not inherit lib/networking's example - self.assertEqual(ci_select.lib_examples('net', REPO), set()) + self.assertEqual(change_impact.lib_examples('net', REPO), set()) def test_build_lib_selects_only_the_using_examples(self): s = self.b(['lib/embedded-cli/embedded_cli.h']) @@ -1583,7 +1571,7 @@ """Pure text-in, families-out: no git, no exec of the parsed module.""" def f(self, head, base=_GD_BASE): - return ci_select.get_deps_changed_families(base, head, REPO) + return change_impact.get_deps_changed_families(base, head, REPO) def test_no_change_selects_nothing(self): self.assertEqual(self.f(_GD_BASE), set()) @@ -1657,7 +1645,7 @@ def test_real_get_deps_parses(self): with open(os.path.join(REPO, 'tools/get_deps.py')) as f: real = f.read() - self.assertEqual(ci_select.get_deps_changed_families(real, real, REPO), set()) + self.assertEqual(change_impact.get_deps_changed_families(real, real, REPO), set()) # a real optional entry bumped resolves to that entry's real family. The commit # is read out of get_deps.py rather than pinned here - a routine dep bump must # not fail this suite, and pinning a hash tests the tree, not the code @@ -1666,7 +1654,7 @@ commit, tokens = get_deps.deps_optional['hw/mcu/nordic/nrfx'][1:3] bumped = real.replace(commit, '0' * len(commit)) self.assertNotEqual(bumped, real) - self.assertEqual(ci_select.get_deps_changed_families(real, bumped, REPO), + self.assertEqual(change_impact.get_deps_changed_families(real, bumped, REPO), set(tokens.split())) @@ -1674,7 +1662,7 @@ """tools/get_deps.py: the changed dep entries' families, or full when unknowable.""" def test_build_selects_the_changed_families(self): - s = ci_select.classify_build(['tools/get_deps.py'], REPO, + s = change_impact.classify_build(['tools/get_deps.py'], REPO, get_deps_families={'stm32f4'}) self.assertFalse(s['full']) self.assertEqual(s['families'], ['stm32f4']) @@ -1682,25 +1670,25 @@ def test_build_without_a_base_is_full(self): # --diff-file mode has no git and so no base content: fail open - self.assertTrue(ci_select.classify_build(['tools/get_deps.py'], REPO)['full']) + self.assertTrue(change_impact.classify_build(['tools/get_deps.py'], REPO)['full']) def test_build_no_dep_entry_changed_selects_nothing(self): - s = ci_select.classify_build(['tools/get_deps.py'], REPO, get_deps_families=set()) + s = change_impact.classify_build(['tools/get_deps.py'], REPO, get_deps_families=set()) self.assertFalse(s['full']) self.assertEqual(s['families'], []) def test_hil_selects_the_changed_families_boards(self): - s = ci_select.classify(['tools/get_deps.py'], REPO, ROSTERS, + s = change_impact.classify(['tools/get_deps.py'], REPO, ROSTERS, get_deps_families={'stm32f4'}) self.assertFalse(s['full']) self.assertEqual(list(s['boards']), ['stm32f407disco']) self.assertEqual(s['families'], ['stm32f4']) def test_hil_without_a_base_is_full(self): - self.assertTrue(ci_select.classify(['tools/get_deps.py'], REPO, ROSTERS)['full']) + self.assertTrue(change_impact.classify(['tools/get_deps.py'], REPO, ROSTERS)['full']) def test_hil_no_dep_entry_changed_selects_nothing(self): - s = ci_select.classify(['tools/get_deps.py'], REPO, ROSTERS, get_deps_families=set()) + s = change_impact.classify(['tools/get_deps.py'], REPO, ROSTERS, get_deps_families=set()) self.assertFalse(s['full']) self.assertEqual(s['boards'], {}) @@ -1709,7 +1697,7 @@ with tempfile.NamedTemporaryFile('w', suffix='.txt', delete=False) as f: f.write('tools/get_deps.py\n') path = f.name - r = subprocess.run([sys.executable, os.path.join(REPO, 'tools/ci_select.py'), + r = subprocess.run([sys.executable, os.path.join(REPO, 'tools/change_impact.py'), '--diff-file', path, os.path.join(REPO, 'test/hil/tinyusb.json')], capture_output=True, text=True) os.unlink(path) @@ -1736,7 +1724,7 @@ elif argv[:2] == ['git', 'merge-base']: self.assertEqual(argv[2:], ['HEADSHA', 'BASESHA']) out = 'MB123\n' - elif argv[:3] == ci_select.GIT_DIFF_ARGV[:3]: + elif argv[:3] == change_impact.GIT_DIFF_ARGV[:3]: out = diff elif argv[:2] == ['git', 'show']: out = _GD_BASE if argv[2].startswith('MB123:') else self.HEAD @@ -1746,10 +1734,10 @@ buf = io.StringIO() argv = [sys.executable, '--base', 'origin/master'] - with mock.patch.object(ci_select.subprocess, 'run', fake_run), \ + with mock.patch.object(change_impact.subprocess, 'run', fake_run), \ mock.patch.object(sys, 'argv', argv), \ contextlib.redirect_stdout(buf), contextlib.redirect_stderr(io.StringIO()): - ci_select.main() + change_impact.main() return json.loads(buf.getvalue()), calls def test_base_mode_reads_the_merge_base_blob(self): @@ -1775,16 +1763,16 @@ return subprocess.CompletedProcess(argv, 0, stdout=out, stderr='') buf = io.StringIO() - with mock.patch.object(ci_select.subprocess, 'run', fake_run), \ + with mock.patch.object(change_impact.subprocess, 'run', fake_run), \ mock.patch.object(sys, 'argv', [sys.executable, '--base', 'origin/master']), \ contextlib.redirect_stdout(buf), contextlib.redirect_stderr(io.StringIO()): - ci_select.main() + change_impact.main() self.assertTrue(json.loads(buf.getvalue())['build']['full']) class TestBuildClassifier(unittest.TestCase): def b(self, files): - return ci_select.classify_build(files, REPO) + return change_impact.classify_build(files, REPO) def test_noncode_and_test_hil_contribute_nothing(self): # rules 1, 2 s = self.b(['docs/info/index.rst', 'README.rst', 'test/hil/hil_test.py', '.claude/skills/hil/SKILL.md']) @@ -1803,8 +1791,8 @@ # survives Task 4's buildability pruning depends on the environment-gated # CI board pick, so the classifier-output assertion must not rely on it self.assertIn('dual/host_info_to_device_cdc', - ci_select.role_examples(REPO, ('device', 'dual'))) - self.assertNotIn('host/bare_api', ci_select.role_examples(REPO, ('device', 'dual'))) + change_impact.role_examples(REPO, ('device', 'dual'))) + self.assertNotIn('host/bare_api', change_impact.role_examples(REPO, ('device', 'dual'))) def test_port_host_rule(self): # rule 4 s = self.b(['src/portable/analog/max3421/hcd_max3421.c']) @@ -1923,7 +1911,7 @@ def test_the_build_machinery_is_still_full(self): # the other side of the same line: these DECIDE what gets built for p in ('.circleci/config.yml', '.github/workflows/build.yml', - '.github/scripts/ci_set_matrix.py', 'tools/ci_select.py', + '.github/scripts/ci_set_matrix.py', 'tools/change_impact.py', 'tools/build_utils.py', '.github/ci-pinned-boards.json'): self.assertTrue(self.b([p])['full'], p) @@ -1936,7 +1924,7 @@ self.assertNotIn('device/hid_composite', s['family_examples']['stm32f4']) # cdc-only there def test_example_names_are_real_dirs(self): - for ex in ci_select.all_examples(REPO): + for ex in change_impact.all_examples(REPO): role, name = ex.split('/') self.assertTrue(os.path.isdir(os.path.join(REPO, 'examples', role, name)), ex) self.assertRegex(ex, r'^(device|dual|host|typec)/[A-Za-z0-9_]+$') @@ -1945,7 +1933,7 @@ class TestBuildPostFilter(unittest.TestCase): def test_kept_examples_are_buildable(self): import build_utils, build as build_py - s = ci_select.classify_build(['src/class/msc/msc_host.c'], REPO) + s = change_impact.classify_build(['src/class/msc/msc_host.c'], REPO) self.assertFalse(s['full']) # families that cannot build a single TUH_MSC example drop out entirely self.assertNotIn('msp430', s['families']) @@ -1962,7 +1950,7 @@ os.chdir(old) def test_unfiltered_family_has_no_map_key(self): - s = ci_select.classify_build(['hw/bsp/stm32f4/family.c'], REPO) + s = change_impact.classify_build(['hw/bsp/stm32f4/family.c'], REPO) self.assertEqual(s['families'], ['stm32f4']) self.assertEqual(s['family_examples'], {}) @@ -1970,19 +1958,19 @@ # build.py's espressif branch builds get_examples('espressif') only (the # examples with an ESP-IDF component), so keeping espressif for a # device/cdc_msc diff spins CircleCI's most expensive leg up to skip everything - s = ci_select.classify_build(['examples/device/cdc_msc/src/main.c'], REPO) + s = change_impact.classify_build(['examples/device/cdc_msc/src/main.c'], REPO) self.assertFalse(s['full']) self.assertNotIn('espressif', s['families']) def test_espressif_survives_an_example_it_does_build(self): - s = ci_select.classify_build(['examples/device/cdc_msc_freertos/src/main.c'], REPO) + s = change_impact.classify_build(['examples/device/cdc_msc_freertos/src/main.c'], REPO) self.assertFalse(s['full']) self.assertIn('espressif', s['families']) def test_ra_survives_the_dual_example_prune(self): # ra's only buildable dual example is gated on only.txt's mcu:ra6m5, which # exists only if the ${MCU_VARIANT} token in FAMILY_MCUS resolves - s = ci_select.classify_build( + s = change_impact.classify_build( ['examples/dual/host_info_to_device_cdc/src/main.c'], REPO) self.assertFalse(s['full']) self.assertIn('ra', s['families'], s['families']) @@ -1990,7 +1978,7 @@ def test_deleted_family_dir_does_not_crash(self): # rule 6 extracts a family from the path; a PR that deletes or renames # hw/bsp/<fam> used to traceback in get_family_boards' scandir - s = ci_select.classify_build(['hw/bsp/no_such_family_xyz/family.cmake'], REPO) + s = change_impact.classify_build(['hw/bsp/no_such_family_xyz/family.cmake'], REPO) self.assertFalse(s['full']) self.assertEqual(s['families'], []) self.assertTrue(any('gone from tree' in r for r in s['reasons']), s['reasons']) @@ -2007,7 +1995,7 @@ # path reached the same branch and the test passed vacuously. real = os.path.join(REPO, 'src/class/bth/bth_device.c') self.assertTrue(os.path.isfile(real), 'the case needs a file that exists') - s = ci_select.classify_build(['src/class/bth/bth_device.c'], REPO) + s = change_impact.classify_build(['src/class/bth/bth_device.c'], REPO) self.assertFalse(s['full']) self.assertEqual(s['families'], []) self.assertTrue(any('no contribution' in r for r in s['reasons']), s['reasons']) @@ -2015,7 +2003,7 @@ self.assertTrue(any('bth' in r for r in s['reasons']), s['reasons']) def test_class_source_with_examples_still_scopes(self): - s = ci_select.classify_build(['src/class/cdc/cdc_device.c'], REPO) + s = change_impact.classify_build(['src/class/cdc/cdc_device.c'], REPO) self.assertFalse(s['full']) def test_no_stdout_pollution(self): @@ -2023,7 +2011,7 @@ import io, contextlib buf = io.StringIO() with contextlib.redirect_stdout(buf): - ci_select.classify_build(['src/class/msc/msc_host.c'], REPO) + change_impact.classify_build(['src/class/msc/msc_host.c'], REPO) self.assertEqual(buf.getvalue(), '') @@ -2038,7 +2026,7 @@ h = sel([p]) self.assertFalse(h['full'], p) self.assertEqual(h['boards'], {}, p) - b = ci_select.classify_build([p], REPO) + b = change_impact.classify_build([p], REPO) self.assertFalse(b['full'], p) self.assertEqual(b['families'], [], p) @@ -2048,7 +2036,7 @@ h = sel([p]) self.assertFalse(h['full'], p) self.assertEqual(h['boards'], {}, p) - self.assertTrue(ci_select.classify_build([p], REPO)['full'], p) + self.assertTrue(change_impact.classify_build([p], REPO)['full'], p) def test_ci_size_scripts_are_a_full_build_matrix_without_hil(self): # run by every pinned build leg (rule 2d): membrowse_cli.py from @@ -2057,7 +2045,7 @@ h = sel([p]) self.assertFalse(h['full'], p) self.assertEqual(h['boards'], {}, p) - self.assertTrue(ci_select.classify_build([p], REPO)['full'], p) + self.assertTrue(change_impact.classify_build([p], REPO)['full'], p) def test_typec_example_builds_but_runs_nothing(self): # examples/typec is compiled by the build matrix and run by no rig board; the @@ -2066,28 +2054,28 @@ h = sel([p]) self.assertFalse(h['full']) self.assertEqual(h['boards'], {}) - b = ci_select.classify_build([p], REPO) + b = change_impact.classify_build([p], REPO) self.assertFalse(b['full']) self.assertTrue(b['families'], 'typec still has to be compiled somewhere') class TestHilExamples(unittest.TestCase): def test_board_test_always_present_and_full_emits(self): - s = ci_select.classify(['src/common/tusb_fifo.c'], REPO, ROSTERS) # full - he = ci_select.hil_examples(s, ROSTERS) + s = change_impact.classify(['src/common/tusb_fifo.c'], REPO, ROSTERS) # full + he = change_impact.hil_examples(s, ROSTERS) self.assertEqual(set(he), {b['name'] for b in ROSTER}) for name, exs in he.items(): self.assertIn('device/board_test', exs) def test_narrowed_board_gets_chosen_tests_only(self): - s = ci_select.classify(['examples/device/cdc_msc/src/main.c'], REPO, ROSTERS) - he = ci_select.hil_examples(s, ROSTERS) + s = change_impact.classify(['examples/device/cdc_msc/src/main.c'], REPO, ROSTERS) + he = change_impact.hil_examples(s, ROSTERS) self.assertEqual(he['stm32f407disco'], ['device/board_test', 'device/cdc_msc']) def test_full_board_gets_its_whole_test_list(self): - s = ci_select.classify(['hw/bsp/stm32f4/boards/stm32f407disco/board.h'], REPO, ROSTERS) - he = ci_select.hil_examples(s, ROSTERS) - want = set(ci_select.board_tests(ROSTER[1])) | {'device/board_test'} + s = change_impact.classify(['hw/bsp/stm32f4/boards/stm32f407disco/board.h'], REPO, ROSTERS) + he = change_impact.hil_examples(s, ROSTERS) + want = set(change_impact.board_tests(ROSTER[1])) | {'device/board_test'} self.assertEqual(set(he['stm32f407disco']), want) self.assertNotIn('raspberry_pi_pico', he) # deselected board: no firmware needed @@ -2097,14 +2085,14 @@ enables: host/bare_api includes class/hid/hid.h with no CFG_TUH_HID.""" def test_build_selects_the_direct_includer(self): - s = ci_select.classify_build(['src/class/hid/hid.h'], REPO) + s = change_impact.classify_build(['src/class/hid/hid.h'], REPO) self.assertFalse(s['full']) self.assertTrue(any('host/bare_api' in exs for exs in s['family_examples'].values())) def test_hil_runs_a_direct_includer_a_roster_lists(self): roster = [{'name': 'raspberry_pi_pico', 'uid': 'u1', 'flasher': {'name': 'openocd'}, 'tests': {'only': ['host/bare_api', 'device/cdc_msc']}}] - s = ci_select.classify(['src/class/hid/hid.h'], REPO, [('test/hil/x.json', roster)]) + s = change_impact.classify(['src/class/hid/hid.h'], REPO, [('test/hil/x.json', roster)]) self.assertEqual(s['boards'].get('raspberry_pi_pico'), ['host/bare_api']) @@ -2121,7 +2109,7 @@ ] def test_duplicate_board_unions_the_test_lists(self): - he = ci_select.hil_examples({'full': True, 'boards': {}}, self.ROSTERS) + he = change_impact.hil_examples({'full': True, 'boards': {}}, self.ROSTERS) self.assertEqual(he['dup_board'], ['device/board_test', 'device/cdc_msc', 'device/hid_boot_interface']) @@ -2129,7 +2117,7 @@ class TestCliJson(unittest.TestCase): def test_build_key_without_rosters(self): - r = subprocess.run([sys.executable, os.path.join(REPO, 'tools/ci_select.py'), + r = subprocess.run([sys.executable, os.path.join(REPO, 'tools/change_impact.py'), '--diff-file', '/dev/null'], capture_output=True, text=True) self.assertEqual(r.returncode, 0, r.stderr) j = json.loads(r.stdout) @@ -2141,7 +2129,7 @@ with tempfile.NamedTemporaryFile('w', suffix='.txt', delete=False) as f: f.write('src/portable/raspberrypi/rp2040/dcd_rp2040.c\n') df = f.name - r = subprocess.run([sys.executable, os.path.join(REPO, 'tools/ci_select.py'), + r = subprocess.run([sys.executable, os.path.join(REPO, 'tools/change_impact.py'), '--diff-file', df, os.path.join(REPO, 'test/hil/tinyusb.json')], capture_output=True, text=True) os.unlink(df) @@ -2403,7 +2391,7 @@ for name in ('.github', 'hw', 'test'): os.symlink(os.path.join(REPO, name), os.path.join(d, name)) os.mkdir(os.path.join(d, 'tools')) - with open(os.path.join(d, 'tools', 'ci_select.py'), 'w') as f: + with open(os.path.join(d, 'tools', 'change_impact.py'), 'w') as f: f.write('raise ImportError("broken selector")\n') for script, args in ((SET_MATRIX, ['--pinned']), (SET_MATRIX, ['--required']), (HIL_SET_MATRIX, [os.path.join(REPO, 'test/hil/tinyusb.json')])): @@ -2503,7 +2491,7 @@ sel = mf(['stm32f4'], boards={'stm32f407disco': 'all'}, required=['stm32f407disco'], hil_examples={'stm32f407disco': ['device/cdc_msc']}) sel['build']['families']['stm32f4'] = {'examples': ['device/cdc_msc']} - self.assertIs(ci_select.check_manifest(sel), sel) + self.assertIs(change_impact.check_manifest(sel), sel) def test_malformed_or_contradictory_manifests_are_refused(self): def bad(edit): @@ -2531,18 +2519,18 @@ } for name, m in cases.items(): with self.subTest(name), self.assertRaises(ValueError): - ci_select.check_manifest(m) + change_impact.check_manifest(m) def test_build_gate(self): gap = {'path': 'src/class/bth/bth_device.c', 'effect': 'gap'} doc = {'path': 'README.rst', 'effect': 'none'} - self.assertFalse(ci_select.build_gate(mf(paths=[doc]))) - self.assertTrue(ci_select.build_gate(mf(paths=[doc, gap]))) # no build compiles it: keep the net - self.assertTrue(ci_select.build_gate(mf(['stm32f4']))) - self.assertTrue(ci_select.build_gate(mf(boards={'stm32f407disco': 'all'}))) - self.assertTrue(ci_select.build_gate(mf(hil_full=True))) + self.assertFalse(change_impact.build_gate(mf(paths=[doc]))) + self.assertTrue(change_impact.build_gate(mf(paths=[doc, gap]))) # no build compiles it: keep the net + self.assertTrue(change_impact.build_gate(mf(['stm32f4']))) + self.assertTrue(change_impact.build_gate(mf(boards={'stm32f407disco': 'all'}))) + self.assertTrue(change_impact.build_gate(mf(hil_full=True))) with self.assertRaises(ValueError): - ci_select.build_gate({'full': True}) + change_impact.build_gate({'full': True}) HIL_SET_MATRIX = os.path.join(REPO, '.github/scripts/hil_ci_set_matrix.py') @@ -2813,7 +2801,7 @@ # the post-configure check still reports both as covered. No collision today, # and the -e lists are machine-generated, so nothing else would notice one. seen = {} - for ex in ci_select.all_examples(REPO): + for ex in change_impact.all_examples(REPO): role, name = ex.split('/', 1) self.assertNotIn(name, seen, f'{ex} and {seen.get(name)}/{name} share a cmake target name; ' @@ -3154,7 +3142,7 @@ def test_identifier_value_is_enabled(self): # examples/host/midi_rx: `#define CFG_TUH_MIDI CFG_TUH_DEVICE_MAX` cfg = os.path.join(REPO, 'examples/host/midi_rx/src/tusb_config.h') - self.assertTrue(ci_select._config_enables(cfg, ['CFG_TUH_MIDI'])) + self.assertTrue(change_impact._config_enables(cfg, ['CFG_TUH_MIDI'])) def test_literal_zero_is_disabled(self): import tempfile @@ -3168,10 +3156,10 @@ '#define CFG_TUD_MIDI 01\n' '#define CFG_TUD_DFU (1)\n') for m in ('CFG_TUD_CDC', 'CFG_TUD_MSC', 'CFG_TUD_HID', 'CFG_TUH_HID'): - self.assertFalse(ci_select._config_enables(cfg, [m]), m) + self.assertFalse(change_impact._config_enables(cfg, [m]), m) for m in ('CFG_TUD_MIDI', 'CFG_TUD_DFU'): - self.assertTrue(ci_select._config_enables(cfg, [m]), m) - self.assertFalse(ci_select._config_enables(cfg, ['CFG_TUD_VIDEO'])) + self.assertTrue(change_impact._config_enables(cfg, [m]), m) + self.assertFalse(change_impact._config_enables(cfg, ['CFG_TUD_VIDEO'])) def test_two_branch_define_reads_on(self): # examples/device/uac2_speaker_fb defines CFG_TUD_HID 1 under @@ -3179,7 +3167,7 @@ # defaults to 1) compiles the HID class in, so a CFG_TUD_HID change must keep # this example on both axes - the #else's zero must not decide it. cfg = os.path.join(REPO, 'examples/device/uac2_speaker_fb/src/tusb_config.h') - self.assertTrue(ci_select._config_enables(cfg, ['CFG_TUD_HID'])) + self.assertTrue(change_impact._config_enables(cfg, ['CFG_TUD_HID'])) def test_any_nonzero_define_wins_over_a_zero_one(self): import tempfile @@ -3190,11 +3178,11 @@ '#define CFG_TUD_MSC 0\n#endif\n' '#if BAR\n#define CFG_TUD_CDC 0\n#else\n' '#define CFG_TUD_CDC (0)\n#endif\n') - self.assertTrue(ci_select._config_enables(cfg, ['CFG_TUD_MSC'])) - self.assertFalse(ci_select._config_enables(cfg, ['CFG_TUD_CDC'])) + self.assertTrue(change_impact._config_enables(cfg, ['CFG_TUD_MSC'])) + self.assertFalse(change_impact._config_enables(cfg, ['CFG_TUD_CDC'])) def test_midi_host_change_selects_midi_rx(self): - s = ci_select.classify_build(['src/class/midi/midi_host.c'], REPO) + s = change_impact.classify_build(['src/class/midi/midi_host.c'], REPO) self.assertFalse(s['full']) self.assertTrue(s['families'], 'a TUH_MIDI change must select some family') self.assertTrue(any('host/midi_rx' in exs @@ -3208,7 +3196,7 @@ family's one-first board cannot build it.""" def test_board_gated_example_keeps_its_family(self): - s = ci_select.classify_build( + s = change_impact.classify_build( ['examples/dual/host_hid_to_device_cdc/src/main.c'], REPO) self.assertFalse(s['full']) self.assertIn('imxrt', s['families'], s['families']) @@ -3231,7 +3219,7 @@ (), 'make')) finally: os.chdir(old) - s = ci_select.classify_build(['examples/device/dfu/src/main.c'], REPO) + s = change_impact.classify_build(['examples/device/dfu/src/main.c'], REPO) self.assertIn('broadcom_64bit', s['families'], s['families']) @@ -3250,8 +3238,8 @@ os.chdir(self.old) def test_only_espressif_narrows_the_pool(self): - allex = list(ci_select.all_examples(REPO)) - for fam in ci_select.all_bsp_families(REPO): + allex = list(change_impact.all_examples(REPO)) + for fam in change_impact.all_bsp_families(REPO): pool = [e for e in allex if e in set(self.build_py.get_examples(fam))] if fam == 'espressif': self.assertNotEqual(pool, allex) # the carve-out is real @@ -3281,7 +3269,7 @@ for files, carve in ((['src/portable/synopsys/dwc2/dcd_dwc2.c'], True), (['src/class/msc/msc_host.c'], True), (['examples/device/cdc_msc_freertos/src/main.c'], False)): - s = ci_select.classify_build(files, REPO) + s = change_impact.classify_build(files, REPO) self.assertFalse(s['full'], files) self.assertIn('espressif', s['families'], files) esp = set(s['family_examples'].get('espressif') or [])
diff --git a/test/hil/test/test_ci_metrics.py b/test/hil/test/test_ci_metrics.py index 787c1ae..89b94e0 100644 --- a/test/hil/test/test_ci_metrics.py +++ b/test/hil/test/test_ci_metrics.py
@@ -13,7 +13,7 @@ os.path.dirname(os.path.abspath(__file__))))) CIRCLECI = os.path.join(REPO, '.circleci') sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) -from test_ci_select import mf # noqa: E402 a manifest v1 fixture +from test_change_impact import mf # noqa: E402 a manifest v1 fixture SENTINEL = 'example-map-default' @@ -65,9 +65,9 @@ self.assertNotIn('ci_set_matrix.py)', tail) def test_the_selector_gate_runs_both_suites(self): - # test_ci_select.py owns the rules; this file owns the sentinel contract the + # test_change_impact.py owns the rules; this file owns the sentinel contract the # very same job rewrites. Gating on one of the two leaves the other unguarded. - for suite in ('test_ci_select.py', 'test_ci_metrics.py'): + for suite in ('test_change_impact.py', 'test_ci_metrics.py'): self.assertIn(suite, self.config, f'{suite} does not gate the CircleCI selector') @@ -86,11 +86,11 @@ # SELECT_JSON="$SELECT_JSON" python3 -c ... E2BIGs at ~128KiB: measured 261KB # for a `git ls-files hw/bsp/**` sweep. Every reader takes the file instead. self.assertNotIn('SELECT_JSON="$SELECT_JSON"', self.build) - self.assertIn('json.load(open("ci_select_out.json"))', self.build) + self.assertIn('json.load(open("selection.json"))', self.build) def test_the_file_is_written_before_its_first_reader(self): - self.assertLess(self.build.index("printf '%s' \"$SELECT_JSON\" > ci_select_out.json"), - self.build.index('json.load(open("ci_select_out.json"))'), + self.assertLess(self.build.index("printf '%s' \"$SELECT_JSON\" > selection.json"), + self.build.index('json.load(open("selection.json"))'), 'the selection file must exist before the step that reads it') def test_pr_derived_env_values_are_character_guarded(self): @@ -358,7 +358,7 @@ self.assertTrue(comment.endswith('_[Full report](https://example/run)_\n')) def _run_block(self, block, sel, setup=None): - """Run a dedented build.yml step block with `sel` as ci_select_out.json in a temp + """Run a dedented build.yml step block with `sel` as selection.json in a temp dir; `setup(dir)` may prepare the dir and return extra env. Returns the block's $GITHUB_OUTPUT as a dict.""" import tempfile @@ -367,7 +367,7 @@ def _run_block_in(self, d, block, sel, setup=None): import subprocess - with open(os.path.join(d, 'ci_select_out.json'), 'w') as fh: + with open(os.path.join(d, 'selection.json'), 'w') as fh: json.dump(sel, fh) out = os.path.join(d, 'gh_output') open(out, 'w').close() @@ -384,7 +384,7 @@ broken script would not). Returns the step's $GITHUB_OUTPUT as a dict.""" import shlex repo = os.path.dirname(CIRCLECI) - i = self.build.index('SELECT_FILE=ci_select_out.json') + i = self.build.index('SELECT_FILE=selection.json') i = self.build.rindex('\n', 0, i) + 1 j = self.build.index('# HIL matrix', i) block = re.sub(r'^ {10}', '', self.build[i:j], flags=re.M) @@ -440,7 +440,7 @@ j = self.build.index('echo "$OUT" >> $GITHUB_OUTPUT', i) j = self.build.index('\n', j) + 1 def setup(d): - for name in ('test', 'tools'): # ci_select, and the roster helpers it imports + for name in ('test', 'tools'): # change_impact, and the roster helpers it imports os.symlink(os.path.join(REPO, name), os.path.join(d, name)) return self._run_block(re.sub(r'^ {10}', '', self.build[i:j], flags=re.M), sel, setup) @@ -458,12 +458,12 @@ os.mkdir(os.path.join(d, 'bin')) with open(os.path.join(d, 'bin', 'python3'), 'w') as fh: fh.write('#!/bin/sh\n' - f'case "$*" in *test_ci_select.py*|*test_build_select.py*) exit {suites_rc} ;;\n' - ' *check_build.py*) echo "$*" > argv; exec cat ci_select_out.json ;; esac\n' + f'case "$*" in *test_change_impact.py*|*test_build_select.py*) exit {suites_rc} ;;\n' + ' *check_build.py*) echo "$*" > argv; exec cat selection.json ;; esac\n' f'exec {shlex.quote(sys.executable)} "$@"\n') os.chmod(os.path.join(d, 'bin', 'python3'), 0o755) if sel is None: - os.unlink(os.path.join(d, 'ci_select_out.json')) + os.unlink(os.path.join(d, 'selection.json')) return {'PATH': os.path.join(d, 'bin') + os.pathsep + os.environ['PATH'], 'EVENT': event, 'BASE_REF': 'master', 'BEFORE': before} with tempfile.TemporaryDirectory() as d: @@ -567,9 +567,9 @@ self.assertIn('check_build.py --select-only --base "origin/$BASE_REF"', job) # the suites ran in check-paths on this commit: not again on the rig runner self.assertIn('SUITES_OK: ${{ needs.check-paths.outputs.selector_suites_ok }}', job) - self.assertNotIn('test_ci_select.py', job) - self.assertIn('--config test/hil/hfp.json > ci_select.json', job) - self.assertIn('ci_select.check_manifest(json.load(open("ci_select.json")))', job) + self.assertNotIn('test_change_impact.py', job) + self.assertIn('--config test/hil/hfp.json > selection.json', job) + self.assertIn('change_impact.check_manifest(json.load(open("selection.json")))', job) def test_the_build_extras_drop_when_the_matrix_falls_open(self): # ci_set_matrix falls open with rc 0, so the example map and family regex must
diff --git a/test/hil/test/test_drivers_coverage.py b/test/hil/test/test_drivers_coverage.py index f88f548..0eb3b29 100644 --- a/test/hil/test/test_drivers_coverage.py +++ b/test/hil/test/test_drivers_coverage.py
@@ -270,7 +270,7 @@ class HilCoverage(unittest.TestCase): - """Spot checks for the ci_select rule-3/4 reuse: family membership + """Spot checks for the change_impact rule-3/4 reuse: family membership (dcd_dwc2, dcd_rp2040) and no rig board at all (dcd_ft9xx).""" def test_dwc2_is_covered_by_stm32f4(self):
diff --git a/test/hil/test/test_hil_util.py b/test/hil/test/test_hil_util.py index d846f10..1a0ad6d 100644 --- a/test/hil/test/test_hil_util.py +++ b/test/hil/test/test_hil_util.py
@@ -97,7 +97,7 @@ class BottomLayer(unittest.TestCase): def test_bad_timeout_env_falls_back(self): - # ci_select (the PR-diff selector) imports hil_util for the example rosters; + # change_impact (the PR-diff selector) imports hil_util for the example rosters; # a malformed HIL_CMD_TIMEOUT must not crash the selector at import and knock # CI back to the full-matrix fallback import subprocess @@ -107,7 +107,7 @@ env={**os.environ, 'HIL_CMD_TIMEOUT': 'bogus'}, capture_output=True, text=True, timeout=30) self.assertEqual(r.returncode, 0, r.stderr) - # the warning must NOT be on stdout: ci_select's stdout is machine-read JSON + # the warning must NOT be on stdout: change_impact's stdout is machine-read JSON self.assertEqual(r.stdout.strip(), '180') self.assertIn('warning', r.stderr) # but a silent fallback hides the misconfiguration @@ -131,15 +131,15 @@ # hil_examples.py used to make this structural (a list of strings cannot grow a # dependency); with the rosters folded into hil_util the invariant needs teeth: # everything the bare GitHub runner imports (selector + this suite) must stay - # stdlib + local. Adding pyserial/pymtp here breaks ci_select on CI. + # stdlib + local. Adding pyserial/pymtp here breaks change_impact on CI. import ast hil_dir = Path(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # ONLY the modules the bare runner can import -- not every stem in the tree. # Globbing the directory allowed `import pymtp` (and hil_test, usbtest, # mtp_test) through, so the pymtp case this test names could never fail: that # module runs ctypes.CDLL(find_library('mtp')) at import and raises where there - # is no libmtp, taking ci_select down with it. - local = {'helper', 'hil_util', 'ci_select', 'hil_flash', + # is no libmtp, taking change_impact down with it. + local = {'helper', 'hil_util', 'change_impact', 'hil_flash', 'hil_lock', 'hil_pool_check', 'build', 'build_utils', 'get_deps', 'family_json', 'hil_report'} allowed = set(sys.stdlib_module_names) | local @@ -147,10 +147,10 @@ # on the bare runner, and its `import serial` is function-local for exactly # this reason -- hoisting it must fail HERE, not on every PR's pre-commit CI # ../../tools/rtt: hil_util exec_module's it at import (helper/hil_util.py's - # loader block), so a non-stdlib import THERE kills ci_select on the bare + # loader block), so a non-stdlib import THERE kills change_impact on the bare # runner just as surely -- and the spec_from_file_location call is invisible to # the ast.Import walk below, which is why it must be listed explicitly - for mod in ('helper/hil_util', 'hil_flash', '../../tools/ci_select', + for mod in ('helper/hil_util', 'hil_flash', '../../tools/change_impact', 'helper/hil_lock', 'helper/hil_pool_check', '../../tools/build', '../../tools/build_utils', '../../tools/rtt', '../../tools/get_deps', '../../tools/family_json',
diff --git a/tools/build.py b/tools/build.py index 11976a6..dad0d42 100755 --- a/tools/build.py +++ b/tools/build.py
@@ -384,7 +384,7 @@ one_first: If True, return only the first board (alphabetical) examples: PR example filter (-e). The one-board pick then prefers a board that can build at least one of them: the family is in the matrix BECAUSE some - board of it builds these examples (ci_select._prune_buildable asks about + board of it builds these examples (change_impact._prune_buildable asks about every board, since CircleCI builds every board), but GHA builds one. Without this, lpc54 selected for host/msc_file_explorer picks lpcxpresso54114 - which every one of those examples skips - and the leg runs to green having @@ -521,7 +521,7 @@ parser.error(f"-e/--example takes 'role/name' (e.g. device/cdc_msc), got '{e}'") # a name no example dir answers to would silently build nothing on every board # and still exit 0 (every row is a Skipped, and main() returns the FAILED count). - # The -e lists are generated - from ci_select's example map and from HIL roster + # The -e lists are generated - from change_impact's example map and from HIL roster # test names - so a stale one must be loud, not green if not os.path.isdir(os.path.join('examples', e)): parser.error(f"-e/--example '{e}': no such example directory examples/{e}")
diff --git a/tools/build_utils.py b/tools/build_utils.py index 625efdd..1fbe505 100755 --- a/tools/build_utils.py +++ b/tools/build_utils.py
@@ -57,7 +57,7 @@ """lru_cache, keyed on the working directory as well as the arguments. Every cached helper below takes repo-RELATIVE paths ('hw/bsp/<fam>', - 'examples/<ex>/skip.txt', or the literal 'hw/bsp' glob), while ci_select._in_repo() + 'examples/<ex>/skip.txt', or the literal 'hw/bsp' glob), while change_impact._in_repo() chdirs around each call so one process can classify more than one tree (e.g. a test fixture). Without the cwd in the key the second tree silently gets the first tree's
diff --git a/tools/ci_select.py b/tools/change_impact.py similarity index 97% rename from tools/ci_select.py rename to tools/change_impact.py index 307846b..301151c 100755 --- a/tools/ci_select.py +++ b/tools/change_impact.py
@@ -1,13 +1,11 @@ #!/usr/bin/env python3 # SPDX-License-Identifier: MIT -"""PR-diff -> CI selection: which rig boards and which tests a change can affect. +"""Change impact: which builds and which HIL tests a set of changed files can affect +(build = board families and their examples, HIL = rig boards and their tests). -Lives in tools/ so it can serve both HIL selection and, from Task 3, build-family -selection. Stdlib-only (runs on bare CI runners; imports hil_util for the example -rosters, never hil_test/pyserial — test_hil_util.BottomLayer enforces the stdlib -closure). Fail-open: any file no rule classifies forces the full matrix. See -docs/superpowers/specs/2026-07-29-hil-pr-scoped-selection-design.md and -docs/superpowers/specs/2026-08-19-ci-build-family-filter-design.md. +Stdlib-only (runs on bare CI runners; imports hil_util for the example rosters, never +hil_test/pyserial — test_hil_util.BottomLayer enforces the stdlib closure). Fail-open: +any file no rule classifies forces the full matrix. JSON: full, boards (name -> 'all' | [tests]), families (bsp families the diff touches, including ones with no rig board - build-only consumers @@ -17,11 +15,11 @@ through `check_build.py --select-only`, the build skill's SKILL.md documents it). THE RULE TABLE. First match wins; answers union per family (build) and per board -(HIL). A CARBON COPY of the table in the design spec above - edit both, or -TestRuleTableIsCarbonOfTheSpec fails. `FAM` = the families whose family.cmake -references the changed path (CMake only; make follows it). `DEV`/`HOST`/`DUAL`/ -`TYPEC`/`ALL` are the example role sets. The Build families column is PRE-PRUNE: -_prune_buildable then intersects each family with what it can actually build. +(HIL). Every row id needs a `# rule N` marker on its branch (TestRuleTableMatchesTheCode). +`FAM` = the families whose family.cmake references the changed path (CMake only; make +follows it). `DEV`/`HOST`/`DUAL`/`TYPEC`/`ALL` are the example role sets. The Build +families column is PRE-PRUNE: _prune_buildable then intersects each family with what it +can actually build. | # | Changed path | Build families | Build examples | HIL boards → tests | | 1 | `docs/`, `.claude/`, `*.md`, `*.rst`, `LICENSE` | — | — | — | @@ -36,7 +34,7 @@ | 5 | `src/portable/<port>/**` (anything else) | `FAM` | `ALL` | `FAM`'s boards → all their tests | | 5b | `src/portable/<port>/**` where `FAM` is empty | — | — | — (empty resolves to nothing on BOTH axes) | | 6 | `hw/bsp/<family>/**` | that family | `ALL` | that family's boards → all tests (a `boards/<board>/` path narrows to that board) | -| 7 | `hw/mcu/<vendor>/**` | `FAM` — empty resolves to nothing (maintainer ruling) | `ALL` | `FAM`'s boards → all tests; empty resolves to nothing (maintainer ruling) ⚠ *see below* | +| 7 | `hw/mcu/<vendor>/**` | `FAM` — empty resolves to nothing (maintainer ruling) | `ALL` | `FAM`'s boards → all tests; empty resolves to nothing (maintainer ruling) | | 8 | `src/class/<cls>/*_device.[ch]` | `ALL` | examples enabling `CFG_TUD_<CLS>` | device-role boards → HIL tests enabling `CFG_TUD_<CLS>` | | 9 | `src/class/<cls>/*_host.[ch]` | `ALL` | examples enabling `CFG_TUH_<CLS>` | host-role boards → HIL tests enabling `CFG_TUH_<CLS>` | | 10 | `src/class/<cls>/**` (shared header) | `ALL` | either, **plus include-edge classes and direct includers** | both roles → same, plus include-edge classes and direct includers | @@ -46,7 +44,7 @@ | 13 | `examples/<role>/<name>/**` | `ALL` | just `<name>` | if `<name>` is a HIL test: all boards → that test; else nothing | | 14 | `examples/device/board_test/**` | `ALL` | just `board_test` | all boards → all tests (HIL parking firmware) | | 15 | `examples/build_system/**`, `examples/CMakeLists.txt`, `examples/<role>/CMakeLists.txt` | `ALL` | `ALL` | all boards → all tests | -| 16 | `src/common/`, `src/osal/`, `src/tusb.[ch]`, `src/tusb_option.h`, `tools/{build,build_utils,ci_select,family_json}.py`, `src/CMakeLists.txt`, `src/tinyusb.mk`, `hw/bsp/{family_support.{cmake,mk},family_rules.mk,zephyr_board_aliases.cmake,board.c,board_api.h,ansi_escape.h}`, `.github/**`, `.circleci/**` | `ALL` | `ALL` | all boards → all tests | +| 16 | `src/common/`, `src/osal/`, `src/tusb.[ch]`, `src/tusb_option.h`, `tools/{build,build_utils,change_impact,family_json}.py`, `src/CMakeLists.txt`, `src/tinyusb.mk`, `hw/bsp/{family_support.{cmake,mk},family_rules.mk,zephyr_board_aliases.cmake,board.c,board_api.h,ansi_escape.h}`, `.github/**`, `.circleci/**` | `ALL` | `ALL` | all boards → all tests | | 16a | `lib/<name>/**` | `ALL` | examples whose own `CMakeLists.txt`/`Makefile` names `lib/<name>` | those examples that are HIL tests, on all boards; empty resolves to nothing | | 16b | `tools/get_deps.py` | families whose `deps_mandatory`/`deps_optional` entries changed | `ALL` | those families' boards → all tests; a logic change, an `'all'` entry, no base content or a changed token naming no family → full | | 17 | anything unclassified (no tracked file reaches this — TestNoTrackedFileIsUnclassified) | `ALL` | `ALL` | all boards → all tests (fail-open) | @@ -118,7 +116,7 @@ r'hw/bsp/family\.json$|' r'.*/[0-9]+-tinyusb[^/]*\.rules$|tools/usb_drivers/|tools/codespell/|' # test/hil/test/ holds the harness's own unit tests, not the harness: nothing on - # the rig runs them (pre-commit does, and build.yml runs test_ci_select.py as the + # the rig runs them (pre-commit does, and build.yml runs test_change_impact.py as the # gate before trusting a selection), so they cannot change what the rig does. # The harness itself stays under _FULL_RE's test/hil/ prefix. r'test/(fuzz|unit-test)/|test/hil/test/|' @@ -158,10 +156,10 @@ # generates the whole CircleCI matrix, same authority as .github/** r'\.circleci/|' # rule 16 says `tools/build*.py`; name the siblings the glob implies. build_utils - # and ci_select decide what gets built, so neither can be trusted to narrow its own + # and change_impact decide what gets built, so neither can be trusted to narrow its own # change; family_json runs inside build.py after every default configure, so a # break in it fails the build the same way - r'tools/(build|build_utils|ci_select|family_json)\.py$|' + r'tools/(build|build_utils|change_impact|family_json)\.py$|' # the make twins of family_support.cmake are the same authority for the make legs r'hw/bsp/(family_support\.(cmake|mk)|family_rules\.mk|zephyr_board_aliases\.cmake|' r'board_api\.h|board\.c|ansi_escape\.h)$|' @@ -358,7 +356,8 @@ def mcu_families(path: str, repo_root: str) -> set: """Families referencing a changed hw/mcu path: longest resolving dir prefix, - hw/mcu/<vendor>/<sub>/... down to hw/mcu/<vendor>.""" + hw/mcu/<vendor>/<sub>/... down to hw/mcu/<vendor>. The same scan as port_families on + a second tree: hw/mcu is mostly gitignored deps, so only its tracked dirs reach a diff.""" parts = path.split('/') for n in range(len(parts) - 1, 2, -1): fams = path_families('/'.join(parts[:n]), repo_root) @@ -1001,7 +1000,7 @@ try: return get_deps_changed_families(base_text_fn(), head_text_fn(), repo_root) except (subprocess.CalledProcessError, OSError) as e: - print(f'ci_select: {GET_DEPS_PATH}: base content unreadable ({e})', file=sys.stderr) + print(f'change_impact: {GET_DEPS_PATH}: base content unreadable ({e})', file=sys.stderr) return None @@ -1172,15 +1171,15 @@ try: files, gd, inp = select_input(repo_root, a.base, a.endpoints, a.worktree, a.diff_file, a.deps_base) except SelectError as e: - print(f'ci_select: {e}', file=sys.stderr) + print(f'change_impact: {e}', file=sys.stderr) sys.exit(2) if a.manifest: m, hil_reasons, build_reasons = manifest(files, repo_root, rosters, gd, inp) for r in build_reasons: - print(f'ci_select[build]: {r}', file=sys.stderr) + print(f'change_impact[build]: {r}', file=sys.stderr) for r in hil_reasons: - print(f'ci_select: {r}', file=sys.stderr) + print(f'change_impact: {r}', file=sys.stderr) print(json.dumps(m)) return @@ -1191,9 +1190,9 @@ s['hil_examples'] = hil_examples(s, rosters) s['build'] = classify_build(files, repo_root, gd) for r in s['build']['reasons']: - print(f'ci_select[build]: {r}', file=sys.stderr) + print(f'change_impact[build]: {r}', file=sys.stderr) for r in s['reasons']: - print(f'ci_select: {r}', file=sys.stderr) + print(f'change_impact: {r}', file=sys.stderr) # reasons go to stderr ONLY - they are a human diagnostic and no consumer reads them # back. They are also ~97% of the payload (a whole-tree diff: 453 KB -> 12 KB), which # build.yml re-parses with ci_set_matrix, hil_ci_set_matrix, an inline python and @@ -1205,8 +1204,7 @@ # ------------------------------------------------------------- -# Build-axis classifier (spec rule table, docs/superpowers/specs/ -# 2026-08-19-ci-build-family-filter-design.md). Independent of the HIL +# Build-axis classifier (the rule table in the module docstring). Independent of the HIL # classifier: same diff, second walk, its own fail-open. # ------------------------------------------------------------- # Both walks recognise an example path with the SAME regex, so a role can never be
diff --git a/tools/drivers_coverage_check.py b/tools/drivers_coverage_check.py index bae3840..c1aeb4a 100644 --- a/tools/drivers_coverage_check.py +++ b/tools/drivers_coverage_check.py
@@ -28,7 +28,7 @@ INFO lines on stdout: each waiver with the pinned boards it suppresses, and drivers with no rig board. -The HIL family mapping reuses tools/ci_select.py's rule-3/4 machinery +The HIL family mapping reuses tools/change_impact.py's rule-3/4 machinery (port_families()/port_option_gates()/board_roles(), the same data and role filter that pick which rig boards a src/portable/ diff selects) rather than a second heuristic. @@ -42,7 +42,7 @@ FAMILY_JSON = os.path.join(REPO, 'hw', 'bsp', 'family.json') PORTABLE = os.path.join(REPO, 'src', 'portable') sys.path.insert(0, os.path.join(REPO, 'tools')) -import ci_select # noqa: E402 +import change_impact # noqa: E402 sys.path.insert(0, os.path.join(REPO, '.github', 'scripts')) import ci_set_matrix # noqa: E402 @@ -91,7 +91,7 @@ port = _driver_port(driver_path, repo_root) if not port: return None, set(), set() - return port, ci_select.port_families(port, repo_root), gates_by_port.get(port, set()) + return port, change_impact.port_families(port, repo_root), gates_by_port.get(port, set()) def _load_object(path): @@ -119,7 +119,7 @@ return None, None, [f'{where}: missing "board"' if not board else f'{where}: "board" must be a string, not {type(board).__name__}'] where = f'{where} ({board})' - family = ci_select.board_family(board, REPO) + family = change_impact.board_family(board, REPO) if family is None: return board, None, [f'{where}: unknown board (no hw/bsp/*/boards/{board})'] errors = [] @@ -213,12 +213,12 @@ def _driver_port(driver_path, repo_root): - """Port dir per ci_select's rule-3/4 extraction, e.g. 'synopsys/dwc2' from + """Port dir per change_impact's rule-3/4 extraction, e.g. 'synopsys/dwc2' from 'src/portable/synopsys/dwc2/dcd_dwc2.c', or None if it doesn't match (it - always should - DriverScan pins the tree shape ci_select._PORT_PATH_RE + always should - DriverScan pins the tree shape change_impact._PORT_PATH_RE expects).""" rel = os.path.relpath(driver_path, repo_root).replace(os.sep, '/') - m = ci_select._PORT_PATH_RE.match(rel) + m = change_impact._PORT_PATH_RE.match(rel) return m.group(1) if m else None @@ -235,24 +235,24 @@ """('INFO', message) for every driver with no board on the HIL rig. A driver is covered when a roster board (a) has the matching role for the driver - device for dcd_*, host for hcd_*/ehci/ohci, checked against - ci_select.board_roles(), exactly the `board_roles(b) & roles` filter + change_impact.board_roles(), exactly the `board_roles(b) & roles` filter rule 3/4 applies - and (b) either its family is in the driver's port's - family set (ci_select.port_families - the same family.cmake references + family set (change_impact.port_families - the same family.cmake references rule 3/4 reads for a src/portable/ diff) or it turns on a build option - that gates the port regardless of family (ci_select.port_option_gates/ + that gates the port regardless of family (change_impact.port_option_gates/ board_options - e.g. analog/max3421's MAX3421_HOST).""" roster_boards = _hil_roster_boards(repo_root) - gates_by_port = ci_select.port_option_gates(repo_root) + gates_by_port = change_impact.port_option_gates(repo_root) driver_paths = list_driver_paths(os.path.join(repo_root, 'src', 'portable')) out = [] for d in sorted(driver_paths): _port, fams, gates = _port_scope(driver_paths[d], gates_by_port, repo_root) - # not ci_select._port_roles: bare ehci.c/ohci.c would read as "both" roles there + # not change_impact._port_roles: bare ehci.c/ohci.c would read as "both" roles there role = 'host' if _is_host_driver(d) else 'device' covered = any( - (ci_select.board_family(b['name'], repo_root) in fams or - (gates and ci_select.board_options(b, repo_root) & gates)) and - role in ci_select.board_roles(b) + (change_impact.board_family(b['name'], repo_root) in fams or + (gates and change_impact.board_options(b, repo_root) & gates)) and + role in change_impact.board_roles(b) for b in roster_boards) if not covered: out.append(('INFO', f'hil: {d} has no board on the rig'))
diff --git a/tools/family_json.py b/tools/family_json.py index 13dc307..13a056a 100644 --- a/tools/family_json.py +++ b/tools/family_json.py
@@ -399,7 +399,7 @@ def changed_cmake(root=ROOT, run=subprocess.run): """The hw/bsp cmake files the tree changes against HEAD, which is what a commit is about to carry; nothing when git cannot say (no git, no repository, no HEAD). --no-renames as - ci_select does: rename detection reports only a rename's destination, so the board or + change_impact does: rename detection reports only a rename's destination, so the board or family a file moved out of would keep its row from before the move.""" try: r = run(['git', '-C', str(root), 'diff', '--no-renames', '--name-only', '-z', 'HEAD', '--', 'hw/bsp'],
diff --git a/tools/rtt.py b/tools/rtt.py index 1a550ba..aea573d 100644 --- a/tools/rtt.py +++ b/tools/rtt.py
@@ -24,7 +24,7 @@ the slice of pyserial the HIL harness uses — read/in_waiting/write/close/timeout, reset_input_buffer, context-manager use, plus an `eof` latch — and are imported by test/hil/helper/hil_util.py, so this file is HARNESS-CRITICAL: a change here -is classified like a test/hil/ harness change (tools/ci_select.py) and runs the +is classified like a test/hil/ harness change (tools/change_impact.py) and runs the console unit tests (pre-commit hil-test hook, test/hil/test/test_hil_rtt.py). Stdlib only — hil_util imports this file, never the other way around. """