blob: ce939a261402ef697200a895306f7d7ad7ad3ba0 [file] [view]
# VeeR tests on the VCK190 FPGA
Tools for running `//target/veer` tests on a Caliptra Subsystem VCK190 FPGA
board, from your workstation, with `bazel test`.
The same test targets also run on the Caliptra emulator. Each test is defined
once and generates a `<name>_emulator` and a `<name>_fpga` target.
## How it works
Every test is a host-side Rust test binary that drives an MCU firmware image
through [`caliptra_mcu_hw_model`]:
| | `<name>_emulator` | `<name>_fpga` |
|---|---|---|
| Test binary runs on | your workstation (x86_64) | the VCK190's ARM cores (aarch64) |
| Hardware model | `ModelEmulated` | `ModelFpgaRealtime`, over UIO |
| MCU image memory map | emulator | FPGA |
| Caliptra ROM/FW, MCU ROM | emulator builds | FPGA builds |
`bazel test --config=fpga` cross-compiles the test binaries and FPGA images on
your workstation, then runs each test on the board through
[`run_remote.sh`](run_remote.sh) (set as `--run_under` in `//.bazelrc`):
1. `run_remote.sh` copies the test's runfiles tree to
`~/openprot-fpga/runfiles/` on the board with `rsync`. The tree contains the
test binary, the signed MCU image, the boot ROMs and firmware, the bitstream,
and [`run_on_board.sh`](run_on_board.sh). Files that haven't changed since the
last test aren't copied again.
2. It runs `run_on_board.sh` on the board as root, over one SSH connection that
is reused across tests. `run_on_board.sh`:
1. Takes `/run/lock/openprot-fpga.lock`, so only one test uses the board at
a time, even across separate `bazel test` invocations.
2. Loads the bitstream into the FPGA, unless it's already loaded. It
compares against `/lib/firmware/caliptra-bitstream.pdi` and
`/dev/shm/fpga-config`, the same files upstream `xtask fpga bootstrap`
uses.
3. Runs the test binary. Output streams back to Bazel, and the test's exit
code becomes the Bazel result.
Infrastructure failures (SSH, missing bitstream, board busy) exit with code 2.
## Board setup
You need:
- A VCK190 running the **Caliptra FPGA image** from caliptra-sw (hostname
`caliptra-fpga`), built by
[this](https://github.com/chipsalliance/caliptra-sw/actions/workflows/fpga-image.yml)
workflow. It provides the kernel modules and runtime bitstream loading. Stock
Ubuntu images are not supported.
- **Key-based SSH** to the board. An alias in `~/.ssh/config` works.
`run_remote.sh` uses `BatchMode=yes`, so it fails instead of prompting.
- **Passwordless `sudo`** on the board, for UIO access and bitstream loading.
- `rsync` on both your workstation and the board.
Then tell Bazel which board to use. Add this to `.bazelrc-site` at the repo
root (it's gitignored):
```
test:fpga --test_env=VCK190_FPGA_HOST=<ssh-host-or-alias>
```
Or pass `--test_env=VCK190_FPGA_HOST=<host>` on each command.
## Running tests
```sh
# All FPGA tests.
bazel test --config=fpga //target/veer/...
# One test, with live output.
bazel test --config=fpga --test_output=streamed \
//target/veer/unittest_runner:host_smoke_test_fpga
# Filter Rust #[test] functions inside one target.
bazel test --config=fpga --test_arg=probe_mci_and_boot \
//target/veer/unittest_runner:host_smoke_test_fpga
# Build everything for the FPGA without a board
bazel build --config=fpga //target/veer/...
# The same tests on the emulator. FPGA tests are skipped by default.
bazel test //target/veer/...
```
`--config=fpga` selects tests tagged `fpga`, runs them one at a time, and gives
each a 600s timeout. Without it, `fpga` tests are filtered out.
## Writing tests
Tests live under `//target/veer/tests/<area>/`, next to the system image they
boot. Load the macros from `//target/veer/tooling:caliptra_test.bzl`.
### 1. Define the image
Use `veer_system_image` from `//target/veer:defs.bzl` instead of a plain
`system_image`. It also defines signed `<name>_image_emulator` and
`<name>_image_fpga` targets that the test macros pick up:
```starlark
veer_system_image(
name = "my_feature",
kernel = ":target",
platform = "//target/veer",
)
```
### 2a. On-device test: the firmware decides pass/fail
If the firmware exits with PASS or FAIL on its own, use
`caliptra_on_device_test`. It boots the image and waits for the exit status;
there's no Rust to write:
```starlark
caliptra_on_device_test(
name = "my_feature_test",
mcu_image = ":my_feature",
)
# -> :my_feature_test_emulator, :my_feature_test_fpga
```
### 2b. Host-driven test: your Rust drives the firmware
If the host has to do something (poke registers, act as the BMC, send
mailbox commands), use `caliptra_test` with your own sources:
```starlark
caliptra_test(
name = "my_host_test",
srcs = ["my_host_test.rs"],
mcu_image = ":my_feature",
)
```
```rust
use caliptra_mcu_hw_model::McuHwModel;
use test_boot_files::BootFiles;
#[test]
fn my_host_test() {
// ROMs, firmware, MCU image, SoC manifest and vendor PK hash for
// `mcu_image`, found through runfiles.
let files = BootFiles::from_env();
let mut hw = caliptra_mcu_hw_model::new(files.init_params()).unwrap();
// ... drive `hw` ...
hw.step_until_exit_success().unwrap();
}
```
Use `caliptra_mcu_hw_model::new_unbooted` if you need to do something before
boot. See [`host_smoke_test.rs`](../../tests/unittest_runner/host_smoke_test.rs)
for an example.
### Options
Both macros take:
- `interfaces`: defaults to `["emulator", "fpga"]`. Use `["emulator"]` for a
test that depends on emulator-only peripherals.
- `tags`, `env`, `data`, `deps`, `timeout`, and other `rust_test` arguments.
They add automatically, so don't set these yourself: the `emulator` / `fpga`
tag, `exclusive` on FPGA tests, `RUST_TEST_THREADS=1`, `CALIPTRA_BOOT_ENV`,
the `caliptra_mcu_hw_model` and `test_boot_files` deps, and (for FPGA) the
board-side files.
## Updating the bitstream
The bitstream is the `@caliptra_fpga_bitstream` `http_file` in
`//MODULE.bazel`. It has to match the `caliptra_mcu_sw` pin: take the URL and
sha256 from `hw/fpga/bitstream_manifests/subsystem.toml` at the pinned commit,
and bump them together with the pin.
## Troubleshooting
| Symptom | Fix |
|---|---|
| `VCK190_FPGA_HOST: is not set` | Set it in `.bazelrc-site` (see [Board setup](#board-setup)). |
| `SSH connection to <host> failed` | Check that `ssh -o BatchMode=yes <host> true` works without a prompt. |
| `board is busy` | Another test held the board lock for over 120s. Wait, or find the stuck process on the board. |
| `unsupported board image` | Flash the Caliptra FPGA image; see [Board setup](#board-setup). |
| Tests fail after someone used `xtask` on the board | `run_on_board.sh` reloads our bitstream if it doesn't match, but `xtask` users will need to bootstrap again afterwards. To force a reload: `ssh <host> sudo rm /dev/shm/fpga-config`. |
## Known gaps
- `//target/veer/tests/interrupts/kernel` is emulator-only: it uses an
emulator-only flash controller to raise an interrupt.
- `//target/veer/tests/threads/kernel:threads_test` is tagged `manual`.
- `//target/veer/syscall_latency:measure_syscall_latency_test` is tagged
`disabled` (kernel exception on process exit).
[`caliptra_mcu_hw_model`]: https://github.com/chipsalliance/caliptra-mcu-sw/tree/main/hw/model