Interoperability Debugging Tool

Overview

The “Interoperability Debugging Tool” (IDT) is a python-based tool that supports a variety of commands that are useful in the context of interoperability testing of Matter devices and app controllers.

Discovery

While in discovery mode, the tool displays all Matter devices that are in commission and/or operational mode. This is useful to have a clear understanding of all Matter devices currently “active” in the testing environment.

See section “4.3. Discovery” of the Matter spec for official documentation.

When run interactively, discovery functions in one of two modes: BLE and DNS-SD.

Capture

While in capture mode, the tool starts capturing all data of interest (e.g. video recording of interactions with the mobile app, logs from all components involved, network packets capture, etc.) while a test is being conducted manually. It also provides feedback to the user on test setup and execution.

When the test completes, capture mode is stopped and all captured data is zipped in a file that can then be sent to all parties involved in investigating any issue uncovered via the manual test. Each ecosystem may implement an analysis that analyzes capture data, displays info to the user, probes the local environment and generates additional artifacts.

Single host installation (no Raspberry Pi)

All features of idt are available on macOS and Linux (tested with Debian based systems).
If you would prefer to execute capture and discovery from a Raspberry Pi, read the next section instead.

The machine running idt should be connected to the same Wi-Fi network used for testing.
Follow the steps below to execute capture and discovery without a Raspberry Pi:

  • From the parent directory of idt, run source idt/scripts/alias.sh.
  • Optionally, run source idt/scripts/setup_shell.sh to install aliases permanently.
  • After idt aliases are available in your environment, calling any idt command will automatically create a new virtual environment and install python dependencies.
    • If you‘re missing non-Python dependencies, you’ll be prompted to install them until they're available.
  • Bluetooth discovery on macOS will require granting the program where idt is run, e.g. terminal emulator or IDE permission to access bluetooth in macOS settings.
    • Failure to do so may result in any of the following:
      • A single abort message and no further output in the terminal.
      • Failure with a relevant stack trace in the terminal.
      • A prompt to allow the application access to bluetooth.

Raspberry Pi installation

Environment overview

The execution environment of IDT when using Raspberry Pi is shown in the figure below.

[TODO] add figure.

The Raspberry Pi is where “discovery” and “capture” are executed.

The “admin” computer is the machine used to connect to and control the RPi, and to fetch artifacts which were created during capture from the RPi.

This directory contains tools for use on both the admin computer and the RPi.

Environment details

  1. idt will be used on both the admin computer and the RPi.
  2. scripts only points to one installation location at a time. It is ideal to maintain a single idt directory on each (admin and RPi) system accordingly.
  3. The expected install location on the RPi is the home directory of the user specified in idt/scripts/vars.sh, which will be generated by running a script in the next section.
  4. Helper scripts may be used on admin computers that support zsh and bash (Linux and macOS).
  5. Windows may be used as the admin computer via tools like PowerShell, MobaXterm and FileZilla.
  6. This setup is intended to work with the admin computer and RPi connected to the same Wi-Fi network, which is also the Wi-Fi network used for testing.
  7. Corporate networks are not expected to be used as test networks.

Prepare the RPi

  1. A >= 128 GB SD card is recommended.
  2. Flash the RPi SD with the debian based distribution of your choice.
  3. Plug the SD into the RPi.
  4. Ensure the RPi is connected to your network, either via ethernet or with Wi-Fi configured in the disk image.
  5. Boot the RPi.

Configure admin computer and push to the RPi

Linux and macOS admin computers

  1. On your admin computer, source the alias script from the parent directory of idt to get idt commands in your current shell.
     source idt/scripts/alias.sh
    • To avoid having to repeat this step for each session, optionally configure automatic aliases permanently.
    • NOTE: Once run, idt commands will be globally and automatically available. If you need to remove the installation, edit the .rc files mentioned in setup_shell.
    source idt/scripts/setup_shell.sh
  2. Run idt_create_vars and follow the prompts to set IDs for the target RPi.
  3. Send idt to the RPi:
    idt_push
  4. ssh to the RPi:
    • NOTE: You may need to wait a few minutes after boot for the ssh server to be available on the RPi. Retry if needed!
    idt_connect

Windows admin computers

  1. Open PowerShell, cd to the directory containing idt and send idt to the RPi:
    scp -r ./idt/* $PIUSER@$PIHOST:/home/$PIUSER/idt
  2. ssh to the RPi, e.g. with MobaXterm
    • NOTE: You may need to wait a few minutes after boot for the ssh server to be available on the RPi. Retry if needed!
    • Use $PIUSER@$PIHOST or $PIUSER@$ip where $ip is the RPi's IP found in your router admin panel.

Configure the RPi

  1. Configure passwords or ssh keys.
  2. Configure Wi-Fi networks if needed.
  3. Set up idt:
    cd ~                               # Go to idt parent dir
source idt/scripts/setup_shell.sh  # Setup atuo aliases
source idt/scripts/alias.sh        # Get aliases now
idt_bootstrap                      # Initial configuration
idt_build                          # Build the container image

Install updates

SCP may not overwrite all files. To clear the idt dir off of the RPi safely between pushes, exit the container and:

idt_clean

NOTE the idt artifacts directory is contained in idt, so running this will delete any artifacts.

Then from the admin computer:

idt_push

User guide

IMPORTANT
idt_ commands are shell aliases helpful for administrative commands.
idt invokes the idt python package.
Output from idt will generally be colorized while output from sub processes is generally not.

RPi users, as needed:

  • For users with Windows admin computers, reconnect e.g., using MobaXterm
  • Other users reconnect ssh to the RPi (from your admin computer):
    idt_connect
  • Run the idt container (from the RPi):
    idt_activate

Capture

IMPORTANT
Ensure you've made it to the log line “Starting real time analysis, press enter to stop!” before launching the app under test.

idt capture -h

usage: idt capture [-h] [--platform {Android}] [--ecosystem {PlayServicesUser,PlayServices,ALL}] [--pcap {t,f}] [--interface {wlp0s20f3,lo,docker0,any}]

options:
  -h, --help            show this help message and exit
  --platform {Android}, -p {Android}
                        Run capture for a particular platform (default Android)
  --ecosystem {PlayServicesUser,PlayServices,ALL}, -e {PlayServicesUser,PlayServices,ALL}
                        Run capture for a particular ecosystem or ALL ecosystems (default ALL)
  --pcap {t,f}, -c {t,f}
                        Run packet capture (default t)
  --interface {wlp0s20f3,lo,docker0,any}, -i {wlp0s20f3,lo,docker0,any}
                        Specify packet capture interface (default any)

For packet capture interface (-i/--interface:

  • On macOS, the only available interface is any.
  • On Linux, idt checks available interfaces from /sys/class/net/ as well as allowing any.

Artifacts

Each ecosystem and platform involved in the capture will have their own subdirectory in the root artifact dir.

Discovery

idt discover -h

usage: idt discover [-h] --type {ble,b,dnssd,d}

options:
  -h, --help            show this help message and exit
  --type {ble,b,dnssd,d}, -t {ble,b,dnssd,d}
                        Specify the type of discovery to execute

BLE

idt discover -t b

mDNS

idt discover -t d

Artifacts

There is a per device log in ble and dnssd subdirectory of the root artifact dir.

Probe

usage: idt probe [-h]

options:
  -h, --help  show this help message and exit

Collect contextually relevant networking info from the local environment and provide artifacts.

Troubleshooting

  • Wireless adb may fail to connect indefinitely depending on network configuration. Use a wired connection if wireless fails repeatedly.
  • Change log level from INFO to DEBUG in root config.py for additional logging.
  • Compiling tcpdump for android may require additional dependencies.
    • If the build script fails for you, try idt_go && source idt/scripts/compilers.sh.
  • You may disable colors and splash by setting enable_color in config.py to False.
  • idt_clean_child will kill any stray tcpdump and adb commands.
    • idt_check_child will look for leftover processes.
    • Not expected to be needed outside of development scenarios.

Project overview

  • The entry point is in idt.py which contains simple CLI parsing with argparse.

capture

  • base contains the base classes for ecosystems and platforms.
  • controller contains the ecosystem and platform producer and controller
  • loader is a generic class loader that dynamically imports classes matching a given super class from a given directory.
  • /platform and /ecosystem contain one package for each platform and ecosystem, which should each contain one implementation of the respective base class.

discovery

  • matter_ble provides a simple ble scanner that shows matter devices being discovered and lost, as well as their VID/PID, RSSI, etc.
  • matter_dnssd provides a simple DNS-SD browser that searches for matter devices and thread border routers.

probe

  • probe contains the base class for (idt's) host platform specific implementation.
    • Reuses the dnssd discovery implementation to build probe targets.
    • Calls platform + addr type specific probe methods for each target.
  • linux and mac contain probe implementations for each host platform.

utils

  • log contains logging utilities used by everything in the project.
  • artifact contains helper functions for managing artifacts.
  • shell contains a simple helper class for background and foreground Bash commands.
  • host_platform contains helper functions for the interacting with the host running idt.

Conventions

  • config.py should be used to hold development configs within the directory where they are needed.
    • It may also hold configs for flaky/cumbersome features that might need to be disabled in an emergency.
    • config.py should not be used for everyday operation.
  • When needed, execute builds in a folder called BUILD within the source tree.
    • idt_clean_all deletes all BUILD dirs and BUILD is in .gitignore.

Extending functionality

Capture

Ecosystem and Platform implementations are dynamically loaded.

For each package in capture/ecosystem, the ecosystem loader expects a module name matching the package name.
This module must contain a single class which is a subclass of capture.base.EcosystemCapture.

/capture/ecosystem/play_services_user contains a minimal example implementation.

As another example, link /res/plugin_demo/ecosystem/demo_ext_ecosystem.

$ idt_go && ln -s $PWD/idt/res/plugin_demo/ecosystem/demo_ext_ecosystem/ idt/capture/ecosystem
$ idt capture -h
usage: idt capture [-h] [--platform {Android}] [--ecosystem {DemoExtEcosystem...

IMPORTANT: Note the following runtime expectations of ecosystems:
analyze_capture() must not block the async event loop excessively and must not interact with standard in

The platform loader functions the same as capture/ecosystem.

For each package in capture/platform, the platform loader expects a module name matching the package name.
This module must contain a single class which is a subclass of capture.base.PlatformLogStreamer.