[Parser Unit Test] Create DataType parser unit tests  (#40477)

* DM XMLs: Add parsing for data types [WIP]

Note that anything with a within-data-type conformance will
have a conformance of optional currently. This is OK since we're
not actually using these for anything expect understanding P
markings currently.

TODO: Still not parsing multi-bit bitfields correctly.
TODO: Needs unit tests
TODO: Probably not passing CI on the test support tests
      because of the new errors with the bitfields.

* Updating changes for XML spec parsing and adding unit tests:
- Enhanced XmlDataTypeComponent with fields for docs, types, and constraints in spec parsing module
- Added support for detecting optional and nullable fields in XML in spec parsing module
- Improved constraint parsing for min/max values and attribute references in spec parsing module
- Added comprehensive test coverage using 1.4.1 cluster spec XML files in TestSpecParsingDataType unit test module
- Improved test output with detailed statistics on XML field usage in TestSpecParsingDataType unit test module and spec parsing module

* Updating TestSpecParsingDataType module:
- Updating some verbiage on comments as needed

* Updating test_metadata.yaml:
- Added TestSpecParsingDataType python3 module as it does not run against an app.

* Restyled by autopep8

* Restyled by isort

* Updating TestSpecParsingDataType unit testing module:
- Resolving linting errors

* Updated TestSpecParsingDataType unit test module:
- Resolving linting errors

* Restyled by autopep8

* Add comprehensive XML validation test for all data types in 1.4.2

Add test_exhaustive_all_fields_in_1_4_2() method to validate every struct,
enum, and bitmap field in all cluster XMLs for version 1.4.2. This test:

- Parses all clusters from /data_model/1.4.2/clusters/
- Validates every struct field has required attributes (name, id, type_info)
- Validates every enum item has required attributes (name, id)
- Validates every bitmap bitfield has required attributes (name, id)
- Skips type_info validation for deprecated fields (e.g., Key field in MonitoringRegistrationStruct)
- Accumulates all issues and reports them at the end instead of failing on first error
- Provides detailed error messages for each validation failure

This ensures full coverage of the data model XMLs and catches any missing
or malformed fields that could affect parsing or downstream analysis.

The test has already identified several clusters with missing type attributes
in ModeOptionStruct fields, which should be addressed in the XML source.

* Restyled by autopep8

* Added exception for possible issue 1.4.2 version Mode cluster XML's ModeOptionStruct struct appears to be missing the type_info attribute

* Restyled by autopep8

* Apply suggestions from code review from Gemini AI

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Fix TestSpecParsingDataType.py test logic for field ID validation:

- Fixed field_id validation to use explicit None check instead of falsy check (field_id=0 is valid but was being treated as empty)
- Added exception handling for fields without type_info that are intentionally missing types (references, ModeOptionStruct fields, etc.)
- Updated test to handle legitimate cases where fields don't have type attributes due to being references or having inferred types

* Restyled by autopep8

* changed chip -> matter for imports

* Restyled by isort

* Updating spec_parsing support module and TestSpecParsingDataType unit test module:
- fixed mypy errors:
-- Line 246: Added raw string prefix to regex pattern
-- Lines 458 & 548: Added type conversion and null checks for ClusterPathLocation calls
-- Line 510: Added null check for max_count.text before assignment
-- Lines 678-680: XmlCluster constructor now includes the required structs, enums, and bitmaps arguments
- Updated for validation of 1.5 XML's instead of 1.4.2

* Updated TestSpecParsingDatatype unit test module:
- Updated copyright date to 2025 from 2024
- Changed verbiage for skipped XML due to missing type_info field for mode
- Updated doc strings to include that we are testing version 1.5 now

* Adding parsing and verification of global data type XML's in version 1.5

* Removed some unneeded debug log outputs

* Restyled by autopep8

* Restyled by isort

* Resolving mypy errors

* Resolving linting errors

* Adding comments for why base=0 in init() of literal class in confomance.py

* Add comment explaining StrEnum usage in DataTypeEnum for XML parsing in spec_parsing.py

* Add comment explaining data_type field in XmlDataType in spec_parsing.py

* Apply suggestions from code review from Gemini-AI-bot

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Broke down _parse_components into 5 different focused helper methods for easier maintenance and provided clear documentation for each one

* Restyled by autopep8

* Resolving mypy error

* Updating spec_parsing.py to resolve Andrei's comments and suggestions:
- Renamed _determine_nullable_status → _isNullableField,, changed it to a static method as it doesn't use self, and changed its call to using the new name
- Renamed _determine_optional_status → _isOptionalField, changed it to a static method as it doesn't use self, and changed its call to using the new name
- Added a clarifying comment to explain why base=0 is used in the _parse_basic_field_attributes method.
- Simplified boolean field checks with direct returns and case-insensitive parsing
- Added comment explaining None vs {} return choice in _parse_field_constraints

* Updating spec_parsing.py to resolve some additional comments/suggestions from Andrei:
- Reduce nesting in _parse_field_conformance using early return pattern
- Add duplicate ID detection in component parsing to catch invalid XML data
- Add duplicate name detection in data type parsing to catch XML conflicts

* Updating unit test TestSpecParsingDataType module:
- Refactoring from suggestion by Gemini AI bot for repetitive skip condition
- Resolved issue where spec_parsing and TestSpecParsingDataType had gone out of sync due to recent updates

* Resolving linting errors

* Restyled by autopep8

* Restyled by autopep8

* Resolving linting error

* Moving TestSpecParsingDataType.py into test_testing directory and adding it back into tests.yaml

* Remove outdated skip conditions from TestSpecParsingDataType for fields that now have proper type_info

---------

Co-authored-by: cecille <cecille@google.com>
Co-authored-by: Restyled.io <commits@restyled.io>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
5 files changed
tree: 707e90e07c28a706b737302b017eb87aa0e90cf9
  1. .devcontainer/
  2. .gemini/
  3. .githooks/
  4. .github/
  5. .vscode/
  6. build/
  7. build_overrides/
  8. config/
  9. credentials/
  10. data_model/
  11. docs/
  12. examples/
  13. integrations/
  14. scripts/
  15. src/
  16. third_party/
  17. zzz_generated/
  18. .actrc
  19. .clang-format
  20. .clang-tidy
  21. .default-version.min
  22. .dir-locals.el
  23. .editorconfig
  24. .gitattributes
  25. .gitignore
  26. .gitmodules
  27. .gn
  28. .matterlint
  29. .mergify.yml
  30. .pre-commit-config.yaml
  31. .prettierrc.json
  32. .pullapprove.yml
  33. .restyled.yaml
  34. .shellcheck_tree
  35. .spellcheck.yml
  36. BUILD.gn
  37. CODE_OF_CONDUCT.md
  38. CODEOWNERS
  39. CONTRIBUTING.md
  40. gn_build.sh
  41. iwyu.imp
  42. kotlin-detect-config.yaml
  43. lgtm.yml
  44. LICENSE
  45. NOTICE
  46. pigweed.json
  47. pyproject.toml
  48. README.md
  49. REVIEWERS.md
  50. SECURITY.md
  51. SPECIFICATION_VERSION
README.md

Documentation links

Matter

Builds

Builds

Android Ameba ASR BouffaloLab Darwin TI CC26X2X7 TI CC32XX EFR32 ESP32 Infineon i.MX Linux K32W with SE051 Linux ARM Linux Standalone nRF Connect SDK QPG STM32 Telink Tizen

Tests

Tests Unit / Integration Tests Cirque QEMU

Tools

ZAP Templates

Documentation

Documentation Build

About

Matter (formerly Project CHIP) creates more connections between more objects, simplifying development for manufacturers and increasing compatibility for consumers, guided by the Connectivity Standards Alliance.

What is Matter?

Matter is a unified, open-source application-layer connectivity standard built to enable developers and device manufacturers to connect and build reliable, and secure ecosystems and increase compatibility among connected home devices. It is built with market-proven technologies using Internet Protocol (IP) and is compatible with Thread and Wi-Fi network transports. Matter was developed by a Working Group within the Connectivity Standards Alliance (Alliance). This Working Group develops and promotes the adoption of the Matter standard, a royalty-free connectivity standard to increase compatibility among smart home products, with security as a fundamental design tenet. The vision that led major industry players to come together to build Matter is that smart connectivity should be simple, reliable, and interoperable.

Matter simplifies development for manufacturers and increases compatibility for consumers.

The standard was built around a shared belief that smart home devices should be secure, reliable, and seamless to use. By building upon Internet Protocol (IP), Matter enables communication across smart home devices, mobile apps, and cloud services and defines a specific set of IP-based networking technologies for device certification.

The Matter specification details everything necessary to implement a Matter application and transport layer stack. It is intended to be used by implementers as a complete specification.

The Alliance officially opened the Matter Working Group on January 17, 2020, and the specification is available for adoption now.

Visit buildwithmatter.com to learn more and read the latest news and updates about the project.

Project Overview

Development Goals

Matter is developed with the following goals and principles in mind:

Unifying: Matter is built with and on top of market-tested, existing technologies.

Interoperable: The specification permits communication between any Matter-certified device, subject to users’ permission.

Secure: The specification leverages modern security practices and protocols.

User Control: The end user controls authorization for interaction with devices.

Federated: No single entity serves as a throttle or a single point of failure for root of trust.

Robust: The set of protocols specifies a complete lifecycle of a device — starting with the seamless out-of-box experience, through operational protocols, to device and system management specifications required for proper function in the presence of change.

Low Overhead: The protocols are practically implementable on low compute-resource devices, such as MCUs.

Pervasive: The protocols are broadly deployable and accessible, by leveraging IP and being implementable on low-capability devices.

Ecosystem-Flexible: The protocol is flexible enough to accommodate deployment in ecosystems with differing policies.

Easy to Use: The protocol provides smooth, cohesive, integrated provisioning and out-of-box experience.

Open: The Project’s design and technical processes are open and transparent to the general public, including non-members wherever possible.

Architecture Overview

Matter aims to build a universal IPv6-based communication protocol for smart home devices. The protocol defines the application layer that will be deployed on devices and the different link layers to help maintain interoperability. The following diagram illustrates the normal operational mode of the stack: Matter Architecture Overview

The architecture is divided into layers to help separate the different responsibilities and introduce a good level of encapsulation among the various pieces of the protocol stack. The vast majority of interactions flow through the stack captured in the following Figure:

Matter Stack Architecture

  1. Application: High-order business logic of a device. For example, an application that is focused on lighting might contain logic to handle turning on/off the bulb as well as its color characteristics.
  1. Data Model: The data layer corresponds to the data and verb elements that help support the functionality of the application. The Application operates on these data structures when there is an intent to interact with the device.
  1. Interaction Model: The Interaction Model layer defines a set of interactions that can be performed between a client and server device. For example, reading or writing attributes on a server device would correspond to application behavior on the device. These interactions operate on the elements defined at the data model layer.
  1. Action Framing: Once an action is constructed using the Interaction Model, it is serialized into a prescribed packed binary format to encode for network transmission.
  1. Security: An encoded action frame is then sent down to the Security Layer to encrypt and sign the payload to ensure that data is secured and authenticated by both sender and receiver of a packet.

  2. Message Framing & Routing: With an interaction encrypted and signed, the Message Layer constructs the payload format with required and optional header fields; which specify the message's properties and some routing information.

  1. IP Framing & Transport Management: After the final payload has been constructed, it is sent to the underlying transport protocol for IP management of the data.

Current Status of Matter

Matter’s design and technical processes are intended to be open and transparent to the general public, including to Working Group non-members wherever possible. The availability of this GitHub repository and its source code under an Apache v2 license is an important and demonstrable step to achieving this commitment. Matter endeavors to bring together the best aspects of market-tested technologies and redeploy them as a unified and cohesive whole-system solution. The overall goal of this approach is to bring the benefits of Matter to consumers and manufacturers as quickly as possible. As a result, what you observe in this repository is an implementation-first approach to the technical specification, vetting integrations in practice. The Matter repository is growing and evolving to implement the overall architecture. The repository currently contains the security foundations, message framing and dispatch, and an implementation of the interaction model and data model. The code examples show simple interactions, and are supported on multiple transports -- Wi-Fi and Thread -- starting with resource-constrained (i.e., memory, processing) silicon platforms to help ensure Matter’s scalability.

How to Contribute

We welcome your contributions to Matter. Read our contribution guidelines here.

Building and Developing in Matter

Instructions about how to build Matter can be found here .

Directory Structure

The Matter repository is structured as follows:

File/FolderContent
buildBuild system support content and built output directories
build_overridesBuild system parameter customization for different platforms
configProject configurations
credentialsDevelopment and test credentials
docsDocumentation, including guides. Visit the Matter SDK documentation page to read it.
examplesExample firmware applications that demonstrate use of Matter
integrations3rd party integrations
scriptsScripts needed to work with the Matter repository
srcImplementation of Matter
third_party3rd party code used by Matter
zzz_generatedZAP generated template code - Revolving around cluster information
BUILD.gnBuild file for the GN build system
CODE_OF_CONDUCT.mdCode of conduct for Matter and contribution to it
CONTRIBUTING.mdGuidelines for contributing to Matter
LICENSEMatter license file
REVIEWERS.mdPR reviewers
gn_build.shBuild script for specific projects such as Android, EFR32, etc.
README.mdThis file

License

Matter is released under the Apache 2.0 license.