Switch DataModel::Provider to a span-based list iterator - flash and ram savings, simpler interface (#37033)

* Copied over the new AttributePathExpandIterator and will incrementally use it (so I can validate tests)

* Rename AttributePathExpandIterator to legacy

* Prepare for using new style iterators ... checking NOT YET enabled though

* Enabled checks ... and unit tests fail, but this now can be debugged

* Fix some of the underlying bugs: read handling logic assumes we are ok to undo

* Unit tests pass now

* Restyle

* Use new iterator in IME

* Update logic to use the new iterator on testRead

* more updates

* Restyle

* Remove the legacy attribute path expand iterator

* Update naming

* Restyle

* Remove extra argument for ReadHandler constructor

* Restyle

* Slight flash improvement

* Fix up includes

* Removed empty line

* added comment on why state is a friend class

* Comment updates

* Restyle, add some comments and add extra checks on validity check only for expansion. This saves a tiny amount of flash (32 bytes)

* Remove an include

* Comment updates, renamed mLastOutputPath to mOutputPath

* Fix one typo

* Re-arrange members of ReadHandler to optimize for memory layout. This saves 8 bytes for struct. We still have a 20-byte padding which I am unsure how to get rid of

* Restyle

* Rename State to Position

* One more rename

* Remove redundant assigment ...we are at a net 0 txt increase now on qpg

* Add more unit tests for non-obvious requirement that wildcard expansion checks path validity, however non-wildcard does not check it

* Update src/app/AttributePathExpandIterator.cpp

Co-authored-by: Tennessee Carmel-Veilleux <tennessee.carmelveilleux@gmail.com>

* Update src/app/AttributePathExpandIterator.h

Co-authored-by: Tennessee Carmel-Veilleux <tennessee.carmelveilleux@gmail.com>

* Update src/app/AttributePathExpandIterator.h

Co-authored-by: Tennessee Carmel-Veilleux <tennessee.carmelveilleux@gmail.com>

* Update src/app/AttributePathExpandIterator.h

Co-authored-by: Tennessee Carmel-Veilleux <tennessee.carmelveilleux@gmail.com>

* Update src/app/ReadHandler.h

Co-authored-by: Tennessee Carmel-Veilleux <tennessee.carmelveilleux@gmail.com>

* Update src/app/ReadHandler.cpp

Co-authored-by: Tennessee Carmel-Veilleux <tennessee.carmelveilleux@gmail.com>

* Update src/app/AttributePathExpandIterator.h

Co-authored-by: Tennessee Carmel-Veilleux <tennessee.carmelveilleux@gmail.com>

* Use different values for the cluster ids for testing

* One more state to position change

* mExpanded is now set during output path returning. Removed 2 more sets to save another tinier amount of .text

* Import metadatalist class and test

* Remove some tests that seem redundant, keep only one

* Start with generated commands, see if we can replace its usage...

* Unit tests for GeneratedCommands pass

* Start with an implementation of accepted commands (no testing yet)

* More tests pass

* Updated AcceptedCommands as well .. unit tests pass

* Restyle

* Fix namespaces

* Slight refactor. Code is still very much ugly

* A bit of refactor, code looks better and tests pass now

* Code compile for semantic tag ... made std::optional support non-trivial destructors

* Update test

* Make chip::Optional be trivially destructible if the underlying type is.

Previous implementation always had a destructor, so it was never
trivially destructible.

* Update src/app/AttributePathExpandIterator.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/AttributePathExpandIterator.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/AttributePathExpandIterator.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/AttributePathExpandIterator.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/InteractionModelEngine.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/ReadHandler.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/AttributePathExpandIterator.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/ReadHandler.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Use mCompletePosition

* Another rename

* Undo submodule update

* Restyle

* Remove extra char

* Remove unused variable

* Update comment text to not sound like graph parsing

* Rename method to be more descriptive

* Remove one more unused variable

* Update peek attribute iterator to rollback and update code logic a bit. Hoping for cleaner code

* Semantic tags conversion is done

* Restyle

* Migrate device types to the new format

* update comment a bit

* Add unused marker for chip errors used for logging only

* Fix descriptor cluster

* Fix microwave oven

* Restyle

* Fix intentional bugprone-use-after-move

* Fix intentional bugprone-use-after-move

* Restyle

* Fix based on clang feedback

* Move endpoints to the new style of iteration

* Fix includes

* Fix includes

* Fix includes

* make it standard that test Providers are for now CodegenDataModelProvider. Saves me some typing as I move things around

* Restyle

* Minor update in logic: do the endpoint selection when next is called

* Allow startup to try to mark attributes dirty even if no provider exists yet

* Fix typo

* start implementing client clusters

* Update ClientCluster logic

* Restyle

* Start defining the server cluster query

* Actually use the new server cluster functionality

* Fix an include

* More include fixes

* implement the get attributes and adapt unit tests

* Restyle

* more updates to cleanup code. I am a bit concerned about O(n^2) attribute access...

* A rename and moved finder methods out of inline. Saves 650 bytes of flash

* Update logic to centralize metadata list code with less templating

* move the generic metadata list to detail, add an assert on trivial destruction

* Fix namespace prefix

* Save 70 bytes by using references in condensed for loops

* Replace count-if because it seems to result in smaller code (46 bytes)

* Another find_if replacement

* Replaced some find_if...we are down to 92 bytes on qpg

* Replaced one more find-if, saving another 88 bytes of flash

* Removed algorithm includes: these are slow and would like compile to fail if used

* Save more flash ... we should be at a net negative now

* This seems to save even more

* More savings by more encode overrides ... this is silly...

* Fix typos

* More explicit casting, removed 64-bit overrides

* Added one more check for freeing memory .Still need to track a leak that darwin finds

* Fix memory leak in assignment

* Self-review: fix includes

* Self-review: fix includes

* Self-review: fix includes

* Self-review: fix includes

* Update src/data-model-providers/codegen/CodegenDataModelProvider.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/data-model-providers/codegen/CodegenDataModelProvider.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/AttributePathExpandIterator.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/AttributePathExpandIterator.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/clusters/descriptor/descriptor.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/clusters/descriptor/descriptor.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/data-model-provider/MetadataList.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Fix spelling for acquire

* Update src/app/data-model-provider/MetadataList.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/data-model-provider/MetadataList.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/data-model-provider/MetadataList.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Fix is_trivially_destructible

* Update src/app/data-model-provider/MetadataSearch.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* More fixes

* Fix includes

* Fix invalid check typo

* Fix comment

* Correct comment

* Updated comment

* Make logic between clusters/attributes/endpoints the same regarding nullopt and wildcards

* Restyle

* Update logic for IsDescentantof

* Help compiler generate efficient code as we keep reusing the same pointer

* clearer logic that we handle all cases

* Fixes

* Use calloc

* Update comment

* Fix include

* Fix casting

* Make cluster count functions from ember public API since they seem reusable

* Also fix dynamic dispatch

* update enumeration entry

* another comment update

* Undo submodule update

* Rename metadata search to metadta lookup

* Update metadata list methods to be all uppercase

* Fix more renames

* Fix invalid cast

* Update src/data-model-providers/codegen/CodegenDataModelProvider.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/data-model-providers/codegen/CodegenDataModelProvider.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/data-model-providers/codegen/CodegenDataModelProvider.cpp

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Update src/app/data-model-provider/MetadataList.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* Address comments

* fix bug

* Update src/app/AttributePathExpandIterator.cpp

Co-authored-by: Mathieu Kardous <84793247+mkardous-silabs@users.noreply.github.com>

* Update src/app/WriteHandler.cpp

Co-authored-by: Mathieu Kardous <84793247+mkardous-silabs@users.noreply.github.com>

* Update src/app/AttributePathExpandIterator.cpp

Co-authored-by: Mathieu Kardous <84793247+mkardous-silabs@users.noreply.github.com>

* Restyle and include update

---------

Co-authored-by: Andrei Litvin <andreilitvin@google.com>
Co-authored-by: Tennessee Carmel-Veilleux <tennessee.carmelveilleux@gmail.com>
Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>
Co-authored-by: Mathieu Kardous <84793247+mkardous-silabs@users.noreply.github.com>
38 files changed
tree: 126d86578cf42d3950dbbb6170d1598706584d4c
  1. .devcontainer/
  2. .githooks/
  3. .github/
  4. .vscode/
  5. build/
  6. build_overrides/
  7. config/
  8. credentials/
  9. data_model/
  10. docs/
  11. examples/
  12. integrations/
  13. scripts/
  14. src/
  15. third_party/
  16. zzz_generated/
  17. .actrc
  18. .clang-format
  19. .clang-tidy
  20. .default-version.min
  21. .dir-locals.el
  22. .editorconfig
  23. .gitattributes
  24. .gitignore
  25. .gitmodules
  26. .gn
  27. .isort.cfg
  28. .mergify.yml
  29. .prettierrc.json
  30. .pullapprove.yml
  31. .restyled.yaml
  32. .shellcheck_tree
  33. .spellcheck.yml
  34. BUILD.gn
  35. CODE_OF_CONDUCT.md
  36. CODEOWNERS
  37. CONTRIBUTING.md
  38. gn_build.sh
  39. iwyu.imp
  40. kotlin-detect-config.yaml
  41. lgtm.yml
  42. LICENSE
  43. NOTICE
  44. pigweed.json
  45. README.md
  46. REVIEWERS.md
  47. ruff.toml
  48. SPECIFICATION_VERSION
README.md

Matter

Builds

Builds

Android Ameba ASR BouffaloLab Darwin TI CC26X2X7 TI CC32XX EFR32 ESP32 Infineon i.MX Linux K32W with SE051 Linux ARM Linux Standalone Linux Standalone Mbed OS MW320 nRF Connect SDK Open IoT SDK QPG STM32 Telink Tizen

Tests

Unit / Integration Tests Cirque QEMU

Tools

ZAP Templates

Documentation

Documentation Build

About

Matter (formerly Project CHIP) creates more connections between more objects, simplifying development for manufacturers and increasing compatibility for consumers, guided by the Connectivity Standards Alliance.

What is Matter?

Matter is a unified, open-source application-layer connectivity standard built to enable developers and device manufacturers to connect and build reliable, and secure ecosystems and increase compatibility among connected home devices. It is built with market-proven technologies using Internet Protocol (IP) and is compatible with Thread and Wi-Fi network transports. Matter was developed by a Working Group within the Connectivity Standards Alliance (Alliance). This Working Group develops and promotes the adoption of the Matter standard, a royalty-free connectivity standard to increase compatibility among smart home products, with security as a fundamental design tenet. The vision that led major industry players to come together to build Matter is that smart connectivity should be simple, reliable, and interoperable.

Matter simplifies development for manufacturers and increases compatibility for consumers.

The standard was built around a shared belief that smart home devices should be secure, reliable, and seamless to use. By building upon Internet Protocol (IP), Matter enables communication across smart home devices, mobile apps, and cloud services and defines a specific set of IP-based networking technologies for device certification.

The Matter specification details everything necessary to implement a Matter application and transport layer stack. It is intended to be used by implementers as a complete specification.

The Alliance officially opened the Matter Working Group on January 17, 2020, and the specification is available for adoption now.

Visit buildwithmatter.com to learn more and read the latest news and updates about the project.

Project Overview

Development Goals

Matter is developed with the following goals and principles in mind:

Unifying: Matter is built with and on top of market-tested, existing technologies.

Interoperable: The specification permits communication between any Matter-certified device, subject to users’ permission.

Secure: The specification leverages modern security practices and protocols.

User Control: The end user controls authorization for interaction with devices.

Federated: No single entity serves as a throttle or a single point of failure for root of trust.

Robust: The set of protocols specifies a complete lifecycle of a device — starting with the seamless out-of-box experience, through operational protocols, to device and system management specifications required for proper function in the presence of change.

Low Overhead: The protocols are practically implementable on low compute-resource devices, such as MCUs.

Pervasive: The protocols are broadly deployable and accessible, by leveraging IP and being implementable on low-capability devices.

Ecosystem-Flexible: The protocol is flexible enough to accommodate deployment in ecosystems with differing policies.

Easy to Use: The protocol provides smooth, cohesive, integrated provisioning and out-of-box experience.

Open: The Project’s design and technical processes are open and transparent to the general public, including non-members wherever possible.

Architecture Overview

Matter aims to build a universal IPv6-based communication protocol for smart home devices. The protocol defines the application layer that will be deployed on devices and the different link layers to help maintain interoperability. The following diagram illustrates the normal operational mode of the stack: Matter Architecture Overview

The architecture is divided into layers to help separate the different responsibilities and introduce a good level of encapsulation among the various pieces of the protocol stack. The vast majority of interactions flow through the stack captured in the following Figure:

Matter Stack Architecture

  1. Application: High-order business logic of a device. For example, an application that is focused on lighting might contain logic to handle turning on/off the bulb as well as its color characteristics.
  1. Data Model: The data layer corresponds to the data and verb elements that help support the functionality of the application. The Application operates on these data structures when there is an intent to interact with the device.
  1. Interaction Model: The Interaction Model layer defines a set of interactions that can be performed between a client and server device. For example, reading or writing attributes on a server device would correspond to application behavior on the device. These interactions operate on the elements defined at the data model layer.
  1. Action Framing: Once an action is constructed using the Interaction Model, it is serialized into a prescribed packed binary format to encode for network transmission.
  1. Security: An encoded action frame is then sent down to the Security Layer to encrypt and sign the payload to ensure that data is secured and authenticated by both sender and receiver of a packet.

  2. Message Framing & Routing: With an interaction encrypted and signed, the Message Layer constructs the payload format with required and optional header fields; which specify the message's properties and some routing information.

  1. IP Framing & Transport Management: After the final payload has been constructed, it is sent to the underlying transport protocol for IP management of the data.

Current Status of Matter

Matter’s design and technical processes are intended to be open and transparent to the general public, including to Working Group non-members wherever possible. The availability of this GitHub repository and its source code under an Apache v2 license is an important and demonstrable step to achieving this commitment. Matter endeavors to bring together the best aspects of market-tested technologies and redeploy them as a unified and cohesive whole-system solution. The overall goal of this approach is to bring the benefits of Matter to consumers and manufacturers as quickly as possible. As a result, what you observe in this repository is an implementation-first approach to the technical specification, vetting integrations in practice. The Matter repository is growing and evolving to implement the overall architecture. The repository currently contains the security foundations, message framing and dispatch, and an implementation of the interaction model and data model. The code examples show simple interactions, and are supported on multiple transports -- Wi-Fi and Thread -- starting with resource-constrained (i.e., memory, processing) silicon platforms to help ensure Matter’s scalability.

How to Contribute

We welcome your contributions to Matter. Read our contribution guidelines here.

Building and Developing in Matter

Instructions about how to build Matter can be found here .

Directory Structure

The Matter repository is structured as follows:

File/FolderContent
buildBuild system support content and built output directories
build_overridesBuild system parameter customization for different platforms
configProject configurations
credentialsDevelopment and test credentials
docsDocumentation, including guides. Visit the Matter SDK documentation page to read it.
examplesExample firmware applications that demonstrate use of Matter
integrations3rd party integrations
scriptsScripts needed to work with the Matter repository
srcImplementation of Matter
third_party3rd party code used by Matter
zzz_generatedZAP generated template code - Revolving around cluster information
BUILD.gnBuild file for the GN build system
CODE_OF_CONDUCT.mdCode of conduct for Matter and contribution to it
CONTRIBUTING.mdGuidelines for contributing to Matter
LICENSEMatter license file
REVIEWERS.mdPR reviewers
gn_build.shBuild script for specific projects such as Android, EFR32, etc.
README.mdThis file

License

Matter is released under the Apache 2.0 license.