Updated the unit test documentation to reflect the transition to PW. (#33492)
* Revised unit testing documentation to refer to PW tests.
* Documentation update
* More doc updates
* More doc updates
* More doc updates
* More doc updates
* More doc updates
* More doc updates
* Updated documentation to reflect PW
* Fixed spelling error
* Replaced gtest.h include.
* Modified MessagingContext and its subclasses as well as AppContext to reflect PW migration.
* Undid last commit.
* Undid last commit.
* Undid last commit.
* Restyled by prettier-markdown
* Changed the includes to be consistent.
* Cleaned up example code comments
---------
Co-authored-by: Restyled.io <commits@restyled.io>
diff --git a/docs/guides/openiotsdk_unit_tests.md b/docs/guides/openiotsdk_unit_tests.md
index 7b9bba5..7560d4d 100644
--- a/docs/guides/openiotsdk_unit_tests.md
+++ b/docs/guides/openiotsdk_unit_tests.md
@@ -6,8 +6,8 @@
sources are built as a static library that can be linked to the unit test
application separately or as a monolithic test library. The common Matter test
library collects all test cases and provides the engine based on
-[Nest Labs Unit Test](https://github.com/nestlabs/nlunit-test) to run them in
-the application.
+[Pigweed Unit Test](https://pigweed.dev/pw_unit_test) to run them in the
+application.
The Open IoT SDK unit tests implementation are located in the
`src/test_driver/openiotsdk/unit-tests` directory. This project builds a
@@ -27,6 +27,7 @@
CoreTests
CredentialsTest
DataModelTests
+ICDServerTests
InetLayerTests
MdnsTests
MessagingLayerTests
@@ -41,7 +42,6 @@
SupportTests
SystemLayerTests
TestShell
-TransportLayerTests
UserDirectedCommissioningTests
```
diff --git a/docs/testing/unit_testing.md b/docs/testing/unit_testing.md
index c539420..64046a4 100644
--- a/docs/testing/unit_testing.md
+++ b/docs/testing/unit_testing.md
@@ -11,85 +11,347 @@
example applications.
- e.g. feature combinations not in example apps.
-## Unit testing in the SDK - nlUnitTest
+## Unit testing in the SDK using pw_unit_test
-The following example gives a small demonstration of how to use nlUnitTest to
-write a unit test
+This SDK uses Pigweed unit test (pw_unit_test), which is an implementation of
+GoogleTest. For more information see the
+[pw_unit_test documentation](https://pigweed.dev/pw_unit_test/) or the
+[GoogleTest documentation](https://google.github.io/googletest/).
+
+### Simple unit tests
+
+The following example demonstrates how to use pw_unit_test to write a simple
+unit test. Each test function is defined using `TEST(NameOfFunction)`. The set
+of test functions in a given source file is called a "suite".
```
-#include <lib/support/UnitTestContext.h>
-#include <lib/support/UnitTestRegistration.h>
-#include <nlunit-test.h>
+#include <pw_unit_test/framework.h>
-class YourTestContext : public Test::AppContext {
- ...
-};
+TEST(YourTestFunction1)
+{
+ // Do some test things here, then check the results using EXPECT_*
+ SomeTypeX foo;
+ foo.DoSomething();
+ EXPECT_EQ(foo.GetResultCount(), 7);
+ foo.DoSomethingElse();
+ EXPECT_EQ(foo.GetResultCount(), 5);
-
-static void TestName(nlTestSuite * apSuite, void * apContext) {
- // If you register the test suite with a context, cast
- // apContext as appropriate
- YourTestContext * ctx = static_cast<YourTestContext *>(aContext);
-
- // Do some test things here, then check the results using NL_TEST_ASSERT
- NL_TEST_ASSERT(apSuite, <boolean condition>)
+ // If you want to abort the rest of the test upon failure, use ASSERT_*
+ SomeTypeY * ptr = foo.GetSomePointer();
+ ASSERT_NE(ptr, nullptr);
+ ptr->DoTheThing(); // Won't reach here if the ASSERT failed.
}
-static const nlTest sTests[] =
+TEST(YourTestFunction2)
{
- NL_TEST_DEF("TestName", TestName), // Can have multiple of these
- NL_TEST_SENTINEL() // If you forget this, you’re going to have a bad time
-};
-
-nlTestSuite sSuite =
-{
- "TheNameOfYourTestSuite", // Test name
- &sTests[0], // The list of tests to run
- TestContext::Initialize, // Runs before all the tests (can be nullptr)
- TestContext::Finalize // Runs after all the tests (can be nullptr)
-};
-
-int YourTestSuiteName()
-{
- return chip::ExecuteTestsWithContext<YourTestContext>(&sSuite); // or “without”
+ // Do some test things here, then check the results using EXPECT_*
+ SomeTypeZ foo;
+ foo.DoSomething();
+ EXPECT_EQ(foo.GetResultCount(), 3);
}
-
-CHIP_REGISTER_TEST_SUITE(YourTestSuiteName)
```
-Each test gets an nlTestSuite object (apSuite) that is passed into the test
-assertions, and a void\* context (apContext) that is yours to do with as you
-require.
-
-The apContext should be derived from
-[Test::AppContext](https://github.com/project-chip/connectedhomeip/blob/master/src/app/tests/AppTestContext.h)
-
See
[TestSpan.cpp](https://github.com/project-chip/connectedhomeip/blob/master/src/lib/support/tests/TestSpan.cpp)
-for a great example of a good unit test.
+for an example of a simple unit test.
-## nlUnitTest - Compiling and running
+In the above example there are no fixtures or setup/teardown behavior.
-- Add to src/some_directory/tests/BUILD.gn
- - chip_test_suite_using_nltest("tests")
- - See for example
- [src/lib/support/tests/BUILD.gn](https://github.com/project-chip/connectedhomeip/blob/master/src/lib/support/tests/BUILD.gn)
-- [./gn_build.sh](https://github.com/project-chip/connectedhomeip/blob/master/gn_build.sh)
- will build and run all tests
+### Test fixtures and setup/teardown behavior
+
+If your tests need fixtures or some kind of setup/teardown you will need to
+define a test context that derives from `::testing::Test`. Each of your test
+functions will be defined with `TEST_F(NameOfTestContext, NameOfFunction)`. The
+following example demonstrates how to use pw_unit_test to write a unit test that
+uses fixtures and setup/teardown behavior.
+
+```
+#include <pw_unit_test/framework.h>
+
+class YourTestContext : public ::testing::Test
+{
+public:
+ // Performs shared setup for all tests in the test suite. Run once for the whole suite.
+ static void SetUpTestSuite()
+ {
+ // Your per-suite setup goes here:
+ sPerSuiteFixture.Init();
+ ASSERT_TRUE(sPerSuiteFixture.WorkingGreat());
+ }
+
+ // Performs shared teardown for all tests in the test suite. Run once for the whole suite.
+ static void TearDownTestSuite()
+ {
+ // Your per-suite teardown goes here:
+ sPerSuiteFixture.Shutdown();
+ }
+
+protected:
+ // Performs setup for each test in the suite. Run once for each test function.
+ void SetUp()
+ {
+ // Your per-test setup goes here:
+ mPerTestFixture.Init();
+ ASSERT_TRUE(mPerTestFixture.WorkingGreat());
+ }
+
+ // Performs teardown for each test in the suite. Run once for each test function.
+ void TearDown()
+ {
+ // Your per-test teardown goes here:
+ mPerTestFixture.Shutdown();
+ }
+
+private:
+ // Your per-suite and per-test fixtures are declared here:
+ static SomeTypeA sPerSuiteFixture;
+ SomeTypeB mPerTestFixture;
+};
+// Your per-suite fixtures are defined here:
+SomeTypeA YourTestContext::sPerSuiteFixture;
+
+TEST_F(YourTestContext, YourTestFunction1)
+{
+ // Do some test things here, then check the results using EXPECT_*
+ mPerTestFixture.DoSomething();
+ EXPECT_EQ(mPerTestFixture.GetResultCount(), 7);
+ sPerSuiteFixture.DoSomething();
+ EXPECT_EQ(sPerSuiteFixture.GetResultCount(), 5);
+
+ // If you want to abort the rest of the test upon failure, use ASSERT_*
+ SomeTypeC * ptr = mPerTestFixture.GetSomePointer();
+ ASSERT_NE(ptr, nullptr);
+ ptr->DoTheThing(); // Won't reach here if the ASSERT failed.
+}
+
+TEST_F(YourTestContext, YourTestFunction2)
+{
+ // Do some test things here, then check the results using EXPECT_*
+ mPerTestFixture.DoSomethingElse();
+ EXPECT_EQ(mPerTestFixture.GetResultCount(), 9);
+}
+```
+
+### A loopback messaging context for convenience
+
+If you need messaging, there is a convenience class
+[Test::AppContext](https://github.com/project-chip/connectedhomeip/blob/master/src/app/tests/AppTestContext.h)
+that you can derive your test context from. It provides a network layer and a
+system layer and two secure sessions connected with each other. The following
+example demonstrates this.
+
+```
+#include <app/tests/AppTestContext.h>
+
+class YourTestContext : public Test::AppContext
+{
+public:
+ // Performs shared setup for all tests in the test suite. Run once for the whole suite.
+ static void SetUpTestSuite()
+ {
+ AppContext::SetUpTestSuite(); // Call parent.
+ VerifyOrReturn(!HasFailure()); // Stop if parent had a failure.
+
+ // Your per-suite setup goes here:
+ sPerSuiteFixture.Init();
+ ASSERT_TRUE(sPerSuiteFixture.WorkingGreat());
+ }
+
+ // Performs shared teardown for all tests in the test suite. Run once for the whole suite.
+ static void TearDownTestSuite()
+ {
+ // Your per-suite teardown goes here:
+ sPerSuiteFixture.Shutdown();
+
+ AppContext::TearDownTestSuite(); // Call parent.
+ }
+
+protected:
+ // Performs setup for each test in the suite. Run once for each test function.
+ void SetUp()
+ {
+ AppContext::SetUp(); // Call parent.
+ VerifyOrReturn(!HasFailure()); // Stop if parent had a failure.
+
+ // Your per-test setup goes here:
+ mPerTestFixture.Init();
+ ASSERT_TRUE(mPerTestFixture.WorkingGreat());
+ }
+
+ // Performs teardown for each test in the suite. Run once for each test function.
+ void TearDown()
+ {
+ // Your per-test teardown goes here:
+ mPerTestFixture.Shutdown();
+
+ chip::app::EventManagement::DestroyEventManagement();
+ AppContext::TearDown(); // Call parent.
+ }
+
+private:
+ // Your per-suite and per-test fixtures are declared here:
+ static SomeTypeA sPerSuiteFixture;
+ SomeTypeB mPerTestFixture;
+};
+// Your per-suite fixtures are defined here:
+SomeTypeA YourTestContext::sPerSuiteFixture;
+
+TEST_F(YourTestContext, YourTestFunction1)
+{
+ // Do some test things here, then check the results using EXPECT_*
+}
+
+TEST_F(YourTestContext, YourTestFunction2)
+{
+ // Do some test things here, then check the results using EXPECT_*
+}
+```
+
+You don't have to override all 4 functions `SetUpTestsuite`,
+`TearDownTestSuite`, `SetUp`, `TearDown`. If you don't need any custom behavior
+in one of those functions just omit it.
+
+If you override one of the setup/teardown functions make sure to invoke the
+parent's version of the function as well. `AppContext::SetUpTestSuite` and
+`AppContext::SetUp` may generate fatal failures, so after you call these from
+your overriding function make sure to check `HasFailure()` and return if the
+parent function failed.
+
+If you don't override any of the setup/teardown functions, you can simply make a
+type alias: `using YourTestContext = Test::AppContextPW;` instead of defining
+your own text context class.
+
+## Best practices
+
+- Try to use as specific an assertion as possible. For example use these
+
+ ```
+ EXPECT_EQ(result, 3);
+ EXPECT_GT(result, 1);
+ EXPECT_STREQ(myString, "hello");
+ ```
+
+ instead of these
+
+ ```
+ EXPECT_TRUE(result == 3);
+ EXPECT_TRUE(result > 1);
+ EXPECT_EQ(strcmp(myString, "hello"), 0);
+ ```
+
+- If you want a test to abort when an assertion fails, use `ASSERT_*` instead
+ of `EXPECT_*`. This will cause the current function to return.
+
+- If a test calls a subroutine which exits due to an ASSERT failing, execution
+ of the test will continue after the subroutine call. This is because ASSERT
+ causes the subroutine to return (as opposed to throwing an exception). If
+ you want to prevent the test from continuing, check the value of
+ `HasFailure()` and stop execution if true. Example:
+
+ ```
+ void Subroutine()
+ {
+ ASSERT_EQ(1, 2); // Fatal failure.
+ }
+
+ TEST(YourTestContext, YourTestFunction1)
+ {
+ Subroutine(); // A fatal failure happens in this subroutine...
+ // ... however execution still continues.
+ print("This gets executed");
+ VerifyOrReturn(!HasFailure());
+ print("This does not get executed");
+ }
+ ```
+
+- If you want to force a fatal failure use `FAIL()`, which will record a fatal
+ failure and exit the current function. This is similar to using
+ `ASSERT_TRUE(false)`. If you want to force a non-fatal failure use
+ `ADD_FAILURE()`, which will record a non-fatal failure and continue
+ executing the current function. This is similar to using
+ `EXPECT_TRUE(false)`.
+
+- `ASSERT_*` and `FAIL` will only work in functions with void return type,
+ since they generate a `return;` statement. If you must use these in a
+ non-void function, instead use `EXPECT_*` or `ADD_FAILURE` and then check
+ `HasFailure()` afterward and return if needed.
+
+- If your test requires access to private/protected members of the underlying
+ class you're testing, you'll need to create an accessor class that performs
+ these operations and is friended to the underlying class. Please name the
+ class `chip::Test::SomethingTestAccess` where `Something` is the name of the
+ underlying class whose private/protected members you're trying to access.
+ Then add `friend class chip::Test::SomethingTestAccess;` to the underlying
+ class. Make sure your test's BUILD.gn file contains
+ `sources = [ "SomethingTestAccess.h" ]`. Before creating a new TestAccess
+ class, check if one already exists. If it does exist but doesn't expose the
+ member you need, you can add a function to that class to do so. Note that
+ you should make these functions as minimal as possible, with no logic
+ besides just exposing the private/protected member.
+ - For an example see `ICDConfigurationDataTestAccess` which is defined in
+ [ICDConfigurationDataTestAccess.h](https://github.com/project-chip/connectedhomeip/blob/master/src/app/icd/server/tests/ICDConfigurationDataTestAccess.h),
+ friends the underlying class in
+ [ICDConfigurationData.h](https://github.com/project-chip/connectedhomeip/blob/master/src/app/icd/server/ICDConfigurationData.h),
+ is included as a source in
+ [BUILD.gn](https://github.com/project-chip/connectedhomeip/blob/master/src/app/icd/server/tests/BUILD.gn),
+ and is used by a test in
+ [TestICDManager.cpp](https://github.com/project-chip/connectedhomeip/blob/master/src/app/icd/server/tests/TestICDManager.cpp).
+ - For another example see `TCPBaseTestAccess` which is defined in
+ [TCPBaseTestAccess.h](https://github.com/project-chip/connectedhomeip/blob/master/src/transport/raw/tests/TCPBaseTestAccess.h),
+ friends the underlying class in
+ [TCP.h](https://github.com/project-chip/connectedhomeip/blob/master/src/transport/raw/TCP.h),
+ is included as a source in
+ [BUILD.gn](https://github.com/project-chip/connectedhomeip/blob/master/src/transport/raw/tests/BUILD.gn),
+ and is used by a test in
+ [TestTCP.cpp](https://github.com/project-chip/connectedhomeip/blob/master/src/transport/raw/tests/TestTCP.cpp).
+
+## Compiling and running
+
+- Add to `src/some_directory/tests/BUILD.gn`
+
+ - Example
+
+ ```
+ chip_test_suite("tests") {
+ output_name = "libSomethingTests"
+
+ test_sources = [
+ "TestSuite1.cpp",
+ "TestSuite2.cpp",
+ // Other test source files go here.
+ ]
+
+ sources = [
+ // Non-test source files go here.
+ ]
+
+ cflags = [ "-Wconversion" ]
+
+ public_deps = [
+ // Dependencies go here.
+ ]
+ }
+ ```
+
+ - Another example:
+ [src/lib/support/tests/BUILD.gn](https://github.com/project-chip/connectedhomeip/blob/master/src/lib/support/tests/BUILD.gn)
+
+- Build and run all tests with
+ [./gn_build.sh](https://github.com/project-chip/connectedhomeip/blob/master/gn_build.sh)
- CI runs this, so any unit tests that get added will automatically be
- added to the CI
+ added to the CI.
- Test binaries are compiled into:
- - out/debug/<host_compiler>/tests
- - e.g. out/debug/linux_x64_clang/tests
-- Tests are run when ./gn_build.sh runs, but you can run them individually in
- a debugger from their location.
+ - `out/debug/<host_compiler>/tests`
+ - e.g. `out/debug/linux_x64_clang/tests`
+- Tests are run when `./gn_build.sh` runs, but you can run them individually
+ in a debugger from their location.
## Debugging unit tests
-- After running ./gn_build.sh, test binaries are compiled into
- - out/debug/<host_compiler>/tests
- - e.g. out/debug/linux_x64_clang/tests
-- Individual binaries, can be run through regular tools:
+- After running `./gn_build.sh`, test binaries are compiled into
+ - `out/debug/<host_compiler>/tests`
+ - e.g. `out/debug/linux_x64_clang/tests`
+- Individual binaries can be run through regular tools:
- gdb
- valgrind
- Your favorite tool that you tell everyone about.
