pw_kvs: Remove temp buffer; redundancy changes

- Instead of copying entries to a large working buffer, incrementally
  read and checksum them and then incrementally copy them. Remove the
  working buffer.
- Find space for all redundant entries before writing any of them. This
  ensures that redundancy is maintained when adding entries to a
  mostly-full KVS.
- Update KeyDescriptors immediately after writing the first copy of
  an entry, instead of after attempting to write all copies.
- Eliminate temporary KeyDescriptor objects. Remove the public
  KeyDescriptor constructors, including the default.

Change-Id: Ia3674e260c9ab0fdc01965563343b2cf5da37adf
5 files changed
tree: ff867904ea593410a0e6ff14eaad17ce32b1b6fa
  1. docs/
  2. pw_assert/
  3. pw_assert_basic/
  4. pw_base64/
  5. pw_bloat/
  6. pw_boot_armv7m/
  7. pw_build/
  8. pw_checksum/
  9. pw_cli/
  10. pw_containers/
  11. pw_cpu_exception/
  12. pw_cpu_exception_armv7m/
  13. pw_docgen/
  14. pw_doctor/
  15. pw_env_setup/
  16. pw_kvs/
  17. pw_log/
  18. pw_log_basic/
  19. pw_minimal_cpp_stdlib/
  20. pw_module/
  21. pw_polyfill/
  22. pw_preprocessor/
  23. pw_presubmit/
  24. pw_protobuf/
  25. pw_protobuf_compiler/
  26. pw_span/
  27. pw_status/
  28. pw_string/
  29. pw_sys_io/
  30. pw_sys_io_baremetal_stm32f429/
  31. pw_sys_io_stdio/
  32. pw_target_runner/
  33. pw_tokenizer/
  34. pw_toolchain/
  35. pw_unit_test/
  36. pw_varint/
  37. pw_watch/
  38. targets/
  39. .clang-format
  40. .gitignore
  41. .gn
  42. .pylintrc
  43. activate.bat
  45. bootstrap.bat
  47. BUILD
  50. CMakeLists.txt
  54. modules.gni
  55. pw_vars_default.gni


Pigweed is an open source collection of embedded-targeted libraries--or as we like to call them, modules. These modules are building blocks and infrastructure that enable faster and more reliable development on small-footprint MMU-less 32-bit microcontrollers like the STMicroelectronics STM32L452 or the Nordic nRF52832.

Pigweed is in the early stages of development, and should be considered experimental. We’re continuing to evolve the platform and add new modules. We value developer feedback along the way.

Pigweed is an open source project with a code of conduct that we expect everyone who interacts with the project to respect.

Getting Started

If you'd like to get set up with Pigweed, please visit the setup guide, and then check out the Pigweed developer's guide for more information on how to get the most out of Pigweed.

What does Pigweed offer?

There are many modules in Pigweed, and this section only showcases a small selection that happen to produce visual output. For more information about the different Pigweed module offerings, refer to “modules” section in the full documentation.


In the web development space, file system watchers are prevalent. These watchers trigger a web server reload on source change, making development much faster. In the embedded space, file system watchers are less prevalent; however, they are no less useful! The Pigweed watcher module makes it easy to instantly compile, flash, and run tests upon save. Combined with the GN-based build which expresses the full dependency tree, only the exact tests affected by a file change are run on saves.

The demo below shows pw_watch building for a STMicroelectronics STM32F429I-DISC1 development board, flashing the board with the affected test, and verifying the test runs as expected. Once this is set up, you can attach multiple devices to run tests in a distributed manner to reduce the time it takes to run tests.

pw watch running on-device tests


Presubmit checks are essential tools, but they take work to set up, and projects don’t always get around to it. The pw_presubmit module provides tools for setting up high quality presubmit checks for any project. We use this framework to run Pigweed’s presubmit on our workstations and in our automated building tools.

The pw_presubmit module includes pw format, a tool that provides a unified interface for automatically formatting code in a variety of languages. With pw format, you can format C, C++, Python, GN, and Go code according to configurations defined by your project. pw format leverages existing tools like clang-format, and it’s simple to add support for new languages.

pw presubmit demo


A classic problem in the embedded space is reducing the time from git clone to having a binary executing on a device. The issue is that an entire suite of tools is needed for non-trivial production embedded projects. For example:

  • A C++ compiler for your target device, and also for your host
  • A build system or three; for example, GN, Ninja, CMake, Bazel
  • A code formatting program like clang-format
  • A debugger like OpenOCD to flash and debug your embedded device
  • A known Python version with known modules installed for scripting
  • A Go compiler for the Go-based command line tools
  • ... and so on

In the server space, container solutions like Docker or Podman solve this; however, in our experience container solutions are a mixed bag for embedded systems development where one frequently needs access to native system resources like USB devices, or must operate on Windows.

env_setup is our compromise solution for this problem that works on Mac, Windows, and Linux. It leverages the Chrome packaging system CIPD to bootstrap a Python installation, which in turn inflates a virtual environment. The tooling is installed into your workspace, and makes no changes to your system. This tooling is designed to be reused by any project.


Unit testing is important, and Pigweed offers a portable library that’s broadly compatible with Google Test. Unlike Google Test, pw_unit_test is built on top of embedded friendly primitives; for example, it does not use dynamic memory allocation. Additionally, it is easy to port to new target platforms by implementing the test event handler interface.