pw_tokenizer: Keep the newest removal date for duplicate tokens

- If entries are added to a database that already contains them, update
  to the newest date removed.
- Have the Database __init__ call self.add(entries) so semantics are
  consistent.
- Restore the DirectoryDatabase test that was deleted because the
  semantics for dates removed with duplicate tokens had not been
  defined.

Change-Id: Id2d6bb60537a51521c2e9ed26c838a009c73294a
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/104681
Reviewed-by: Carlos Chinchilla <cachinchilla@google.com>
Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com>
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
3 files changed
tree: 5695fa33af5355e7735a556e007bbe511440fd2f
  1. build_overrides/
  2. docker/
  3. docs/
  4. pw_allocator/
  5. pw_analog/
  6. pw_android_toolchain/
  7. pw_arduino_build/
  8. pw_assert/
  9. pw_assert_basic/
  10. pw_assert_log/
  11. pw_assert_tokenized/
  12. pw_assert_zephyr/
  13. pw_base64/
  14. pw_bloat/
  15. pw_blob_store/
  16. pw_bluetooth/
  17. pw_bluetooth_hci/
  18. pw_boot/
  19. pw_boot_cortex_m/
  20. pw_build/
  21. pw_build_info/
  22. pw_build_mcuxpresso/
  23. pw_bytes/
  24. pw_checksum/
  25. pw_chrono/
  26. pw_chrono_embos/
  27. pw_chrono_freertos/
  28. pw_chrono_stl/
  29. pw_chrono_threadx/
  30. pw_chrono_zephyr/
  31. pw_cli/
  32. pw_console/
  33. pw_containers/
  34. pw_cpu_exception/
  35. pw_cpu_exception_cortex_m/
  36. pw_crypto/
  37. pw_digital_io/
  38. pw_docgen/
  39. pw_doctor/
  40. pw_env_setup/
  41. pw_file/
  42. pw_function/
  43. pw_fuzzer/
  44. pw_hdlc/
  45. pw_hex_dump/
  46. pw_i2c/
  47. pw_i2c_mcuxpresso/
  48. pw_interrupt/
  49. pw_interrupt_cortex_m/
  50. pw_interrupt_zephyr/
  51. pw_kvs/
  52. pw_libc/
  53. pw_log/
  54. pw_log_android/
  55. pw_log_basic/
  56. pw_log_null/
  57. pw_log_rpc/
  58. pw_log_string/
  59. pw_log_tokenized/
  60. pw_log_zephyr/
  61. pw_malloc/
  62. pw_malloc_freelist/
  63. pw_metric/
  64. pw_minimal_cpp_stdlib/
  65. pw_module/
  66. pw_multisink/
  67. pw_package/
  68. pw_persistent_ram/
  69. pw_polyfill/
  70. pw_preprocessor/
  71. pw_presubmit/
  72. pw_protobuf/
  73. pw_protobuf_compiler/
  74. pw_random/
  75. pw_result/
  76. pw_ring_buffer/
  77. pw_router/
  78. pw_rpc/
  79. pw_snapshot/
  80. pw_software_update/
  81. pw_span/
  82. pw_spi/
  83. pw_status/
  84. pw_stm32cube_build/
  85. pw_stream/
  86. pw_string/
  87. pw_symbolizer/
  88. pw_sync/
  89. pw_sync_baremetal/
  90. pw_sync_embos/
  91. pw_sync_freertos/
  92. pw_sync_stl/
  93. pw_sync_threadx/
  94. pw_sync_zephyr/
  95. pw_sys_io/
  96. pw_sys_io_arduino/
  97. pw_sys_io_baremetal_lm3s6965evb/
  98. pw_sys_io_baremetal_stm32f429/
  99. pw_sys_io_emcraft_sf2/
  100. pw_sys_io_mcuxpresso/
  101. pw_sys_io_stdio/
  102. pw_sys_io_stm32cube/
  103. pw_sys_io_zephyr/
  104. pw_system/
  105. pw_target_runner/
  106. pw_thread/
  107. pw_thread_embos/
  108. pw_thread_freertos/
  109. pw_thread_stl/
  110. pw_thread_threadx/
  111. pw_tls_client/
  112. pw_tls_client_boringssl/
  113. pw_tls_client_mbedtls/
  114. pw_tokenizer/
  115. pw_tool/
  116. pw_toolchain/
  117. pw_trace/
  118. pw_trace_tokenized/
  119. pw_transfer/
  120. pw_unit_test/
  121. pw_varint/
  122. pw_watch/
  123. pw_web_ui/
  124. pw_work_queue/
  125. targets/
  126. third_party/
  127. ts/
  128. zephyr/
  129. .bazelignore
  130. .bazelrc
  131. .clang-format
  132. .clang-tidy
  133. .eslintrc.json
  134. .gitattributes
  135. .gitignore
  136. .gn
  137. .prettierrc.js
  138. .pylintrc
  139. activate.bat
  140. Android.bp
  141. AUTHORS
  142. bootstrap.bat
  143. bootstrap.sh
  144. BUILD.bazel
  145. BUILD.gn
  146. BUILDCONFIG.gn
  147. CMakeLists.txt
  148. jest.config.ts
  149. Kconfig.zephyr
  150. LICENSE
  151. modules.gni
  152. OWNERS
  153. package-lock.json
  154. package.json
  155. PIGWEED_MODULES
  156. PW_PLUGINS
  157. README.md
  158. rollup.config.js
  159. tsconfig.json
  160. WORKSPACE
README.md

Pigweed

Pigweed is an open source collection of embedded-targeted libraries–or as we like to call them, modules. These modules are building blocks and infrastructure that enable faster and more reliable development on small-footprint MMU-less 32-bit microcontrollers like the STMicroelectronics STM32L452 or the Nordic nRF52832.

For more information please see our website: https://pigweed.dev/.

Links