pw_tokenizer: Handle finding multiple CSVs in a directory database

- Handle multiple untracked files by using the most recently modified
  file.
- Check if HEAD is in an ancestor of the specified commit rather than
  HEAD~, which was unintentionally being used. This prevents updating a
  previously committed CSV.
- Move the _new_entries() function to the Database class as
  difference().
- Use the .pw_tokenizer.csv extension in directory databases to indicate
  that the CSVs are for pw_tokenizer.
- Rename DatabaseFile.create() to DatabaseFile.load() to more accurately
  express what it does.
- Remove some unnecessary functions.

Fixes: b/243040287
Change-Id: I63b4479847937f8ebaced568389d4eaa601c1c96
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/107965
Reviewed-by: Anthony DiGirolamo <tonymd@google.com>
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
4 files changed
tree: 10f6bc0a3a37c6638c28db1e339fffb4f560e3d7
  1. build_overrides/
  2. docker/
  3. docs/
  4. pw_allocator/
  5. pw_analog/
  6. pw_android_toolchain/
  7. pw_arduino_build/
  8. pw_assert/
  9. pw_assert_basic/
  10. pw_assert_log/
  11. pw_assert_tokenized/
  12. pw_assert_zephyr/
  13. pw_base64/
  14. pw_bloat/
  15. pw_blob_store/
  16. pw_bluetooth/
  17. pw_bluetooth_hci/
  18. pw_boot/
  19. pw_boot_cortex_m/
  20. pw_build/
  21. pw_build_info/
  22. pw_build_mcuxpresso/
  23. pw_bytes/
  24. pw_checksum/
  25. pw_chrono/
  26. pw_chrono_embos/
  27. pw_chrono_freertos/
  28. pw_chrono_stl/
  29. pw_chrono_threadx/
  30. pw_chrono_zephyr/
  31. pw_cli/
  32. pw_compilation_testing/
  33. pw_console/
  34. pw_containers/
  35. pw_cpu_exception/
  36. pw_cpu_exception_cortex_m/
  37. pw_crypto/
  38. pw_digital_io/
  39. pw_docgen/
  40. pw_doctor/
  41. pw_env_setup/
  42. pw_file/
  43. pw_function/
  44. pw_fuzzer/
  45. pw_hdlc/
  46. pw_hex_dump/
  47. pw_i2c/
  48. pw_i2c_mcuxpresso/
  49. pw_interrupt/
  50. pw_interrupt_cortex_m/
  51. pw_interrupt_zephyr/
  52. pw_kvs/
  53. pw_libc/
  54. pw_log/
  55. pw_log_android/
  56. pw_log_basic/
  57. pw_log_null/
  58. pw_log_rpc/
  59. pw_log_string/
  60. pw_log_tokenized/
  61. pw_log_zephyr/
  62. pw_malloc/
  63. pw_malloc_freelist/
  64. pw_metric/
  65. pw_minimal_cpp_stdlib/
  66. pw_module/
  67. pw_multisink/
  68. pw_package/
  69. pw_persistent_ram/
  70. pw_polyfill/
  71. pw_preprocessor/
  72. pw_presubmit/
  73. pw_protobuf/
  74. pw_protobuf_compiler/
  75. pw_random/
  76. pw_result/
  77. pw_ring_buffer/
  78. pw_router/
  79. pw_rpc/
  80. pw_rust/
  81. pw_snapshot/
  82. pw_software_update/
  83. pw_span/
  84. pw_spi/
  85. pw_status/
  86. pw_stm32cube_build/
  87. pw_stream/
  88. pw_string/
  89. pw_symbolizer/
  90. pw_sync/
  91. pw_sync_baremetal/
  92. pw_sync_embos/
  93. pw_sync_freertos/
  94. pw_sync_stl/
  95. pw_sync_threadx/
  96. pw_sync_zephyr/
  97. pw_sys_io/
  98. pw_sys_io_arduino/
  99. pw_sys_io_baremetal_lm3s6965evb/
  100. pw_sys_io_baremetal_stm32f429/
  101. pw_sys_io_emcraft_sf2/
  102. pw_sys_io_mcuxpresso/
  103. pw_sys_io_stdio/
  104. pw_sys_io_stm32cube/
  105. pw_sys_io_zephyr/
  106. pw_system/
  107. pw_target_runner/
  108. pw_thread/
  109. pw_thread_embos/
  110. pw_thread_freertos/
  111. pw_thread_stl/
  112. pw_thread_threadx/
  113. pw_tls_client/
  114. pw_tls_client_boringssl/
  115. pw_tls_client_mbedtls/
  116. pw_tokenizer/
  117. pw_tool/
  118. pw_toolchain/
  119. pw_trace/
  120. pw_trace_tokenized/
  121. pw_transfer/
  122. pw_unit_test/
  123. pw_varint/
  124. pw_watch/
  125. pw_web/
  126. pw_work_queue/
  127. targets/
  128. third_party/
  129. ts/
  130. zephyr/
  131. .bazelignore
  132. .bazelrc
  133. .clang-format
  134. .clang-tidy
  135. .eslintrc.json
  136. .gitattributes
  137. .gitignore
  138. .gn
  139. .prettierrc.js
  140. .pylintrc
  141. activate.bat
  142. Android.bp
  143. AUTHORS
  144. bootstrap.bat
  145. bootstrap.sh
  146. BUILD.bazel
  147. BUILD.gn
  148. BUILDCONFIG.gn
  149. CMakeLists.txt
  150. jest.config.ts
  151. Kconfig.zephyr
  152. LICENSE
  153. modules.gni
  154. OWNERS
  155. package-lock.json
  156. package.json
  157. PIGWEED_MODULES
  158. PW_PLUGINS
  159. README.md
  160. rollup.config.js
  161. tsconfig.json
  162. WORKSPACE
README.md

Pigweed

Pigweed is an open source collection of embedded-targeted libraries–or as we like to call them, modules. These modules are building blocks and infrastructure that enable faster and more reliable development on small-footprint MMU-less 32-bit microcontrollers like the STMicroelectronics STM32L452 or the Nordic nRF52832.

For more information please see our website: https://pigweed.dev/.

Links