Google Git
Sign in
pigweed / pigweed / pigweed / 36a1ef7322c44fc5a9566783674a435a49897abf
commit36a1ef7322c44fc5a9566783674a435a49897abf[log] [tgz]
authorArmando Montanez <amontanez@google.com>Thu Jan 13 17:23:45 2022 -0800
committerCQ Bot Account <pigweed-scoped@luci-project-accounts.iam.gserviceaccount.com>Sat Jan 15 00:14:59 2022 +0000
treebb028c57c249553bc0ab7e0107af8cff997a556f
parent49f21314ece3a2346fb5adf773ce123221e2712a [diff]
pw_tokenizer: Variable length hash

Renames pw_tokenizer_65599_fixed_length_hash to pw_tokenizer_65599_hash
and changes the function to do variable length tokenization by default.
This makes it easier to generate unbounded length tokens in addition to
fixed-length tokens.

Change-Id: I7dac54fbf39efa523708e9c1db49a28859078240
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/78881
Reviewed-by: Wyatt Hepler <hepler@google.com>
Pigweed-Auto-Submit: Armando Montanez <amontanez@google.com>
Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com>
4 files changed
tree: bb028c57c249553bc0ab7e0107af8cff997a556f
  1. build_overrides/
  2. docker/
  3. docs/
  4. pw_allocator/
  5. pw_analog/
  6. pw_android_toolchain/
  7. pw_arduino_build/
  8. pw_assert/
  9. pw_assert_basic/
  10. pw_assert_log/
  11. pw_assert_zephyr/
  12. pw_base64/
  13. pw_bloat/
  14. pw_blob_store/
  15. pw_bluetooth_hci/
  16. pw_boot/
  17. pw_boot_cortex_m/
  18. pw_build/
  19. pw_build_info/
  20. pw_build_mcuxpresso/
  21. pw_bytes/
  22. pw_checksum/
  23. pw_chrono/
  24. pw_chrono_embos/
  25. pw_chrono_freertos/
  26. pw_chrono_stl/
  27. pw_chrono_threadx/
  28. pw_chrono_zephyr/
  29. pw_cli/
  30. pw_console/
  31. pw_containers/
  32. pw_cpu_exception/
  33. pw_cpu_exception_cortex_m/
  34. pw_crypto/
  35. pw_docgen/
  36. pw_doctor/
  37. pw_env_setup/
  38. pw_file/
  39. pw_function/
  40. pw_fuzzer/
  41. pw_hdlc/
  42. pw_hex_dump/
  43. pw_i2c/
  44. pw_interrupt/
  45. pw_interrupt_cortex_m/
  46. pw_interrupt_zephyr/
  47. pw_kvs/
  48. pw_libc/
  49. pw_log/
  50. pw_log_basic/
  51. pw_log_null/
  52. pw_log_rpc/
  53. pw_log_tokenized/
  54. pw_log_zephyr/
  55. pw_malloc/
  56. pw_malloc_freelist/
  57. pw_metric/
  58. pw_minimal_cpp_stdlib/
  59. pw_module/
  60. pw_multisink/
  61. pw_package/
  62. pw_persistent_ram/
  63. pw_polyfill/
  64. pw_preprocessor/
  65. pw_presubmit/
  66. pw_protobuf/
  67. pw_protobuf_compiler/
  68. pw_random/
  69. pw_result/
  70. pw_ring_buffer/
  71. pw_router/
  72. pw_rpc/
  73. pw_snapshot/
  74. pw_software_update/
  75. pw_span/
  76. pw_spi/
  77. pw_status/
  78. pw_stm32cube_build/
  79. pw_stream/
  80. pw_string/
  81. pw_symbolizer/
  82. pw_sync/
  83. pw_sync_baremetal/
  84. pw_sync_embos/
  85. pw_sync_freertos/
  86. pw_sync_stl/
  87. pw_sync_threadx/
  88. pw_sync_zephyr/
  89. pw_sys_io/
  90. pw_sys_io_arduino/
  91. pw_sys_io_baremetal_lm3s6965evb/
  92. pw_sys_io_baremetal_stm32f429/
  93. pw_sys_io_mcuxpresso/
  94. pw_sys_io_stdio/
  95. pw_sys_io_stm32cube/
  96. pw_sys_io_zephyr/
  97. pw_system/
  98. pw_target_runner/
  99. pw_thread/
  100. pw_thread_embos/
  101. pw_thread_freertos/
  102. pw_thread_stl/
  103. pw_thread_threadx/
  104. pw_tls_client/
  105. pw_tls_client_boringssl/
  106. pw_tls_client_mbedtls/
  107. pw_tokenizer/
  108. pw_tool/
  109. pw_toolchain/
  110. pw_trace/
  111. pw_trace_tokenized/
  112. pw_transfer/
  113. pw_unit_test/
  114. pw_varint/
  115. pw_watch/
  116. pw_web_ui/
  117. pw_work_queue/
  118. targets/
  119. third_party/
  120. zephyr/
  121. .dockerignore ⇨ .gitignore
  122. activate.sh ⇨ bootstrap.sh
  123. .bazelignore
  124. .bazelrc
  125. .clang-format
  126. .clang-tidy
  127. .eslintrc.json
  128. .gitattributes
  129. .gitignore
  130. .gn
  131. .prettierrc.js
  132. .pylintrc
  133. activate.bat
  134. Android.bp
  135. AUTHORS
  136. bootstrap.bat
  137. bootstrap.sh
  138. BUILD.bazel
  139. BUILD.gn
  140. BUILDCONFIG.gn
  141. CMakeLists.txt
  142. Kconfig.zephyr
  143. LICENSE
  144. modules.gni
  145. OWNERS
  146. package.json
  147. PW_PLUGINS
  148. README.md
  149. tsconfig.json
  150. WORKSPACE
  151. yarn.lock
README.md

Pigweed

Pigweed is an open source collection of embedded-targeted libraries–or as we like to call them, modules. These modules are building blocks and infrastructure that enable faster and more reliable development on small-footprint MMU-less 32-bit microcontrollers like the STMicroelectronics STM32L452 or the Nordic nRF52832.

For more information please see our website: https://pigweed.dev/

Links