[roll pigweed] pw_tokenizer: Support tokenized string expressions

Test: Verified PW_TOKENIZE_STRING and PW_TOKENIZE_STRING_EXPR have
 identical size reports (+8 bytes FLASH).
Test: See details in testing done comment in the code review.
Original-Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/104960

https://pigweed.googlesource.com/pigweed/pigweed
pigweed Rolled-Commits: e6425959d009d5c..fca61fd0dacc4b4
Roller-URL: https://ci.chromium.org/b/8806016620187291617
CQ-Do-Not-Cancel-Tryjobs: true
Change-Id: Ia3e4108f04edc499c2c55bd6e11f63d220eb4d75
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/zephyr-integration/+/106370
Bot-Commit: Pigweed Integration Roller <pigweed-integration-roller@pigweed.google.com.iam.gserviceaccount.com>
Commit-Queue: Pigweed Integration Roller <pigweed-integration-roller@pigweed.google.com.iam.gserviceaccount.com>
1 file changed
tree: de0e9e7286c43bcc81ffb1c4a80438d3e0538b62
  1. .gitmodules
  2. test.sh