docs_builder: Copy files with rsync

If you run 'cp foo bar', the result depends on whether 'bar' exists and
whether it's a regular file or a folder. If it doesn't exist or is a
regular file, 'bar' is replaced with 'foo'. If it's a folder, 'foo' is
inserted into 'bar' as 'bar/foo'. 'gsutil cp' has this same behavior,
which causes uploads to 'gs://pigweed-docs-try/<change-id>' to not
always end up in the same location. 'gsutil rsync' does not have this
issue, so switching to it.

Bug: b/352130983
Change-Id: I46f70f1cb12f79c2239e2ccd4448b73592df0b4a
Reviewed-on: https://pigweed-review.googlesource.com/c/infra/recipes/+/220788
Pigweed-Auto-Submit: Rob Mohr <mohrr@google.com>
Presubmit-Verified: CQ Bot Account <pigweed-scoped@luci-project-accounts.iam.gserviceaccount.com>
Lint: Lint 🤖 <android-build-ayeaye@system.gserviceaccount.com>
Commit-Queue: Auto-Submit <auto-submit@pigweed-service-accounts.iam.gserviceaccount.com>
Reviewed-by: Ted Pudlik <tpudlik@google.com>
1 file changed
tree: 59ca2a2aa034d5a478a513cf2e9d9205d6e3c990
  1. infra/
  2. recipe_modules/
  3. recipes/
  4. scripts/
  5. .editorconfig
  6. .gitignore
  7. .style.yapf
  8. AUTHORS
  9. cipd.ensure
  10. LICENSE
  11. OWNERS
  12. presubmit.sh
  13. pyproject.toml
  14. README.md
  15. recipes.py
README.md

Pigweed Recipes

This repository contains recipes for Pigweed.

A recipe is a Python script that runs a series of commands, using the recipe engine framework from the LUCI project. We use recipes to automatically check out, build, and test Pigweed and downstream projects using Pigweed in continuous integration jobs. The commands the recipes use are very similar to the ones you would use as a developer to check out, build, and test Pigweed in your local environment.

See go/pigweed-recipe-docs for complete documentation and a guide for getting started with writing recipes.

Getting the Code

The recommended way to get the source code is with git.

git clone https://pigweed.googlesource.com/infra/recipes

In most cases you will need a Chromium depot_tools checkout in your PATH as well.

git clone https://chromium.googlesource.com/chromium/tools/depot_tools ~/depot_tools
echo 'export PATH="$PATH:$HOME/depot_tools"' >> ~/.bashrc

Running Tests

./presubmit.sh runs three different sets of tests:

  • Recipe expectation tests (./recipes.py test train)
  • Formatting (./black --diff --check .)
  • Dependencies (.recipe_deps/fuchsia/scripts/cleanup_deps.py --check)

The formatting check will tell you what‘s wrong but not fix it. For that you need to run ./black .. Similarly, the dependencies check will tell you what’s wrong but you'll need to edit the files to fix issues.

If not using ./presubmit.sh you'll need to run ./scripts/ensure_black.sh before ./black is present.