bisector: Retry api.gitiles.log()
Retry api.gitiles.log() if it fails, but wait a bit first since failures
likely come from quota issues.
Also increase the wait time before a gitiles query.
Gitiles quota was hit last night in http://go/bbid/8720547962191514753.
Bug: b/401921575
Change-Id: I1532eca032ef90a948820d1d650339b899eeaf3d
Reviewed-on: https://pigweed-review.googlesource.com/c/infra/recipes/+/275632
Reviewed-by: Ted Pudlik <tpudlik@google.com>
Lint: Lint 🤖 <android-build-ayeaye@system.gserviceaccount.com>
Pigweed-Auto-Submit: Rob Mohr <mohrr@google.com>
Commit-Queue: Rob Mohr <mohrr@google.com>
diff --git a/recipes/bisector.py b/recipes/bisector.py
index 1778b1e..21f2f10 100644
--- a/recipes/bisector.py
+++ b/recipes/bisector.py
@@ -200,16 +200,26 @@
if remote not in repos:
# Avoid hitting gitiles quota. (And there's no need for
# this builder to run quickly.)
- api.time.sleep(3.0)
+ api.time.sleep(10.0)
- log = api.gitiles.log(
- url=remote.url,
- treeish=remote.branch,
- limit=200,
- test_data=[
- {'id': commit40(x)} for x in range(200)
- ],
- )
+ def gitiles_log():
+ return api.gitiles.log(
+ url=remote.url,
+ treeish=remote.branch,
+ limit=200,
+ test_data=[
+ {'id': commit40(x)} for x in range(200)
+ ],
+ )
+
+ # If the gitiles query fails it's likely a quota issue. Wait
+ # two minutes and try again. Only retry once since this could
+ # be a permissions issue.
+ try:
+ log = gitiles_log()
+ except api.step.StepFailure: # pragma: no cover
+ api.time.sleep(2 * 60.0)
+ log = gitiles_log()
repos[remote] = [x['id'] for x in log]
