)]}'
{
  "commit": "5f26e2a87fd4492e4f9dddc153268eeca7dab56b",
  "tree": "85b140367d4750b7d3ea9c1ea1d2121c7b1061ee",
  "parents": [
    "25caa1938b13131e9cb9824c069c644c654ff493"
  ],
  "author": {
    "name": "Sergey Shevchenko",
    "email": "sergeygs@google.com",
    "time": "Wed Jan 03 17:23:23 2024 -0800"
  },
  "committer": {
    "name": "Copybara-Service",
    "email": "copybara-worker@google.com",
    "time": "Wed Jan 03 17:24:10 2024 -0800"
  },
  "message": "#Centipede Seeder: More RAM-efficient source element sampling\n\nAvoid a spike in peak RSS \u0026 VSize usage and speed up source shards processing.\n\nThe previous version std::sampled the elements themselves, inserting copies into the destination.\n\nThe new version std::shuffles and resizes a source\u0027s elements\u0027 indices to achieve the same effect of sampling, then moves selected elements from the source\u0027s vector to the destination vector.\n\nExample rusage profile for one large shard shows RSS reduction of 30%, from 36GB to 24GB, and sampling time decrease from 40 seconds to 2.\n\nBEFORE:\n\n```\nRSS:\n\nseed_corpus_maker_lib.cc:254 @ 14:49:18.95 [P.0:S.80 Done reading ]     25.22G [#################################-----------------]\nseed_corpus_maker_lib.cc:271 @ 14:49:20.07 [P.0:S.81 Done merging ]     25.22G [#################################-----------------]\nseed_corpus_maker_lib.cc:154 @ 14:49:24.50 [P.0:S.82 Timelapse    ]     26.22G [###################################---------------]\nseed_corpus_maker_lib.cc:154 @ 14:49:34.50 [P.0:S.83 Timelapse    ]     29.22G [#######################################-----------]\nseed_corpus_maker_lib.cc:154 @ 14:49:44.50 [P.0:S.84 Timelapse    ]     32.22G [############################################------]\nseed_corpus_maker_lib.cc:154 @ 14:49:54.50 [P.0:S.85 Timelapse    ]     35.22G [################################################--]\nseed_corpus_maker_lib.cc:307 @ 14:49:59.00 [P.0:S.86 Done sampling]     36.22G [##################################################]\nseed_corpus_maker_lib.cc:154 @ 14:50:04.50 [P.0:S.87 Timelapse    ]     36.22G [##################################################]\nseed_corpus_maker_lib.cc:154 @ 14:50:14.50 [P.0:S.88 Timelapse    ]     36.22G [##################################################]\nseed_corpus_maker_lib.cc:154 @ 14:50:24.50 [P.0:S.89 Timelapse    ]     36.22G [##################################################]\nseed_corpus_maker_lib.cc:154 @ 14:50:34.50 [P.0:S.90 Timelapse    ]     36.22G [##################################################]\nseed_corpus_maker_lib.cc:154 @ 14:50:44.50 [P.0:S.91 FINAL        ]     36.22G [##################################################]\n```\n\nAFTER:\n\n```\nRSS:\n\nseed_corpus_maker_lib.cc:256 @ 21:05:28.54 [P.0:S.74 Done reading  ]     24.07G [#################################################-]\nseed_corpus_maker_lib.cc:276 @ 21:05:29.61 [P.0:S.75 Done merging  ]     24.26G [#################################################-]\nseed_corpus_maker_lib.cc:316 @ 21:05:31.14 [P.0:S.76 Done sampling ]     24.26G [#################################################-]\nseed_corpus_maker_lib.cc:324 @ 21:05:31.57 [P.0:S.77 Done appending]     24.26G [#################################################-]\nseed_corpus_maker_lib.cc:156 @ 21:05:33.96 [P.0:S.78 Timelapse     ]     24.26G [#################################################-]\nseed_corpus_maker_lib.cc:156 @ 21:05:43.96 [P.0:S.79 Timelapse     ]     24.26G [#################################################-]\nseed_corpus_maker_lib.cc:156 @ 21:05:53.96 [P.0:S.80 FINAL         ]     24.26G [##################################################]\n```\n\nPiperOrigin-RevId: 595544166\n",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "4b06185648435250a8d5c66f01ca7d0dd51331df",
      "old_mode": 33188,
      "old_path": "centipede/seed_corpus_maker_lib.cc",
      "new_id": "82813023704c164c6a7ce20029e64e54eb25152d",
      "new_mode": 33188,
      "new_path": "centipede/seed_corpus_maker_lib.cc"
    },
    {
      "type": "modify",
      "old_id": "4b52676bb64c625da6fcd09f9545c8c0d68346fc",
      "old_mode": 33188,
      "old_path": "centipede/seed_corpus_maker_lib_test.cc",
      "new_id": "e7e40a3c7a1dc30b4e742ca1d2e0e8bc6d4b52fd",
      "new_mode": 33188,
      "new_path": "centipede/seed_corpus_maker_lib_test.cc"
    }
  ]
}