feat(pip_repository): Support pip parse cycles (#1166)
This patch reworks the `pip_repository` machinery to allow users to
manually annotate groups of libraries which form packaging cycles in
PyPi and must be simultaneously installed.
The strategy here is to transform any dependencies `A` and `B` which
have dependencies and are mutually dependent
```mermaid
graph LR;
A-->B;
A-->D;
A-->E;
B-->A;
B-->F;
B-->G;
```
into a new "dependency group" `C` which has `A*` and `B*` as
dependencies, defined as `A` and `B` less any direct dependencies which
are members of the group. This is viable _for python_ because Python
files just need to be emplaced into a runfiles directory for the
interpreter. We don't actually have a true hard dependency between the
build definition of `A` requiring the build product `B` be available
which requires that the build product of `A` be available.
```mermaid
graph LR
C-->A*;
A*-->D;
A*-->E;
C-->B*;
B*-->F;
B*-->G;
```
This gets us most of the way there, as a user can now safely write
`requirement("A")` and we can provide them with `C`, which has the
desired effect of pulling in `A`, `B` and their respective transitives.
There is one remaining problem - a user writing `deps =
[requirement("A"), requirement("B")]` will take a double direct
dependency on `C`. So we need to insert a layer of indirection,
generating `C_A` and `C_B` which serve only as unique aliases for `C` so
that we can support the double dependency. Our final dependency graph
then is as follows
```mermaid
graph LR
C_A-->C;
C_B-->C;
C-->A*;
A*-->D;
A*-->E;
C-->B*;
B*-->F;
B*-->G;
```
Addresses #1076, #1188
## To do
- [x] Get rebased
- [x] Get re-validated manually
- [x] Buildifier
- [x] Get CI happy
- [x] Update documentation
- [x] Update changelog
---------
Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>diff --git a/examples/pip_parse/WORKSPACE b/examples/pip_parse/WORKSPACE
index 79aca14..415d064 100644
--- a/examples/pip_parse/WORKSPACE
+++ b/examples/pip_parse/WORKSPACE
@@ -24,6 +24,19 @@
# can be passed
# environment = {"HTTPS_PROXY": "http://my.proxy.fun/"},
name = "pypi",
+
+ # Requirement groups allow Bazel to tolerate PyPi cycles by putting dependencies
+ # which are known to form cycles into groups together.
+ experimental_requirement_cycles = {
+ "sphinx": [
+ "sphinx",
+ "sphinxcontrib-qthelp",
+ "sphinxcontrib-htmlhelp",
+ "sphinxcontrib-devhelp",
+ "sphinxcontrib-applehelp",
+ "sphinxcontrib-serializinghtml",
+ ],
+ },
# (Optional) You can provide extra parameters to pip.
# Here, make pip output verbose (this is usable with `quiet = False`).
# extra_pip_args = ["-v"],
@@ -44,6 +57,7 @@
# (Optional) You can set quiet to False if you want to see pip output.
#quiet = False,
requirements_lock = "//:requirements_lock.txt",
+ requirements_windows = "//:requirements_windows.txt",
)
load("@pypi//:requirements.bzl", "install_deps")
