Avoid excessive BES event size (#79)

The files in requested output groups are announced as events on the
Build Event Stream (BES), with declared directories expanded into the
list of files they contain. Since mypy caches contain information about
all transitive dependencies, including the cache directory in the `mypy`
output group thus resulted in a very large number of (duplicated) files
announced in build events when the aspect is applied to an entire repo,
causing excessive upload times and BES sizes in the GBs.

This is fixed by dropping the cache directory from the `mypy` output
group. It is still accessible via the `MypyCacheInfo` provider.
1 file changed
tree: c9b24c233cbbd9c95362fbed28b97d67296333ad
  1. .bcr/
  2. .github/
  3. .vscode/
  4. examples/
  5. mypy/
  6. .bazelignore
  7. .bazelrc
  8. .bazelversion
  9. .gitattributes
  10. .gitignore
  11. BUILD.bazel
  12. LICENSE
  13. MODULE.bazel
  14. readme.md
  15. WORKSPACE.bazel
readme.md

rules_mypy

An aspect to instrument py_* targets with mypy type-checking.

Compared to bazel-mypy-integration, this ruleset aims to make a couple of improvements:

  • Propagation of the mypy cache between dependencies within a repository to avoid exponential type-checking work
  • Robust (and automated) support for including 3rd party types/stubs packages

[!WARNING]
rules_mypy's build actions produce mypy caches as outputs, and these may contain large file counts and that will only grow as a dependency chain grows. This may have an impact on the size and usage of build and/or remote caches.

Usage

This aspect will run over any py_binary, py_library or py_test.

Setup is significantly easier with bzlmod, we recommend and predominantly support bzlmod, though these rules should work without issue in non-bzlmod setups, albeit with more work to configure.

Bzlmod Setup

Add rules_mypy to your MODULE.bazel:

bazel_dep(name = "rules_mypy", version = "0.0.0")

Optionally, configure a types repository:

Many Python packages have separately published types/stubs packages. While mypy (and these rules) will work without including these types, this ruleset provides some utilities for leveraging these types to improve mypy's type checking.

types = use_extension("@rules_mypy//mypy:types.bzl", "types")
types.requirements(
    name = "pip_types",
    # `@pip` in the next line corresponds to the `hub_name` when using
    # rules_python's `pip.parse(...)`.
    pip_requirements = "@pip//:requirements.bzl",
    # also legal to pass a `requirements.in` here
    requirements_txt = "//:requirements.txt",
)
use_repo(types, "pip_types")

Configure mypy_aspect.

Define a new aspect in a .bzl file (such as ./tools/aspects.bzl):

load("@pip_types//:types.bzl", "types")
load("@rules_mypy//mypy:mypy.bzl", "mypy")

mypy_aspect = mypy(types = types)

Update your .bazelrc to include this new aspect:

# register mypy_aspect with Bazel
build --aspects //tools:aspects.bzl%mypy_aspect

# optionally, default enable the mypy checks
build --output_groups=+mypy

Customizing mypy

Configuring mypy with mypy.ini

mypy's behavior may be customized using a mypy config file file. To use a mypy config file, pass a label for a valid config file to the mypy aspect factory:

mypy_aspect = mypy(
    mypy_ini = "@@//:mypy.ini",
    types = types,
)

[!NOTE] The label passed to mypy_ini needs to be absolute (a prefix of @@ means the root repo).

[!NOTE] mypy.ini files should likely contain the following lines to suppress type-checking 3rd party modules.

follow_imports = silent
follow_imports_for_stubs = True

Changing the version of mypy and/or including plugins

To customize the version of mypy, use rules_python's requirements resolution and construct a custom mypy CLI:

# in a BUILD file
load("@pip//:requirements.bzl", "requirements") # '@pip' must match configured pip hub_name
load("@rules_mypy//mypy:mypy.bzl", "mypy", "mypy_cli")

mypy_cli(
    name = "mypy_cli",
    mypy_requirement = requirement("mypy"),
)

And in your aspects.bzl (or similar) file:

load("@rules_mypy//mypy:mypy.bzl", "mypy")

mypy_aspect = mypy(
    mypy_cli = ":mypy_cli",
    types = types,
)

Further, to use mypy plugins referenced in any config file, use the deps attribute of mypy_cli:

# in a BUILD file
load("@pip//:requirements.bzl", "requirement") # '@pip' must match configured pip hub_name
load("@rules_mypy//mypy:mypy.bzl", "mypy", "mypy_cli")

mypy_cli(
    name = "mypy_cli",
    mypy_requirement = requirement("mypy"),
    deps = [
        requirement("pydantic"),
    ],
)

Skipping Targets

Skip running mypy on targets by tagging with no-mypy, or customize the tags that will suppress mypy by providing a list to the suppression_tags argument of the mypy aspect initializer:

load("@rules_mypy//mypy:mypy.bzl", "mypy")

mypy_aspect = mypy(
    suppression_tags = ["no-mypy", "no-checks"],
    types = types,
)

Running in opt-in mode

To add type checking to a codebase incrementally, configure a list of opt-in tags that will suppress running mypy by default unless a target is tagged explicitly with one of the opt-in tags.

load("@rules_mypy//mypy:mypy.bzl", "mypy")

mypy_aspect = mypy(
    opt_in_tags = ["typecheck"],
    types = types,
)