Test disabling aspect on protos attr to compile rules
34 files changed
tree: e9501835d2a555e274d27c164f1b78c076a9be8a
  1. .bazelci/
  2. .github/
  3. android/
  4. cpp/
  5. csharp/
  6. d/
  7. docs/
  8. example/
  9. go/
  10. grpc-gateway/
  11. internal/
  12. java/
  13. js/
  14. nodejs/
  15. objc/
  16. php/
  17. protobuf/
  18. python/
  19. ruby/
  20. rust/
  21. scala/
  22. swift/
  23. test_workspaces/
  24. third_party/
  25. tools/
  26. .bazelignore
  27. .bazelrc
  28. .gitignore
  29. _config.yml
  30. BUILD.bazel
  31. CHANGELOG.md
  32. defs.bzl
  33. LICENSE
  34. Makefile
  35. README.md
  36. repositories.bzl
  37. WORKSPACE
README.md

Announcements 📣

  • 2021/02/09: Version 4.0 of Bazel has been released recently and is supported without update for most languages. Changes for full Bazel 4.0 compatibility and updated dependencies are in progress.
  • 2020/10/11: Version 2.0.0 has been released with updated Protobuf and gRPC versions. For some languages this may not be a drop-in replacement and it may be necessary to update your WORKSPACE file due to changes in dependencies; please see the above linked release notes for details or the language specific rules pages. If you discover any problems with the new release, please open a new issue.

Contents:

Overview

These rules provide Protocol Buffers (Protobuf) and gRPC rules for a range of languages and services.

Each supported language ({lang} below) is generally split into four rule flavours:

  • {lang}_proto_compile: Provides generated files from the Protobuf protoc plugin for the language. e.g for C++ this provides the generated *.pb.cc and *.pb.h files.

  • {lang}_proto_library: Provides a language-specific library from the generated Protobuf protoc plugin outputs, along with necessary dependencies. e.g for C++ this provides a Bazel native cpp_library created from the generated *.pb.cc and *pb.h files, with the Protobuf library linked. For languages that do not have a ‘library’ concept, this rule may not exist.

  • {lang}_grpc_compile: Provides generated files from both the Protobuf and gRPC protoc plugins for the language. e.g for C++ this provides the generated *.pb.cc, *.grpc.pb.cc, *.pb.h and *.grpc.pb.h files.

  • {lang}_grpc_library: Provides a language-specific library from the generated Protobuf and gRPC protoc plugins outputs, along with necessary dependencies. e.g for C++ this provides a Bazel native cpp_library created from the generated *.pb.cc, *.grpc.pb.cc, *.pb.h and *.grpc.pb.h files, with the Protobuf and gRPC libraries linked. For languages that do not have a ‘library’ concept, this rule may not exist.

Therefore, if you are solely interested in the generated source code artifacts, use the {lang}_{proto|grpc}_compile rules. Otherwise, if you want a ready-to-go library, use the {lang}_{proto|grpc}_library rules.

Installation

Add rules_proto_grpc to your WORKSPACE file and then look at the language specific examples linked below:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_proto_grpc",
    urls = ["https://github.com/rules-proto-grpc/rules_proto_grpc/archive/2.0.0.tar.gz"],
    sha256 = "d771584bbff98698e7cb3cb31c132ee206a972569f4dc8b65acbdd934d156b33",
    strip_prefix = "rules_proto_grpc-2.0.0",
)

load("@rules_proto_grpc//:repositories.bzl", "rules_proto_grpc_toolchains", "rules_proto_grpc_repos")
rules_proto_grpc_toolchains()
rules_proto_grpc_repos()

load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()

It is recommended that you use the tagged releases for stable rules. Master is intended to be ‘ready-to-use’, but may be unstable at certain periods. To be notified of new releases, you can use GitHub's ‘Watch Releases Only’ on the repository.

Important: You will also need to follow instructions in the language-specific README.md for additional workspace dependencies that may be required.

Rules

LanguageRuleDescription
Androidandroid_proto_compileGenerates an Android protobuf .jar artifact (example)
Androidandroid_grpc_compileGenerates Android protobuf+gRPC .jar artifacts (example)
Androidandroid_proto_libraryGenerates an Android protobuf library using android_library from rules_android (example)
Androidandroid_grpc_libraryGenerates Android protobuf+gRPC library using android_library from rules_android (example)
C++cpp_proto_compileGenerates C++ protobuf .h & .cc artifacts (example)
C++cpp_grpc_compileGenerates C++ protobuf+gRPC .h & .cc artifacts (example)
C++cpp_proto_libraryGenerates a C++ protobuf library using cc_library, with dependencies linked (example)
C++cpp_grpc_libraryGenerates a C++ protobuf+gRPC library using cc_library, with dependencies linked (example)
C#csharp_proto_compileGenerates C# protobuf .cs artifacts (example)
C#csharp_grpc_compileGenerates C# protobuf+gRPC .cs artifacts (example)
C#csharp_proto_libraryGenerates a C# protobuf library using core_library from rules_dotnet. Note that the library name must end in .dll (example)
C#csharp_grpc_libraryGenerates a C# protobuf+gRPC library using core_library from rules_dotnet. Note that the library name must end in .dll (example)
Dd_proto_compileGenerates D protobuf .d artifacts (example)
Dd_proto_libraryGenerates a D protobuf library using d_library from rules_d (example)
Gogo_proto_compileGenerates Go protobuf .go artifacts (example)
Gogo_grpc_compileGenerates Go protobuf+gRPC .go artifacts (example)
Gogo_proto_libraryGenerates a Go protobuf library using go_library from rules_go (example)
Gogo_grpc_libraryGenerates a Go protobuf+gRPC library using go_library from rules_go (example)
grpc-gatewaygateway_grpc_compileGenerates grpc-gateway .go files (example)
grpc-gatewaygateway_openapiv2_compileGenerates grpc-gateway OpenAPI v2 .json files (example)
grpc-gatewaygateway_grpc_libraryGenerates grpc-gateway library files (example)
Javajava_proto_compileGenerates a Java protobuf srcjar artifact (example)
Javajava_grpc_compileGenerates a Java protobuf+gRPC srcjar artifact (example)
Javajava_proto_libraryGenerates a Java protobuf library using java_library (example)
Javajava_grpc_libraryGenerates a Java protobuf+gRPC library using java_library (example)
JavaScriptjs_proto_compileGenerates JavaScript protobuf .js and .d.ts artifacts (example)
JavaScriptjs_grpc_node_compileGenerates JavaScript protobuf + gRPC-node .js and .d.ts artifacts (example)
JavaScriptjs_grpc_web_compileGenerates JavaScript protobuf + gRPC-Web .js and .d.ts artifacts (example)
JavaScriptjs_proto_libraryGenerates a JavaScript protobuf library using js_library from rules_nodejs (example)
JavaScriptjs_grpc_node_libraryGenerates a Node.js protobuf + gRPC-node library using js_library from rules_nodejs (example)
JavaScriptjs_grpc_web_libraryGenerates a JavaScript protobuf + gRPC-Web library using js_library from rules_nodejs (example)
Objective-Cobjc_proto_compileGenerates Objective-C protobuf .m & .h artifacts (example)
Objective-Cobjc_grpc_compileGenerates Objective-C protobuf+gRPC .m & .h artifacts (example)
Objective-Cobjc_proto_libraryGenerates an Objective-C protobuf library using objc_library (example)
PHPphp_proto_compileGenerates PHP protobuf .php artifacts (example)
PHPphp_grpc_compileGenerates PHP protobuf+gRPC .php artifacts (example)
Pythonpython_proto_compileGenerates Python protobuf .py artifacts (example)
Pythonpython_grpc_compileGenerates Python protobuf+gRPC .py artifacts (example)
Pythonpython_grpclib_compileGenerates Python protobuf+grpclib .py artifacts (supports Python 3 only) (example)
Pythonpython_proto_libraryGenerates a Python protobuf library using py_library from rules_python (example)
Pythonpython_grpc_libraryGenerates a Python protobuf+gRPC library using py_library from rules_python (example)
Pythonpython_grpclib_libraryGenerates a Python protobuf+grpclib library using py_library from rules_python (supports Python 3 only) (example)
Rubyruby_proto_compileGenerates Ruby protobuf .rb artifacts (example)
Rubyruby_grpc_compileGenerates Ruby protobuf+gRPC .rb artifacts (example)
Rubyruby_proto_libraryGenerates a Ruby protobuf library using ruby_library from rules_ruby (example)
Rubyruby_grpc_libraryGenerates a Ruby protobuf+gRPC library using ruby_library from rules_ruby (example)
Rustrust_proto_compileGenerates Rust protobuf .rs artifacts (example)
Rustrust_grpc_compileGenerates Rust protobuf+gRPC .rs artifacts (example)
Rustrust_proto_libraryGenerates a Rust protobuf library using rust_library from rules_rust (example)
Rustrust_grpc_libraryGenerates a Rust protobuf+gRPC library using rust_library from rules_rust (example)
Scalascala_proto_compileGenerates a Scala protobuf .jar artifact (example)
Scalascala_grpc_compileGenerates Scala protobuf+gRPC .jar artifacts (example)
Scalascala_proto_libraryGenerates a Scala protobuf library using scala_library from rules_scala (example)
Scalascala_grpc_libraryGenerates a Scala protobuf+gRPC library using scala_library from rules_scala (example)
Swiftswift_proto_compileGenerates Swift protobuf .swift artifacts (example)
Swiftswift_grpc_compileGenerates Swift protobuf+gRPC .swift artifacts (example)
Swiftswift_proto_libraryGenerates a Swift protobuf library using swift_library from rules_swift (example)
Swiftswift_grpc_libraryGenerates a Swift protobuf+gRPC library using swift_library from rules_swift (example)

Example Usage

These steps walk through the actions required to go from a raw .proto file to a C++ library. Other languages will have a similar high-level layout.

Step 1: Write a Protocol Buffer .proto file (example: thing.proto):

syntax = "proto3";

package example;

import "google/protobuf/any.proto";

message Thing {
    string name = 1;
    google.protobuf.Any payload = 2;
}

Step 2: Write a BAZEL.build file with a proto_library target:

proto_library(
    name = "thing_proto",
    srcs = ["thing.proto"],
    deps = ["@com_google_protobuf//:any_proto"],
)

In this example we have a dependency on a well-known type any.proto, hence the proto_library to proto_library dependency ("@com_google_protobuf//:any_proto")

Step 3: Add a cpp_proto_compile target

NOTE: In this example thing.proto does not include service definitions (gRPC). For protos with services, use the cpp_grpc_compile rule instead.

# BUILD.bazel
load("@rules_proto_grpc//cpp:defs.bzl", "cpp_proto_compile")

cpp_proto_compile(
    name = "cpp_thing_proto",
    protos = [":thing_proto"],
)

But wait, before we can build this, we need to load the dependencies necessary for this rule (see cpp/README.md):

Step 4: Load the workspace macro corresponding to the build rule.

# WORKSPACE
load("@rules_proto_grpc//cpp:repositories.bzl", "cpp_repos")

cpp_repos()

We're now ready to build the target:

Step 5: Build it!

$ bazel build //example/proto:cpp_thing_proto
Target //example/proto:cpp_thing_proto up-to-date:
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.h
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.cc

If we were only interested in the generated file artifacts, the cpp_grpc_compile rule would be fine. However, for convenience we‘d rather have the outputs compiled into a C++ library. To do that, let’s change the rule from cpp_proto_compile to cpp_proto_library:

# BUILD.bazel
load("@rules_proto_grpc//cpp:defs.bzl", "cpp_proto_library")

cpp_proto_library(
    name = "cpp_thing_proto",
    protos = [":thing_proto"],
)
$ bazel build //example/proto:cpp_thing_proto
Target //example/proto:cpp_thing_proto up-to-date:
  bazel-bin/example/proto/libcpp_thing_proto.a
  bazel-bin/example/proto/libcpp_thing_proto.so  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.h
  bazel-genfiles/example/proto/cpp_thing_proto/example/proto/thing.pb.cc

This way, we can use //example/proto:cpp_thing_proto as a dependency of any other cc_library or cc_binary target as per normal.

NOTE: The cpp_proto_library target implicitly calls cpp_proto_compile, and we can access that rule's by adding _pb at the end of the target name, like bazel build //example/proto:cpp_thing_proto_pb

Developers

Code Layout

Each language {lang} has a top-level subdirectory that contains:

  1. {lang}/README.md: Generated documentation for the language rules

  2. {lang}/repositories.bzl: Macro functions that declare repository rule dependencies for that language

  3. {lang}/{rule}.bzl: Rule implementations of the form {lang}_{kind}_{type}, where kind is one of proto|grpc and type is one of compile|library

  4. {lang}/BUILD.bazel: proto_plugin() declarations for the available plugins for the language

  5. example/{lang}/{rule}/: Generated WORKSPACE and BUILD.bazel demonstrating standalone usage of the rules

  6. {lang}/example/routeguide/: Example routeguide example implementation, if possible

Rule Generation

To help maintain consistency of the rule implementations and documentation, all of the rule implementations are generated by the tool //tools/rulegen. Changes in the main README.md should be placed in tools/rulegen/README.header.md or tools/rulegen/README.footer.md. Changes to generated rules should be put in the source files (example: tools/rulegen/java.go).

Developing Custom Plugins

Generally, follow the pattern seen in the multiple language examples in this repository. The basic idea is:

  1. Load the plugin rule: load("@rules_proto_grpc//:defs.bzl", "proto_plugin")
  2. Define the rule, giving it a name, options (not mandatory), tool and outputs. tool is a label that refers to the binary executable for the plugin itself
  3. Choose your output type (pick one!):
    • outputs: A list of strings patterns that predicts the pattern of files generated by the plugin. For plugins that produce one output file per input proto file
    • out: The name of a single output file generated by the plugin
    • output_directory: Set to true if your plugin generates files in a non-predictable way. e.g. if the output paths depend on the service names within the files
  4. Create a compilation rule and aspect using the following template:
load("@rules_proto//proto:defs.bzl", "ProtoInfo")
load(
    "@rules_proto_grpc//:defs.bzl",
    "ProtoLibraryAspectNodeInfo",
    "ProtoPluginInfo",
    "proto_compile_aspect_attrs",
    "proto_compile_aspect_impl",
    "proto_compile_attrs",
    "proto_compile_impl",
)

# Create aspect
example_aspect = aspect(
    implementation = proto_compile_aspect_impl,
    provides = [ProtoLibraryAspectNodeInfo],
    attr_aspects = ["deps"],
    attrs = dict(
        proto_compile_aspect_attrs,
        _plugins = attr.label_list(
            doc = "List of protoc plugins to apply",
            providers = [ProtoPluginInfo],
            default = [
                Label("//<LABEL OF YOUR PLUGIN>"),
            ],
        ),
        _prefix = attr.string(
            doc = "String used to disambiguate aspects when generating outputs",
            default = "example_aspect",
        ),
    ),
    toolchains = ["@rules_proto_grpc//protobuf:toolchain_type"],
)

# Create compile rule to apply aspect
_rule = rule(
    implementation = proto_compile_impl,
    attrs = dict(
        proto_compile_attrs,
        protos = attr.label_list(
            mandatory = False,  # TODO: set to true in 4.0.0 when deps removed below
            providers = [ProtoInfo, ProtoLibraryAspectNodeInfo],
            doc = "List of labels that provide a ProtoInfo (such as rules_proto proto_library)",
        ),
        deps = attr.label_list(
            mandatory = False,
            providers = [ProtoInfo, ProtoLibraryAspectNodeInfo],
            aspects = [example_aspect],
            doc = "DEPRECATED: Use protos attr",
        ),
        _plugins = attr.label_list(
            doc = "List of protoc plugins to apply",
            providers = [ProtoPluginInfo],
            default = [
                Label("//<LABEL OF YOUR PLUGIN>"),
            ],
        ),
    ),
    toolchains = [str(Label("//protobuf:toolchain_type"))],
)

# Create macro for converting attrs and passing to compile
def example_compile(**kwargs):
    _rule(
        verbose_string = "{}".format(kwargs.get("verbose", 0)),
        **kwargs
    )


License

This project is derived from stackb/rules_proto under the Apache 2.0 license and this project therefore maintains the terms of that license

Contributing

Contributions are very welcome. Please see CONTRIBUTING.md for further details.