Bazel build: Keep generated sources and Python runtime in the same directory.

Users often encounter a Python import error when trying to build Python
protos if protobuf is installed locally on the machine. In this case,
Python ends up looking in the wrong directory when importing files (see
bazelbuild/bazel#1209 and tensorflow/tensorflow#2021). It seems that the
problem is caused by Python getting confused when there are Python
source files that are meant to be part of the same package but are
in separate directories.

Prior to #1233, the Bazel build setup would copy the Python
runtime sources and all generated sources for the builtin protos into
the root directory (assuming that the protobuf tree is vendored in a
google/protobuf directory).

With #1233, the two sets of sources are kept in their respective
directories but both `src/` and `python/` are added to the `PYTHONPATH`
using the new `imports` attribute of the Bazel Python rules. However,
both the runtime sources and the generated sources are under the same
package: `google.protobuf`, causing Python to become confused when
trying to import modules that are in the other directory.

This patch adds a workaround to the Bazel build to add a modified
version of the original `internal_copied_filegroup` macro to copy the
`.proto` files under `src/` to `python/` before building the
`py_proto_library` targets for the builtin protos. This ensures that the
generated sources for the builtin protos will be in the same directory
as the corresponding runtime sources.

This patch was tested with the following:
* All Python tests in protobuf
* All Python tests in tensorflow
* All tests in [Skydoc](https://github.com/bazelbuild/skydoc)
* Importing protobuf as `//google/protobuf`
* Importing and binding targets under `//external`
* Importing protobuf as `//third_party/protobuf`
2 files changed
tree: da62e7b53018d4e0dd0408774a082765e354c0a5
  1. benchmarks/
  2. cmake/
  3. conformance/
  4. csharp/
  5. docs/
  6. editors/
  7. examples/
  8. java/
  9. javanano/
  10. jenkins/
  11. js/
  12. m4/
  13. more_tests/
  14. objectivec/
  15. php/
  16. protoc-artifacts/
  17. python/
  18. ruby/
  19. src/
  20. util/
  21. .gitignore
  22. .travis.yml
  23. appveyor.bat
  24. appveyor.yml
  25. autogen.sh
  26. BUILD
  27. CHANGES.txt
  28. configure.ac
  29. CONTRIBUTORS.txt
  30. generate_descriptor_proto.sh
  31. gmock.BUILD
  32. LICENSE
  33. Makefile.am
  34. post_process_dist.sh
  35. protobuf-lite.pc.in
  36. protobuf.bzl
  37. protobuf.pc.in
  38. Protobuf.podspec
  39. README.md
  40. six.BUILD
  41. tests.sh
  42. update_file_lists.sh
  43. WORKSPACE
README.md

Protocol Buffers - Google's data interchange format

Build Status Build status

Copyright 2008 Google Inc.

https://developers.google.com/protocol-buffers/

Overview

Protocol Buffers (a.k.a., protobuf) are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data. You can find protobuf's documentation on the Google Developers site.

This README file contains protobuf installation instructions. To install protobuf, you need to install the protocol compiler (used to compile .proto files) and the protobuf runtime for your chosen programming language.

Protocol Compiler Installation

The protocol compiler is written in C++. If you are using C++, please follow the C++ Installation Instructions to install protoc along with the C++ runtime.

For non-C++ users, the simplest way to install the protocol compiler is to download a pre-built binary from our release page:

https://github.com/google/protobuf/releases

In the downloads section of each release, you can find pre-built binaries in zip packages: protoc-$VERSION-$PLATFORM.zip. It contains the protoc binary as well as a set of standard .proto files distributed along with protobuf.

If you are looking for an old version that is not available in the release page, check out the maven repo here:

http://repo1.maven.org/maven2/com/google/protobuf/protoc/

These pre-built binaries are only provided for released versions. If you want to use the github master version at HEAD, or you need to modify protobuf code, or you are using C++, it's recommended to build your own protoc binary from source.

If you would like to build protoc binary from source, see the C++ Installation Instructions.

Protobuf Runtime Installation

Protobuf supports several different programming languages. For each programming language, you can find instructions in the corresponding source directory about how to install protobuf runtime for that specific language:

LanguageSource
C++ (include C++ runtime and protoc)src
Javajava
Pythonpython
Objective-Cobjectivec
C#csharp
JavaNanojavanano
JavaScriptjs
Rubyruby
Gogolang/protobuf
PHPTBD

Usage

The complete documentation for Protocol Buffers is available via the web at:

https://developers.google.com/protocol-buffers/