Nanopb: Overview

Nanopb is an ANSI-C library for encoding and decoding messages in Google's Protocol Buffers format with minimal requirements for RAM and code space. It is primarily suitable for 32-bit microcontrollers.

Documentation version

This documentation applies for nanopb 0.4.0 and later versions. For documentation of older releases, see here.

Overall structure

For the runtime program, you always need pb.h for type declarations and pb_common.h/c for base functions. Depending on whether you want to encode, decode, or both, you also need pb_encode.h/c or pb_decode.h/c.

The high-level encoding and decoding functions take a pointer to pb_msgdesc_t structure, which describes the fields of a message structure. Usually you want these autogenerated from a .proto file. The tool script nanopb_generator.py accomplishes this.

Image: Nanopb generator flow

So a typical project might include these files:

  1. Nanopb runtime library:

    • pb.h
    • pb_common.h and pb_common.c (always needed)
    • pb_decode.h and pb_decode.c (needed for decoding messages)
    • pb_encode.h and pb_encode.c (needed for encoding messages)

  2. Protocol description (you can have many):

    • person.proto (just an example)
    • person.pb.c (autogenerated, contains message descriptors)
    • person.pb.h (autogenerated, contains type declarations and macros)

Features and limitations

Features

  1. Pure C runtime
  2. Small code size (5--20 kB depending on processor and compilation options, plus any message definitions)
  3. Small ram usage (typically ~1 kB stack, plus any message structs)
  4. Allows specifying maximum size for strings and arrays, so that they can be allocated statically.
  5. No malloc needed: everything can be allocated statically or on the stack. Optional malloc support available.
  6. You can use either encoder or decoder alone to cut the code size in half.
  7. Support for most protobuf features, including: all data types, nested submessages, default values, repeated and optional fields, oneofs, packed arrays, extension fields.
  8. Callback mechanism for handling messages larger than can fit in available RAM.
  9. Extensive set of tests.

Limitations

  1. Some speed has been sacrificed for code size.
  2. Encoding is focused on writing to streams. For memory buffers only it could be made more efficient.
  3. The deprecated Protocol Buffers feature called “groups” is not supported.
  4. Fields in the generated structs are ordered by the tag number, instead of the natural ordering in .proto file. (Since nanopb-0.4.2 this can be configured with sort_by_tag setting.)
  5. Unknown fields are not preserved when decoding and re-encoding a message.
  6. Reflection (runtime introspection) is not supported. E.g. you can't request a field by giving its name in a string.
  7. Numeric arrays are always encoded as packed, even if not marked as packed in .proto.
  8. Cyclic references between messages are supported only in callback and malloc mode.
  9. Nanopb doesn't have a stable ABI (application binary interface) between versions, so using it as a shared library (.so / .dll) requires extra care.

Getting started

For starters, consider this simple message:

message Example {
    required int32 value = 1;
}

Save this in message.proto and compile it:

user@host:~$ python nanopb/generator/nanopb_generator.py message.proto

You should now have in message.pb.h:

typedef struct {
   int32_t value;
} Example;

extern const pb_msgdesc_t Example_msg;
#define Example_fields &Example_msg

Then you have to include the nanopb headers and the generated header:

#include <pb_encode.h>
#include "message.pb.h"

Now in your main program do this to encode a message:

Example mymessage = {42};
uint8_t buffer[10];
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
pb_encode(&stream, Example_fields, &mymessage);

After that, buffer will contain the encoded message. The number of bytes in the message is stored in stream.bytes_written. You can feed the message to protoc --decode=Example message.proto to verify its validity.

For a complete example of the simple case, see examples/simple/simple.c. For a more complex example with network interface, see the examples/network_server subdirectory.

Compiler requirements

Nanopb should compile with most ansi-C compatible compilers. It however requires a few header files to be available:

  1. string.h, with these functions: strlen, memcpy, memset
  2. stdint.h, for definitions of int32_t etc.
  3. stddef.h, for definition of size_t
  4. stdbool.h, for definition of bool
  5. limits.h, for definition of CHAR_BIT

If these header files do not come with your compiler, you can use the file extra/pb_syshdr.h instead. It contains an example of how to provide the dependencies. You may have to edit it a bit to suit your custom platform.

To use the pb_syshdr.h, define PB_SYSTEM_HEADER as "pb_syshdr.h" (including the quotes). Similarly, you can provide a custom include file, which should provide all the dependencies listed above.

Running the test cases

Extensive unittests and test cases are included under the tests folder.

To build the tests, you will need the scons build system. The tests should be runnable on most platforms. Windows and Linux builds are regularly tested. The tests also support embedded targets: STM32 (ARM Cortex-M) and AVR builds are regularly tested.

In addition to the build system, you will also need a working Google Protocol Buffers protoc compiler, and the Python bindings for Protocol Buffers.

Easiest way to install dependencies is to use the Python package manager pip, which works on all platforms supported by Python:

pip3 install scons protobuf grpcio-tools