Skip to content

Protocol Buffers for Serialization

Introduction

Protocol Buffers is a compact binary serialization format with very fast cross-platform and cross-language implementations. It is very useful when you need to quickly move data between memory and some other location, for example a file, communications over networks, and fast message passing between robots (though nowadays ROS and ROS2 handle serialization and message passing for us in most cases). At MItHRIL, we use protocol buffers ("protobufs") to serialize messages passed between robots and computers over UDP; for example, to perform message passing between motion capture systems and Unity for external tracking of VR headsets and physical props.

Protobuf was first designed, implemented, and maintained by Google for communications between network peers on their internal network. Since then, it has been implemented in a number of different languages, including the protobuf-net implementation built and maintained by Marc Gravell. Much of the documentation on using protobufs can be found on Google's official protocol buffers webpage.

Info: serialization is the process of converting an object (of any language) into bytes and storing them in a persistent memory system (a file on disk, a messaging queue, or a database), so the data can be reused and the object recreated on the same or a different machine. In deserialization, the stored bytes are converted back into an object.

The Protocol Buffer Definition File (.proto)

The protocol buffer "definition file" contains the schema definition of the data we want to serialize, stored in a human-readable file with the extension .proto. Here's an example definition file for passing motion capture data:

syntax = "proto3";
package mocap;

message Object {
  string name = 1;
  string type = 2;
  int32 id = 3;
  double x = 4;
  double y = 5;
  double z = 6;
  double qx = 7;
  double qy = 8;
  double qz = 9;
  double qw = 10;
  double vx = 11;
  double vy = 12;
  double vz = 13;
  double framerate = 14;
}

message MocapData {
  string system = 1;
  int32 count = 2;
  int64 frame = 3;
  repeated Object objects = 4;
}

A closer look at what each part of the schema does:

  • syntax = "proto3": represents the Protobuf version in use (here, the latest version 3, so the schema can use all syntax valid for version 3).
  • package mocap;: used for conflict resolution, e.g. if multiple classes/members share the same name.
  • message Object: the name of the base class for the object that would be created/recreated.
  • string name = 1; string type = 2; etc.: the attributes of the Object class, along with the data type and the position of the tag in the schema. Each attribute in a class must have a unique position, represented by an integer; a new tag added later should use position 15. This position integer matters for keeping the actual data compact and allowing for schema evolution.

The MocapData class has its own attributes and position integers; note that Object is used as the type for the attribute in position 4, and the keyword repeated denotes that multiple items of type Object will be stored in that field (in the original application, to track multiple objects using mocap and send them to Unity).

Download and Install the Protobuf Compiler

Once the message is defined, download and install the protobuf binary, which is used to auto-generate the code for the MocapData class:

  1. Download the protobuf binaries from https://github.com/protocolbuffers/protobuf/releases/ (the appropriate compressed folder, e.g. .zip for Windows).
  2. Unpack and store the folder in your development libraries and tools folder.
  3. Copy the path for the bin folder containing protoc.exe (the protobuf compiler executable) and add it to your PATH environment variable.

Once installed, verify with:

protoc --version

This should print the installed version of protoc. You can then auto-generate serializer code specific to whichever language you're using, since protobuf is cross-language:

protoc --java_out=[path_to_output_dir] mocap_data.proto    # outputs java code
protoc --cpp_out=[path_to_output_dir] mocap_data.proto     # outputs c++ code
protoc --csharp_out=[path_to_output_dir] mocap_data.proto  # outputs c# code (useful for Unity)
protoc --python_out=[path_to_output_dir] mocap_data.proto  # outputs python code

This produces new code file(s) (the serializer) that you can incorporate into your project to serialize and deserialize data based on your definition file.

Using Protocol Buffers in Your Code

Google has good language guides specific to each supported language (https://protobuf.dev/getting-started/). Briefly, in Python:

After generating the serializer script with protoc (for the example above, likely mocap_data_pb2.py), import it into your code:

import mocap_data_pb2

Then use the MocapData class as if it defined each field of the Message base class as a regular field:

# Create an instance of MocapData using a protocol buffer defined by mocap_data_pb2
mocap_data = mocap_data_pb2.MocapData()

# Set the frame number for the MocapData
mocap_data.system = "vicon"
mocap_data.count = 3
mocap_data.frame = 10

for i in range(mocap_data.count):
    # Create a new ObjectData instance and fill its attributes
    object_data = mocap_data.objects.add()
    object_data.name = "Object" + str(i)
    object_data.x = 0
    object_data.y = 0
    object_data.z = 0
    object_data.qw = 1
    object_data.qx = 0
    object_data.qy = 0
    object_data.qz = 0

These assignments are not adding arbitrary new fields to a generic Python object: assigning a field not defined in the .proto file raises an AttributeError, assigning a field to a value of the wrong type raises a TypeError, and reading a field's value before it has been set returns the default value.

MocapData.no_such_field = 1  # raises AttributeError
MocapData.count = "3"        # raises TypeError

For more on exactly what members the protocol compiler generates for a given field definition, see the Python generated code reference.

Standard Message Methods

Each message class also contains methods to check or manipulate the entire message, including:

  • IsInitialized(): checks if all required fields have been set.
  • __str__(): returns a human-readable representation of the message, useful for debugging (invoked via str(message) or print message).
  • CopyFrom(other_msg): overwrites the message with the given message's values.
  • Clear(): clears all elements back to the empty state.

To inspect what's in a message so far: print(mocap_data).

Serialization and Deserialization

Each protocol buffer class has methods for writing and reading messages using the protocol buffer binary format, including:

  • SerializeToString(): serializes the message and returns it as a string (the bytes are binary, not text; str is used only as a convenient container).
  • ParseFromString(data): parses a message from the given string.

For the mocap application, obtaining serialized byte data ready to send to a robot or VR headset over UDP:

serialized_data = mocap_data.SerializeToString()

To deserialize a received MocapData message:

mocap_data = ParseFromString(data)

Troubleshooting

Caveats for Using Protobuf with Unity

In Unity, generated classes reference the external library Google.Protobuf, which can cause errors if the library isn't found, so it needs to be included in the Unity project:

  1. Download the Google.Protobuf C# dynamic linked library (dll) via the NuGet package.
  2. The downloaded file has a .nuget extension, an alias for .zip; change the extension and unzip it.
  3. Copy Google.Protobuf.dll from ./lib/net45/ (or the appropriate target such as .NET 4.6 or .NET Standard 1.0/2.0, matching your Unity project's settings) to your Unity project's Assets/Plugins folder.

Version Problems

Over the years, Google.Protobuf has required more .NET dependencies not included in Unity; for example, it may ask for System.Memory.dll (for the Span class, not yet supported by Unity), which in turn may ask for System.Runtime.CompilerServices.Unsafe. These libraries aren't included in Unity because something doesn't fully work or behaves incorrectly on some platforms Unity supports, so it's best not to use anything in these libraries beyond what's needed to satisfy Google.Protobuf.dll.