You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
305 lines
10 KiB
305 lines
10 KiB
.. _module-pw_protobuf_compiler:
|
|
|
|
--------------------
|
|
pw_protobuf_compiler
|
|
--------------------
|
|
The Protobuf compiler module provides build system integration and wrapper
|
|
scripts for generating source code for Protobuf definitions.
|
|
|
|
Generator support
|
|
=================
|
|
Protobuf code generation is currently supported for the following generators:
|
|
|
|
+-------------+----------------+-----------------------------------------------+
|
|
| Generator | Code | Notes |
|
|
+-------------+----------------+-----------------------------------------------+
|
|
| pw_protobuf | ``pwpb`` | Compiles using ``pw_protobuf``. |
|
|
+-------------+----------------+-----------------------------------------------+
|
|
| Nanopb | ``nanopb`` | Compiles using Nanopb. The build argument |
|
|
| | | ``dir_pw_third_party_nanopb`` must be set to |
|
|
| | | point to a local nanopb installation. |
|
|
+-------------+----------------+-----------------------------------------------+
|
|
| Nanopb RPC | ``nanopb_rpc`` | Compiles pw_rpc service and client code for |
|
|
| | | nanopb. Requires a nanopb installation. |
|
|
+-------------+----------------+-----------------------------------------------+
|
|
| Raw RPC | ``raw_rpc`` | Compiles raw binary pw_rpc service code. |
|
|
+-------------+----------------+-----------------------------------------------+
|
|
| Go | ``go`` | Compiles using the standard Go protobuf |
|
|
| | | plugin with gRPC service support. |
|
|
+-------------+----------------+-----------------------------------------------+
|
|
| Python | ``python`` | Compiles using the standard Python protobuf |
|
|
| | | plugin, creating a ``pw_python_package``. |
|
|
+-------------+----------------+-----------------------------------------------+
|
|
|
|
GN template
|
|
===========
|
|
The ``pw_proto_library`` GN template is provided by the module.
|
|
|
|
It defines a collection of protobuf files that should be compiled together. The
|
|
template creates a sub-target for each supported generator, named
|
|
``<target_name>.<generator>``. These sub-targets generate their respective
|
|
protobuf code, and expose it to the build system appropriately (e.g. a
|
|
``pw_source_set`` for C/C++).
|
|
|
|
For example, given the following target:
|
|
|
|
.. code-block::
|
|
|
|
pw_proto_library("test_protos") {
|
|
sources = [ "my_test_protos/test.proto" ]
|
|
}
|
|
|
|
``test_protos.pwpb`` compiles code for pw_protobuf, and ``test_protos.nanopb``
|
|
compiles using Nanopb (if it's installed).
|
|
|
|
Protobuf code is only generated when a generator sub-target is listed as a
|
|
dependency of another GN target.
|
|
|
|
GN permits using abbreviated labels when the target name matches the directory
|
|
name (e.g. ``//foo`` for ``//foo:foo``). For consistency with this, the
|
|
sub-targets for each generator are aliased to the directory when the target name
|
|
is the same. For example, these two labels are equivalent:
|
|
|
|
.. code-block::
|
|
|
|
//path/to/my_protos:my_protos.pwpb
|
|
//path/to/my_protos:pwpb
|
|
|
|
``pw_python_package`` subtargets are also available on the ``python`` subtarget:
|
|
|
|
.. code-block::
|
|
|
|
//path/to/my_protos:my_protos.python.lint
|
|
//path/to/my_protos:python.lint
|
|
|
|
**Arguments**
|
|
|
|
* ``sources``: List of input .proto files.
|
|
* ``deps``: List of other pw_proto_library dependencies.
|
|
* ``inputs``: Other files on which the protos depend (e.g. nanopb ``.options``
|
|
files).
|
|
* ``prefix``: A prefix to add to the source protos prior to compilation. For
|
|
example, a source called ``"foo.proto"`` with ``prefix = "nested"`` will be
|
|
compiled with protoc as ``"nested/foo.proto"``.
|
|
* ``strip_prefix``: Remove this prefix from the source protos. All source and
|
|
input files must be nested under this path.
|
|
* ``python_package``: Label of Python package to which to add the proto modules.
|
|
|
|
**Example**
|
|
|
|
.. code-block::
|
|
|
|
import("$dir_pw_protobuf_compiler/proto.gni")
|
|
|
|
pw_proto_library("my_protos") {
|
|
sources = [
|
|
"my_protos/foo.proto",
|
|
"my_protos/bar.proto",
|
|
]
|
|
}
|
|
|
|
pw_proto_library("my_other_protos") {
|
|
sources = [ "some/other/path/baz.proto" ] # imports foo.proto
|
|
|
|
# This removes the "some/other/path" prefix from the proto files.
|
|
strip_prefix = "some/other/path"
|
|
|
|
# This adds the "my_other_protos/" prefix to the proto files.
|
|
prefix = "my_other_protos"
|
|
|
|
# Proto libraries depend on other proto libraries directly.
|
|
deps = [ ":my_protos" ]
|
|
}
|
|
|
|
source_set("my_cc_code") {
|
|
sources = [
|
|
"foo.cc",
|
|
"bar.cc",
|
|
"baz.cc",
|
|
]
|
|
|
|
# When depending on protos in a source_set, specify the generator suffix.
|
|
deps = [ ":my_other_protos.pwpb" ]
|
|
}
|
|
|
|
From C++, ``baz.proto`` included as follows:
|
|
|
|
.. code-block:: cpp
|
|
|
|
#include "my_other_protos/baz.pwpb.h"
|
|
|
|
From Python, ``baz.proto`` is imported as follows:
|
|
|
|
.. code-block:: python
|
|
|
|
from my_other_protos import baz_pb2
|
|
|
|
Proto file structure
|
|
--------------------
|
|
Protobuf source files must be nested under another directory when they are
|
|
compiled. This ensures that they can be packaged properly in Python. The first
|
|
directory is used as the Python package name, so must be unique across the
|
|
build. The ``prefix`` option may be used to set this directory.
|
|
|
|
Using ``prefix`` and ``strip_prefix`` together allows remapping proto files to
|
|
a completely different path. This can be useful when working with protos defined
|
|
in external libraries. For example, consider this proto library:
|
|
|
|
.. code-block::
|
|
|
|
pw_proto_library("external_protos") {
|
|
sources = [
|
|
"//other/external/some_library/src/protos/alpha.proto",
|
|
"//other/external/some_library/src/protos/beta.proto,
|
|
"//other/external/some_library/src/protos/internal/gamma.proto",
|
|
]
|
|
strip_prefix = "//other/external/some_library/src/protos"
|
|
prefix = "some_library"
|
|
}
|
|
|
|
These protos will be compiled by protoc as if they were in this file structure:
|
|
|
|
.. code-block::
|
|
|
|
some_library/
|
|
├── alpha.proto
|
|
├── beta.proto
|
|
└── internal
|
|
└── gamma.proto
|
|
|
|
.. _module-pw_protobuf_compiler-add-to-python-package:
|
|
|
|
Adding Python proto modules to an existing package
|
|
--------------------------------------------------
|
|
By default, generated Python proto modules are organized into their own Python
|
|
package. These proto modules can instead be added to an existing Python package
|
|
declared with ``pw_python_package``. This is done by setting the
|
|
``python_package`` argument on the ``pw_proto_library`` and the
|
|
``proto_library`` argument on the ``pw_python_package``.
|
|
|
|
For example, the protos declared in ``my_protos`` will be nested in the Python
|
|
package declared by ``my_package``.
|
|
|
|
.. code-block::
|
|
|
|
pw_proto_library("my_protos") {
|
|
sources = [ "hello.proto ]
|
|
prefix = "foo"
|
|
python_package = ":my_package"
|
|
}
|
|
|
|
pw_python_pacakge("my_package") {
|
|
generate_setup = {
|
|
name = "foo"
|
|
version = "1.0"
|
|
}
|
|
sources = [ "foo/cool_module.py" ]
|
|
proto_library = ":my_protos"
|
|
}
|
|
|
|
The ``hello_pb2.py`` proto module can be used alongside other files in the
|
|
``foo`` package.
|
|
|
|
.. code-block:: python
|
|
|
|
from foo import cool_module, hello_pb2
|
|
|
|
Working with externally defined protos
|
|
--------------------------------------
|
|
``pw_proto_library`` targets may be used to build ``.proto`` sources from
|
|
existing projects. In these cases, it may be necessary to supply the
|
|
``strip_prefix`` argument, which specifies the protobuf include path to use for
|
|
``protoc``. If only a single external protobuf is being compiled, the
|
|
``python_module_as_package`` option can be used to override the requirement that
|
|
the protobuf be nested under a directory. This option generates a Python package
|
|
with the same name as the proto file, so that the generated proto can be
|
|
imported as if it were a standalone Python module.
|
|
|
|
For example, the ``pw_proto_library`` target for Nanopb sets
|
|
``python_module_as_package`` to ``nanopb_pb2``.
|
|
|
|
.. code-block::
|
|
|
|
pw_proto_library("proto") {
|
|
strip_prefix = "$dir_pw_third_party_nanopb/generator/proto"
|
|
sources = [ "$dir_pw_third_party_nanopb/generator/proto/nanopb.proto" ]
|
|
python_module_as_package = "nanopb_pb2"
|
|
}
|
|
|
|
In Python, this makes ``nanopb.proto`` available as ``import nanopb_pb2`` via
|
|
the ``nanopb_pb2`` Python package. In C++, ``nanopb.proto`` is accessed as
|
|
``#include "nanopb.pwpb.h"``.
|
|
|
|
The ``python_module_as_package`` feature should only be used when absolutely
|
|
necessary --- for example, to support proto files that include
|
|
``import "nanopb.proto"``.
|
|
|
|
CMake
|
|
=====
|
|
CMake provides a ``pw_proto_library`` function with similar features as the
|
|
GN template. The CMake build only supports building firmware code, so
|
|
``pw_proto_library`` does not generate a Python package.
|
|
|
|
**Arguments**
|
|
|
|
* ``NAME``: the base name of the libraries to create
|
|
* ``SOURCES``: .proto source files
|
|
* ``DEPS``: dependencies on other ``pw_proto_library`` targets
|
|
* ``PREFIX``: prefix add to the proto files
|
|
* ``STRIP_PREFIX``: prefix to remove from the proto files
|
|
* ``INPUTS``: files to include along with the .proto files (such as Nanopb
|
|
.options files)
|
|
|
|
**Example**
|
|
|
|
.. code-block:: cmake
|
|
|
|
include($ENV{PW_ROOT}/pw_build/pigweed.cmake)
|
|
include($ENV{PW_ROOT}/pw_protobuf_compiler/proto.cmake)
|
|
|
|
pw_proto_library(my_module.my_protos
|
|
SOURCES
|
|
my_protos/foo.proto
|
|
my_protos/bar.proto
|
|
)
|
|
|
|
pw_proto_library(my_module.my_protos
|
|
SOURCES
|
|
my_protos/foo.proto
|
|
my_protos/bar.proto
|
|
)
|
|
|
|
pw_proto_library(my_module.my_other_protos
|
|
SOURCES
|
|
some/other/path/baz.proto # imports foo.proto
|
|
|
|
# This removes the "some/other/path" prefix from the proto files.
|
|
STRIP_PREFIX
|
|
some/other/path
|
|
|
|
# This adds the "my_other_protos/" prefix to the proto files.
|
|
PREFIX
|
|
my_other_protos
|
|
|
|
# Proto libraries depend on other proto libraries directly.
|
|
DEPS
|
|
my_module.my_protos
|
|
)
|
|
|
|
add_library(my_module.my_cc_code
|
|
foo.cc
|
|
bar.cc
|
|
baz.cc
|
|
)
|
|
|
|
# When depending on protos in a source_set, specify the generator suffix.
|
|
target_link_libraries(my_module.my_cc_code PUBLIC
|
|
my_module.my_other_protos.pwpb
|
|
)
|
|
|
|
These proto files are accessed in C++ the same as in the GN build:
|
|
|
|
.. code-block:: cpp
|
|
|
|
#include "my_other_protos/baz.pwpb.h"
|