Delta V Software

Protocol Buffers

Documentation


PrevUpHomeNext

Protocol Buffers is an open source project released by Google, which uses a specialized language to define serialization schemas, from which actual serialization code is generated. The official Protocol Buffers compiler can generate code in three languages (C++, Java and Python), and various third party projects are available to provide support for other languages.

RCF provides native marshaling support for classes generated by Protocol Buffers, which can be enabled by defining the preprocessor symbol RCF_USE_PROTOBUF. With this symbol defined, classes produced by the Protocol Buffers compiler, can be used directly in RCF interface declarations, and will be serialized and deserialized according to their Protocol Buffer schemas.

For more information on building RCF with Protocol Buffer support, see Appendix - Building.

As an example, consider this .proto file.

// Person.proto

message Person {
  required int32 id = 1;
  required string name = 2;
  optional string email = 3;
}

Running the Protocol Buffer compiler (in C++ mode) over Person.proto yields the two files Person.pb.h and Person.pb.cc, containing the implementation of the C++ class Person.

// Person.pb.h

class Person : public ::google::protobuf::Message {
// ...
}

We include this code in our program, and can then use the class Person in our RCF interfaces. RCF will detect that the Person class is a Protocol Buffer class, and will use Protocol Buffer functions to serialize and deserialize Person instances.

#include <RCF/../../test/protobuf/Person.pb.h>

RCF_BEGIN(I_X, "I_X")
    RCF_METHOD_R1(Person, echo, const Person &)
RCF_END(I_X)

In RCF interfaces, we are free to mix native C++ classes, with Protocol Buffer classes.

RCF_BEGIN(I_Y, "I_Y")
    RCF_METHOD_V4(void , func, int, const std::vector<std::string> &, Person &, RCF::ByteBuffer)
RCF_END(I_Y)

The serialization and deserialization code generated by Protocol Buffers is highly optimized. However, to make the most of performance, it may be necessary to recycle Protocol Buffer objects from call to call, rather than creating new ones. That way the memory held by a Protocol Buffer object can be reused, rather than being freed and then re-allocated.

RCF provides server-side object caching, for this purpose. For example, to enable caching of the Person class:

RCF::ObjectPool & cache = RCF::getObjectPool();

// Enable server-side caching for Person.
// * Don't cache more than 10 Person objects.
// * Call Person::Clear() before putting a Person object into the cache.
cache.enableCaching<Person>(10, boost::bind(&Person::Clear, _1));

By default, RCF uses its own internal message protocol for request and response headers, error messages, handshakes, and so on. However, it also supports using a Protocol Buffer-based message protocol, where the message protocol is defined using Protocol Buffer schemas. This Protocol Buffer-based message protocol (henceforth referred to as the PB-generated message protocol) makes it possible to write code in languages other than C++, communicating directly with RCF servers and clients.

To build RCF with support for the PB-generated message protocol, the preprocessor symbol RCF_USE_PROTOBUF must be defined.

To illustrate the use of the PB-generated message protocol, we will write a C++ server, and make calls to it from a Python script. The RCF interface we'll use, is as follows:

RCF_BEGIN(I_X, "I_X")
    RCF_METHOD_R1(Person, echo, const Person &)
RCF_END(I_X)

The server implementation is trivial:

class X
{
public:
    Person echo(const Person & person)
    {
        return person;
    }
};

We start the server on a fixed port number:

X x;
RCF::RcfServer server( RCF::TcpEndpoint(50001) );
server.bind<I_X>(x);
server.start();

To write the Python script that calls the server, we need to look a little closer at the PB-generated message protocol.

Conceptually, a RCF request message consists of a sequence of Protocol Buffer objects. Each encoded object is preceded by a little-endian 4 byte size field. The entire message is also preceded by a little-endian 4 byte size field. Specifically,

  • The first object in the message is a FilterHeader, specifying any message filters that have been used to encode the message. Typically, this header will simply specify that no message filters have been used.
  • The second object is a RequestHeader, containing details of the remote call, such as which function to call on which object, whether the request is oneway or twoway, etc.
  • The remaining objects are the user defined in-parameters of the remote call.

RCF response messages are constructed in essentially the same way:

  • The first object in the message is a FilterHeader, specifying any message filters that have been used to encode the message. Typically, this header will simply specify that no message filters have been used.
  • The second object is a ResponseHeader, containing error and exception information.
  • The remaining objects are the user defined out-parameters of the remote call.

The Protocol Buffer declarations of the FilterHeader, RequestHeader and ResponseHeader classes are available in the RcfMessages.proto file in the include/RCF/protobuf folder of the distribution.

Implementing this protocol in Python is straightforward. Here is Python code to send a request to the server:

import socket
import struct
import sys
import RcfMessages_pb2
import Person_pb2

# Send the request.

# Encode the filter header.
filterHeader                = RcfMessages_pb2.FilterHeader()
filterHeader.unfilteredLen  = 0
msg                         = filterHeader.SerializeToString()
msgLen                      = len(msg)
msgLenPacked                = struct.pack('<i', msgLen)
data                        = data + msgLenPacked + msg

# Encode the request header.
request                     = RcfMessages_pb2.RequestHeader()
request.token               = 0;
request.subInterface        = "I_X";
request.fnId                = 1;
request.serializationProtocol = 1;
request.oneway              = False;
request.close               = False;
request.service             = "I_X";
request.rcfRuntimeVersion   = 5;
request.pingBackInterval    = 0;
request.archiveVersion      = 0;
msg                         = request.SerializeToString()
msgLen                      = len(msg)
msgLenPacked                = struct.pack('<i', msgLen)
data                        = data + msgLenPacked + msg

# Encode a single in-parameter.
person1                     = Person_pb2.Person()
person1.id                  = 123
person1.name                = 'Bob'
person1.email               = 'bob@example.com'
msg                         = person1.SerializeToString()
msgLen                      = len(msg)
msgLenPacked                = struct.pack('<i', msgLen)
data                        = data + msgLenPacked + msg

# Length prefix for the entire message.
dataLen                     = len(data)
dataLenPacked               = struct.pack('<i', dataLen)
data                        = dataLenPacked + data

# Send message.
HOST                        = 'localhost'
PORT                        = 50001
s                           = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.send(data)

And here is Python code to receive the response:

# Receive the response

# Receive message.
data                        = s.recv(4)
dataLen                     = struct.unpack('<i', data)[0]
data                        = s.recv(dataLen)
s.close()

# Decode the filter header.
pos                         = 0
len                         = struct.unpack('<i', data[pos:pos+4])[0]
msg                         = data[pos+4:pos+4+len]
filterHeader.ParseFromString(msg)

# Decode the response header.
pos                         = pos + 4 + len
len                         = struct.unpack('<i', data[pos:pos+4])[0]
msg                         = data[pos+4:pos+4+len]
response                    = RcfMessages_pb2.ResponseHeader()
response.ParseFromString(msg)

# Decode the out-parameter.
pos                         = pos + 4 + len
len                         = struct.unpack('<i', data[pos:pos+4])[0]
msg                         = data[pos+4:pos+4+len]
person2                     = Person_pb2.Person()
person2.ParseFromString(msg)

The RcfMessages.proto file also contains declarations for Protocol Buffer objects that can be used to interact with other functionality built in to RCF, such as creating remote objects, creating remote session objects, requesting transport filters, and requesting subscriptions.


PrevUpHomeNext