Protocol Buffers is an open source project released by Google, which uses a specialized language to define serialization schemas, from which actual serialization code is generated. The official Protocol Buffers compiler can generate code in three languages (C++, Java and Python), and various third party projects are available to provide support for other languages.
RCF provides native marshaling support for classes generated by Protocol
Buffers, which can be enabled by defining the preprocessor symbol RCF_USE_PROTOBUF. With this symbol defined,
classes produced by the Protocol Buffers compiler, can be used directly in
RCF interface declarations, and will be serialized and deserialized according
to their Protocol Buffer schemas.
For more information on building RCF with Protocol Buffer support, see Appendix - Building.
As an example, consider this .proto file.
// Person.proto message Person { required int32 id = 1; required string name = 2; optional string email = 3; }
Running the Protocol Buffer compiler (in C++ mode) over Person.proto
yields the two files Person.pb.h and Person.pb.cc, containing the implementation of the
C++ class Person.
// Person.pb.h class Person : public ::google::protobuf::Message { // ... }
We include this code in our program, and can then use the class Person in our RCF interfaces. RCF will
detect that the Person class
is a Protocol Buffer class, and will use Protocol Buffer functions to serialize
and deserialize Person instances.
#include <RCF/../../test/protobuf/Person.pb.h> RCF_BEGIN(I_X, "I_X") RCF_METHOD_R1(Person, echo, const Person &) RCF_END(I_X)
In RCF interfaces, we are free to mix native C++ classes, with Protocol Buffer classes.
RCF_BEGIN(I_Y, "I_Y") RCF_METHOD_V4(void , func, int, const std::vector<std::string> &, Person &, RCF::ByteBuffer) RCF_END(I_Y)
The serialization and deserialization code generated by Protocol Buffers is highly optimized. However, to make the most of performance, it may be necessary to recycle Protocol Buffer objects from call to call, rather than creating new ones. That way the memory held by a Protocol Buffer object can be reused, rather than being freed and then re-allocated.
RCF provides server-side
object caching, for this purpose. For example, to enable caching of
the Person class:
RCF::ObjectPool & cache = RCF::getObjectPool(); // Enable server-side caching for Person. // * Don't cache more than 10 Person objects. // * Call Person::Clear() before putting a Person object into the cache. cache.enableCaching<Person>(10, boost::bind(&Person::Clear, _1));
By default, RCF uses its own internal message protocol for request and response headers, error messages, handshakes, and so on. However, it also supports using a Protocol Buffer-based message protocol, where the message protocol is defined using Protocol Buffer schemas. This Protocol Buffer-based message protocol (henceforth referred to as the PB-generated message protocol) makes it possible to write code in languages other than C++, communicating directly with RCF servers and clients.
To build RCF with support for the PB-generated message protocol, the preprocessor
symbol RCF_USE_PROTOBUF must
be defined.
To illustrate the use of the PB-generated message protocol, we will write a C++ server, and make calls to it from a Python script. The RCF interface we'll use, is as follows:
RCF_BEGIN(I_X, "I_X") RCF_METHOD_R1(Person, echo, const Person &) RCF_END(I_X)
The server implementation is trivial:
class X { public: Person echo(const Person & person) { return person; } };
We start the server on a fixed port number:
X x; RCF::RcfServer server( RCF::TcpEndpoint(50001) ); server.bind<I_X>(x); server.start();
To write the Python script that calls the server, we need to look a little closer at the PB-generated message protocol.
Conceptually, a RCF request message consists of a sequence of Protocol Buffer objects. Each encoded object is preceded by a little-endian 4 byte size field. The entire message is also preceded by a little-endian 4 byte size field. Specifically,
FilterHeader,
specifying any message filters that have been used to encode the message.
Typically, this header will simply specify that no message filters have
been used.
RequestHeader,
containing details of the remote call, such as which function to call on
which object, whether the request is oneway or twoway, etc.
RCF response messages are constructed in essentially the same way:
FilterHeader,
specifying any message filters that have been used to encode the message.
Typically, this header will simply specify that no message filters have
been used.
ResponseHeader,
containing error and exception information.
The Protocol Buffer declarations of the FilterHeader,
RequestHeader and ResponseHeader classes are available in
the RcfMessages.proto file in the include/RCF/protobuf folder of the distribution.
Implementing this protocol in Python is straightforward. Here is Python code to send a request to the server:
import socket import struct import sys import RcfMessages_pb2 import Person_pb2 # Send the request. # Encode the filter header. filterHeader = RcfMessages_pb2.FilterHeader() filterHeader.unfilteredLen = 0 msg = filterHeader.SerializeToString() msgLen = len(msg) msgLenPacked = struct.pack('<i', msgLen) data = data + msgLenPacked + msg # Encode the request header. request = RcfMessages_pb2.RequestHeader() request.token = 0; request.subInterface = "I_X"; request.fnId = 1; request.serializationProtocol = 1; request.oneway = False; request.close = False; request.service = "I_X"; request.rcfRuntimeVersion = 5; request.pingBackInterval = 0; request.archiveVersion = 0; msg = request.SerializeToString() msgLen = len(msg) msgLenPacked = struct.pack('<i', msgLen) data = data + msgLenPacked + msg # Encode a single in-parameter. person1 = Person_pb2.Person() person1.id = 123 person1.name = 'Bob' person1.email = 'bob@example.com' msg = person1.SerializeToString() msgLen = len(msg) msgLenPacked = struct.pack('<i', msgLen) data = data + msgLenPacked + msg # Length prefix for the entire message. dataLen = len(data) dataLenPacked = struct.pack('<i', dataLen) data = dataLenPacked + data # Send message. HOST = 'localhost' PORT = 50001 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((HOST, PORT)) s.send(data)
And here is Python code to receive the response:
# Receive the response # Receive message. data = s.recv(4) dataLen = struct.unpack('<i', data)[0] data = s.recv(dataLen) s.close() # Decode the filter header. pos = 0 len = struct.unpack('<i', data[pos:pos+4])[0] msg = data[pos+4:pos+4+len] filterHeader.ParseFromString(msg) # Decode the response header. pos = pos + 4 + len len = struct.unpack('<i', data[pos:pos+4])[0] msg = data[pos+4:pos+4+len] response = RcfMessages_pb2.ResponseHeader() response.ParseFromString(msg) # Decode the out-parameter. pos = pos + 4 + len len = struct.unpack('<i', data[pos:pos+4])[0] msg = data[pos+4:pos+4+len] person2 = Person_pb2.Person() person2.ParseFromString(msg)
The RcfMessages.proto file also contains declarations for
Protocol Buffer objects that can be used to interact with other functionality
built in to RCF, such as creating remote objects, creating remote session
objects, requesting transport filters, and requesting subscriptions.