Serialization

Table of Contents

• Standard C++ Types
• User-defined Types
• Binary Data
• Portability
• Serialization To and From Disk

Serialization is a fundamental part of every remote call. Remote call arguments need to be serialized into a binary format so they can be transmitted across the network, and once received, they need to be deserialized from a binary format back into regular C++ objects.

RCF provides a built-in serialization framework which handles serialization of common C++ types automatically, and can be customized to serialize arbitrary user-defined C++ types.

Standard C++ Types

Serialization of fundamental C++ types (char, int, double, etc) is handled automatically by RCF. Serialization of a number of other common and standard C++ types requires inclusion of the relevant header file:

Type	Serialization header to include
std::string, std::wstring, std::basic_string<>	#include <SF/string.hpp>
std::vector<>	#include <SF/vector.hpp>
std::list<>	#include <SF/list.hpp>
std::deque<>	#include <SF/deque.hpp>
std::set<>	#include <SF/set.hpp>
std::map<>	#include <SF/map.hpp>
std::pair<>	#include <SF/utility.hpp>
std::bitset<>	#include <SF/bitset.hpp>
std::auto_ptr<>	#include <SF/auto_ptr.hpp>
std::unique_ptr<>	#include <SF/unique_ptr.hpp>
std::shared_ptr<>	#include <SF/shared_ptr.hpp>
std::tuple<>	#include <SF/tuple.hpp>
std::array<>	#include <SF/array.hpp>
boost::scoped_ptr<>	#include <SF/boost/scoped_ptr.hpp>
boost::shared_ptr<>	#include <SF/boost/shared_ptr.hpp>
boost::intrusive_ptr<>	#include <SF/boost/intrusive_ptr.hpp>
boost::any	#include <SF/boost/any.hpp>
boost::tuple<>	#include <SF/boost/tuple.hpp>
boost::variant<>	#include <SF/boost/variant.hpp>
boost::array<>	#include <SF/boost/array.hpp>

C++ enums are serialized automatically, as integers. Serialization of C++11 enum classes requires the use of a helper macro:

// Legacy C++ enum. Automatically serialized as 'int'.
enum Suit
{
    Heart = 1,
    Diamond = 2,
    Club = 2,
    Spade = 2
};

// C++11 enum class with custom base type (8 bit integer).
enum class Colors : std::int8_t 
{ 
    Red = 1, 
    Green = 2, 
    Blue = 3 
};

// Use SF_SERIALIZE_ENUM_CLASS() to specify the base type of the enum class.
SF_SERIALIZE_ENUM_CLASS(Colors, std::int8_t)

User-defined Types

If you take a class of your own, and use it in an RCF interface:

class Point3D
{
public:
    double mX;
    double mY;
    double mZ;
};

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(Point3D, Echo, const Point3D &)
RCF_END(I_Echo)

, you'll end up with an compiler error similar to this one:

..\\..\\..\\..\\..\include\SF\Serializer.hpp(324) : error C2039: 'serialize' : is not a member of 'Point3D'
        C6.cpp(13) : see declaration of 'Point3D'
        ..\\..\\..\\..\\..\include\SF\Serializer.hpp(336) : see reference to function template instantiation 'void SF::serializeInternal<T>(SF::Archive &,T &)' being compiled
        with
        \[
            T=U
        \]
        <snip>

The compiler is telling us that it couldn't find any serialization code for the class Point3D. We need to provide a serialize() function, either as a member function:

class Point3D
{
public:
    double mX;
    double mY;
    double mZ;

    // Internal serialization.
    void serialize(SF::Archive &ar)
    {
        ar & mX & mY & mZ;
    }
};

, or as a free function in either the same namespace as Point3D, or in the SF namespace:

// External serialization.
void serialize(SF::Archive &ar, Point3D &point)
{
    ar & point.mX & point.mY & point.mY;
}

The code in the serialize() function specifies which members to serialize.

The serialize() function is used both for serialization and deserialization. In some cases, you may want to use different logic, depending on whether the code is serializing or deserializing an object. For example, the following snippet implements serialization of the boost::gregorian::date class, by representing it as a string:

// Serialization code for boost::gregorian::date can be placed either in
// the boost::gregorian::date namespace (where the path class is defined), or 
// in the SF namespace. Here we've chosen the SF namespace.
namespace SF {

    void serialize(SF::Archive & ar, boost::gregorian::date & dt)
    {
        if (ar.isWrite())
        {
            // Code for serializing.
            std::ostringstream os;
            os << dt;
            std::string s = os.str();
            ar & s;
        }
        else
        {
            // Code for deserializing.
            std::string s;
            ar & s;
            std::istringstream is(s);
            is >> dt;
        }
    }

}

Binary Data

To send a chunk of binary data, you can use the RCF::ByteBuffer class:

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(RCF::ByteBuffer, Echo, RCF::ByteBuffer)
RCF_END(I_Echo)

class EchoImpl
{
public:
    RCF::ByteBuffer Echo(RCF::ByteBuffer byteBuffer)
    {
        return byteBuffer;
    }
};

} 

int main()
{
    EchoImpl echoImpl;
    RCF::RcfServer server( RCF::TcpEndpoint(0));
    server.bind<I_Echo>(echoImpl);
    server.start();

    int port = server.getIpServerTransport().getPort();

    // Create and fill a 500 kb byte buffer.
    RCF::ByteBuffer byteBuffer(500*1024);
    for (std::size_t i=0; i<byteBuffer.getLength(); ++i)
    {
        byteBuffer.getPtr()[i] = char(i % 256);
    }

    RcfClient<I_Echo> client(( RCF::TcpEndpoint(port) ));

    // Echo it.
    RCF::ByteBuffer byteBuffer2 = client.Echo(byteBuffer);

    return 0;
}

std::string or std::vector<char> could be used for the same purpose. However, serialization and marshaling of RCF::ByteBuffer is significantly more efficient (see Performance). With RCF::ByteBuffer, no copies at all will be made of the data, on either end of the wire.

Portability

Because C++ does not prescribe the sizes of its fundamental types, there is potential for serialization errors when servers and clients are deployed on different platforms. For example, consider the following interface:

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(std::size_t, Echo, std::size_t)
RCF_END(I_Echo)

The Echo() method uses std::size_t as a parameter and return value.

Unfortunately, std::size_t has different meanings on different platforms. For example, the 32 bit Visual C++ compiler considers std::size_t to be a 32 bit type, while the 64 bit Visual C++ compiler considers std::size_t to be a 64 bit type. Consequently, if a remote call is made from a 32 bit client to a 64 bit server, using a method with std::size_t parameters, a runtime serialization error will be raised.

The same issue arises when using the type long with 32 and 64 bit versions of gcc. 32-bit gcc considers long to be a 32 bit type, while 64-bit gcc considers long to be a 64-bit type.

The correct approach in these situations is to use typedefs for arithmetic types whose bit sizes are guaranteed. In particular, the standard C++ <cstdint> header provides a number of useful typedefs, including the following.

Type	Description
std::int16_t	16 bit signed integer
std::uint16_t	16 bit unsigned integer
std::int32_t	32 bit signed integer
std::uint32_t	32 bit unsigned integer
std::int64_t	64 bit signed integer
std::uint64_t	64 bit unsigned integer

To ensure portability, the example with std::size_t, above, should be rewritten as:

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(std::uint32_t, Echo, std::uint32_t)
    RCF_METHOD_R1(std::uint64_t, Echo, std::uint64_t)
RCF_END(I_Echo)

Serialization To and From Disk

RCF serialization can be used to serialize objects to and from disk. To do so, use SF::OBinaryStream and SF::IBinaryStream:

    std::string filename = "data.bin";

    X x1;
    X x2;

    // Write x1 to a file.
    {
        std::ofstream fout(filename.c_str(), std::ios::binary);
        SF::OBinaryStream os(fout);
        os << x1;
    }

    // Read x2 from a file.
    {
        
        std::ifstream fin(filename.c_str(), std::ios::binary);
        SF::IBinaryStream is(fin);
        is >> x2;
    }

For more advanced topics on serialization, see Advanced Serialization.