So, what is Remote Call Framework (RCF), and what would you use it for?
RCF is a C++ library that allows you to make remote calls between C++ processes. RCF provides a clean and simple solution for getting multiple C++ processes talking to each other, with a minimum of overhead and a minimum of complexity.
The simplest way to understand how RCF works is to just jump in and get started with an example. This tutorial presents a simple RCF-based client and server, and subsequently uses them to introduce fundamental RCF features.
We'll start off with a simple TCP-based client and server. We want to write a server exposing a print service, which receives messages from clients and prints them to standard output.
This is the server:
And this is the client:
The code should be fairly self-explanatory.
To build the code, copy it into a C++ source file, and then compile it along with RCF.cpp
(available in the distribution at <rcf_distro>/src/RCF/RCF.cpp
, where <rcf_distro>
is the location at which you have unpacked the RCF distribution). You will also need to add <rcf_distro>/include
to your compilers include directories. More information on building is available in Building RCF.
Let's now start the server:
, followed by the client:
In the server window, you should now see:
So far, so good.
To make it easier to run and debug the code, we can rewrite our print service application so that the client and server run in a single process:
Running this program yields the output:
Let's summarize what we have at this point:
127.0.0.1
) interface.127.0.0.1:50001
, and communicating in clear text with the server.std::cout
.The rest of the tutorial will build on this example, to illustrate some of the fundamental features of RCF.
RCF interfaces are defined in C++, directly in your code. You define RCF interfaces much as you would define any other C++ code, and then bind those interfaces directly to your C++ client and server code.
In the example above, we defined the I_PrintService
interface:
This code defines the RCF interface I_PrintService
. The interface contains a single method Print()
. The Print()
method has a void return type and takes a single const std::string &
parameter.
The RCF_METHOD_<xx>()
macros are used to define remote methods. RCF_METHOD_V1()
defines a remote method with a void return type and one parameter. RCF_METHOD_R2()
defines a remote method with a non-void return type and two parameters, and so on. The RCF_METHOD_<xx>()
macros are defined for methods with void and non-void return types, taking up to 15 parameters.
On the server, the RCF interface is bound to a servant object:
The server will now route incoming remote calls on the I_PrintService
interface, to the printService
object.
Let's add a few more methods to the I_PrintService
interface. We'll add methods to print a list of strings, and return the number of characters printed:
Having added these methods to the I_PrintService
interface, we also need to implement them in the PrintService
servant object:
Here is how we would call the new Print()
methods:
As part of executing a remote call, RCF serializes the parameters of the remote call into a byte sequence and transmits them over a network connection. On the other side of the connection, RCF deserializes the parameters from a byte sequence back into actual C++ objects.
RCF handles serialization of most standard C++ data types automatically. All you need to do is include the relevant serialization header for the data types you are using. A table of standard C++ data types, along with their RCF serialization headers, is available in the section on Serialization.
You can also use your own application data types as parameters in remote calls. In that case you will need to provide serialization functions for those data types.
So far we've only used standard C++ data types in the I_PrintService
interface. Imagine now that we want to add a Print()
method to the I_PrintService
interface, that takes an application-specific LogMessage
structure, with LogMessage
defined as follows:
Before we can use the LogMessage
class in a RCF interface, we need to provide a RCF serialization function for it.
Serialization functions are generally quite simple to write. They can be provided as member functions, or as free standing global functions within the same namespace as the class being serialized.
The serialization function for a data type essentially specifies which members take part in serialization, and is called whenever RCF needs to serialize or deserialize an object of that type.
Serialization functions can be member functions, or free standing functions. Here is a serialization member function for LogMessage
:
If we wanted to use a free standing function instead, it would look like this:
And that's it. We can now add a method to the I_PrintService
interface:
, and call it from a client:
Remote calls are always made through a RcfClient<>
instance. Each RcfClient<>
instance contains a RCF::ClientStub
, which can be accessed by calling RcfClient<>::getClientStub()
. RCF::ClientStub
is where almost all client side configuration of remote calls takes place.
Two important client side settings are the connection timeout and the remote call timeout. The connection timeout determines how long the client will wait while trying to establish a network connection to the server, while the remote call timeout determines how long the client will wait for a remote call response to return from the server.
To change these settings, call the RCF::ClientStub::setConnectTimeoutMs()
and RCF::ClientStub::setRemoteCallTimeoutMs()
functions:
Another commonly used client-side configuration setting is the remote call progress callback, configured through RCF::ClientStub::setRemoteCallProgressCallback()
. This allows you to monitor or cancel a remote call while it is in progress:
There are many other client-side configuration settings, documented in RCF::ClientStub
.You can read more about client side configuration in Client-side programming.
On the client-side every RcfClient<>
instance controls a single network connection to the server. On the server-side, RCF maintains a session (RCF::RcfSession
) for every such connection to the server. The RcfSession
of a client connection is available to server-side code through the global RCF::getCurrentRcfSession()
function.
You can use RcfSession
to maintain application data specific to a particular client connection. Arbitrary C++ objects can be stored in a session by calling RCF::RcfSession::createSessionObject<>()
or RCF::RcfSession::getSessionObject<>()
.
For instance, here is how we would associate a PrintServiceSession
object with each client connection to the I_PrintService
interface, to tracks the numer of calls made on that connection:
Distinct RcfClient<>
instances will have distinct connections to the server. Here we are calling Print()
three times from two RcfClient<>
instances:
, resulting in the following output:
The server session and any associated session objects are destroyed when the client connection is closed.
See Server-side programming for more about server sessions.
In RCF, the transport layer handles transmission of messages across network connections. The transport layer is determined by the endpoint parameters passed to the RcfServer
and RcfClient<>
constructors. So far we've been using RCF::TcpEndpoint
, to specify a TCP transport.
By default, when you specify a RCF::TcpEndpoint
with only a port number, RCF will use 127.0.0.1
as the IP address. So the following two snippets are equivalent:
, as are the following:
127.0.0.1
is the IPv4 loopback address. A server listening on 127.0.0.1
will only be accessible to clients on the same machine as the server. It's likely you'll want to run clients across the network, in which case the server will need to listen on an externally visible network address. To do that, specify 0.0.0.0
(for IPv4), or ::0
(for IPv6) instead, which will make the server listen on all available network interfaces:
RCF supports a number of other endpoint types as well. To run a server and client over UDP, use RCF::UdpEndpoint
:
It's usually not practical to use UDP for two-way (request/response) messaging, as the unreliable semantics of UDP mean that messages may not be delivered, or may be delivered out of order. UDP is useful in one-way messaging scenarios, where the server application logic is resilient to messages being lost or arriving out of order.
To run a server and client over named pipes, use RCF::Win32NamedPipeEndpoint
(for Windows named pipes) or RCF::UnixLocalEndpoint
(for UNIX local domain sockets):
HTTP and HTTPS endpoints are also available, if you need to tunnel remote calls through the HTTP protocol. The most common reason for doing so is to traverse network infrastructure such as HTTP proxies:
Multiple transports can be configured for a single RcfServer
. For example, to configure a server that accepts TCP, UDP and named pipe connections:
See Transports for more information on transports.
RCF provides a number of options for encryption and authentication of remote calls. Encryption and authentication are provided as transport protocols, layered on top of the transport.
RCF supports the following transport protocols:
The NTLM and Kerberos transport protocols are only supported on Windows. The SSL transport protocol is supported on all platforms, but for non-Windows builds requires RCF to be built with OpenSSL support (see Building RCF).
A RcfServer
can be configured to require its clients to use certain transport protocols:
On the client-side, RCF::ClientStub::setTransportProtocol()
is used to configure transport protocols. For instance, to enable the NTLM protocol on a client connection:
In this example, the client will authenticate itself to the server and encrypt its connection using the NTLM protocol, using the implicit Windows credentials of the logged on user.
To provide explicit credentials, use RCF::ClientStub::setUserName()
and RCF::ClientStub::setPassword()
:
The Kerberos protocol can be configured similarly. The Kerberos protocol requires the client to supply the Service Principal Name (SPN) of the server. You can do this by calling RCF::ClientStub::setKerberosSpn()
:
If a client attempts to call Print()
without configuring one of the transport protocols required by the server, they will get an error:
From within the server-side implementation of Print()
, you can retrieve the transport protocol in use on the current session. If the transport protocol is NTLM or Kerberos, you can also retrieve the Windows user name of the client, and even impersonate the client:
RCF also supports using SSL as the transport protocol. RCF provides two SSL implementations, one based on OpenSSL, and the other based on the Windows Schannel security package. The Windows Schannel implementation is used automatically on Windows, while the OpenSSL implementation is used if RCF_FEATURE_OPENSSL=1
is defined when building RCF (see Building RCF).
SSL-enabled servers require a SSL certificate to present to clients. The mechanics of certificate configuration vary depending on which implementation (Schannel or OpenSSL) is being used. Here is an example using the Schannel SSL implementation:
RCF::PfxCertificate
is used to load PKCS #12 certificates, from .pfx
and .p12
files. RCF also provides the RCF::StoreCertificate
class, to load certificates from a Windows certificate store.
If instead you are using OpenSSL, you can use the RCF::PemCertificate
class to load PEM certificates, from .pem
files.
RCF supports transport level compression of remote calls, using zlib. Compression is configured independently of transport protocols, using ClientStub::setEnableCompression()
. The compression stage is applied immediately before the transport protocol stage. In other words, remote call data is compressed before being encrypted.
This is an example of a client connecting to a server through an HTTP proxy, using NTLM for authentication and encryption, and with compression enabled:
For more information see Transport protocols.
By default a RcfServer
uses a single thread to handle incoming remote calls, and consequently remote calls are dispatched serially, one after the other. Even if you have multiple transports configured in your server, the transports will all be serviced by a single thread.
This is normally the safest way to run a server, as there is no need for thread synchronization in your server-side application code. You should also be aware that a single-threaded server does not at all limit the number of clients that can concurrently access your server. It only limits the number of remote calls that can execute concurrently. If the remote calls in your application execute quickly, then most likely a single threaded server will be more than sufficient to handle a large number of clients.
However, it is also common to have remote calls that do take some time to execute. For example, your server-side code may be making requests to other backend servers, with potentially significant latency. If the backend server is slow to respond, then the RCF server thread will be blocked for some time and other client requests will be delayed. In this situation, you would need a multi-threaded server in order to be able to continue responding to other clients.
To configure a multi-threaded RcfServer
, use RCF::RcfServer::setThreadPool()
to assign a thread pool to the server. A RCF thread pool can be configured either with a fixed number of threads:
, or a varying number of threads, depending on server load:
Thread pools configured through RCF::RcfServer::setThreadPool()
are shared across all the transports within the RcfServer
. If you want to configure dedicated thread pools for particular transports, you can do so using RCF::ServerTransport::setThreadPool()
:
RCF normally executes remote calls synchronously, and will block the client thread until the call completes. RCF also allows you to execute remote calls asynchronously. Rather than blocking the client thread, the client thread is instead notified at a later point when the remote call has completed.
Asynchronous remote calls are performed in RCF using the RCF::Future<>
template. Essentially, for any parameter or return type T
in a remote call, if you instead pass in a Future<T>
, the remote call will be executed asynchronously. Multiple Future<T>
instances can be passed in to a remote call. You can configure the Future<T>
instances so that you receive a callback on completion, or you can use RCF::Future::wait()
or RCF::Future::ready()
to check the status of the remote call. Once the remote call has completed, you can use RCF::Future::operator*()
to dereference a Future<T>
instance and retrieve the actual parameter value.
Given this RCF interface:
, here is an example of a client that makes an asynchronous calls and then polls for the result to become available:
Of course, polling for completion is not all that different from making a blocking call. A more powerful technique is to assign a completion callback to the remote call. A completion callback is a function object which RCF will call once the remote call is complete. Here is a simple example:
Note that we've passed both the Future<int>
return value and a reference counted client (PrintServicePtr
) to the callback. If we were to destroy the client on the main thread, the asynchronous call would be automatically canceled, and the callback would not be called at all.
Asynchronous remote calls are useful in many circumstances. Here is an example of a client calling Print()
once every 10 seconds on 50 different servers:
If we had to write this code using synchronous calls, we would need 50 threads, each connecting to one server. By using asynchronous calls, we can manage all 50 connections with a single thread. The remote calls to the 50 servers are all completed on a background RCF thread, and the connections are automatically destroyed when the main thread goes out of scope.
For more information, see Asynchronous Remote Calls.
Imagine that we have a number of I_PrintService
servers on our network, and that we want our client to make the same Print()
remote call to all of them. To do so using regular two-way remote calls would be tedious. First, we would have to somehow maintain a list of currently available I_PrintService
servers. Assuming we had such a list, we would then have to manually call Print()
on each server, with the same message.
This kind of scenario is much better suited to a messaging paradigm commonly known as publish/subscribe. The publish/subscribe paradigm is based on a single publisher maintaining a number of publishing topics, and multiple subscribers subscribing to those topics. Whenever the publisher publishes a message on a topic, that message is sent out to all the subscribers subscribing to that particular topic.
The major advantage of the publish/subscribe system is that the publishing code does not need to know anything about the subscribers. The subscribers themselves are responsible for registering for topics they are interested in, while the publisher simply publishes messages.
RCF makes it easy to set up a publish/subscribe system. Publish/subscribe functionality is provided by the RCF::RcfServer::createPublisher()
and RCF::RcfServer::createSubscription()
functions. For example, here is code for a publisher publishing a Print()
call once every second:
Creating a subscription to this publisher is similarly simple:
Published calls are sent out as one way messages to all listening subscribers.
For more information on publish/subscribe messaging, see Publish/subscribe.
File downloads and uploads are common in distributed systems. For smaller files, you can just load them into a std::string
(or better, a RCF::ByteBuffer
) and send them on their way. This approach breaks down though, for files big enough to exceed the maximum message length of your connection.
Rather than requiring you to split the file into chunks and send the chunks through one by one, RCF provides dedicated functions for downloading and uploading files. These functions take care of the necessary chunking logic, as well as the low level details of concurrent network and disk I/O that are necessary to maximize throughput. RCF also provides the ability to resume interrupted transfers, as well as limiting the amount of bandwidth a file transfer consumes.
For example, here is how we would implement a PrintFile()
method on the I_PrintService
interface:
Notice that the PrintFile()
method takes a upload identifier in the form of a string. To call PrintFile()
, the client needs to first use RCF::ClientStub::uploadFile()
, to upload the relevant file and obtain a string identifier for the upload. The string identifier is then passed to PrintFile()
, which locates the upload on the server, prints the file and then deletes it.
Downloading files is done similarly. Here is how we would implement a GetPrintSummary()
method, whose purpose is to return a potentially large file to the client:
GetPrintSummary()
configures a download on the server, and then returns the download identifier to the client. The client then calls RCF::ClientStub::downloadFile()
with the download identifier, to download the file:
For more information, see File transfers.