Transports

Table of Contents

• Transport Access
• Transport Configuration
  • Maximum Incoming Message Lengths
  • Connection Limits
  • System Port Selection
  • IP-based Access Rules
  • IPv4/IPv6
  • Local Address Bindings for Clients
  • Socket Level Access
• Transport Implementations
  • TCP
  • UDP
    • Multicasting
    • Broadcasting
    • Address Sharing
    • Server Discovery
  • Win32 Named Pipes
  • UNIX Domain Sockets
  • HTTP/HTTPS
    • Reverse proxies

Transport Access

Server and client transports are responsible for the actual transmission and reception of network messages. A RcfServer contains one or more server transports, while a client contains only a single client transport.

To access the server transports of a RcfServer, capture the return value of RcfServer::addEndpoint():

    RCF::RcfServer server;

    RCF::ServerTransport & serverTransportTcp = server.addEndpoint( 
        RCF::TcpEndpoint(50001) );

    RCF::ServerTransport & serverTransportUdp = server.addEndpoint( 
        RCF::UdpEndpoint(50002) );

    server.start();

Alternatively, if the RcfServer only has a single server transport, you can access it by calling RCF::RcfServer::getServerTransport():

    RCF::RcfServer server( RCF::TcpEndpoint(50001) );
    RCF::ServerTransport & serverTransport = server.getServerTransport();

On the client side, the client transport is availablable through RCF::ClientStub::getTransport():

    RcfClient<I_Echo> client( RCF::TcpEndpoint(50001) );
    RCF::ClientTransport & clientTransport = client.getClientStub().getTransport();

Transport Configuration

Maximum Incoming Message Lengths

For server-side transports, it is generally necessary to set an upper limit on the size of incoming network messages. Without an upper limit, it is possible for malformed requests to cause arbitrarily sized memory allocations on the server.

The maximum incoming message length setting of a RCF server transport defaults to 1 Mb, and can be changed by calling RCF::ServerTransport::setMaxIncomingMessageLength() :

    RCF::RcfServer server( RCF::TcpEndpoint(0) );

    // Set max message length to 5 Mb.
    server.getServerTransport().setMaxIncomingMessageLength(5*1024*1024);

Similarly, there is a maximum incoming message length setting for client transports. It defaults to 1 Mb and can be changed by calling RCF::ClientTransport::setMaxIncomingMessageLength() :

    RcfClient<I_Echo> client( RCF::TcpEndpoint(123) );

    // Set max message length to 5 Mb.
    client.getClientStub().getTransport().setMaxIncomingMessageLength(5*1024*1024);

As client transports only receive network messages from a peer they have connected to, the risk of malformed packets is not as great as for server transports.

It is possible to query a RcfClient<> for the sizes of the latest request and response messages sent:

        RcfClient<I_Echo> client( RCF::TcpEndpoint(50001) );
        client.Echo("1234");

        // Retrieve request and response size of the previous call.
        RCF::ClientTransport & transport = client.getClientStub().getTransport();

        std::size_t requestSize = transport.getLastRequestSize();
        std::size_t responseSize = transport.getLastResponseSize();

Connection Limits

To set the maximum number of simultaneous connections to a RCF server transport:

    RCF::RcfServer server( RCF::TcpEndpoint(0) );

    // Allow at most 100 clients to be connected at any time.
    server.getServerTransport().setConnectionLimit(100);

This setting is not relevant to the UDP server transport, as there is no concept of connections within the UDP protocol.

System Port Selection

For IP-based server transports, you can allow the local system to assign a server port number automatically, by specifying 0 as the port number. When the server starts, the system will find a free port and assign it to the server. The port number can subseqently be retrieved through RCF::IpServerTransport::getPort():

    RCF::RcfServer server( RCF::TcpEndpoint(0) );
    server.start();

    int port = server.getIpServerTransport().getPort();
    RcfClient<I_Echo> client(( RCF::TcpEndpoint(port) ));

IP-based Access Rules

For IP-based server transports, client access can be allowed or denied, based on the IP addresses of the clients.

To configure IP rules for allowing clients, use RCF::IpServerTransport::setAllowIps():

    RCF::RcfServer server( RCF::TcpEndpoint(0) );

    RCF::IpServerTransport & ipTransport = 
        dynamic_cast<RCF::IpServerTransport &>(server.getServerTransport());

    std::vector<RCF::IpRule> rules;

    // Match 11.22.33.* (24 significant bits).
    rules.push_back( RCF::IpRule( RCF::IpAddress("11.22.33.0"), 24) );

    ipTransport.setAllowIps(rules);

    server.start();

    // Access will be granted to clients connecting from IP addresses matching 11.22.33.* .
    // All other clients will be denied.

To configure IP rules for denying clients, use RCF::IpServerTransport::setDenyIps():

    RCF::RcfServer server( RCF::TcpEndpoint(0) );

    RCF::IpServerTransport & ipTransport = 
        dynamic_cast<RCF::IpServerTransport &>(server.getServerTransport());

    std::vector<RCF::IpRule> rules;

    // Match 11.*.*.* (8 significant bits).
    rules.push_back( RCF::IpRule( RCF::IpAddress("11.0.0.0"), 8) );

    // Match 12.22.*.* (16 significant bits).
    rules.push_back( RCF::IpRule( RCF::IpAddress("12.22.0.0"), 16) );

    ipTransport.setDenyIps(rules);

    server.start();

    // Access will be denied to clients connecting from IP addresses matching 11.*.*.*  and 12.22.*.* .
    // All other clients will be allowed.

IPv4/IPv6

RCF supports both IPv4 and IPv6. IPv6 support is enabled by default in RCF, but you can define RCF_FEATURE_IPV6=0 to disable it (see Building RCF).

For example, to run a server and client over a loopback IPv4 connection:

        // Specifying an explicit IPv4 address to bind to.
        RCF::RcfServer server( RCF::TcpEndpoint("127.0.0.1", 50001) );
        server.start();

        // Specifying an explicit IPv4 address to bind to.
        RcfClient<I_Echo> client( RCF::TcpEndpoint("127.0.0.1", 50001) );

To run a server and client over a loopback IPv6 connection instead, specify ::1 instead of 127.0.0.1:

        // Specifying an explicit IPv6 address to bind to.
        RCF::RcfServer server( RCF::TcpEndpoint("::1", 50001) );
        server.start();

        // Specifying an explicit IPv6 address to bind to.
        RcfClient<I_Echo> client( RCF::TcpEndpoint("::1", 50001) );

RCF uses the POSIX getaddrinfo() function to resolve IP addresses. getaddrinfo() can return either IPv4 or IPv6 addresses, depending on the configuration of the local system and network. So the following client will use either IPv4 or IPv6, depending on how the local system and network have been configured:

        // Will resolve to either IPv4 or IPv6, depending on what the system 
        // resolves machine.domain to.

        RcfClient<I_Echo> client( RCF::TcpEndpoint("machine.domain", 50001) );

You can force IPv4 or IPv6 resolution by using the RCF::IpAddressV4 and RCF::IpAddressV6 classes:

        // Force IPv4 address resolution.
        RCF::IpAddressV4 addr_4("machine.domain", 50001);
        RcfClient<I_Echo> client_4(( RCF::TcpEndpoint(addr_4) ));

        // Force IPv6 address resolution.
        RCF::IpAddressV6 addr_6("machine.domain", 50001);
        RcfClient<I_Echo> client_6(( RCF::TcpEndpoint(addr_6) ));

On machines with dual IPv4/IPv6 stacks, you will probably want your server to listen on both IPv4 and IPv6 addresses. To do this portably, you should listen on both 0.0.0.0 and ::0:

        // Listen on port 50001, on both IPv4 and IPv6.
        RCF::RcfServer server;
        server.addEndpoint( RCF::TcpEndpoint("0.0.0.0", 50001) );
        server.addEndpoint( RCF::TcpEndpoint("::0", 50001) );
        server.start();

On some platforms, it is sufficient to listen only on ::0, as the system will translate incoming IPv4 connections into IPv6 connections with a special class of IPv6 addresses.

Local Address Bindings for Clients

When a RcfClient<> connects to a server using an IP-based transport, the default behavior is to allow the system to decide which local network interface and local port to use. In some circumstances, you may want to explicitly set the local network interface a client should bind to.

You can do so by calling RCF::IpClientTransport::setLocalIp(), before connecting:

        RcfClient<I_Echo> client( RCF::TcpEndpoint("127.0.0.1", 50001) );
        
        RCF::IpClientTransport & ipTransport = 
            client.getClientStub().getIpTransport();

        // Force client to bind to a particular local network interface (127.0.0.1).
        ipTransport.setLocalIp( RCF::IpAddress("127.0.0.1", 0) );

        client.getClientStub().connect();

After a RcfClient<> has connected, you can determine which local network interface and port it is bound to, by calling RCF::IpClientTransport::getAssignedLocalIp():

        RcfClient<I_Echo> client( RCF::TcpEndpoint("127.0.0.1", 50001) );

        RCF::IpClientTransport & ipTransport = 
            client.getClientStub().getIpTransport();

        client.getClientStub().connect();

        // Find out which local network interface the client is bound to.
        RCF::IpAddress localIp = ipTransport.getAssignedLocalIp();
        std::string localInterface = localIp.getIp();
        int localPort = localIp.getPort();

Socket Level Access

RCF provides access to the underlying OS primitives, such as sockets and handles, of client and server transports. For example:

        // Client-side.

        RcfClient<I_Echo> client( RCF::TcpEndpoint("127.0.0.1", 50001) );
        client.getClientStub().connect();

        RCF::TcpClientTransport & tcpClientTransport = 
            dynamic_cast<RCF::TcpClientTransport &>( 
                client.getClientStub().getTransport() );

        // Obtain client socket handle.
        int sock = tcpClientTransport.getNativeHandle();

        // Server-side.

        RCF::NetworkSession & networkSession = RCF::getCurrentRcfSession().getNetworkSession();
        
        RCF::TcpNetworkSession & tcpNetworkSession = 
            dynamic_cast<RCF::TcpNetworkSession &>(networkSession);

        // Obtain server socket handle.
        int sock = tcpNetworkSession.getNativeHandle();

This can be useful if you need to set custom socket options.

Transport Implementations

This sections covers the various transport types that are implemented in RCF.

TCP

TCP endpoints are represented in RCF by the RCF::TcpEndpoint class, constructed from an IP address and a port number.

        RCF::RcfServer server( RCF::TcpEndpoint("0.0.0.0", 50001) );
        server.start();

        RcfClient<I_Echo> client( RCF::TcpEndpoint("127.0.0.1", 50001) );

Server transports interpret the IP address as the local network interface to listen on. So for example "0.0.0.0" should be specified in order to listen on all available IPv4 network interfaces, and "127.0.0.1" should be specified to listen only on the loopback IPv4 interface. If no IP address is specified, "127.0.0.1" is assumed.

UDP

Like RCF::TcpEndpoint, RCF::UdpEndpoint is constructed from an IP address and a port.

        RCF::RcfServer server( RCF::UdpEndpoint("0.0.0.0", 50001) );
        server.start();

        RcfClient<I_Echo> client( RCF::UdpEndpoint("127.0.0.1", 50001) );

RCF::UdpEndpoint also contains some extra functionality, to deal with multicasting and broadcasting.

Multicasting

RCF::UdpEndpoint can be configured to listen on a multicast IP address:

        // Listen on multicast address 232.5.5.5, on port 50001, on all network interfaces.
        RCF::UdpEndpoint udpEndpoint("0.0.0.0", 50001);
        udpEndpoint.listenOnMulticast("232.5.5.5");
        RCF::RcfServer server(udpEndpoint);

        server.start();

Note that the server still needs to specify a local network interface to listen on.

To send multicast messages, specify a multicast IP address and port when creating the client:

            RcfClient<I_Echo> client( RCF::UdpEndpoint("232.5.5.5", 50001) );
            client.Echo(RCF::Oneway, "ping");

Broadcasting

To send broadcast messages, specify a broadcast IP address and port:

            RcfClient<I_Echo> client( RCF::UdpEndpoint("255.255.255.255", 50001) );
            client.Echo(RCF::Oneway, "ping");

Address Sharing

RCF's UDP server transport can be configured to share its address binding, so that multiple RcfServer's can listen on the same port of the same interface. This is enabled by default when listening on multicast addresses, but can also be enabled when listening on non-multicast addresses. This can be useful if multiple processes on the same machine need to listen to the same broadcasts:

        EchoImpl echoImpl;

        RCF::UdpEndpoint udpEndpoint1("0.0.0.0", 50001);
        udpEndpoint1.enableSharedAddressBinding();
        RCF::RcfServer server1(udpEndpoint1);

        server1.bind<I_Echo>(echoImpl);
        server1.start();

        RCF::UdpEndpoint udpEndpoint2("0.0.0.0", 50001);
        udpEndpoint2.enableSharedAddressBinding();
        RCF::RcfServer server2(udpEndpoint2);

        server2.bind<I_Echo>(echoImpl);
        server2.start();

        // This broadcast message will be received by both servers.
        RcfClient<I_Echo> client( RCF::UdpEndpoint("255.255.255.255", 50001) );
        client.Echo(RCF::Oneway, "ping");

Server Discovery

In situations where servers are started on dynamically assigned ports, multicasting and broadcasting can be a useful means of communicating server IP addresses and ports to clients. For example:

// Interface for broadcasting port number of a TCP server.
RCF_BEGIN(I_Broadcast, "I_Broadcast")
    RCF_METHOD_V1(void, ServerIsRunningOnPort, int)
RCF_END(I_Broadcast)

// Implementation class for receiving I_Broadcast messages.
class BroadcastImpl
{
public:
    BroadcastImpl() : mPort()
    {}
    void ServerIsRunningOnPort(int port)
    {
        mPort = port;
    }
    int mPort;
};

// A server thread runs this function, to broadcast the server location once 
// per second.
void broadcastThread(int port, const std::string &multicastIp, int multicastPort)
{
    RcfClient<I_Broadcast> client( 
        RCF::UdpEndpoint(multicastIp, multicastPort) );

    client.getClientStub().setRemoteCallMode(RCF::Oneway);

    // Broadcast 1 message per second.
    while (true)
    {
        client.ServerIsRunningOnPort(port);
        RCF::sleepMs(1000);
    }
}

        // ***** Server side ****

        // Start a server on a dynamically assigned port.
        EchoImpl echoImpl;
        RCF::RcfServer server( RCF::TcpEndpoint(0));
        server.bind<I_Echo>(echoImpl);
        server.start();

        // Retrieve the port number.
        int port = server.getIpServerTransport().getPort();        

        // Start broadcasting the port number.
        RCF::ThreadPtr broadcastThreadPtr(new RCF::Thread(
            [=]() { broadcastThread(port, "232.5.5.5", 50001); }));

        // ***** Client side ****

        // Clients will listen for the broadcasts before doing anything else.
        
        RCF::UdpEndpoint udpEndpoint("0.0.0.0", 50001);
        udpEndpoint.listenOnMulticast("232.5.5.5");
        RCF::RcfServer clientSideBroadcastListener(udpEndpoint);

        BroadcastImpl broadcastImpl;
        clientSideBroadcastListener.bind<I_Broadcast>(broadcastImpl);

        clientSideBroadcastListener.start();

        // Wait for a broadcast message.
        while (!broadcastImpl.mPort)
        {
            RCF::sleepMs(1000);
        }

        // Once the clients know the port number, they can connect.
        RcfClient<I_Echo> client( RCF::TcpEndpoint(broadcastImpl.mPort));
        client.Echo("asdf");
    

Note that here we are actually using a multicast address to broadcast information to clients. If multicasting had been unavailable on this particular network, we could also have used an IP broadcast address instead of an IP multicast address.

Win32 Named Pipes

RCF supports Win32 named pipe transports. RCF::Win32NamedPipeEndpoint takes one constructor parameter, which is the name of the named pipe, with or without the leading \\.\pipe\ prefix.

        RCF::RcfServer server( RCF::Win32NamedPipeEndpoint("PrintSvrPipe") );
        server.start();

        RcfClient<I_Echo> client( RCF::Win32NamedPipeEndpoint("PrintSvrPipe") );

An advantage of using Win32 named pipes, is that they allow easy authentication of clients. A server using a Win32 named pipe server transport can authenticate its clients through the RCF::Win32NamedPipeImpersonator class, which uses the Windows API function ImpersonateNamedPipeClient() to impersonate the client:

RCF_BEGIN(I_Echo, "I_Echo")
    RCF_METHOD_R1(std::string, Echo, const std::string &)
RCF_END(I_Echo)

class EchoImpl
{
public:
    std::string Echo(const std::string & s)
    {
        // Impersonate client.
        RCF::Win32NamedPipeImpersonator impersonator(RCF::getCurrentRcfSession());
        std::cout << "Client user name: " << RCF::getMyUserName();
        return s;
    }
};

If we had used a TCP connection to 127.0.0.1 instead, we would have needed to enable Kerberos or NTLM authentication to securely determine the clients user name (see Transport protocols).

UNIX Domain Sockets

UNIX domain sockets function analogously to Win32 named pipes, and allow efficient communication between servers and clients on the same machine. RCF::UnixLocalEndpoint takes one parameter, which is the name of the UNIX domain socket. The name must be a valid filesystem path. For servers, the program must have sufficient privilege to create the given path, and the file must not already exist. For clients, the program must have sufficient privilege to access the given path.

Here is a simple example:

    RCF::RcfServer server( RCF::UnixLocalEndpoint("/home/xyz/MySocket"));
    server.start();

    RcfClient<I_Echo> client( RCF::UnixLocalEndpoint("/home/xyz/MySocket"));

HTTP/HTTPS

RCF supports tunneling remote calls over the HTTP and HTTPS protocols. In particular, remote calls can be directed through HTTP and HTTPS proxies.

HTTPS is essentially the HTTP protocol layered on top of the SSL protocol. As such, configuration of the SSL aspects of HTTPS, is done in the same way as for the SSL transport protocol (see Transport protocols).

Server-side

To setup a server with an HTTP endpoint, use RCF::HttpEndpoint:

        RCF::RcfServer server( RCF::HttpEndpoint("0.0.0.0", 80) );
        PrintService printService;
        server.bind<I_PrintService>(printService);
        server.start();

Similarly, for an HTTPS endpoint, use RCF::HttpsEndpoint:

        RCF::RcfServer server( RCF::HttpsEndpoint("0.0.0.0", 443) );
        PrintService printService;
        server.bind<I_PrintService>(printService);

        server.setCertificate( RCF::CertificatePtr( new RCF::PfxCertificate(
            "C:\\serverCert.p12", 
            "password", 
            "CertificateName") ) );

        server.start();

Client-side

Client side configuration is similar, using RCF::HttpEndpoint for a HTTP client:

        RcfClient<I_PrintService> client( RCF::HttpEndpoint("printsvr.acme.com", 80) );
        client.Print("Hello World");

, and RCF::HttpsEndpoint for a HTTPS client:

        RcfClient<I_PrintService> client( RCF::HttpsEndpoint("printsvr.acme.com", 443) );
        client.getClientStub().setCertificateValidationCallback(&schannelValidate);
        client.Print("Hello World");

Finally, to direct remote calls through a HTTP or HTTPS proxy, use the RCF::ClientStub::setHttpProxy() and RCF::ClientStub::setHttpProxyPort() functions:

            client.getClientStub().setHttpProxy("web-proxy.acme.com");
            client.getClientStub().setHttpProxyPort(8080);
            client.Print("Hello World");

Reverse proxies

HTTP reverse proxies are common on the Internet, and are used to provide functionality such as load balancing and SSL offloading for back-end HTTP servers.

Reverse proxies are generally transparent to the client and destination servers. You can place a reverse proxy between a RCF client and a server, and this provides an out-of-the-box mechanism for distributing load across a set of RCF servers.

In load-balancing scenarios, you may want a RCF client to continue sending requests to the same back-end server that it initially connects to. To accomplish this, you will need to use a reverse proxy with support for session affinity. Session affinity is normally implemented by the reverse proxy, by inserting a special HTTP cookie in the initial HTTP response of the communication stream. The RCF client will then automatically include this cookie on subsequent requests, allowing the reverse proxy to route subsequent requests through to the same back-end server.

If you have multiple RCF client connections, you can configure them to connect to the same back-end server, by retrieving the HTTP cookies from the first connection and subsequently applying them to the other connections (see RCF::ClientStub::setHttpCookies()).

Another reverse proxying feature you can utilize with RCF is SSL offloading. With SSL offloading , the RCF client connects using a HTTPS connection, but the HTTPS connection is decrypted by the reverse proxy, and the decrypted HTTP stream is forwarded on to the back-end server. This allows secure communication between the client and the back-end server, with the reverse proxy assuming the load of SSL encryption/decryption. This also has the advantage of centralizing certificate configuration, from multiple back-end servers, to the reverse proxy server.

RCF has been tested with the following reverse proxies:

• IIS ARR
• nginx