SCTP Multi-homing in Linux

Introduction

After exploring how SCTP multi-homing works, it's time to see how to use this feature in Linux. This post will show how to implement multi-homing for the client-server application, used up to now. Baseline code for the development will be the 'one-to-many_advanced' branch, used in SCTP specific socket functions in Linux post. It already uses SCTP's advanced interface so only a few modifications are required to get the job done.

Implementing multi-homing means that the server should bind to more than one IP address and the client - to connect to more than one destination IP address. Both applications can also operate on single IP address. Mixed scenario is also possible, e.g. multi-homed server can communicate with non multi-homed client.

The code in this post uses one-to-many styles sockets, however the approach is the same for one-to-one sockets. I am pretty sure that you will be able to handle the latter case by yourself, so I will not discuss it. Anyway do not hesitate to leave a comment, in case you have got any questions.

How to implement multi-homing

You should remember from SCTP specific socket functions in Linux post that sctp_connectx() and sctp_bindx() can work with more than one IP address. This means that it is mandatory to use these functions in a multi-homed application. Everything else, like link monitoring and path switch-over) is done by the kernel.

On the server side sctp_bindx() is called to specify which IP addresses and ports should be used. The addresses are passed via addrs parameter (which is a pointer to an array of struct sockaddr) and addrcnt is the elements count.

On the client side, sctp_connectx() is used to connect to a multi-homed server. This function has also got addrs parameter (pointer to array of struct sockaddr) and addrcnt (the elements count). If specific local IP addresses should be used for the association establishment, sctp_bindx() should also be called to specify them. If random local port number should be used (which is a good idea in most of the cases) sin_port in struct sockaddr should be set to zero.

Code review

To explore multi-homing, proper network environment is needed. The setup from the previous post is good enough, so I will use it again. The Vagrantfile is here. The code for this post is in one-to-many_multihoming branch. Check it out:

git checkout one-to-many_multihoming

Both client and server have got command line parameters for the server port, local and peer IP addresses. In real world applications it's very important to implement user friendly handling of those parameters. However the main purpose of this project is to demonstrate how SCTP is used and I try to keep the code as focused on this topic as possible. For this reason I decided to sacrifice the proper command line arguments handling and implemented a really basic solution. For example the list of local/remote addresses is represented by a semi-colon separated string. If a space/tab is put between the addresses - the application will return an error. I hope you will excuse this sloppiness for the sake of simplicity.

The code which parses the command line arguments is based on getopt. I consider it not relevant to this post and I will not discuss it at all.

if(sctp_bindx(server_fd, (struct sockaddr*)&bind_addr, 1, SCTP_BINDX_ADD_ADDR) == -1) {

if(sctp_bindx(server_fd, (struct sockaddr*)&local_addrs, local_addr_count, SCTP_BINDX_ADD_ADDR) == -1) {

int generate_default_addresses(struct sockaddr_in* bind_addrs, int* count, const int server_port)
{
    memset(bind_addrs, 0, sizeof(struct sockaddr_in));
    bind_addrs->sin_family = ADDR_FAMILY;
    bind_addrs->sin_port = htons(server_port);
    bind_addrs->sin_addr.s_addr = INADDR_ANY;

    *count = 1;

    return 0;
}

if(local_ipaddr_list != NULL) {
    if(sctp_bindx(client_fd, (struct sockaddr*)&local_addrs, local_addr_count, SCTP_BINDX_ADD_ADDR) == -1) {
        perror("bind");
        return 9;
    }
}

if(local_ipaddr_list != NULL) {
    if(parse_addresses(local_ipaddr_list, strlen(local_ipaddr_list), 0, local_addrs, MAX_MULTIHOMING_ADDRESSES, &local_addr_count)) {
        return 7;
    }
}

sctp_assoc_t assoc_id = 0;
if(sctp_connectx(client_fd, (struct sockaddr*)&remote_addrs, remote_addr_count, &assoc_id) == -1) {
    perror("sctp_connectx");
    return 11;
}

int sctp_sendmsg(int sd, const void * msg, size_t len,
                        struct sockaddr *to, socklen_t tolen,
                        uint32_t ppid, uint32_t flags,
                        uint16_t stream_no, uint32_t timetolive,
                        uint32_t context);

Few words about one-to-one style sockets

Everything discussed up to now is also valid for one-to-one sockets. Binding is done with sctp_bindx() and connecting to the server - with sctp_connectx(). For sending data sctp_send() can be used, which is the SCTP alternative of send(). Here is its declaration:

int sctp_send(int sd, const void * msg, size_t len,
                    const struct sctp_sndrcvinfo *sinfo,
                    uint32_t flags);

It is like sctp_sendmsg(), but without the address parameter. This function is very convenient for the cases when you don't have to pick destination IP addresses and you don't want to use notifications.

Run the code

Now let's test the client and the server. Run the server on alpha:

./server '192.168.35.10;192.168.45.10' 4444

And the client on beta:

./client -l '192.168.35.11;192.168.45.11' -r '192.168.35.10;192.168.45.10' -p 4444

They will exchange some messages and the client will quit. You can investigate the association with Wireshark. What you see should be similar to the results in the previous post.

Conclusion

In many posts I have advertised multi-homing as one of the SCTP's killer features. I avoided this topic many times and promised dedicated posts for it. I did this on purpose and I hope you are now convinced that with the appropriate foundations multi-homing is easy to implement. In many cases you might not need multi-homing support in your application, but my advise is to design your code in a way which allows you to enable it with minimum effort. If your app uses SCTP, in 99% of the cases someone will want to use multi-homing with it.

Thanks for reading and don't hesitate to comment, if you have got any questions!