SCTP specific socket functions in Linux

Preface

In the previous posts we have used socket related functions common for protocols other than SCTP. For example with connect() you can connect to a remote peer with either TCP or SCTP socket. recvmsg() and sendmsg() can also be used with UDP and SCTP. This is normal, because SCTP has features similar to TCP (connection oriented) and UDP (message based). However SCTP's unique features deserve specific API. In this post we will review some SCTP only functions for sending and receiving messages, establishing associations and extracting local and peer IP addresses from the association.

The sample code for this post is in two separate branches - one-to-many_advanced and one-to_many_peeloff. For the following material you need the first branch, so let's check it out:

git checkout one-to-many_advanced

All the functions, described in this post, are in libsctp and you need to link your application to it. This is compiler specific and offtopic for this post, so I will not talk about it. You can see how I did this with cmake in the project's CMakeLists.txt.

Obtaining association id

For one-to-many style sockets, some of the functions we are about to review require association id as an input parameter. One easy way to get it is to subscribe for sctp_association_event notification and to wait for SCTP_ASSOC_CHANGE event. Then you can obtain the association id from sac_assoc_id parameter of struct sctp_assoc_change. Check the previous post for more details about notifications.

Getting local and remote association addresses

Let's start with something trivial - getting local and peer IP address(es) of an association. There are two functions for this job. sctp_getpaddrs() returns the peer IP addresses of the association and it is described in Section 9.3. Respectively sctp_getladdrs() returns the local IP addresses and you can find more information about it in Section 9.5. They both have got similar signatures:

int sctp_getpaddrs(int sd, sctp_assoc_t id, struct sockaddr **addrs);
int sctp_getladdrs(int sd, sctp_assoc_t id, struct sockaddr **addrs);

Their input parameters are:

• sd - The socket for the association.

• id - If the socket is one-to-many style, this parameter represents the association id. For one-to-one sockets it is ignored.

• addrs - Empty pointer which will be assigned to an array, containing the local/peer IP addresses. The size of the array can be obtained from the return value of the function.

On success both functions return the number of IP addresses saved in addrs array. If the socket is not bound/connected the return value is 0. On error, -1 is returned and addrs is undefined.

One very important thing! Note that you pass to the function the address of a pointer variable. Both functions allocate memory for the array with the addresses and it is your job to free it, when you no longer need the addresses. For your convenience there are dedicated functions for this - sctp_freepaddrs() (described in Section 9.4) and sctp_freeladdrs() (described in Section 9.6). They also have got similar signatures:

int sctp_freepaddrs(struct sockaddr *addrs);
int sctp_freeladdrs(struct sockaddr *addrs);

Again, don't forget this functions or you will get memory leaks!

Now let's see how to use this functions in practice. In common.h there are two functions - get_assoc_local_addresses() and get_assoc_peer_addresses(). They are identical, but we will review both of them. First is get_assoc_local_addresses():

On line 3 we declare a NULL pointer to a struct sockaddr. It will be passed to sctp_getpaddrs(). On lines 6 and 7 the text buffer is prepared. It is filled in with zeros and the size is decremented with 1, in order to leave space for the NULL terminator. On line 9 sctp_getpaddrs() is called. Remember that its last parameter was a pointer to a struct sockaddr pointer (struct sockaddr**) and we have to pass the address of the pointer (&addrs) we defined on line 3. On lines 14-16 the address(es) are processed and saved in the text buffer. This has nothing to do with the SCTP API, so I will skip this functions. The last thing we need to to is to free the array with the addresses. We do this just after they are saved in the text buffer - on line 18 with sctp_freepaddrs().

get_assoc_peer_addresses() is identical:

The label of the log message is changed and we call sctp_getladdrs() and sctp_freeladdrs() in order to work with the local association addresses.

Binding and connecting

The SCTP specific API provides two new functions for binding to local address and connecting to a peer - sctp_bindx() (described in Section 9.1) and sctp_connectx() (described in Section 9.9). The main difference between them and bind()/connect() is that the sctp_ versions accept array of IP addresses as input parameters. These functions are useful when you want to use SCTP's multi-homing feature, but you can also use them with a single IP address.

The signature of sctp_bindx() is:

int sctp_bindx(int sd, struct sockaddr *addrs, int addrcnt, int flags);

Its input parameters are:

• sd - SCTP socket.

• addrs - Pointer to the array of with IP addresses.

• addrcnt - How many IP addresses are located in the array.

• flags - Can be SCTP_BINDX_ADD_ADDR, if you want to add bind addresses and SCTP_BINDX_REM_ADDR, to remove addresses.

On success 0 is returned. If there is an error, sctp_bindx() returns -1. Another difference between sctp_bindx() and bind() is that the former can be called more than once, however you can't remove all local IP addresses from an association.

Let's see how this function is used in server's main():

Pretty straightforward. We bind to only one IP address so we pass it directly to sctp_bindx(). The size of the array is 1 and we use SCTP_BINDX_ADD_ADDR flag, because we want to bind addresses.

sctp_connectx() has got the following signature:

int sctp_connectx(int sd, struct sockaddr *addrs, int addrcnt, sctp_assoc_t *id);

Its parameters are:

• sd - SCTP socket.

• addrs - Pointer to the array of with IP addresses.

• addrcnt - How many IP addresses are located in the array.

• id - Pointer to a sctp_assoc_t. If it is not NULL, the id of the newly created association will be written there.

sctp_connectx() returns 0 on success and -1 on error. Here is a sample code from client.c:

Almost identical to connect(). The only difference is that we pass a pointer to sctp_assoc_t and on successful connect the association ID is logged on the screen.

Sending and receiving data

We already saw how to set and retrieve some SCTP specific parameters with ancillary control messages and sendmsg()/recvmsg(). Now we are about to see an easier way to use these parameters thanks to sctp_sendmsg() (described in Section 9.7) and sctp_recvmsg() (described in Section 9.8). Both functions wrap the socket interface from the previous post, so I where necessary I will redirect to sections from there.

I want to make one clarification before we go on. According to RFC 6458, sctp_recvmsg() and sctp_sendmsg() are deprecated and you should use sctp_sendv() and sctp_recvv() instead. However for some reason they are not implemented in neither Linux, nor FreeBSD. I was unable to find out why so if you know something about this please leave a comment.

Let's have a look at sctp_sendmsg()'s signature:

ssize_t sctp_sendmsg(int sd, const void *msg, size_t len, const struct sockaddr *to, socklen_t tolen,
                     uint32_t ppid, uint32_t flags, uint16_t stream_no, uint32_t timetolive, uint32_t context);

The return value is the number of bytes sent or -1 in case of an error. There are quite a lot of parameters, but most of them are already familiar:

• sd - SCTP socket.

• msg - The buffer with the message payload.

• len - The length of the buffer.

• to - Destination IP address.

• tolen - Length of the destination IP address.

• ppid - Payload protocol ID.

• flags - The same as snd_flags in struct sctp_sndinfo.

• stream_no - Output stream ID.

• timetolive - Time to live (in milliseconds).

• context - Context id, which is sent to the upper layer if an error occurs during the message sending.

If you need more details about some of the parameters, check my previous post and especially Ancillary data that can be passed to sendmsg(). Now let's have a look at some code:

The example is self-explanatory. The string "OK" is sent over the wire, without setting any of the SCTP specific parameters.

Let's move on to sctp_recvmsg() and its signature:

ssize_t sctp_recvmsg(int sd, void *msg, size_t len, struct sockaddr *from, socklen_t *fromlen,
                        struct sctp_sndrcvinfo *sinfo, int *msg_flags);

On success the function returns the number of bytes read. Return value of -1 indicates an error. Its input parameters are:

• sd - SCTP socket.

• msg - Pointer to a user provided buffer for the incoming message.

• len - The size of the buffer.

• from - Pointer to a user provided buffer, which will be filled in with the source IP address of the incoming message.

• fromlen - Size of the source IP address buffer.

• sinfo - Pointer to a structure, which contains SCTP specific parameters. It is similar to struct sctp_rcvinfo, so I won't describe it. For more information check Section 5.3.2.

• flags - The values are the same as the ones described in Receiving messages with recvmsg() section from SCTP Linux API: One-to-many style interface post.

You can see code with sctp_recvmsg() in the next section, where the function get_connection() is listed.

Peel off

Peel off is a function call which extracts an association from one-to-many style socket and allows you to work with it individually in one-to-one style. Function's name is sctp_peeloff(), it is described in Section 9.2 and its definition is:

int sctp_peeloff(int sd, sctp_assoc_t assoc_id);

On success the function returns the new socket, which is in one-to-one style. On error, -1 is returned. The function has got the following input parameters:

• sd - One-to-many SCTP socket.

• assoc_id - Association ID of the target, that we want to peel off. As we already discussed, this ID can be obtained via SCTP_ASSOC_CHANGE event or sctp_connectx().

Let's see an example. In the branch one-to_many_peeloff I have written a SCTP server which waits for new connections on one-to-many style sockets, extracts the newly created association and spawns a thread which handles it in one-to-one style. First let's switch to this branch:

git checkout one-to_many_peeloff

All the modifications in this branch are in the server. It initially creates one-to-many (SOCK_SEQPACKET) socket and waits for a connection. Then when the association is established the server peels it off to one-to-one style socket and passes it to a new thread, which handles the client. This scenario has a lot of drawbacks from scalability point of view, but it is simple enough to see sctp_peeloff() in action.

Let's start with server's main(). The only change there is that in the main loop get_connection() is called :

Now let's see what it does:

This is nothing more than the get_message() function with modified while loop. The idea here is to use sctp_recvmsg() to get the notification about new association, to extract the association id (with get_association_id()) and then peel off the association in handle_client(). I want to remark that if your purpose is to peel off each association, it is totally pointless to use one-to-many style sockets! Stick with one-to-one and safe yourself from doing all the extra work. But this project is only for demonstration purposes, so we don't bother.

get_association_id() does some error checking and returns the association id from struct sctp_assoc_change:

We have done this before, so let's see what happens in handle_client():

On lines 5-8 we allocate memory for the new socket on the heap, because we have to pass it to the client handling thread. Then on lines 10-15 we peel off the association. We are in one-to-one style now and we don't want to receive notifications anymore. They are disabled on lines 17-21. Finally on lines 23-45 a thread is created for the new client.

The function client_thread() handles the messages from the clients. Here is its implementation:

Our first job is to save the socket on the stack deallocate the memory. The rest is identical to the server behavior you have seen so far.

Conclusion

In this post we saw how to access most of the SCTP features in a simplified and more convenient manner. On the projects I was working on so far, this interface in combination with the notifications were sufficient to get the job done. I recommend you to keep your life simple and stick with the sctp_XXX functions. Fall back to ancillary data only in case you need something really specific. Thanks for reading!