SCTP Linux API: One-to-many style interface

Receiving messages with recvmsg()

get_message() uses recvmsg() in a loop to get a message from the network. According to man recvmsg(2) recvmsg() has got the following definition:

ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);

The function accepts three parameters and returns the number of bytes read. The first parameter is the file descriptor (returned from socket()). The third one is flags, which I will not discuss here. If you need information about them check man recvmsg(2). In the sample code this parameter is set to 0, which means that no flags are set.

The interesting here is how to correctly fill in struct msghdr. It is described in man recvmsg(2) like this:

struct msghdr {
    void *msg_name;           /* ptr to socket address structure */
    socklen_t msg_namelen;    /* size of socket address structure */
    struct iovec *msg_iov;    /* scatter/gather array */
    int msg_iovlen;           /* # elements in msg_iov */
    void *msg_control;        /* ancillary data */
    socklen_t msg_controllen; /* ancillary data buffer length */
    int msg_flags;            /* flags on received message */
};

Generally speaking it contains three arrays (msg_name, msg_iov and msg_control) and a flag variable. msg_name is a buffer with the IP address and port of the sender. This field should point to struct sockaddr_in (for IPv4) or struct sockaddr_in6 (for IPv6). The sample code handles only the IPv4 case. msg_namelen is the size of the address buffer.

msg_iov contains the buffer(s) where the payload of the message will be saved. It is an array of struct iovec which man recvmsg(2) defines this way:

struct iovec {                    /* Scatter/gather array items */
    void  *iov_base;              /* Starting address */
    size_t iov_len;               /* Number of bytes to transfer */
};

iov_base points to a buffer with preallocated memory and iov_len is its size.

msg_control is a buffer with ancillary data and msg_controllen is its length. It allows the programmer to receive various SCTP protocol parameters. Working with this data requires handling of different set of data structures and I will discuss it in another post.

Finally msg_flags contains message related flags. Don't confuse this with recvmsg() flags! The message flags are strictly specific for the currently received message. The most important of them is MSG_EOR, which indicates end of message. It is guaranteed that each call of recvmsg() will return data from single message. However if the buffers supplied aren't big enough, the message will be retrieved with multiple recvmsg() calls. MSG_EOR flag being set is an indication that the whole message is retrieved. You can find the rest of the flags in man recvmsg(2).

Finally, let's see a code sample from get_message(), which fills in struct msghdr:

On lines 1-3 a statically allocated buffer is defined. We expect to receive string, so as a precaution its size is decremented with one for the null terminator.

Lines 5-8 declare struct iovec. iov_base is set to the preallocated buffer and iov_len - to its size.

Lines 10-15 declare struct msghdr. msg_iov is set to the address of the structure we just created. iovlen is set to 1, because there is only one struct iovec. If we have created an array of struct iovec, here we would have passed the number of its elements. msg_name is set to the address of struct sockaddr_in (passed as parameter) and msg_namelen is just sizeof(struct sockaddr_in).

Lines 17-31 handle message receiving. recvmsg() is called in an infinite loop. On each call we print data to the screen. Please note that in the real world, the programmer usually needs to move the payload to a bigger buffer, until the whole message is received. In this post however we want to focus on the socket API, so it is just printed out. Also note how we check if MSG_EOR is set after each call (line 24). When this happens we break out from the loop.

Sending messages with sendmsg()

send_reply() function sends response to the client, which is again simple string. This job is done by sendmsg(). It is very similar to recvmsg(). It accepts the same parameters, but the flags are a bit different. You can read about them in man sendmsg(2). The meaning of struct msghdr fields are also a bit different:

• msg_name and msg_namelen represent the IP address and port of the receiver of the message.

• msg_iov and msg_iovlen are the payload of the message.

• msg_control and msg_controllen again contains ancillary data.

• msg_flags are the flags, defined for recvmsg().

By default sendmsg() is blocking, which means we should not bother to check if the whole message is sent. This of course is not true for non-blocking sockets or when fancy sendmsg() flags are set. Here is a sample code from send_reply(), which handles the message sending:

It is pretty similar to the one in get_message() so I believe it doesn't need any explanations.