Source and Sink in Crypto++

This post builds upon A Brief Introduction to Crypto++. I suggest to have a quick look at it before diving into Source and Sink classes.

Sources and sinks are quite similar, so we will review them together. Just a refresher: a source provides an input for a Crypto++ pipeline, while a sink terminates it by saving the result to an output. The input and output of a pipeline are usually data containers (e.g., arrays, vectors, strings) or files.

Sources

Sources implement the Source base class. The following implementations exist:

• FileSource
• StringSource
• ArraySource (which is a typedef of StringSource)
• VectorSource
• RandomNumberSource

StringSource and VectorSource write the result to the corresponding STL containers. Their constructor parameters are self-explanatory as long as you follow Crypto++'s convention: a pointer means 'pass ownership,' while a reference means 'retain ownership.'

StringSource has a constructor that accepts a pointer to a byte array:

StringSource (const byte *string, size_t length, bool pumpAll, BufferedTransformation *attachment=NULL)

In Crypto++, byte is a typedef for unsigned char, so StringSource (and ArraySource) can work with either a std::string or an unsigned char array.

FileSource gets input data from a file (an instance of std::ifstream). RandomNumberSource uses an RNG (random number generator) to generate input data, but I'll leave this for Chapter 6, which is dedicated to RNGs.

Sinks

Sinks implement the Sink base class. Their implementations are similar to the Source ones:

• StringSink - writes data to a std::string or an array.
• VectorSink - writes data to a std::vector.
• RandomNumberSink - adds additional entropy to an RNG (more on RNGs in Chapter 6).
• ArraySink - unlike its ArraySource counterpart, it has a separate implementation. It uses a fixed-size buffer to write to. Once the buffer is full, ArraySink silently discards any additional data.
• BitBucket - swallows and discards everything passed to it.
• FileSink - writes to a file (an instance of std::ostream).

Each Sink also implements BufferedTransformation, allowing it to be used to terminate pipelines, as we will see in a moment.

You should keep in mind that sinks writing to STL containers will resize them as needed. This behavior might lead to unexpected memory allocations. To avoid the performance penalty associated with such allocations, it is a good practice to preallocate sufficient memory for the output buffer.

Code Sample - StringSource and VectorSink

Let's see some examples. The first one reads a std::string with StringSource and saves it to a std::vector with a VectorSink:

#include <iostream>
#include <string>
#include <vector>

#include <cryptopp/cryptlib.h>
#include <cryptopp/filters.h>

int main() {
  const std::string input{"INPUT"};
  std::vector<uint8_t> output;

  (void)CryptoPP::StringSource(input, true, new CryptoPP::VectorSink(output));

  if (output != std::vector<uint8_t>({73, 78, 80, 85, 84})) {
    std::cout << "Output should match input" << std::endl;
  }

  return 0;
}

The signature of the StringSource constructor is:

StringSource(
  const std::string &string,
  bool pumpAll,
  BufferedTransformation *attachment = NULLPTR
)

The first parameter is a reference to the input string. There are overrides accepting const byte* (aka const unsigned char*) and const char* pointers. It's important to know that Source implementations doesn't take ownership over any of these const pointers. Remember that in A Brief Introduction to Crypto++ we said that Crypto++ classes don't take ownership over pointers to primitive types such as char and int. So in this case if the input was allocated dynamically it's our responsibility to handle the memory deallocation.

The pumpAll parameter is a boolean indicating whether all the input should be pumped to the output. Usually, you want this parameter set to true unless you have a very large input or want manual flow control. The final parameter (attachment) is a pointer to a BufferedTransformation instance, which in our case is VectorSink.

Usually, BufferedTransformation is implemented for filters, but in order to terminate a Crypto++ pipeline with a sink, the Sink base class also implements BufferedTransformation. In other words, anywhere you can put a filter, you can also put a sink to terminate the pipeline.

Now let's see how the final element of the pipeline, the VectorSink, is initialized. VectorSink is actually a typedef:

DOCUMENTED_TYPEDEF(StringSinkTemplate<std::vector<byte> >, VectorSink);

It has a simple constructor:

StringSinkTemplate(T &output)

Note that T in our case is std::vector<byte>. So VectorSink accepts a reference to a std::vector<unsigned char>. Sinks usually don't take ownership of their inputs because you want to use them after the pipeline finishes. Consuming them would leave you empty-handed.

Also note that VectorSink will resize the output buffer if it is not big enough so again - it's a good practice to preallocate enough memory for the outputs to avoid performance penalties from unnecessary memory allocations.

Code Sample - FileSource and FileSink

Let's have a look at another example involving files:

#include <iostream>
#include <string>
#include <vector>

#include <cryptopp/cryptlib.h>
#include <cryptopp/files.h>
#include <cryptopp/filters.h>

int main() {
  const std::string input{"INPUT"};
  std::vector<uint8_t> result;

  {
    auto output = std::ofstream("/tmp/test-file", std::ios::binary);
    (void)CryptoPP::StringSource(input, true, new CryptoPP::FileSink(output));
  }

  {
    auto input = std::ifstream("/tmp/test-file", std::ios::binary);
    (void)CryptoPP::FileSource(input, true, new CryptoPP::VectorSink(result));
  }

  if (result != std::vector<uint8_t>({73, 78, 80, 85, 84})) {
    std::cout << "Output should match input" << std::endl;
  }

  return 0;
}

This sample contains two pipelines. The first one reads a std::string with a StringSource and saves it to a file with a FileSink. The second one reads the same file with a FileSource and saves it to a VectorSink. We already saw how StringSource and VectorSink work, so in this sample, we'll focus on the file source/sink.

The FileSink constructor looks like this:

FileSink(std::ostream &out)

Note that FileSink doesn't take ownership of the std::ostream object, so it is our responsibility to deallocate it (if needed) and more importantly to close the output file. That's why each pipeline is enclosed in a separate scope. std::ostream closes the file on deallocation, and we want the file closed before the second pipeline starts.

FileSource's constructor is quite similar to StringSource's:

FileSource(std::istream &in, bool pumpAll, BufferedTransformation *attachment = NULLPTR)

We still have attachment and pumpAll parameters, but the input parameter here is a reference to std::istream. Again, closing the file is our responsibility, which is why this pipeline is in a separate scope as well.

Conclusion

Source and Sink are fundamental elements of a Crypto++ pipeline since they start and terminate it. Crypto++ provides implementations that work with std::string, std::vector, byte arrays, and files. If you need something more complex (e.g., a source reading from a network socket), you can implement your own Source or Sink, but this is beyond the scope of this book.

Each Sink is also a filter since they both implement BufferedTransformation. This makes sense because wherever you can plug in a filter, it should also be possible to terminate the pipeline with a Sink.

The book