The term serialization is unfortunately also used in the context of transactions, with a completely different meaning. To avoid overloading the word, we’ll stick with encoding in these notes, even though serialization is perhaps more common.
Sometimes encoding/decoding is not needed—for example, when a database operates directly on compressed data loaded from disk, as discussed in “Query Execution: Compilation and Vectorization”. There are also zero-copy data formats that are designed to be used both at runtime and on disk/on the network, without an explicit conversion step, such as Cap’n Proto and FlatBuffers.
However, most systems need to convert between in-memory objects and flat byte sequences. As this is such a common problem, there are a myriad libraries and encoding formats to choose from. Let’s do a brief overview.