Skip to content

notexOPENOverview

Notex/Systems/Encoding and Evolution

Systems / NOTES1 MIN READ

Encoding and Evolution

Always free to read

Programs usually work with data in (at least) two representations:

06

PART 1 / DATA FOUNDATIONS

Encoding and Evolution

13 LESSONS · 49 MIN

06.01Formats for Encoding DataPrograms usually work with data in (at least) two representations:READING1 MIN
06.02Terminology clashThe term serialization is unfortunately also used in the context of transactions, with a completely different meaning. To avoid overloading the word, we’ll stick with encoding…READING1 MIN
06.03Language-Specific FormatsMany programming languages come with built-in support for encoding in-memory objects into byte sequences. For example, Java has java.io.Serializable, Python has pickle, and…READING1 MIN
06.04JSON, XML, and Binary VariantsWhen moving to standardized encodings that can be written and read by many programming languages, JSON and XML are the obvious contenders: they are widely known and widely…READING6 MIN
06.05Protocol BuffersProtocol Buffers (protobuf) is a binary encoding library developed at Google. It is similar to Apache Thrift, which was originally developed by Facebook; most of what this…READING4 MIN
06.06AvroApache Avro is another binary encoding format, with some interesting differences from Protocol Buffers. It was started in 2009 as a subproject of Hadoop, as a result of…READING8 MIN
06.07The Merits of SchemasAs we’ve seen, Protocol Buffers and Avro both use a schema to describe a binary encoding format. Their schema languages are much simpler than XML Schema or JSON Schema, which…READING2 MIN
06.08Modes of DataflowAt the beginning of this topic we said that whenever you want to send some data to another process with which you don’t share memory—for example, when you want to send data…READING1 MIN
06.09Dataflow Through DatabasesIn a database, the process that does the writing encodes the data, and the process that does the reading decodes it. Just one process may be accessing the database, in which…READING3 MIN
06.10Dataflow Through Services: REST and RPCWhen you have processes that need to communicate over a network, you can arrange that communication in a few ways. The most common arrangement is to have two roles: clients and…READING12 MIN
06.11Durable Execution and WorkflowsBy definition, service-based architectures have multiple services that are all responsible for different portions of an application. Consider a payment processing application…READING4 MIN
06.12Event-Driven ArchitecturesIn this final section, we will briefly look at event-driven architectures, which are another way encoded data can flow from one process to another. In this context, a request…READING4 MIN
06.13Recap: Encoding and EvolutionIn this topic we looked at several ways of turning data structures into bytes on the network or on disk. We saw how the details of these encodings affect not only their…READING2 MIN

Read complete module notes →

KEEP EXPLORING

One note leads to another.

Back to the library →