Clocks and time are important. Applications depend on clocks in various ways, to answer questions like the following:
-
Has this request timed out yet?
-
What’s the 99th percentile response time of this service?
-
How many queries per second did this service handle on average in the last five minutes?
-
How long did the user spend on our site?
-
When was this article published?
-
At what date and time should the reminder email be sent?
-
When does this cache entry expire?
-
What is the timestamp on this error message in the log file?
Questions 1–4 measure durations (e.g., the time interval between a request being sent and a response being received), whereas questions 5–8 describe points in time (events that occur on a particular date, at a particular time).
In a distributed system, time is a tricky business, because communication is not instantaneous; it takes time for a message to travel across the network from one machine to another. The time when a message is received is always later than the time when it is sent, but because of variable delays in the network, we don’t know how much later. This fact sometimes makes it difficult to determine the order in which things happened when multiple machines are involved.
Moreover, each machine on the network has its own clock, which is a hardware device—usually a quartz crystal oscillator. These devices are not perfectly accurate, so each machine has its own notion of time, which may be slightly faster or slower than on other machines. It is possible to synchronize clocks to some degree; the most commonly used mechanism is the Network Time Protocol (NTP), which allows the computer clock to be adjusted according to the time reported by a group of servers. The servers in turn get their time from a more accurate time source, such as a GPS receiver.