Most discussions of software performance consider two main types of metric:
Response time
The elapsed time from the moment when a user makes a request until they receive the requested answer. The unit of measurement is seconds (or milliseconds, or microseconds).
Throughput
The number of requests per second, or the data volume per second, that the system is processing. For a given allocation of hardware resources, there is a maximum throughput that can be handled. The unit of measurement is “somethings per second.”
In the social network case study, “posts per second” and “timeline writes per second” are throughput metrics, whereas “time it takes to load the home timeline” and “time until a post is delivered to followers” are response time metrics.
Throughput and response time are often related. An example of such a relationship for an online service is sketched in Figure 2-3. The service has a low response time when request throughput is low, but response time increases as load increases. This is because of queueing: when a request arrives on a highly loaded system, the CPU is likely already in the process of handling an earlier request, and therefore the incoming request needs to wait until the earlier request has been completed. As throughput approaches the maximum that the hardware can handle, queueing delays increase sharply.
A graph illustrates the relationship between throughput and response time, showing that response time increases sharply as throughput nears the hardware's capacity due to queueing.
Figure 2-3. As the throughput of a service approaches its capacity, the response time increases dramatically because of queueing.