Everybody has an intuitive idea of what it means for something to be reliable or unreliable. For software, typical expectations include the following:
-
The application performs the function that the user expected.
-
The application can tolerate the user making mistakes or using the software in unexpected ways.
-
Its performance is good enough for the required use case, under the expected load and data volume.
-
The system prevents any unauthorized access and abuse.
If all those things together mean “working correctly,” then we can understand reliability as meaning, roughly, “continuing to work correctly, even when things go wrong.” To be more precise about things going wrong, we will distinguish between faults and failures:
Fault
A fault occurs when a particular part of a system stops working correctly—for example, if a single hard drive malfunctions, or a single machine crashes, or an external service (that the system depends on) has an outage.
Failure
A failure occurs when the system as a whole stops providing the required service to the user—in other words, when it does not meet the SLO.
The distinction between faults and failures can be confusing because they are the same thing, just at different levels. For example, if a hard drive stops working, we say that the hard drive has failed; if the system consists of only that one hard drive, it has stopped providing the required service and thus has also failed. However, if the system consists of multiple hard drives, the failure of a single hard drive is only a fault from the point of view of the bigger system, and the bigger system might be able to tolerate that fault by having a copy of the data on another hard drive.