In today's digital era, it is essential for companies to be prepared to handle complex systems and deliver a seamless user experience to their customers. To achieve this, IT managers need a clear, accurate view of the health of the systems under their responsibility. It is in this context that the four golden signals emerge as a key approach to systems monitoring.

The four golden signals are latency, traffic, errors, and saturation. If you can measure only four metrics of your user-facing system, focus on these four indicators.

Latency: This metric represents the time it takes to serve a request. It's important to distinguish between the latency of successful requests and the latency of failed requests. For example, an HTTP 500 error can be served quickly due to a lost connection to a database or another critical backend resource. However, since an HTTP 500 error indicates a failed request, including 500s in your overall latency count can lead to inaccurate calculations. On the other hand, a slow failure is even worse than a fast failure. That's why it's important to track error latency rather than simply filter errors out.

Traffic: This is a measure of the demand placed on your system, measured by a high-level metric specific to the system. For a web service, this measurement is usually in HTTP requests per second, broken down by request type (for example, static versus dynamic content). For an audio streaming system, the measurement might focus on network I/O rate or concurrent sessions. For a key-value storage system, the measurement might be in transactions and retrievals per second.

Errors: The rate of requests that fail, whether explicitly (for example, HTTP 500s), implicitly (for example, a successful HTTP 200 response that is paired with the wrong content), or by policy (for example, "if you've committed to one-second response times, any request that takes longer than one second is an error"). Monitoring these error rates is important for identifying weak points in the system and ensuring the quality of the service delivered to users.

Saturation: How "full" your service is. A measure of the fraction of your system in use, emphasizing the most constrained resources (for example, in a memory-constrained system, show memory; in an I/O-constrained system, show I/O). Note that many systems degrade in performance before they reach 100% utilization, so having a utilization target is essential. In complex systems, saturation can be complemented with a higher-level load measurement: can your service properly handle double the traffic, handle only 10% more traffic, or handle even less traffic than it currently receives? For very simple services that have no parameters changing the complexity of the request (for example, "Give me a nonce" or "I need a globally unique monotonic number") and that rarely change configuration, a static value from a load test may be adequate. However, as discussed in the previous paragraph, most services need to use indirect signals such as CPU utilization or network bandwidth that have a known upper bound. Increases in latency are often an early sign of saturation. Measuring your 99th-percentile latency over some small window (for example, one minute) can give a very early signal of saturation.

These four golden signals are crucial for IT management, as they provide an overall view of the challenges the system faces. Monitoring the four signals lets IT managers quickly identify problems, determine their severity, and make informed decisions to resolve them. By maintaining an overall view of system performance, IT managers can anticipate potential problems before they occur, ensuring business continuity and preventing service disruptions. In addition, monitoring the four signals lets IT managers identify areas for improvement in the system, which can help boost its efficiency and scalability.

Turn your IT operation into a thriving business with Infraops. With an impressive success rate, Infraops is the ideal choice for companies seeking professional, reliable solutions to monitor and operate their critical IT environments. Our highly skilled team delivers tailored solutions that ensure system availability, data protection, and customer satisfaction. Join our satisfied clients and experience the difference Infraops can make for your business. Contact us.

.

‍