Assuming that you have a meaningful quality indicator, you can then set an objective for your organization to strive for. As it is commonly referred to in the SRE community, trying to ensure anything is reliable 100% of the time is a fool’s errand. In the information age, nothing runs in a vacuum, everything depends on something else - mistakes are made, things break, failures happen and unforeseen events take place. It's all part of the journey, that's ok! As long as these don't happen too often... SLOs serve just that, for you to determine how often you are allowed to fail while ensuring your users are still happy.

So let's get technical, how do you go from an indicator to an objective? Besides having an SLI, there are two things you need to define:

The first is the quality target you are aiming for. This is simply a value (between 0 and 100) in respect to your SLI's success ratio, equation 1. This threshold defines the number of failures that are comfortable enduring at a certain point in time.

The second is the time window during which the objective will track on, which involves picking its size and type.

WINDOW TYPE

The size of your window will define how long term the your SLO-based decision making process should be.

Shorter time windows are better for short-term decisions. If you missed your SLO last week then you can prioritize small optimizations, bug fixes, and reducing technical debt so that you can do better during the next few weeks. Longer periods allow you to be more strategic on the general direction your team is heading. "Should I have my engineers focus on moving our back-end to another framework that's more reliable since they're always complaining about the amount of trouble that the current one presents or should I have them automate a pipeline block with a new ML model ?" Simply put: do you want to increase the stability of your back-end with a better framework, or increase the amount of uncertainty with automation? Well, one week of data doesn't provide you with enough information to allow you to make such a big decision. Regardless of your choice, your engineers will have their hands full for the next few weeks so you want to make sure that during that amount of time your service meets your quality standards.

WINDOW SIZE

Regarding the window type: there are two types of windows: rolling windows which are continuously moving as time passes, and static windows, which are bound to calendar periods (e.g., a week, a month, a year). They both have their advantages and limitations. You can even decide to use both simultaneously as long as you are well aware of how to correctly interpret each one.

Rolling windows are more aligned with user experience.

As a rule of thumb, rolling windows should be defined in terms of weeks to consistently include the same number of weekends since it is common for the amount of traffic to vary significantly between weekends and weekdays.

Static windows are more aligned with the usual planning within an organization and overall project calendar, for example, you may wish to evaluate whether you were able to achieve your objective monthly or quarterly so you can plan.

Don't worry if you are feeling a bit overwhelmed by the number of decisions you need to make to establish an SLO. SRE is still an emerging field and there are no absolute right answers! Each service is a service, each organization is an organization. Recall what we said previously, this is a journey, not a destination! Start by choosing something that you can understand. As soon as you have at least one SLO up and running you'll immediately start to get valuable data about your service! You can iterate and make the appropriate changes as you start to get a feel for how these metrics evolve specifically in your service. Over time you can (and should!) re-think and re-iterate your SLO configurations to continuously improve their usefulness to your organization.

Error budgets are the most advanced part of the reliability stack because they rely on the two previous building blocks being meaningful.

As previously stated, when you define your SLO's target you are basically defining two states for your service: your success ratio is either acceptable, in which case you are in budget, or not, in which case you are out of budget, the image below illustrates the two scenarios:

As you can see, your error budget is automatically defined once you set your target:

What you are interested in measuring is the remaining error budget (REB), meaning the wiggle room you still have left until your good event ratio breaks the quality target that you have defined. As a way to generalize, this is measures as the percentage of the total error budget that has been consumed, as expressed by the equation bellow.

Simply put, you can think of EB as representations of how much your service is allowed to fail until it breaks your definition of quality.

EB allow you to say things such as "We can only suffer 3000 more errors until we break our budget this month", or "We only have 2 more hours of budget this quarter". The REB can be a powerful tool for decision-making, it provides you with an intuitive way to identify whether you should focus on developing new features or whether you should solidify the current state of your service. The figure below presents the most basic of algorithms that an organization can employ regarding EB (borrowed from Hidalgo et al. [1]).

A common mistake SRE beginners make when thinking about error budgets is to simply think of them as gas tanks that get filled at the beginning of your SLO window and decrease as time goes on and bad events occur. While it is true that your budget is full when your period starts and that bad events consume budget, you must remember that the consistent occurrence of good events can also provide you with budget. Only with time and practice can you truly grasp the dynamics evolved in an error budget, it varies according to your SLI, your SLO's time window type, and your own service as well.

Now for a more practical way of understanding how the dynamics behind the concepts that we have just introduced relate to one another. These notions can be grasped by thoughtful analysis of the mathematical formulas presented throughout the article, nonetheless, examples and visual representations are usually good for a deeper understanding.

Imagine the common case where HTTP requests with a 200 code are the good events and 500 code are bad events, and you had established a target value of 95%. For those of you with no web development background, HTTP codes are just a simple way of indication whether a request was fulfilled correctly or not.

At a given point, let's say tA you have a total of 100k requests from which 99k are successful (HTTP 200), and 1k are unsuccessful (HTTP 500). The percentage of good events is therefore 99%.

Suddenly, for some unforeseen circumstance your platform misbehaves for a period of time, during which you endure 3k unsuccessful requests. At the end of that period, let's say at tB, your percentage of good events is 99.000/103.000≈96%.

You are able to fix the problem and things go back to normal, some time passes and from then on you were able to respond successfully to another 2k requests, at which point, tC, your percentage of good events is 101.000/104.000≈97%.

Congratulations! If you got here, hopefully, you now have a better understanding of the fundamental concepts behind SRE. Nonetheless, you are just getting started! There is a lot more stuff built on top of these building blocks that can help your organization make even more informed decisions and keep you up to the date with the reliability of your service. As we said at the beginning of this article SLOs are a journey, not a destination, and there are always ways to improve! We here at [Rely.io](https://Rely.io/) are here to help you every step of the way.

Now that you understand how your error budgets evolve with time, it can be helpful to monitor how fast (or slow) you are losing budget or when is it predicted to run out, this is usually known as the rate of error budget consumption (or burn rate). Moreover, you may even want to set up policies for being alerted when specific events occur so you can act on them as quickly as possible! You may learn about this topics in our upcoming blog articles!

[1] Implementing Service Level Objectives: A Practical Guide to SLIs, SLOs, and Error Budgets" by Alex Hidalgo | 2020

[2] "The Site Reliability Workbook: Practical Ways to Implement SRE" by Betsy Beyer, Niall Richard Murphy, et al. | 2018

[3] "Site Reliability Engineering: How Google Runs Production Systems" by Niall Richard Murphy, Betsy Beyer, et al. | 2016