How to Optimize API Rate Limits

This is a complementary blog post to a video from the ShopifyDevs YouTube channel. It is the final post from a three-part series created by Zameer Masjedee, a Solutions Engineer with Shopify Plus. See part one, An Introduction to Rate Limits, part two, API Rate Limits and Working with GraphQL, and part three, Implementing API Rate Limits in Your App.

In this post, I’ll continue showing my work in the application in a development store introduced to you in part three of this series, Implementing API Rate Limits in Your App. First, I’ll teach you how to capture rate limit information that is crucial to running your app. Then I’ll walk you through the best practices on how to optimize your app so as to responsibly use your limits. 

By the end of this article, you’ll be able to responsibly consume your rate limit, avoid 429 responses, and build a fast, functional app.

How to capture rate limit information

In this section we’re going to go through how to capture all the necessary information you’ll need regarding your rate limit. We’ll begin by showing you how to create your initial rawRequest, and then we’ll walk through the various elements you’ll need to include in your request, including creating the initial structure. 

Creating your request

We begin with using a rawRequest to capture all that information. It looks like this: 

We’re going to say await GraphQLClient.rawRequest(productUpdateMutation, productInput. 

Formulating your then clause

We’re also going to use a then clause, which allows us to wait until this execution is done to do some other business logic. We’ll go over why we’re structuring the query this way a bit below, but for now, know that the skeleton of your then clause should look like this:

then(({ errors, data, extensions, headers, status })) => {

}

Inside this clause is where we define what needs to happen, first logging the response: console.log(JSON.stringify(data, undefined, 2)). The other thing we want to know is how much API rate limit we still have available. The log for that request looks like this: 

console.log(There are ${extensions.cost.throttleStatus.currentlyAvailable} rate limit points available/n) 

Your completed, final then clause should look like this: 

It should be nested inside your await clause. So, your whole rawRequest should look like this:

Now with our extensions we can take a look at the cost to see how much rate limit we still have available. Make sure to save everything, clear your console, and then press run. 

You can see that when we made the initial request, we had 958 rate limit points left because we were making a query for 10 different products. In order to find the mutations in your terminal, you want to scroll down to just below all that data we got back from our request. 

It’s showing us that we started with 969 rate limit points. By the time we made that second mutation, we actually had 975 rate limit points. Then it increased again by seven points, to 982.

This is our leaky bucket algorithm in action. As you might be able to recall, we get 50 more points every second. We’re observing that in the fluctuation of rate limit points available above. In the amount of time it takes us to make the mutation, wait for the response, print the response and make the next mutation, our rate limit is changing.

You might also like: An Introduction to Rate Limits. 

Recall that a mutation takes just 10 points. This explains why, in less than a second, we're able to refill our rate limit faster than we're able to use it. This means that in theory I should be able to let this run for as long as I have products available and I'm not going to eat into all of my rate limit. It will not run into a rate limit error if it's able to constantly refill it faster than it's able to consume it.

The only problem with this structure is that while you're not wasting your rate limit, you’re also not optimizing it either. Shopify gives you 1,000 rate limit points—use it! That's how you can operate your app as efficiently as possible. 

"Shopify gives you 1,000 rate limit points—use it! That's how you can operate your app as efficiently as possible."

Optimizing by removing your await clause

One thing we can do to achieve this is to remove the await from our rawRequest. 

When working with Node.js, the await allows us to make an asynchronous request that has some sort of future or Promise associated with it. There's something that this request will ultimately return. By using the await command, we’re saying whether we want to wait for that to complete or not.

If we chose not to use it, our rawRequest could execute as fast as we iterate through the for loop, making multiple asynchronous requests in a row. That would definitely consume your API rate limit faster.

We’re going to take a look at that in action, but before doing so we need to make some changes to our rawRequest. To implement best practices, we’re going to add a catch clause at the bottom of our rawRequest because we’re expecting an error, and we want to log that error. It should look like this:

The other thing we want to change is the number of products we’re querying for. Instead of trying to retrieve 10 products at once, we’re going to make the request to retrieve 100 products at once. 

Once you’ve made those adjustments, you want to save it, clear your terminal, and run it again. 

Hitting your rate limit

There should be quite a few changes happening in your terminal. You won’t be able to see the entire history, but you should see a number of errors, some requests being made, and finally the rate limit points that are left.

Now, clearly if we’re running into errors this method doesn't work well. This isn’t good production level code. We’ll need to put some safeguards in place to prevent those errors.

However, what it does do is bring our rate limit down to three. It eats up all of that 1,000, and that's a good thing. It's consuming as much as it can. But the problem is that it's consuming more than it should. 

We need to find a way to optimize it, without going over our rate limit. That's what we're going to do next.

Optimizing your rate limit by pairing synchronous and asynchronous requests

One way to optimize your rate limit is by pairing up a sequence of synchronous and asynchronous requests. Every time we go through our for loop, we don't want to wait for every single request. That's not fast enough. We’re going to wait for every other request.

This means that effectively in a second, or the amount of time it takes for a single request to process, we’re sending two requests instead. As a result, we should see our rate limit decreasing over time. 

In our example, we're keeping track of the index of this loop using our edge parameter. On it we’re going to create a little conditional if statement.  

Using a modulus function, we’re saying that if our edge is even or every other edge is equal to (==) zero (0), we want to wait for our rawRequest to complete. If it's not (else), then we’re going to want to run it asynchronously. This means that we’re putting two requests in, but we’re only waiting for every other one. 

Remember to save your new conditional statement, clear your terminal, and put it to work.

You can see that the API rate limit is increasing pretty slowly, right? But over time, if you have an endless list of products and let it run indefinitely, you’ll see it start to consume your API rate limit. However, we're still sitting around 500 to 568. This indicates that it's not as fast as it can be.

We can do better. We can make more requests. 

Before doing so, we’re going to implement some best practices by associating our conditional statement to a variable in our async function section. We’re going to write const numberOfParallelRequests = 3, and paste it down into our conditional. 

When you run it again, your rate limit should go down a lot faster. You should be getting into the 400s and lower. This means that my app is moving a lot faster, which is great. But it’s still not reaching its rate limit before it's done iterating those 100 products. 

How do we make sure we’re running as close to the rate limit as possible? 

Getting as close to your rate limit as possible: setting up your safety net 

So far, we’ve achieved a bit of active optimization, but we can definitely do better. Our goal is to make our numberOfParallelRequests as high as possible without actually running into our rate limit by getting our available limit to zero.

The first thing we’re going to do is implement a safety net. This ensures that even if you do consume a lot of your API rate limit and it begins to approach zero, you’re not going to make a request when you’re running close to your rate limit. 

In the following example I’ll show you how. 

We're already capturing and returning our available rate limit in our logs, under console.log(There are ${extensions.cost.throttleStatus.currentlyAvailable} rate limit points available\n). 

So we’re just going to tie that log to a variable, that way we can have it on hand to use. 

In order to do that you’re going to create a new variable in your main function. In this example, we’re going to set that new variable to zero availableRateLimit = 0, for now.

Then you’re going to set your availableRateLimit variable in both your rawRequests' callback functions under the conditional. It’ll look like this: 

Great! This gives us a way to keep track of how much we have left. Now we have to set our threshold. 

The threshold prevents us from making requests when we’re below a set rate limit. We want to avoid going below zero and hitting those 429 responses. 

In order to do that, we’re just going to define another variable. We’re going to call it rateLimitThreshold, and we’re going to set it to 50. Now, 50 isn’t an arbitrary number. We can do some mental math here. We know that every mutation costs 10 query points and I'm doing about three of them at once. That would be 30 points used in the amount of time it takes to do one request. Therefore, 30 is less than 50. Given that the amount of time we’re waiting for requests is roughly about a second, this math should check out. 

Now I expect that sometimes we’ll be off and it'll approach getting below 50, but that's when our business logic will protect us. We'll go over and implement that shortly.

Safeguarding with thresholds

So what is that business logic?

Now we want to take a look to see what's that value that's stored in my availableRateLimit variable. After I've made those requests, what do I have left? If that's less than my rateLimitThreshold, the acceptable threshold that we’ve defined for ourselves, then we want to tell our applications, "Wait a second. Hold on. We don't have room to keep making these requests indefinitely."

"The goal is to build good software, so that we can prevent making any requests that we know are going to hit our rate limit."

We’re doing this because we want to avoid hitting a bunch of 429s, which might actually make Shopify think we're a bot. The goal is to build good software, so that we can prevent making any requests that we know are going to hit a 429.

In order to do so we’re going to write in the below clause:

This is saying that if the availableRateLimit is less than (<) the rateLimitThreshold, we’re writing a log to wait. We want our app to wait. To do that in Node.js, we’re going to implement a new promise, await new Promise. Within this promise, we’ll pass in a resolve parameter, and we’ll define what takes place within the promise. 

In this example, it's going to be a timeout setTimeout. For the timeout, we’ll pass in a function to define what happens once that time out is complete. In this case, we’re going to respond with resolve(‘Rate limit wait’). Then we’re going to insert another log: console.lot(‘Done waiting - continuing requests’).

The last thing you want to provide into your setTimeout (function) is an integer for the timeout length. The quantity is milliseconds, so we’re going to pass in 1,000, which means that we're waiting one second in total.

You might also like: API Rate Limits and working with GraphQL. 

This is our failsafe to ensure we never hit our rate limit! This conditional is now stating that if we ever have less than the acceptable 50 points available, instead of going through the entire loop again, it’s going to wait one second, and then make those requests. 

Before running that failsafe to see how it works in practice, I want you to change your query to fetch 249 products, because 250 would put your allowance of 1,000 points per query.

Alright, go ahead and run it! 

When you drop below 48, you should see a log that looks something like this: 

The above is a log saying we hit our threshold, had to wait one second, and now we’re done and continuing our requests. 

One thing to note is that when we are running these promises this way, there's a little bit of non-sequential output of the logs. This is due to our synchronous and asynchronous rawRequests.

Each rawRequest executes its then callback immediately once it completes. That being said, the end result of ensuring we're not making more requests when we're below our threshold holds true. If you scroll through your terminal, you’ll see a lot of output, but no actual errors, which is great.

Running the final query

One thing I want to make clear is that implementing logic within your application is only one example of how to ensure you're optimizing your rate limit without going over. It's definitely not the only solution. 

In a production app, you probably have a lot more available to you. 

1. You can implement a queuing model where you drop your request into a queue that shares resources between multiple threads. 
2. You could defer some execution and drop some ids into a database that you have a cron job or scheduled job for. Later iterating over each record and making those requests at a regular cadence (eg. once a day, an hour, etc).

The method outlined above should give you a sense of how we understand what our rate limit availability is, and what you can do with it, such as implementing a failsafe for when things get too close. 

Now, if we run this execution of this JS one more time, we see we have 50 rate limit points available. As we wait, our number increases again up to the hundreds, because every second we wait we get 50 more.

Each one of those lines is a product update, but I'm never concerned that it's going to hit my rate limit, it's always taken care of, and that's the goal. You want to ensure that you're optimizing and maximizing your APIs and not going over, while building a fast app that gets the job done for the merchants that you're developing for.

I hope you found this post educational and it helped you get a better understanding of what API rate limiting looks like, as well as the best practices for working with it.