The Shopify API is a powerful resource that allows you to add additional functionality to any Shopify store. As a store grows in size and complexity, it is important to ensure that you use the Shopify API wisely and effectively.
In this article, I’ll walk you through seven critical tips to maximize your use of the Shopify API. These tips should help your API jobs run faster and more efficiently, so you can make the most out of using the Shopify API.
You might also like: You Can Now Build a Complete Ecommerce Store On Any Website or Mobile App
1. Create an API usage plan
The first step in maximizing your usage requires that you do some upfront planning on how you will interact with the APIs. That means creating an API usage plan. By mapping out your intended usage upfront, you'll be better suited to identify issues and streamline your efforts.
Your API plan should include:
- What jobs need to be done?
- Product updates
- Inventory updates
- Order Fulfillments
- How many API calls (roughly) will it take on average for each of these jobs?
- How often do you need to run each job type?
The Shopify API is built for extracting data quickly. However, for the most part, updates require an API call for each item being updated.
For example, based on a 1,000 product store:
- Getting all 1,000 products will take 4 API calls, at 250 each
- Updating all 1,000 products will take 1,000 API calls since you need to do them one at a time
2. Schedule jobs concurrently
To help manage your API usage, you will need some sort of job scheduler that is capable of:
- Running jobs in the background
- Running multiple jobs at once
- (Optionally) Retrying jobs that failed
Running jobs concurrently will reduce the total time it takes to complete your jobs as opposed to running them as a series. The Shopify API call limit only allows for two requests per second. However, thanks to its leaky bucket formula, this limit allows bursts up to 40 API calls.
If you are using Ruby, I personally use and recommend Sidekiq as a job scheduler.
3. Synchronize requests around a shared API call budget
Each request to the Shopify API returns the remaining call limit available as a header in the response:
When you run multiple background jobs concurrently, each of those jobs will consume one of your API calls.
When running jobs concurrently, it is not sufficient to only check the header for the remaining call limit, since this limit is being consumed by many workers at the same time.
For example, given this scenario:
If all three API calls track their own Call Limit, they would think one API call was still available. All three would attempt to sent the request, but two of them would receive the dreaded 429 - Too Many Requests error.
The less-than-ideal approach would be to just run each job until it hits the 429 - Too Many Requests error. Then you would sleep for some arbitrary amount of time and try again.
A better approach is to use a shared counter to track the remaining calls. This prevents you from wasting time or resources by sending requests to Shopify only to discover there are no API calls available. To implement this approach, each Shopify API call should do the following:
- Check the shared remaining budget to make sure there is an API call available
- Decrement the shared remaining budget
- When the Shopify response is received, set the shared remaining budget based on what Shopify returns
Here is a Ruby example which uses a shared credits_remaining, protected with a Mutex so only one thread can modify it at a time:
To test this functionality, we can spin up 42 threads to hit our limit quickly:
This results in the following output:
It is worth pointing out that this technique is not perfect. Shopify responses may be processed out of order as you can see in the sample output. This would give you an incorrect current total api call limit.
As a result, plan on leaving a bit of a buffer (in the example I used five as a buffer) and adding support to retry failed API calls.
Note: If you are not using Ruby or Sidekiq, you could using something like Redis to track your shared budget counter.
4. Schedule your jobs efficiently
You can maximize your API call budget by using a few simple scheduling techniques:
- Start jobs at different time blocks
- Group smaller / faster jobs together
- Keep larger jobs / slower jobs separate to allow them the full API limit
Scheduling your jobs more efficiently will allow you to spread your workload more evenly throughout the day. Each group of jobs should complete before the next one finishes, or you may end up growing your job queue faster than you can finish it.
You might also like: App Development: Working Together for the Greater Good
5. Only request only what you need
When requesting data from Shopify’s APIs, it is always a good idea to only ask for the specific elements you need. Most API list views have a fields option you can use to limit the information sent back.
For example, if you want to get a list of Product IDs and titles, you can request just those values like so:
This has two main benefits:
- Less data needs to be sent, so the payload should be faster to receive
- Less work needs to be performed by Shopify to construct the data
In particular, the less work for Shopify is probably the biggest benefit. If Shopify can build the response faster, that should reduce the overall clock time it takes to run each request. This should result in a net increase in your effective API budget.
6. Only update things that changed
Similar to only requesting what you need, you should only send Shopify the fields that have changed.
For example, if you need to update the inventory of variants, you can send just the variant id and inventory_quantity data like so:
Also, notice you can batch update variant data for products, which will also greatly reduce the total API calls to get this job done.
Sending less data will make your API calls return much faster, since Shopify will not need to update many records in it’s database or perform additional processing.
Note: In particular, sending image urls every time will cause a lot of extra work for Shopify to reprocess the same images.
7. Beware of storing data in too many places
As mentioned previously, Shopify streamlines the retrieval of data, but updates require a call for each item being updated. As a result, if you are not careful, you may need to make a lot of extra API calls to update each piece of data.
The most common place this may occur is through the use of metafields. Metafields allow you to store additional data at many places throughout the Shopify API including: Products, Variants, Orders, etc.
To highlight this issue, let’s assume you have a store with 1,000 products, each with five variants. Let’s say you want to store two additional attributes with each variant: Width and Height.
If you add these as variant metafields, this will require:
- 5,000 API calls to update the Width metafield for each variant.
- 5,000 API calls to update the Height metafield for each variant.
However, another approach I’ve been using is to store these attributes as a JSON Hash inside a product metafield like so:
The key for the Hash in this example is the variant SKU. This reduces the number of API calls needed to update these values from 10,000 down to 1,000. I’d say that is an impressive reduction.
To obtain values for a specific SKU, you can look it up like so:
Note: At the time of this writing, you can create variant metafields in bulk if you send them as part of a Product update. However, you can not modify existing variant metafields in bulk.
Start maximizing your Shopify API usage
This article covered seven critical tips for maximizing your use of the Shopify API. These tips included:
- Create an API usage plan
- Schedule jobs concurrently
- Synchronize requests around a shared API call budget
- Schedule your jobs efficiently
- Only request only what you need
- Only update things that changed
- Beware of storing data in too many places
If you follow these as a guideline, you should be able to stretch your Shopify API budget a little further and simplify your overall API usage.
You can access the full code used to create these examples on Github.
What are your favourite tips for using Shopify API? Share them in the comments below.
You might also like: Getting Your App in the Shopify App Store
