
Lander Audian
October 7, 2024
•
15 min read
In Retool, dynamic resources open up powerful possibilities by allowing developers to seamlessly swap between different resources—such as REST APIs—based on logic or credentials. Imagine having multiple API credentials for different environments or users. With dynamic resources, you can easily switch between them without duplicating configurations.
In this post, we'll explore how dynamic resources work, use cases, and tips to maximize efficiency in your Retool apps.
Dynamic resources enable flexibility by allowing your Retool app to dynamically choose which resource to call, based on logic you define. These resources can be anything supported by Retool, such as databases, REST APIs, or even Google Sheets.
For instance, if you have multiple API credentials (for different clients or environments), you can define multiple REST API resources in Retool and switch between them as needed.
IMPORTANT: One thing to remember is that once a query is set to run dynamic resources, it will determine the current type of the resource - if it’s either a REST API, Retool DB, Google Sheets, or something else. Once the query is saved with that type of resource, you should ONLY switch with the resource IDs that has the same type!
To utilize dynamic resources in Retool, you first define multiple resources, such as REST APIs, databases, or other integrations. You then implement control logic or use variables within your app to determine which resource to call for a particular action. At runtime, Retool dynamically selects and uses the appropriate resource based on this logic, allowing you to switch between resources seamlessly without the need to manually adjust settings.
When developing in multiple environments (development, staging, production), switching between different API credentials is crucial. With dynamic resources, you can set up separate REST APIs for each environment and switch between them depending on your app’s state or user selection.
If you have multiple APIs providing the same data in different regions, dynamic resources in Retool allow you to route requests to the closest API, reducing latency. This helps distribute traffic evenly, improving performance and preventing any one server from being overloaded.
Pooling credentials involves rotating between multiple sets of API credentials to distribute the load of requests and avoid hitting the rate limits imposed by an API provider. This approach can be useful when working with services that limit the number of requests per account within a certain time frame. By dynamically switching between different credentials, you can ensure that your app maintains consistent performance even when faced with these constraints.
If you're working with APIs that impose strict rate limits, pooling credentials can help distribute your requests across multiple accounts or API keys.
Think of it like a busy supermarket with multiple cashiers, where each cashier can only handle a limited number of customers per hour. If you have a long line of customers waiting to check out, relying on just one cashier would quickly hit a bottleneck. However, by opening more cash registers (each representing a different cashier), you can spread the workload, allowing more customers to check out at the same time.
In the same way, when you're working with APIs that have strict rate limits, each API key is like a cashier with a limited capacity. By pooling credentials (like opening more cash registers), you can distribute your API requests across multiple accounts or API keys, ensuring that no single key hits its limit too quickly, and your "customers" (requests) keep flowing smoothly.
I guess you might say, isn’t that also like load balancing already? Well yeah, the GIF above also works for load balancing, but here’s some key differences of load balancing vs resource pooling:
Now, let’s see how we can pool multiple resources in Retool.
We can set a query to be running a resource dynamically by clicking the button with the ƒ symbol beside resource selection dropdown.
Before you start, you'll need to create the pool of resources you’ll be using, which is a separate API resource for each set of credentials you plan to use.
IMPORTANT: To overcome API rate limits, we basically need to have a set of resources which are REST APIs with same base URL endpoint. These REST API resources will be making requests to the endpoint, only having different credentials or keys configured for each, thus, a pool of resources.
While Retool still doesn’t have a simple way to view resource ID or the list of resources’ IDs, a way we could get them currently is by going inside any app, adding the query to use the resource we want to see the resource ID of and simply clicking the the button with the ƒ symbol (Basically, how we set them to use dynamic resource).
IMPORTANT: We must save the resource IDs somewhere, either in a variable, transformer, config variable or whichever we prefer to store it in to pull up the ids later on.
ℹ Utilizing config variables is a good way to add or remove resources without having to go in the app.
ℹ Setting a resource ID automatically detects the resource type upon saving the query. So make sure that we are saving the correct resource IDs, which are the REST API resources we’ve just created.
To dynamically switch between resources, we will use a variables to track the important states within the app we are building, such as current resource ID . Each time you make a request, the states will be updated.
var_resourceId
)We must use a state variable and not additionalScope
as it’ll not work for resource id switching.
var_currentIdx
, default = 0). This will be the index to reference the resource IDs from the list. (From the list of resource IDs we stored, from an earlier step.)Two ways in which we could set the rotation of the resource IDs are:
const nextIndex = (currentIndex + 1) % resources.length
const nextIndex = Math.floor(Math.random() * resources.length)
Here’s a simple pseudocode to preview of how the entire logic flow works:
// LIST OF RESOURCE IDS
// you can pull them from anywhere you stored it in from earlier step.
/*
when using config vars, you have to convert them to array as config vars
doesn't support arrays. (e.g) assuming they are strings separated by comma:
configVars.RESOURCE_LIST.split(',')
*/
const resources = ['resourceId1', 'resourceId2', 'resourceId3', ..., 'resourceId10']
// OR when using environment config variables, assuming the list is named RESOURCE_LIST
const resources = retoolContext.configVars.RESOURCE_LIST.split(',')
Written in a JS query in Retool
var_resourceId
and var_currentIdx
// CURRENT RESOURCE ID
const currentIndex = var_currentIdx.value //0
await var_resourceId.setValue(resources[currentIndex]) //resourceId1
// PREPARING NEXT RESOURCE ID TO USE
// 1st method - Moving on to the next resource ID in the list every time.
const nextIndex = (currentIndex + 1) % resources.length //1
//OR
// 2nd method - Getting a random resource ID from the list every time.
const nextIndex = Math.floor(Math.random() * resources.length) //random
// ASSIGN NEXT INDEX TO BE USED FOR NEXT RUN
await var_currentIdx.setValue(nextIndex) //1
//... Finally, run the dynamic query in an event handler.
Written in a JS query in Retool
Finally, with the value of var_resourceId
pointing to the correct resource ID we can run the dynamic query in the event handler OR, if we want to run the query within the code block above instead of in an event handler, we can use .trigger()
at the end but we’ll need to have the following setting checked for the setting of variable values to take effect first:
Toggle to keep the variables we used .setValue()
with, to use the new value after that line in the JS query. (Can be found in “Advanced” tab)
Here’s a preview of the code above adjusted for that:
// CURRENT RESOURCE ID
const currentIndex = var_currentIdx.value //0
await var_resourceId.setValue(resources[currentIndex]) //resourceId1
// RUN THE QUERY HERE
// ...
// PREPARING NEXT RESOURCE ID TO USE
// 1st method - Moving on to the next resource ID in the list every time.
const nextIndex = (currentIndex + 1) % resources.length //1
//OR
// 2nd method - Getting a random resource ID from the list every time.
const nextIndex = Math.floor(Math.random() * resources.length) //random
// ASSIGN NEXT INDEX TO BE USED FOR NEXT RUN
await var_currentIdx.setValue(nextIndex) //1
Written in a JS query in Retool
Here’s a shorter one that works using random method (with variable sync setting enabled):
In this version we don’t even need a variable for the current index as we’ll be using a random index each time, hence no need to save current or next index.
// CURRENT RESOURCE ID
const randomIndex = Math.floor(Math.random() * resources.length)
await var_resourceId.setValue(resources[randomIndex]) // random resourceId
// RUN THE QUERY HERE
// ...
Written in a JS query in Retool
That’s some of the few ways for the flow or cycle of pooling REST API resources to be used on each dynamic query run.
There are certainly more ways to pool resources in Retool other than the one above, it’s just a matter of one’s creativity in regards to implementation as Retool allows limitless methods or ways of implementing something.
This adaptability not only streamlines workflows but also enhances the application's ability to handle various scenarios efficiently. With the ability to dynamically switch between different resources based on user needs or system conditions, applications become more resilient. As developers increasingly embrace these dynamic capabilities, they can build solutions that ultimately leads to a better user experience and improved application reliability.
You’ve reached the end of this blog. I hope this was helpful and has given you some useful insights!
Thanks for sticking around! Till next time!
Ready to bring your dashboard vision to life? Contact Retoolers today, and let us help you create a powerful, intuitive dashboard that meets your exact requirements.