Retool Inventory Management System: Lambda Function Guide (P3)

What is Lambda Function?

We introduced the concept of Event-Driven Architecture (EDA) in the first post of this series. In an event-driven system, events are constantly being generated and received, and we need a way to handle those events properly. Often, this involves running specific logic based on the event.

In a serverless environment, this is where Function-as-a-Service (FaaS) comes in. Instead of managing entire servers, the service provider gives us just the function. This means we only focus on the function's logic and not on the infrastructure behind it.

AWS Lambda is the FaaS offering from Amazon Web Services. It allows us to write code, and AWS handles the rest, such as scaling, high availability, and fault tolerance. You still have some flexibility to adjust settings, and you only pay for the actual time your code is running.

How does Lambda Function Works?

In this blog, we’ll explore how AWS Lambda works while intertwining the restocking use case of RetoolerStock.

Function Permission

IAM Resource Policy: Who Can Trigger the Function?

An IAM resource policy governs who or which services can invoke the Lambda function. In simpler terms, it acts as a gatekeeper, defining which AWS services are allowed to trigger the function. This is particularly important when dealing with event-driven systems, like the one in RetoolerStock.

RetoolerStock Scenario: DynamoDB Triggers Lambda

In RetoolerStock, DynamoDB stores the inventory data, and DynamoDB Streams capture any changes made to the stock levels (e.g., when an item is sold or received). These changes need to trigger a Lambda function that automatically checks stock levels and decides whether a restocking workflow should be initiated.

For this trigger to happen, the Lambda function must have an IAM resource policy that allows DynamoDB Streams to invoke it. Without this permission, even though DynamoDB has updates, it won’t be able to "call" the Lambda function to process those updates.

Example Policy for DynamoDB Triggering Lambda:

The IAM resource policy might look something like this, where DynamoDB is granted permission to invoke the Lambda function:

‍

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "dynamodb.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:region:account-id:function:function-name"
    }
  ]
}

‍

This policy ensures that DynamoDB can trigger the Lambda function whenever stock levels change in the database, enabling real-time decision-making for restocking.

Other AWS Services Triggering Lambda in RetoolerStock

Besides DynamoDB Streams, other AWS services in RetoolerStock also trigger Lambda functions. For example:

• S3: When product images or files are uploaded to S3, a Lambda function may process the file and link it to the inventory data.
• EventBridge: For complex workflows, such as automatically scheduling restocking when certain conditions are met (e.g., a special sale is approaching), EventBridge triggers a Lambda function to handle these tasks asynchronously.
• Or in simplest scenario, that’s API Gateway.

In each of these cases, the Lambda function needs a resource policy that allows these services (S3, EventBridge) to invoke it. Each service must be explicitly granted permission to trigger the Lambda function, ensuring that only authorized services can interact with it.

IAM Execution Role: What Can the Lambda Function Do?

The IAM execution role determines what actions the Lambda function is allowed to perform when it runs. In the RetoolerStock application, Lambda functions need to interact with various AWS services to perform their tasks, such as reading data from DynamoDB, sending alerts via SNS.

RetoolerStock Scenario: Checking Stock Levels and Sending Alerts

Let’s walk through an example of how the IAM execution role comes into play in RetoolerStock.

1. DynamoDB Stream Triggers Lambda: A change in stock levels is captured by DynamoDB Streams, which triggers the Lambda function.
2. Lambda Checks Stock Levels: The Lambda function needs to access the DynamoDB table to read the updated inventory data. This requires the execution role to have permission to read from DynamoDB.
3. Low Stock Alert via SNS: If the stock level is below a certain threshold, the Lambda function will send a low-stock alert through SNS to notify the warehouse manager. For this, the execution role must have permission to publish messages to SNS.

Example IAM Execution Role:

Here’s an example of what the execution role might look like for this workflow:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:Scan"
      ],
      "Resource": "arn:aws:dynamodb:region:account-id:table/InventoryTable"
    },
    {
      "Effect": "Allow",
      "Action": "sns:Publish",
      "Resource": "arn:aws:sns:region:account-id:LowStockAlerts"
    }
  ]
}

Learn more about IAM policy and Trust policy.

Learn more about Least Privilege Principle.

Invocation Models

AWS Lambda can be triggered using different ways (invocation models) depending on the use case and the type of interaction we need.

Synchronous Invocation: API Gateway

Synchronous invocation is when a service or user waits for the Lambda function to complete and return a result before continuing. This model is ideal for real-time operations where immediate feedback is required.

RetoolerStock Use Case: Manually Checking and Updating Stock Levels

In RetoolerStock, warehouse managers may manually check and update stock levels through Retool dashboard. The Retool application has a resource to integrated with API Gateway - which basically, a REST resource, which serves as the entry point for user-initiated requests.

1. API Gateway Trigger: The request made is sent to API Gateway, which triggers the Lambda function responsible for updating the stock.
2. Lambda Execution: The Lambda function executes synchronously, retrieving the updated stock level from the user, checking for errors or inconsistencies, and making the appropriate changes to the DynamoDB inventory table.
3. Immediate Response: Once the function completes, it sends a response back to API Gateway, which forwards the result (e.g., confirmation of the stock update) to the user.

Asynchronous Invocation: S3, SNS, EventBridge

Asynchronous invocation allows Lambda functions to handle tasks in the background, where the initiating service does not wait for the function to complete. This is useful for tasks that don’t require an immediate response, like processing data or sending notifications.

RetoolerStock Use Case: Low-Stock Alerts and Restocking Workflow

In RetoolerStock, several tasks happen asynchronously to ensure efficient inventory management without blocking other processes:

• Low-Stock Alerts via SNS: When stock levels fall below a certain threshold, the system automatically sends an alert to warehouse staff using SNS. This alert notifies them that an item is running low and may need to be restocked.
• Restocking Workflow with EventBridge: When the system detects that an item is below the minimum stock level, EventBridge triggers a Lambda function that initiates a restocking workflow. This workflow might involve multiple steps, including creating a purchase order, notifying suppliers, and updating the inventory once new stock arrives.

In both of these examples, Lambda is invoked asynchronously, meaning the system doesn’t need to wait for the function to finish before moving on to the next task. Here’s a more detailed look at how this works:

1. Low-Stock Alert: When a product’s stock level drops below a defined threshold in DynamoDB, a stream of changes is captured by DynamoDB Streams. This event triggers a Lambda function that checks the stock level.
2. SNS Notification: If the stock level is low, the Lambda function sends an alert via SNS to the warehouse staff. Since this is an asynchronous operation, the SNS service immediately begins processing the alert, while the Lambda function moves on without waiting for the notification to be sent.
3. EventBridge Restocking Workflow: At the same time, EventBridge can also trigger another Lambda function to handle restocking. This function could send out purchase orders or update the inventory system, depending on the workflow configuration

Polling + Event Source Mapping: SQS and DynamoDB Streams

An event source mapping is a resource within AWS Lambda that reads items from specified event sources and invokes a Lambda function with batches of records.

In the Asynchronous Invocation model, the initiating service (such as S3, SNS, or EventBridge) sends an event directly to the Lambda function. Lambda immediately starts processing the event without waiting for the consumer to explicitly request it. This works well for background tasks where real-time processing isn’t critical. Once the event is sent, AWS takes care of executing the Lambda function, and the event doesn’t persist in any long-term stream after being handled.

By contrast, in the Polling Invocation model, Lambda doesn’t automatically receive events. Instead, events are placed into a stream or queue, such as DynamoDB Streams, SQS (Simple Queue Service), or Kinesis Streams. Lambda continuously polls this stream to check for new events. The key difference here is that the consumer (Lambda function) decides when to process the event by polling the stream, rather than receiving the event directly upon its creation.

• Streams Act as Buffers: In the polling model, streams like DynamoDB Streams or Kinesis Streams act as a buffer where events are stored temporarily. Consumers (Lambda functions) can then poll the stream to process those events at their own pace. This is useful in scenarios where multiple consumers are processing the same stream independently. For example, in RetoolerStock, you might have two separate Lambda functions consuming the same DynamoDB stream, but one Lambda might be used for real-time stock updates, while the other is used for analytics or auditing.
• Independence of Consumers: Each consumer has its own position in the stream. One Lambda function might be processing the most recent events in real-time, while another might be reading older events, depending on its needs or backlog. The events remain in the stream for a limited period (24 hours to 7 days for DynamoDB Streams or Kinesis), allowing consumers to access them even if they don’t process them immediately.

Lambda functions, invoked asynchronously in polling or async models, let you handle execution outcomes. You can set destinations based on success or failure. When the function succeeds, trigger actions like logging. When it fails, send the result to a dead-letter queue (DLQ), retry, or analyze it later. This setup helps manage errors and ensures smoother handling of failed tasks.

How to Write a Lambda Function

Writing a Lambda function is straightforward. Lambda functions can be authored in various supported languages, or you can use a custom runtime to write them in an unsupported language.

In a Lambda function, you need to define a handler method. This method receives an event—which serves as the main parameter, carrying data from other services (the structure of the event depends on the source)—and an optional context object that contains information about the current execution environment.

Each function should focus on doing one thing to improve reusability and composability for more complex use cases. Additionally, the function should only include what it needs to reduce cold start times.

What If We Need Reuse Code between Multiple Lambda Function or Using Custom Libraries/Dependencies?

AWS Lambda enables developers to efficiently reuse code and manage dependencies by leveraging Lambda Layers.

For example, in a Node.js application, we can package all the required dependencies into a layer. Here’s how it works:

1. Install Dependencies: First, install the necessary dependencies (e.g., node_modules folder) for your Node.js project.
2. Create a Layer: ZIP the node_modules folder and any other shared code you'd like to reuse across Lambda functions.
3. Upload the Layer: Use the AWS Management Console or CLI to create a new Lambda Layer from the ZIP file.
4. Attach the Layer: Attach this layer to any Lambda function that needs access to these dependencies.

This approach ensures that you can reuse common code or dependencies across multiple functions without needing to include them in every Lambda deployment package.

Read more about Node.js Lambda Function Layers here: https://docs.aws.amazon.com/lambda/latest/dg/nodejs-layers.html

Function Lifecycle - Optimization

Before diving into writing Lambda functions, it's essential to understand that with every invocation, AWS Lambda creates a new instance of the function - new execution environment. This allows for multiple functions to process concurrently. However, this also means that each invocation is stateless and must be treated as new.

Every Lambda function goes through three stages in its lifecycle:

1. Initialize: AWS Lambda sets up the environment, loads dependencies, and runs any code outside the handler function.
2. Invoke: AWS executes the function by passing in the event and context.
3. Shutdown: After the function runs, the environment stays alive briefly to handle any additional invocations before shutting down due to inactivity.

For each invocation, the Lambda function initializes, runs the code, and shuts down. Initially, this process may seem inefficient, and to some extent, it is. However, notice that Lambda waits a short time before shutting down the function. If another invocation occurs during this period, the existing instance is reused, skipping the initialization phase. This is known as a warm start, which results in lower latency since no initialization is required. In contrast, a cold start occurs when a new instance is created, including the initialization phase.

To optimize performance and reduce cold starts, you can:

• Minimize initialization time by reducing dependencies and keeping the code outside the handler function minimal.
• Use provisioned concurrency to keep the function "warm" and ready for immediate execution.

How to Configure a Lambda Function

• Memory: You can configure the memory for a Lambda function up to 10GB. AWS charges based on the amount of memory you allocate, not on how much is actually used. For example, if you allocate 10GB but only use 2GB, you’ll still be charged for the full 10GB. Charges are also based on how long the function runs.
• Timeout: A Lambda function can run for a maximum of 900 seconds (15 minutes). AWS charges based on the exact time in milliseconds that the function executes.

It's important to note that AWS only starts charging once your code begins to run, meaning charges apply from the beginning of the invoke phase, not during the initialization phase. Optimizing the initialization phase helps reduce latency, but does not impact billing.

To minimize costs, it's essential to balance the allocated memory and the function's timeout setting for optimal performance at the lowest acceptable cost.

• Concurrency

Concurrency determines how many instances of your Lambda function can run at the same time, which directly impacts its performance and ability to scale. When a function is invoked, Lambda creates an instance to handle the event. If another request comes in while the first is still running, Lambda spins up a new instance. The total number of instances running simultaneously is the function’s concurrency.

Types of Concurrency

1. Unreserved Concurrency: This is the default pool of concurrency available for all your functions. At least 100 instances are always unreserved, ensuring that functions without any specific allocation can still run.
2. Reserved Concurrency: Reserved concurrency allows you to allocate a fixed number of concurrent instances for a specific Lambda function. There are two primary use cases for this:
   • Guaranteed Resources: By reserving concurrency, you ensure that no other function can use these allocated instances. This is particularly useful for critical functions that require consistent performance and availability.
   • Cost Management: If a function is not critical or if downstream services have limited concurrency capabilities, you may want to cap its concurrency to save costs. In such cases, it doesn’t make sense to allocate excessive resources to the Lambda function.
3. Provisioned Concurrency: This pre-initializes (warm-up) a specific number of runtime environments so your function is ready to handle traffic instantly, reducing latency. You pay for the provisioned concurrency and the time it remains active. It's useful when you expect high traffic and need fast response times but can be scaled down to save costs when demand drops.

RetoolerStock Function Examples

‍

Read more about AWS CloudWatch

Learn more about AWS Lambda through 20 videos of ServerlessLand: https://serverlessland.com/content/service/lambda/guides/aws-lambda-fundamentals/what-is-aws-lambda

Learn more about: Tags, alias

Get in Touch

Ready to bring your dashboard vision to life? Contact Retoolers today, and let us help you create a powerful, intuitive dashboard that meets your exact requirements.