Event-driven architectures with message brokers need careful attention to security best practices. Amazon MQ for RabbitMQ combined with AWS Lambda enables serverless event processing. However, implementing defense in depth and least privilege principles necessitates a clear understanding of networking requirements. This is particularly important when working with different subnet types and their impact on service connectivity.
This post explores the networking aspects of Lambda event source mapping for Amazon MQ for RabbitMQ. Learn how deployment options influence your networking setup and security posture to make informed architectural decisions. These networking concepts are essential for building secure, scalable solutions, regardless of your experience level with message brokers.
For clarity in this post, when we refer to “RabbitMQ”, we mean Amazon MQ for RabbitMQ.
Prerequisites
The following prerequisites are necessary to complete this post:
- An Amazon Web Services (AWS) account
- Basic understanding of AWS networking concepts
- Familiarity with Amazon MQ for RabbitMQ
- Basic knowledge of Lambda
Furthermore, to enable setup of the discussed architectures, this post is accompanied by a GitHub repository that uses AWS Cloud Development Kit (AWS CDK).
Repository prerequisites
The following prerequisites are necessary for the repository:
- AWS Command Line Interface (AWS CLI)
- NodeJS/NPM
- AWS CDK v2
Repository setup
Clone the https://github.com/aws-samples/sample-amazonmq-rabbitmq-lambda-esm repository. This repository contains all the necessary code and instructions to create relevant architectures using AWS CDK.
Install dependencies and build
Install the necessary NPM dependencies by running the following commands:
Amazon MQ for RabbitMQ networking deployment options
Public accessibility is the primary networking differentiator when deploying a RabbitMQ broker in AWS. Although the broker operates in the Amazon MQ service account, the networking configuration varies based on this choice.
Public broker
When you deploy a publicly accessible broker, Amazon MQ provisions all networking components in the service account. The service provides a DNS name that resolves to an IP address of the Network Load Balancer (NLB) in that account. This configuration doesn’t support security groups. All security measures must be implemented through the RabbitMQ broker’s authentication and authorization mechanisms. The following diagram shows this communication flow.
Figure-1 DNS resolution of a public Amazon MQ for RabbitMQ broker.
Private broker
A private broker routes networking through a Amazon Virtual Private Cloud (Amazon VPC) in your account. Amazon MQ uses AWS PrivateLink to provision VPC Endpoints, which serve as entry points for broker communication.
The following diagram shows how client applications communicate with RabbitMQ:
- The client application connects to Amazon Route 53 Resolver
- Route 53 Resolver resolves the DNS name to the VPC Endpoint’s IP address
- The client communicates with the broker through PrivateLink
- Security groups protect the VPC Endpoint’s Elastic Network Interfaces (ENIs)
Figure-2 DNS resolution of a private Amazon MQ for RabbitMQ broker.
A private broker deployment offers two networking options:
- Custom VPC configuration – Specify:
- Subnets for VPC Endpoint creation
- At least one security group to protect the VPC Endpoints
- Default VPC configuration – Leave VPC options blank to use:
- Default VPC
- Default security group
Amazon MQ for RabbitMQ Lambda event source mapping building blocks
RabbitMQ solutions offer two approaches for message processing:
- Create a custom client to read messages from broker queues
- Use Lambda functions with event source mapping (ESM) for automated message retrieval
The ESM is a Lambda service resource that reads the messages from the broker and invokes the Lambda function synchronously. In the remainder of this post, we refer to this Lambda function as listener.
ESM connectivity depends on the following:
- The listener’s AWS Identity and Access Management (IAM) Role for access permissions
- RabbitMQ broker networking components
For public brokers, ESM uses public connectivity. For private brokers, ESM:
- Assumes the listener’s IAM Role
- Creates ENIs in the same subnets as the broker’s VPC Endpoints
- Uses the same security groups that protect the VPC Endpoints
The listener’s IAM Role must include these Amazon Elastic Compute Cloud (Amazon EC2) permissions:
- CreateNetworkInterface
- DeleteNetworkInterface
- DescribeNetworkInterfaces
- DescribeSecurityGroups
- DescribeSubnets
- DescribeVpcs
To view ESM ENIs:
- Open the AWS Management Console
- Navigate to EC2 > Network Interfaces
- Look for ENIs with the following naming pattern:
AWS Lambda VPC ENI-armq-<ACCOUNT_ID>-<ESM_ID>-<remainder>where:
- ACCOUNT_ID – The AWS account number containing the ESM
- ESM_ID – The unique identifier of the ESM
The following image shows example ESM ENIs.
Figure-3 An example list of interfaces that Amazon MQ for RabbitMQ creates for private brokers.
Disabling or deleting the ESM removes the ESM components.
An enabled ESM needs connectivity to the following:
- AWS Security Token Service: For IAM role assumption
- AWS Secrets Manager: For RabbitMQ credentials
- RabbitMQ broker: For queue access
- AWS Lambda: For listener invocation
Because the ESM queue polling process follows these steps:
- Assumes the listener’s IAM Role
- Retrieves RabbitMQ credentials from Secrets Manager
- Establishes broker communication
- Invokes the listener when messages are present
You have two options to enable private broker connectivity to support the queue polling process:
- Deploy VPC endpoints in ESM subnets for:
- AWS Security Token Service (AWS STS)
- Secrets Manager
- Lambda
- Deploy NAT gateway in ESM subnets
ESM networking configuration options
The following sections detail ESM networking configurations for different deployment scenarios.
Option 1: Public broker
In this approach all network communication happens on the Amazon MQ service’s side. The ESM, when enabled, uses public connectivity.
To observe the architecture implemented in your account go to the cloned repository root location, make sure that you are signed in with AWS CLI and run the following:
Option 2: Private broker in a default VPC
Deploying a private RabbitMQ broker without specifying the VPC informs the Amazon MQ service to pick the default VPC for setting up the networking and then the public subnet(s) in that VPC. The default security group is used for securing the broker’s VPC Endpoints.
Creating the ESM provisions dedicated ENIs in the public subnets where the RabbitMQ broker’s VPC Endpoints reside with the default security group applied. The default security group allows itself for inbound traffic on all protocols and full port range, thus the ESM can route traffic through the VPC Endpoint.
Although the subnet is public with internet gateway access, the ESM ENIs operate in private address space, preventing direct communication with AWS services. To enable proper communication, create VPC Endpoints for AWS STS, Secrets Manager, and Lambda. These endpoints allow the ESM to communicate with AWS services through private IP addresses within your VPC. The following diagram shows the complete communication path from the ESM to the broker.
Figure-4 Networking configuration and request flow for a private broker provisioned in the default VPC.
To observe the architecture implemented in your account, go to the cloned repository root location, make sure that you are signed in with AWS CLI, and run the following
Option 3: Private broker in a Custom VPC with NAT
When deploying a private RabbitMQ broker in a custom VPC, specify either a single subnet for a standalone broker or multiple subnets for a cluster deployment. The deployment also needs a security group for the VPC Endpoint ENIs.
Configure the security group with a self-referencing inbound rule on the AMQP port. This configuration enables communication between the ESM and the RabbitMQ VPC Endpoints’ ENIs.
The following diagram shows how ESM resources communicate through networking components when deployed in a private subnet with NAT gateway. This architecture demonstrates the complete communication path from the ESM to the broker.
Figure-5 Networking configuration and request flow for a private broker provisioned in a private VPC subnet with NAT.
To observe the architecture implemented in your account, go to the cloned repository root location, make sure that you are signed in with AWS CLI, and run the following:
Option 4: Private broker in a Custom VPC with isolated subnets
This configuration builds upon the previous architecture but introduces isolated subnets. These subnets restrict all internet connectivity, permitting only internal VPC network traffic. Although the broker networking components mirror Option 3, the isolation introduces more considerations.
The security group still needs an open AMQP port for queue operations, but the subnet isolation prevents the ESM from directly accessing AWS services. To address this limitation, deploy VPC Endpoints for AWS STS, Secrets Manager, and Lambda within the isolated subnets. These endpoints create a private communication path for the ESM to interact with essential AWS services without needing internet access.
The following diagram shows the communication architecture for ESM resources deployed in isolated subnets. It demonstrates how VPC Endpoints enable secure communication between the ESM and AWS services while maintaining network isolation. This architecture makes sure that the ESM can fulfill its message processing responsibilities without compromising security through internet exposure.
Figure-6 Networking configuration and request flow for a private broker provisioned in an isolated VPC subnet.
To observe the architecture implemented in your account, go to the cloned repository root location, make sure that you are signed in with AWS CLI, and run the following:
Option 5: Private broker in a Custom VPC with public subnets
The final configuration places the broker in public subnets while maintaining the core deployment requirements from the previous options. Despite the public subnet placement, the ESM’s networking behavior presents an important consideration: ESM ENIs operate in private address space, preventing direct internet communication even with an internet gateway present.
This architecture necessitates VPC Endpoints for AWS service communication, similar to Option 2. Any attempts to route ESM traffic through the internet gateway fail because the ENIs operate in private address space. Understanding this limitation is crucial for proper deployment planning.
The following diagram shows the ESM communication architecture in public subnets. Despite the different subnet type, this configuration mirrors the isolated subnet approach in its use of VPC Endpoints. These endpoints enable the ESM to communicate with AWS STS, Secrets Manager, and Lambda services through private, secure connections within the VPC.
Figure-7 Networking configuration and request flow for a private broker provisioned in a public VPC subnet.
To observe the architecture implemented in your account, go to the cloned repository root location, make sure that you are signed in with AWS CLI, and run the following:
Cleaning up
To prevent unexpected AWS charges, remove resources you’ve created. The following AWS CDK command helps you safely remove all deployed resources:
Conclusion
This post explored the relationship between AWS Lambda event source mapping and RabbitMQ networking configurations. We examined various deployment scenarios, from public brokers to isolated subnets, each presenting unique considerations for secure and effective implementation.
Understanding these networking patterns enables you to make informed architectural decisions when deploying Amazon MQ for RabbitMQ with Lambda event source mapping. Whether choosing public accessibility or implementing private networking with VPC Endpoints, understanding the consequences of choosing specific networking configurations allows you to apply security best practices while meeting your application’s messaging needs. As you implement these patterns, consider your specific security requirements and operational needs to choose the most appropriate configuration for your use case.
Take the next step in optimizing your serverless messaging architecture. Dive in to the AWS documentation, experiment with the RabbitMQ and Lambda integration patterns discussed, and discover how these networking configurations can elevate the security and performance of your own applications. Start implementing these strategies today to build more robust, scalable solutions.