RabbitMQ Cluster With Terraform On AWS

Konrad Rotkiewicz
31 July 2017 · 7 min read

Creating RabbitMQ cluster is tricky. It is easy to do manually and hard if you’d like to automate it.

TL;DR Use this Terraform configuration to create RabbitMQ cluster in less than 5 minutes.

The simplest cluster requires 2 nodes and a load balancer. In AWS we are going to use ELB as a load balancer and put nodes in Auto Scaling group, so that if a node goes down (or became unhealthy) it will be replaced by a new one.

Our setup will be:


Using Terraform we can create Launch Configuration, Auto Scaling group and ELB. This is our ELB configuration:

resource "aws_elb" "elb" {
name = "rabbit-elb"
listener {
instance_port = 5672
instance_protocol = "tcp"
lb_port = 5672
lb_protocol = "tcp"
listener {
instance_port = 15672
instance_protocol = "http"
lb_port = 80
lb_protocol = "http"
health_check {
interval = 30
unhealthy_threshold = 10
healthy_threshold = 2
timeout = 3
target = "TCP:5672"
subnets = ["${var.subnet_ids}"]
idle_timeout = 3600
internal = true
security_groups = ["${aws_security_group.rabbitmq_elb.id}"]
tags {
Name = "rabbitmq"
view raw elb.tf hosted with ❤ by GitHub

ELB is configured to replace a node if it becomes unhealthy after 5 minutes (there are 10 checks every 30 seconds). It listens on ports 5672 (AMQP) and 80 proxied to 15672 (HTTP interface).

What about nodes ? We use cloud-init to initialize a node and in there we configure RabbitMQ to run in Docker.

After RabbitMQ is running, the node has to join the cluster. To do that we call rabbitmqctl join_cluster <node> for each of the nodes.

To find out what are the other nodes in the cluster we prepared a bash script that query nodes in our Auto Scaling group:

#!/usr/bin/env bash
export AWS_DEFAULT_REGION='${region}'
DNSES=$(aws ec2 describe-instances --filters "Name=tag:aws:autoscaling:groupName,Values=rabbitmq" "Name=instance-state-name,Values=running" | jq ".Reservations[].Instances[].PrivateDnsName" | xargs)
for dns in $DNSES; do
hostname=($${dns//./ })
if [ "$hostname" != "$HOSTNAME" ]; then
HOSTNAMES+=( $hostname )
view raw 1.sh hosted with ❤ by GitHub

And then a script to join to these nodes:

#!/usr/bin/env sh
for run in {1..3}; do
sleep $[ ( $RANDOM % 10 ) + 1 ]s
rabbitmqctl stop_app
for hostname in $HOSTNAMES; do
rabbitmqctl join_cluster rabbit@$hostname
if [ $st -ne 0 ] && [ $st -ne 130 ]; then # 130 is "already joined"
NEW_HOSTNAMES+=( $hostname )
rabbitmqctl start_app
if [ $${#HOSTNAMES[@]} -eq 0 ]; then
exit 0
view raw 2.sh hosted with ❤ by GitHub

The tricky part here is that to join a cluster, you have to stop the node first. So there is a chance that other node could also be stopped as well. To mitigate this problem we set sleep for some random amount of seconds before stopping the server Also, in case of errors, we perform sane amount of retries.

Last thing that requires explanation is that querying nodes in given Auto Scalinggroup. To to be able to do so you need to associate _IAM_ role with autoscaling:DescribeAutoScalingInstances and ec2:DescribeInstancespolicies. This can be done by our Terraform configuration automatically:

resource "aws_iam_role" "role" {
name = "rabbitmq"
assume_role_policy = "${data.aws_iam_policy_document.policy_doc.json}"
resource "aws_iam_role_policy" "policy" {
name = "rabbitmq"
role = "${aws_iam_role.role.id}"
policy = <<EOF
"Version": "2012-10-17",
"Statement": [
"Effect": "Allow",
"Action": [
"Resource": [
resource "aws_iam_instance_profile" "profile" {
name = "rabbitmq"
role = "${aws_iam_role.role.name}"
view raw iam.tf hosted with ❤ by GitHub

Using this Terraform configuration we successfully deployed many RabbitMQ clusters with up to 4 nodes.

Leave a comment if you find this useful or a question in case of troubles. Cheers!

Share on
Related posts
Convox As A Solid Kubernetes Alternative

Convox As A Solid Kubernetes Alternative

Are you an AWS user considering moving to Kubernetes ? Think twice before going down that road as there is a better alternative – Convox. AWS ECS To be more precise, the alternative to Kubernetes on…
6 min read
Elastic Beanstalk Is Outdated, Stay Away From It.

Elastic Beanstalk Is Outdated, Stay Away From It.

Three months ago a new client came to us with a project hosted on Amazon Elastic Beanstalk (EB). Since then we’ve learned a lot about this technology and today I’d like to share some thoughts with you…
3 min read

Talk to us about your project

Get in touch with us and find out how we can help you develop your software
Contact us