search string side menu

ELMA365 On-Premises > Prepare infrastructure > Databases > High availability infrastructure / RabbitMQ cluster

RabbitMQ cluster

To work with the ELMA365 application, RabbitMQ version 3.9.15 or higher is required. The article describes the installation of RabbitMQ 3.12.0 and Erlang 25.3.2.2-1 for Ubuntu Linux 20.04 and 22.04. You can also refer to the guide in the official RabbitMQ documentation

Installation consists of six steps:

  1. Prepare nodes (servers).
  2. Install RabbitMQ.
  3. Prepare RabbitMQ cluster.
  4. Configure RabbitMQ.
  5. Configure HAproxy rabbitmq block).
  6. Connect to RabbitMQ.

Step 1: Prepare nodes (servers)

начало внимание

The minimum number of servers to organize a cluster is three.

конец внимание

  1. Create three nodes (servers) with sequentially numbered host names:
  • rabbitmq-server1.your_domain;
  • rabbitmq-server2.your_domain;
  • rabbitmq-server3.your_domain.
  1. Create the necessary host name mappings in DNS. If this is not possible, add the required entries to /etc/hosts.

Step 2: Install RabbitMQ

  1. Install the necessary packages:

sudo apt-get install curl gnupg apt-transport-https -y

  1. Import all necessary keys:

curl -1sLf "https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA" | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null
curl -1sLf https://ppa1.novemberain.com/gpg.E495BB49CC4BBE5B.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg > /dev/null
curl -1sLf https://ppa1.novemberain.com/gpg.9F4587F226208342.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/rabbitmq.9F4587F226208342.gpg > /dev/null

  1. Add RabbitMQ repositories:

sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
deb [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-erlang/deb/ubuntu $(lsb_release -cs) main
deb-src [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-erlang/deb/ubuntu $(lsb_release -cs) main
 
deb [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-server/deb/ubuntu $(lsb_release -cs) main
deb-src [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-server/deb/ubuntu $(lsb_release -cs) main
EOF

  1. Update the package cache:

sudo apt-get update -y

  1. Install Erlang packages:

sudo apt-get install -y erlang-base=1:25.3.2.2-1 erlang-asn1=1:25.3.2.2-1 erlang-crypto=1:25.3.2.2-1 erlang-eldap=1:25.3.2.2-1 erlang-ftp=1:25.3.2.2-1 erlang-inets=1:25.3.2.2-1 erlang-mnesia=1:25.3.2.2-1 erlang-os-mon=1:25.3.2.2-1 erlang-parsetools=1:25.3.2.2-1 erlang-public-key=1:25.3.2.2-1 erlang-runtime-tools=1:25.3.2.2-1 erlang-snmp=1:25.3.2.2-1 erlang-ssl=1:25.3.2.2-1 erlang-syntax-tools=1:25.3.2.2-1 erlang-tftp=1:25.3.2.2-1 erlang-tools=1:25.3.2.2-1 erlang-xmerl=1:25.3.2.2-1

  1. Install rabbitmq-server and its dependencies:

sudo apt-get install rabbitmq-server=3.12.0-1 -y --fix-missing

  1. Start rabbitmq-server:

sudo systemctl enable --now rabbitmq-server

Step 3: Prepare RabbitMQ cluster

  1. On each rabbitmq node, create the file /etc/rabbitmq/rabbitmq-env.conf and add the environment variables RABBITMQ_NODENAME and RABBITMQ_USE_LONGNAME.

Example content for rabbitmq-server1.your_domain:

RABBITMQ_NODENAME=rabbit@rabbitmq-server1.your_domain
RABBITMQ_USE_LONGNAME=true

Example content for rabbitmq-server2.your_domain:

RABBITMQ_NODENAME=rabbit@rabbitmq-server2.your_domain
RABBITMQ_USE_LONGNAME=true

Example content for rabbitmq-server3.your_domain:

RABBITMQ_NODENAME=rabbit@rabbitmq-server3.your_domain
RABBITMQ_USE_LONGNAME=true

  1. Copy Cookie /var/lib/rabbitmq/.erlang.cookie from the first node rabbitmq-server1.your_domain to all the other nodes in the cluster.

Начало внимание

For the RabbitMQ cluster to work, all nodes participating in the cluster must have the same content in the /var/lib/rabbitmq/.erlang.cookie file.

Конец внимание

  1. Restart the RabbitMQ service on each node:

sudo systemctl restart rabbitmq-server

  1. Stop the application on nodes rabbitmq-server2.your_domain and rabbitmq-server3.your_domain for subsequent joining to the cluster:

sudo rabbitmqctl stop_app

  1. Reset rabbitmq on nodes rabbitmq-server2.your_domain and rabbitmq-server3.your_domain:

sudo rabbitmqctl reset

  1. Join nodes rabbitmq-server2.your_domain and rabbitmq-server3.your_domain to the cluster:

sudo rabbitmqctl join_cluster rabbit@rabbitmq-server1.your_domain

  1. Start the application on nodes rabbitmq-server2.your_domain and rabbitmq-server3.your_domain:

sudo rabbitmqctl start_app

  1. Check the cluster status:

sudo rabbitmqctl cluster_status

Step 4: Configure RabbitMQ

начало внимание

For the password, the following characters are allowed:

  • Uppercase Latin letters: A to Z
  • Lowercase Latin letters: a to z
  • Digits: 0 to 9
  • Symbols: -_

Reserved (invalid) symbols:

! * ' ( ) ; : @ & = + $ , / ? % # [ ]

конец внимание

  1. On each node, enable the necessary plugins:

sudo rabbitmq-plugins enable \
rabbitmq_management

  1. Create vhost nd give the user access to it, executing commands on the first node rabbitmq-server1.your_domain:

sudo rabbitmqctl add_vhost elma365vhost
sudo rabbitmqctl add_user elma365user SecretPassword
sudo rabbitmqctl set_permissions -p elma365vhost elma365user ".*" ".*" ".*"
sudo rabbitmqctl set_user_tags elma365user administrator

  1. Create a policy that allows mirroring queues for all nodes in the cluster, executing the command on the first node rabbitmq-server1.your_domain:

sudo rabbitmqctl set_policy -p 'elma365vhost' MirrorAllQueues ".*" '{"ha-mode":"all"}'

To view the configured policies, use the command:

sudo rabbitmqctl list_policies --vhost elma365vhost

Enabling TLS/SSL in RabbitMQ

To enable TLS/SSL support in RabbitMQ, you need to edit the configuration file /etc/rabbitmq/rabbitmq.conf:

  1. Add SSL configuration to the rabbit (port 5671) and management (port 15671) blocks.
  1. In the ssl_options.certfile and management.ssl.certfile parameters, specify the path to the server certificate file.
  1. In the ssl_options.keyfile и management.ssl.keyfile parameters, specify the path to the private key file.
  1. In the ssl_options.cacertfile и management.ssl.cacertfile parameters, specify the path to the root CA file.

listeners.tcp = none
listeners.ssl.default = 5671
ssl_options.cacertfile = /path/to/ca_certificate.pem
ssl_options.certfile = /path/to/server_certificate.pem
ssl_options.keyfile = /path/to/server_key.pem
ssl_options.verify = verify_peer
ssl_options.fail_if_no_peer_cert = true
 
management.ssl.port = 15671
management.ssl.cacertfile = /path/to/ca_certificate.pem
management.ssl.certfile = /path/to/server_certificate.pem
management.ssl.keyfile = /path/to/server_key.pem

  1. Restart the RabbitMQ service:

sudo systemctl restart rabbitmq-server

For more details on configuring TLS/SSL in RabbitMQ, refer to the official RabbitMQ documentation.
 
For enabling TLS/SSL between RabbitMQ nodes "Securing Cluster (Inter-node)," you need to:

  1. Combine the public and private keys of the node into one file, for example, in combined_keys.pem:

cat server_certificate.pem server_key.pem > combined_keys.pem

  1. Extract the path to the Erlang TLS library and export it to the environment variable ERL_SSL_PATH в файл /etc/rabbitmq/rabbitmq-env.conf:

erl -noinput -eval 'io:format("ERL_SSL_PATH=~s~n", [filename:dirname(code:which(inet_tls_dist))])' -s init stop > /tmp/ssl-path.txt
cat /tmp/ssl-path.txt /etc/rabbitmq/rabbitmq-env.conf > /tmp/new-rabbitmq-env.conf
mv -f /tmp/new-rabbitmq-env.conf /etc/rabbitmq/rabbitmq-env.conf

  1. In the /etc/rabbitmq/rabbitmq-env.conf file, add parameters for the environment variables SERVER_ADDITIONAL_ERL_ARGS and RABBITMQ_CTL_ERL_ARGS:

SERVER_ADDITIONAL_ERL_ARGS="-pa $ERL_SSL_PATH \
  -proto_dist inet_tls \
  -ssl_dist_opt server_certfile /path/to/combined_keys.pem \
  -ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"
RABBITMQ_CTL_ERL_ARGS="-pa $ERL_SSL_PATH \
  -proto_dist inet_tls \
  -ssl_dist_opt server_certfile /path/to/combined_keys.pem \
  -ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"

  1. Restart the RabbitMQ service on each node:

sudo systemctl restart rabbitmq-server

For more details on configuring TLS/SSL between RabbitMQ nodes, refer to the official RabbitMQ documentation.

Step 5: Configure HAProxy (rabbitmq block)

Load balancing between RabbitMQ cluster nodes will be performed using HAProxy. Configure it according to Configure HAProxy for RabbitMQ.

Step 6: Connect to RabbitMQ

Connection string to connect to the RabbitMQ cluster (via HAProxy):

amqp://elma365user:SecretPassword@haproxy-server.your_domain:5672/elma365vhost

Connection string to connect to the RabbitMQ cluster with TLS/SSL (via HAProxy with TLS/SSL):

amqps://elma365user:SecretPassword@haproxy-server.your_domain:5671/elma365vhost

configure-redis.html minio-cluster.html

Found a typo? Highlight the text, press ctrl + enter and notify us