rabbitmq: cluster on docker swarm can not be assembled

rabbitmq cluster on docker swarm can not be assembled

Symptoms

oqpqhlailzzn   myproject_rabbitmq-2.1  rabbitmq:3.7-management-alpine   ip-10-103-32-185.us-east-1.compute.internal   Shutdown            Failed 22 seconds ago        "task: non-zero exit (69)"   
ngo5qnu2n4lk   myproject_rabbitmq-3.1  rabbitmq:3.7-management-alpine   ip-10-103-47-234.us-east-1.compute.internal   Shutdown            Failed 22 seconds ago        "task: non-zero exit (69)"

Reason 1: different values from previous installation

Check values in files on master and slave nodes

# on volume
/data/rabbitmq/.erlang.cookie
# in container
/var/lib/rabbitmq/.erlang.cookie

Solution: remove .erlang.cookie at all nodes, deploy stack again

Reason 2: hostname of master not equal value of master hostname in script on slave nodes

# in docker-compose.yml was:
services:
  ...
  rabbitmq-0:
  ...
    hostname: "{{.Node.Hostname}}-{{.Service.Name}}-{{.Task.Slot}}"

# on master node container
bash-5.0# hostname
myproject-us-dev-rabbit-0-myproject_rabbitmq-0-1

# script on slave nodes
...
rabbitmqctl join_cluster rabbit@rabbitmq-0
....

Solution: set correct name of container in docker-compose.yml