Skip to content

Error when secondary galera servers are stopped and .my.cnf is present #395

@JCallicoat

Description

@JCallicoat

During incremental upgrade, secondary galera servers (galera[1:]) are shutdown by the mariadb-slaves-shutdown.yml play. An error is then encountered in the galera_server galera_server_cluster_state.yml tasks on galera[1]:

TASK [galera_server : Fail if galera_cluster_name doesnt match provided value] *****************************************************************************************************************************
fatal: [infra02_galera_container-6e8c2b47]: FAILED! => {"changed": false, "msg": "The galera_cluster_name variable does not match what is set in mysql. Check your galera_cluster_name setting in your user_*.yml files in "/etc/openstack_deploy" and compare to the current value in "/etc/mysql/conf.d/cluster.cnf" on the host, and the "wsrep_cluster_name" value set in your running galera cluster. To ignore the cluster state set '-e galera_ignore_cluster_state=true'.\n"}

Adding a debug task shows this is because the mysql command is trying to connect to mysql through local socket, which fails because the server is shutdown from mariadb-slaves-shutdown.yml.

- name: Print cluster name
  debug:
    msg: "Cluster name is {{ _cluster_name }}"

TASK [galera_server : Print cluster name] ******************************************************************************************************************************************************************
ok: [infra02_galera_container-6e8c2b47] => {
"msg": "Cluster name is {'msg': 'non-zero return code', 'cmd': ['mysql', '--silent', '--skip-column-names', '--connect-timeout=10', '-e', 'SHOW VARIABLES LIKE "wsrep_cluster_name";'], 'stdout': '', 'stderr': "ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)", 'rc': 1, 'start': '2021-03-27 23:09:05.538996', 'end': '2021-03-27 23:09:05.549118', 'delta': '0:00:00.010122', 'changed': False, 'failed': False, 'stdout_lines': [], 'stderr_lines': ["ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)"], 'failed_when_result': False}"
}

Upon further investigation, a /root/.my.cnf file is present specifying to use localhost:

root@infra02-galera-container-6e8c2b47:~# cat .my.cnf
[client]
host = localhost
user = root
password = <redacted>

It looks like this file is dropped by rpc-maas maas-infra-galera.yml play.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions