T O P

[资源分享]     高可用 | Xenon 实现 MySQL 高可用架构 常用操作篇

  • By - 楼主

  • 2021-09-03 18:00:35
  • 原创:知数堂

    上一篇文章,我们详细介绍了 Xenon 实现 MySQL 高可用架构的部署过程。接下来本篇将介绍 Xenon 的常用操作,帮助大家在完成环境搭建之后,能把 Xenon 熟练的用起来,以更好的对 MySQL 高可用架构进行管理。

    | 1 启动集群

    1.1 启动 MySQL

    # db1:
    [mysql@db1 ~]$ nohup mysqld_safe --defaults-file=/etc/my.cnf &
    [1] 5526
    [mysql@db1 ~]$ nohup: ignoring input and appending output to ‘nohup.out’
     
    # db2:
    [mysql@db2 ~]$ nohup mysqld_safe --defaults-file=/etc/my.cnf &
    [1] 3637
    [mysql@db2 ~]$ nohup: ignoring input and appending output to 'nohup.out'
    mysql –uroot –p    # 开启增强半同步
    set global  rpl_semi_sync_slave_enabled=1;
    stop  slave io_thread;
    start slave  io_thread;
    

    db3 的启动步骤同 db2。

    1.2 启动 Xenon

    db1:
    [mysql@db1 ~]$ cd /usr/local/xenon/bin/
    [mysql@db1 bin]$ ./xenon -c /etc/xenon/xenon.json > /etc/xenon/log/xenon.log 2>&1 &
    

    db2、db3 的启动方式同 db1。

    注意:当 MySQL 未启动时,启动 Xenon 会自动启动 MySQL。

    1.3 检查半同步复制

    启动时,需要检查半同步复制是否开启。

    (product)root@localhost [(none)]> show variables like '%semi%';
    +-------------------------------------------+------------+
    | Variable_name                             | Value      |
    +-------------------------------------------+------------+
    | rpl_semi_sync_master_enabled              | ON         |
    | rpl_semi_sync_master_timeout              | 10000      |
    | rpl_semi_sync_master_trace_level          | 32         |
    | rpl_semi_sync_master_wait_for_slave_count | 1          |
    | rpl_semi_sync_master_wait_no_slave        | ON         |
    | rpl_semi_sync_master_wait_point           | AFTER_SYNC |
    | rpl_semi_sync_slave_enabled               | OFF        |
    | rpl_semi_sync_slave_trace_level           | 32         |
    ……
    

    主要参数:主节点rpl_semi_sync_master_enabled 的状态为 ON,从节点 rpl_semi_sync_slave_enabled 的状态为 ON,表示半同步复制正常开启。

    | 2 帮助命令

    通过 xenoncli 的帮助命令查看可选的命令。

    [mysql@db1 bin]$ ./xenoncli  help
    A simple command line client for xenon
     
    Usage:
      xenoncli [command]
    
    Available Commands:
      cluster     cluster related commands
      init        init the xenon config file
      mysql       mysql related commands
      perf        perf related commands
      raft        raft related commands
      version     Print the version number of xenon client
      xenon       xenon related commands
     
    Use "xenoncli [command] --help" for more information about a command.
    

    举例说明,查看 cluster 的操作命令帮助。

    [mysql@db1 bin]$  ./xenoncli cluster -h
    cluster related commands
    Usage:
      xenoncli cluster [command]
     
    Available Commands:
      add         add peers to leader(if there is no leader, add to local)
      gtid        show cluster gtid status
      log         merge cluster xenon.log from logdir
      mysql       show cluster mysql status
      raft        show cluster raft status
      remove      remove peers from leader(if there is no leader, remove from local)
      status      show cluster status
      xenon       show cluster xenon status
     
    Use "xenoncli cluster [command] --help" for more information about a command.
    

    | 3 查看集群

    3.1 查看状态

    xenoncli cluster status
    
    xenoncli cluster mysql
    
    xenoncli cluster xenon
    

    file

    file

    3.2 检查集群一致状态

    xenoncli cluster raft
    

    file

    3.3 检查集群 GTID 状态

    xenoncli cluster gtid
    

    file

    | 4 关闭集群

    xenoncli mysql stopmonitor
    xenoncli mysql shutdown
    pkill xenon
    

    | 5 添加/删除节点

    5.1 添加节点

    xenoncli cluster add 192.168.0.2:8801,192.168.0.3:8801,192.168.0.5:8801
    

    Xenon 允许添加已经添加过的节点。如果节点已经在集群里面,将直接被忽视,不做任何操作。

    5.2 删除节点

    xenoncli cluster  remove 192.168.0.2:8801,192.168.0.3:8801,192.168.0.5:8801
    

    | 6 备份 MySQL 到本地

    [root@db1 backup]# xenoncli mysql backup
    Usage:
      xenoncli mysql backup --to=backupdir [flags]
    [mysql@db1 bin]$ ./xenoncli mysql backup --to=/backup/data
     2020/02/10 04:28:43.924306       [WARNING]     rebuildme.found.best.slave[10.10.10.11:8801].leader[10.10.10.10:8801]
     2020/02/10 04:28:43.924576       [WARNING]     S1-->found.the.best.backup.host[10.10.10.11:8801]....
     2020/02/10 04:28:43.945828       [WARNING]     S2-->rm.and.mkdir.backupdir[/backup/data]
     2020/02/10 04:28:43.945845       [WARNING]     S3-->xtrabackup.begin....
     2020/02/10 04:28:43.946692       [WARNING]     rebuildme.backup.req[&{From: BackupDir:/backup/data SSHHost:10.10.10.10 SSHUser:mysql SSHPasswd:123456 SSHPort:22 IOPSLimits:100000 XtrabackupBinDir:/usr/bin}].from[10.10.10.11:8801]
     2020/02/10 04:29:06.360412       [WARNING]     S3-->xtrabackup.end....
     2020/02/10 04:29:06.360444       [WARNING]     S4-->apply-log.begin....
     2020/02/10 04:29:11.010835       [WARNING]     S4-->apply-log.end....
     2020/02/10 04:29:11.011275       [WARNING]     completed OK!
     2020/02/10 04:29:11.011300       [WARNING]     backup.all.done....
    

    因 xenoncli 会尝试先删除 backupdir 再 mkdir,故不建议使用 /backup 根目录路径。若使用这种路径,rm 可以成功,mkdir 会失败。

    | 7 Rebuild 重建

    选择最佳状态的从节点,进行 XtraBackup 流式重建,from 参数可指定重建源节点。

    • 作用:重建当前节点 MySQL
    • 场景:节点坏掉需要快速重建
    // 在要做重建的节点执行: 
    xenoncli mysql rebuildme
    
    // 如果想以指定节点为准重建,执行: 
    xenoncli mysql -h
    
    // 基于指定节点复制重建当前 MySQL 节点
    [mysql@db1 bin]$ ./xenoncli mysql rebuildme --from=10.10.10.11:8801
     2020/02/11 01:19:40.907645       [WARNING]     =====prepare.to.rebuildme=====
                            IMPORTANT: Please check that the backup run completes successfully.
                                       At the end of a successful backup run innobackupex
                                       prints "completed OK!".
     
     2020/02/11 01:19:40.908418       [WARNING]     S1-->check.raft.leader
     2020/02/11 01:19:40.911586       [WARNING]     S2-->prepare.rebuild.from[10.10.10.11:8801]....
     2020/02/11 01:19:40.913050       [WARNING]     S3-->check.bestone[10.10.10.11:8801].is.OK....
     2020/02/11 01:19:40.913066       [WARNING]     S4-->set.learner
     2020/02/11 01:19:40.915499       [WARNING]     S5-->stop.monitor
     2020/02/11 01:19:40.916758       [WARNING]     S6-->kill.mysql
     2020/02/11 01:19:40.947717       [WARNING]     S7-->check.bestone[10.10.10.11:8801].is.OK....
     2020/02/11 01:19:40.950867       [WARNING]     S8-->rm.datadir[/backup/data]
     2020/02/11 01:19:40.950879       [WARNING]     S9-->xtrabackup.begin....
     2020/02/11 01:19:40.951632       [WARNING]     rebuildme.backup.req[&{From: BackupDir:/backup/data SSHHost:10.10.10.10 SSHUser:mysql SSHPasswd:123456 SSHPort:22 IOPSLimits:100000 XtrabackupBinDir:/usr/bin}].from[10.10.10.11:8801]
     2020/02/11 01:19:51.200092       [WARNING]     S9-->xtrabackup.end....
     2020/02/11 01:19:51.200116       [WARNING]     S10-->apply-log.begin....
     2020/02/11 01:19:56.666199       [WARNING]     S10-->apply-log.end....
     2020/02/11 01:19:56.666209       [WARNING]     S11-->start.mysql.begin...
     2020/02/11 01:19:56.666757       [WARNING]     S11-->start.mysql.end...
     2020/02/11 01:19:56.666766       [WARNING]     S12-->wait.mysqld.running.begin....
     2020/02/11 01:19:59.680987       [WARNING]     wait.mysqld.running...
     2020/02/11 01:19:59.709382       [WARNING]     S12-->wait.mysqld.running.end....
     2020/02/11 01:19:59.709395       [WARNING]     S13-->wait.mysql.working.begin....
     2020/02/11 01:19:59.710943       [WARNING]     S13-->wait.mysql.working.end....
     2020/02/11 01:19:59.710955       [WARNING]     S14-->stop.and.reset.slave.begin....
     2020/02/11 01:19:59.731108       [WARNING]     S14-->stop.and.reset.slave.end....
     2020/02/11 01:19:59.731121       [WARNING]     S15-->reset.master.begin....
     2020/02/11 01:19:59.739414       [WARNING]     S15-->reset.master.end....
     2020/02/11 01:19:59.739455       [WARNING]     S15-->set.gtid_purged[1520b6dd-2fb1-11ea-ab64-080027d70146:1-14
    ].begin....
     2020/02/11 01:19:59.742273       [WARNING]     S15-->set.gtid_purged.end....
     2020/02/11 01:19:59.742284       [WARNING]     S16-->enable.raft.begin...
     2020/02/11 01:19:59.743512       [WARNING]     S16-->enable.raft.done...
     2020/02/11 01:19:59.743534       [WARNING]     S17-->wait[3000 ms].change.to.master...
     2020/02/11 01:19:59.743551       [WARNING]     S18-->start.slave.begin....
     2020/02/11 01:19:59.763468       [WARNING]     S18-->start.slave.end....
     2020/02/11 01:19:59.763478       [WARNING]     completed OK!
    

    | 8 手动 Failover/switchover

    手动触发主从切换。

    (product)root@localhost [(none)]> set global read_only=on;
    Query OK, 0 rows affected (0.00 sec)
     
    [mysql@db1 bin]$ ./xenoncli raft trytoleader
     2020/02/11 01:48:51.219975       [WARNING]     [10.10.10.10:8801].prepare.to.propose.this.raft.to.leader
     2020/02/11 01:48:51.255617       [WARNING]     [10.10.10.10:8801].propose.done
    [mysql@db1 bin]$ ./xenoncli  cluster status
    +------------------+-------------------------------+---------+---------+--------------------------+---------------------+----------------+------------------+
    |        ID        |             Raft              | Mysqld  | Monitor |          Backup          |        Mysql        | IO/SQL_RUNNING |     MyLeader     |
    +------------------+-------------------------------+---------+---------+--------------------------+---------------------+----------------+------------------+
    | 10.10.10.10:8801 | [ViewID:4 EpochID:0]@LEADER   | RUNNING | ON      | state:[NONE]
                | [ALIVE] [READWRITE] | [true/true]    | 10.10.10.10:8801 |
    |                  |                               |         |         | LastError:               |                     |                |                  |
    +------------------+-------------------------------+---------+---------+--------------------------+---------------------+----------------+------------------+
    | 10.10.10.11:8801 | [ViewID:4 EpochID:0]@FOLLOWER | RUNNING | ON      | state:[NONE]
                | [ALIVE] [READONLY]  | [true/true]    | 10.10.10.10:8801 |
    |                  |                               |         |         | LastError:               |                     |                |                  |
    +------------------+-------------------------------+---------+---------+--------------------------+---------------------+----------------+------------------+
    | 10.10.10.18:8801 | [ViewID:4 EpochID:0]@FOLLOWER | RUNNING | ON      | state:[NONE]
                | [ALIVE] [READONLY]  | [true/true]    | 10.10.10.10:8801 |
    |                  |                               |         |         | LastError:               |                     |                |                  |
    +------------------+-------------------------------+---------+---------+--------------------------+---------------------+----------------+------------------+
    

    | 9 MySQL Stack Info

    查看 MySQL 如何调用堆栈信息。Quick Stack 功能速度快,对流程几乎没有影响。

    xenoncli perf -h
    perf related commands
     
    Usage:
      xenoncli perf [command]
     
    Available Commands:
      quickstack  capture the stack of mysqld using quickstack
    

    | 10 Raft + Operation

    不发起选主 enable 是恢复,disable 指该 Xenon 节点只复制。

    xenoncli raft -h
    raft related commands
     
    Usage:
      xenoncli raft [command]
     
    Available Commands:
      add                add peers to local
      disable            enable the node out control of raft
      disablepurgebinlog disable leader to purge binlog
      enable             enable the node in control of raft
      enablepurgebinlog  enable leader to purge binlog(default)
      nodes              show raft nodes
      remove             remove peers from local
      status             status in JSON(state(LEADER/CANDIDATE/FOLLOWER/IDLE))
      trytoleader        propose this raft as leader
    

    | 11 启动增强半同步检查

    # 启动半同步复制检查
    xenoncli raft enablechecksemisync
    
    # 启动或关闭半同步复制检查
    xenoncli raft disablechecksemisync
    

    我们可以看到半同步的参数是:

    (product)root@localhost [(none)]> show variables like '%rpl%';
    +-------------------------------------------+---------------------+
    | Variable_name                             | Value               |
    +-------------------------------------------+---------------------+
    | rpl_semi_sync_master_enabled              | ON                  |
    | rpl_semi_sync_master_timeout              | 1000000000000000000 |
    | rpl_semi_sync_master_trace_level          | 32                  |
    | rpl_semi_sync_master_wait_for_slave_count | 1                   |
    | rpl_semi_sync_master_wait_no_slave        | ON                  |
    | rpl_semi_sync_master_wait_point           | AFTER_SYNC          |
    | rpl_semi_sync_slave_enabled               | ON                  |
    | rpl_semi_sync_slave_trace_level           | 32                  |
    | rpl_stop_slave_timeout                    | 31536000            |
    +-------------------------------------------+---------------------+
    

    若通过 set global 修改半同步的 timeout,将自动订正为无限大;若基于运维临时调整,需要先执行 xenoncli mysql disablechecksemisync 再修改;若运维完,需要恢复自动订正的话,执行 enablechecksemisync

    // 例如:修改 timeout 参数为 10,
    (product)root@localhost [(none)]> set global rpl_semi_sync_master_timeout=10;
    Query OK, 0 rows affected (0.00 sec)
     
    (product)root@localhost [(none)]> show variables like '%rpl%';
    +-------------------------------------------+------------+
    | Variable_name                             | Value      |
    +-------------------------------------------+------------+
    | rpl_semi_sync_master_enabled              | ON         |
    | rpl_semi_sync_master_timeout              | 10         |
    | rpl_semi_sync_master_trace_level          | 32         |
    | rpl_semi_sync_master_wait_for_slave_count | 1          |
    | rpl_semi_sync_master_wait_no_slave        | ON         |
    | rpl_semi_sync_master_wait_point           | AFTER_SYNC |
    | rpl_semi_sync_slave_enabled               | ON         |
    | rpl_semi_sync_slave_trace_level           | 32         |
    | rpl_stop_slave_timeout                    | 31536000   |
    +-------------------------------------------+------------+
    9 rows in set (0.00 sec)
     
    // 过一会儿又自动变为无穷大了
    (product)root@localhost [(none)]> show variables like '%rpl%';
    +-------------------------------------------+---------------------+
    | Variable_name                             | Value               |
    +-------------------------------------------+---------------------+
    | rpl_semi_sync_master_enabled              | ON                  |
    | rpl_semi_sync_master_timeout              | 1000000000000000000 |
    | rpl_semi_sync_master_trace_level          | 32                  |
    | rpl_semi_sync_master_wait_for_slave_count | 1                   |
    | rpl_semi_sync_master_wait_no_slave        | ON                  |
    | rpl_semi_sync_master_wait_point           | AFTER_SYNC          |
    | rpl_semi_sync_slave_enabled               | ON                  |
    | rpl_semi_sync_slave_trace_level           | 32                  |
    | rpl_stop_slave_timeout                    | 31536000            |
    +-------------------------------------------+---------------------+
    9 rows in set (0.00 sec)
    

    | 指令分类

    查询集群信息:

    xenoncli cluster 指令
    

    操作本机的:

    xenoncli raft 指令
    xenoncli perf quickstack
    xenoncli mysql backup | cancelbackup | kill | rebuildme | shutdown | start | startmonitor |status | stopmonitor | sysvar
    

    先尝试连接到 Leader 执行,如果没有 Leader,则在本机执行:

    xenoncli mysql changepassword | createsuperuser | createuser | createuserwithgrants |
    dropuser | getuser
    

    下期预告

    下期我们将介绍 Xenon 配置文件相关的使用。

    推荐阅读

    本帖子中包含资源

    您需要 登录 才可以下载,没有帐号?立即注册