Quantcast
Channel: Planet MySQL
Viewing all articles
Browse latest Browse all 1081
↧

MySQL Fabricつらい(障害ノードの戻しとか)

November 6, 2014, 12:14 am
$
0
0
マスター昇格編 でマスターをkillしても、スレーブがちゃんとマスターに昇格してくれるところまでは良かったんですが、


$ mysqlfabric group lookup_servers my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                         server_uuid         address  status       mode weight
------------------------------------ --------------- ------- ---------- ------
47cf54df-63fc-11e4-942e-fa163e020fd0 127.0.0.1:20886  FAULTY READ_WRITE    1.0
4b7036a9-63fc-11e4-942e-fa163e020fd0 127.0.0.1:20887 PRIMARY READ_WRITE    1.0



(旧)マスターを起動したあとでも、待てど暮らせどファームには勝手に戻ってこない。


$ ./m
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.6.21-log Source distribution

Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

master [localhost] {msandbox} ((none)) > show slave status\G
Empty set (0.00 sec)


$ mysqlfabric group promote my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

GroupError: There is no valid candidate that can be automatically chosen in group (my_first_fabric). Please, choose one manual
ly.

killしてもう一度起動したマスターには、当然今のマスターの情報はない。生きてる間にpromoteならFabricサーバーがログインしてゴニョゴニョしてくれるんだろうけど、そもそも接続できなかったんだから当たり前。スレーブがいないんだからもちろんpromoteもできない。


$ mysqlfabric group demote my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid finished success result
------------------------------------ -------- ------- ------
dec31088-00b1-4a33-bf36-622a9e1f302a        1       1      1

state success          when                                                   description
----- ------- ------------- -------------------------------------------------------------
    3       2   1.41526e+09 Triggered by <mysql .fabric.events.event="" 0x26c38d0="" at="" object="">.
    4       2   1.41526e+09                       Executing action (_block_write_demote).
    5       2   1.41526e+09                        Executed action (_block_write_demote).
    3       2   1.41526e+09 Triggered by <mysql .fabric.events.event="" 0x26c39d0="" at="" object="">.
    4       2   1.41526e+09                       Executing action (_wait_slaves_demote).
    5       2   1.41526e+09                        Executed action (_wait_slaves_demote).


$ mysqlfabric group health my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid is_alive    status is_not_running is_not_configured io_not_running sql_not_running io_err
or sql_error
------------------------------------ -------- --------- -------------- ----------------- -------------- --------------- ------
-- ---------
47cf54df-63fc-11e4-942e-fa163e020fd0        1    FAULTY              0                 1              0               0    Fal
se     False
4b7036a9-63fc-11e4-942e-fa163e020fd0        1 SECONDARY              0                 1              0               0    Fal
se     False

issue
-----


$ mysqlfabric group lookup_servers my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                         server_uuid         address    status       mode weight
------------------------------------ --------------- --------- ---------- ------
47cf54df-63fc-11e4-942e-fa163e020fd0 127.0.0.1:20886    FAULTY READ_WRITE    1.0
4b7036a9-63fc-11e4-942e-fa163e020fd0 127.0.0.1:20887 SECONDARY  READ_ONLY    1.0


$ mysqlfabric group promote my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid finished success result
------------------------------------ -------- ------- ------
01a40c48-347d-4fa8-887c-2cde33a353ad        1       1      1

state success          when                                                   description
----- ------- ------------- -------------------------------------------------------------
    3       2   1.41526e+09 Triggered by <mysql .fabric.events.event="" 0x2552e50="" at="" object="">.
    4       2   1.41526e+09                      Executing action (_define_ha_operation).
    5       2   1.41526e+09                       Executed action (_define_ha_operation).
    3       2   1.41526e+09 Triggered by <mysql .fabric.events.event="" 0x26bc7d0="" at="" object="">.
    4       2   1.41526e+09                      Executing action (_find_candidate_fail).
    5       2   1.41526e+09                       Executed action (_find_candidate_fail).
    3       2   1.41526e+09 Triggered by <mysql .fabric.events.event="" 0x26bc710="" at="" object="">.
    4       2   1.41526e+09                     Executing action (_check_candidate_fail).
    5       2   1.41526e+09                      Executed action (_check_candidate_fail).
    3       2   1.41526e+09 Triggered by <mysql .fabric.events.event="" 0x26bc750="" at="" object="">.
    4       2   1.41526e+09                          Executing action (_wait_slave_fail).
    5       2   1.41526e+09                           Executed action (_wait_slave_fail).
    3       2   1.41526e+09 Triggered by <mysql .fabric.events.event="" 0x26c3890="" at="" object="">.
    4       2   1.41526e+09                      Executing action (_change_to_candidate).
    5       2   1.41526e+09                       Executed action (_change_to_candidate).


$ mysqlfabric group health my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid is_alive  status is_not_running is_not_configured io_not_running sql_not_running io_error
 sql_error
------------------------------------ -------- ------- -------------- ----------------- -------------- --------------- --------
 ---------
47cf54df-63fc-11e4-942e-fa163e020fd0        1  FAULTY              0                 1              0               0    False
     False
4b7036a9-63fc-11e4-942e-fa163e020fd0        1 PRIMARY              0                 0              0               0    False
     False

issue
-----



demote/promoteしてみたけどコレジャナイ。
じゃあ、remove/addで登録しなおすか。


$ mysqlfabric group remove my_first_fabric 47cf54df-63fc-11e4-942e-fa163e020fd0
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid finished success result
------------------------------------ -------- ------- ------
51509e3b-d1a7-4876-a52d-e25364c46d8d        1       1      1

state success          when                                                   description
----- ------- ------------- -------------------------------------------------------------
    3       2   1.41526e+09 Triggered by .
    4       2   1.41526e+09                            Executing action (_remove_server).
    5       2   1.41526e+09                             Executed action (_remove_server).


$ mysqlfabric group lookup_servers my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                         server_uuid         address  status       mode weight
------------------------------------ --------------- ------- ---------- ------
4b7036a9-63fc-11e4-942e-fa163e020fd0 127.0.0.1:20887 PRIMARY READ_WRITE    1.0


$ mysqlfabric group add my_first_fabric 127.0.0.1:20886
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid finished success result
------------------------------------ -------- ------- ------
f50a682b-dab6-4094-9f54-72c72569e71a        1       1      1

state success          when                                                   description
----- ------- ------------- -------------------------------------------------------------
    3       2   1.41526e+09 Triggered by .
    4       2   1.41526e+09                               Executing action (_add_server).
    5       2   1.41526e+09                                Executed action (_add_server).


$ mysqlfabric group lookup_servers my_first_fabric                            [896/2897]
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                         server_uuid         address    status       mode weight
------------------------------------ --------------- --------- ---------- ------
47cf54df-63fc-11e4-942e-fa163e020fd0 127.0.0.1:20886 SECONDARY  READ_ONLY    1.0
4b7036a9-63fc-11e4-942e-fa163e020fd0 127.0.0.1:20887   PRIMARY READ_WRITE    1.0


$ ./check_slaves.a
master
              Master_Log_File: mysql-bin.000003
          Read_Master_Log_Pos: 1548
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
          Exec_Master_Log_Pos: 1548
slave # 1
             File: mysql-bin.000003
         Position: 1548

あ、ちゃんとスレーブとしてファームに組み込まれた。


$ make_multiple_sandbox 5.6.21
installing node 1
installing node 2
installing node 3
group directory installed in $HOME/sandboxes/multi_msb_5_6_21

$ ./use_all "select @@port"
# server: 1:
@@port
14722
# server: 2:
@@port
14723
# server: 3:
@@port
14724

$ vim node?/my.sandbox.cnf
..
log-slave-updates
gtid-mode= ON
enforce-gtid-consistency

$ export MYCLIENT_OPTIONS="-uroot"
$ ./use_all "grant all on *.* to fabric@localhost identified by 'fabric_password'"
$ ./restart_all

$ mysqlfabric group add my_first_fabric 127.0.0.1:14722
$ mysqlfabric group add my_first_fabric 127.0.0.1:14723
$ mysqlfabric group add my_first_fabric 127.0.0.1:14724

$ mysqlfabric group health my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid is_alive    status is_not_running is_not_configured io_not_running sql_not_running io_err
or sql_error
------------------------------------ -------- --------- -------------- ----------------- -------------- --------------- ------
-- ---------
01250ed8-6584-11e4-9e28-fa163e020fd0        1 SECONDARY              0                 0              0               0    Fal
se     False
47cf54df-63fc-11e4-942e-fa163e020fd0        1 SECONDARY              0                 0              0               0    Fal
se     False
4b7036a9-63fc-11e4-942e-fa163e020fd0        1   PRIMARY              0                 0              0               0    Fal
se     False
f9fe3cee-6583-11e4-9e28-fa163e020fd0        1 SECONDARY              0                 0              0               0    Fal
se     False
fd8c1c45-6583-11e4-9e28-fa163e020fd0        1 SECONDARY              0                 0              0               0    Fal
se     False

issue
-----


$ ./n1
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 28
Server version: 5.6.21-log Source distribution

Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

node1 [localhost] {root} ((none)) > show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| d1                 |
| mysql              |
| performance_schema |
| test               |
+--------------------+
5 rows in set (0.01 sec)

おおー。GTIDだから、マスターにバイナリーログさえ残ってればmysqlfabric group addするより前のバイナリーログもちゃんと適用される。
ジェネラルログを覗くと、


76 Connect   fabric@localhost on
                   76 Query     SET NAMES 'utf8' COLLATE 'utf8_general_ci'
                   76 Query     SET @@session.autocommit = ON
                   76 Query     SELECT @@GLOBAL.SERVER_UUID as SERVER_UUID
                   76 Query     SELECT @@GLOBAL.SERVER_ID as SERVER_ID
                   76 Query     SELECT @@GLOBAL.VERSION as VERSION
                   76 Query     SELECT @@GLOBAL.GTID_MODE as GTID_MODE
                   76 Query     SELECT @@GLOBAL.LOG_BIN as LOG_BIN
                   76 Query     SELECT @@GLOBAL.READ_ONLY as READ_ONLY
                   76 Query     SHOW GRANTS
                   76 Query     STOP SLAVE
                   76 Query     SHOW SLAVE STATUS
                   76 Query     SHOW SLAVE STATUS
                   76 Query     CHANGE MASTER TO MASTER_HOST = '127.0.0.1' MASTER_USER = 'fabric' MASTER_PASSWORD =  MASTER_PORT = 20887
                   76 Query     SET @@GLOBAL.READ_ONLY = ON
                   76 Query     SELECT @@GLOBAL.READ_ONLY as READ_ONLY
                   76 Query     START SLAVE
                   76 Query     SHOW SLAVE STATUS
                   76 Query     SHOW SLAVE STATUS

うむ。シャードをゴニョゴニョするときはmysqldumpを使う(/etc/mysql/fabric.cfgにmysqldumpのパスを書くパラメーターがある)けど、単に追加するだけの時はFabricサーバーは特に面倒を見ないと。とはいえGTID前提だから、XtraBackupでサクッと作ってサクッとmysqlfabric group addでおしまいと。

( ´-｀).oO（や、便利なんだけどその前提条件がな。。


【2014/11/06 18:16】
途中からだいぶ脱線していた気がするけれど、もともとの目的だった"status: FAULTYからstatus: SECONDARYに戻す"は、

@yoku0825 FAULTYになったサーバはmysqlfabric server set_statusでSPAREにしてから同じコマンドでSECONDARYにできたりはします。1.4の時は。。。

— R.Kajiyama (@RKajiyama) 2014, 11月 6



$ mysqlfabric server set_status 4b7036a9-63fc-11e4-942e-fa163e020fd0 SPARE
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid finished success result
------------------------------------ -------- ------- ------
aa09ecf8-2f55-4c11-bc25-53b0ae19b3b0        1       1      1

state success          when                                                   description
----- ------- ------------- -------------------------------------------------------------
    3       2   1.41527e+09 Triggered by .
    4       2   1.41527e+09                        Executing action (_set_server_status).
    5       2   1.41527e+09                         Executed action (_set_server_status).


$ mysqlfabric group health my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid is_alive  status is_not_running is_not_configured io_not_running sql_not_running io_error sql_error
------------------------------------ -------- ------- -------------- ----------------- -------------- --------------- -------- ---------
47cf54df-63fc-11e4-942e-fa163e020fd0        1 PRIMARY              0                 0              0               0    False     False
4b7036a9-63fc-11e4-942e-fa163e020fd0        1   SPARE              0                 0              0               0    False     False

issue
-----


$ mysqlfabric server set_status 4b7036a9-63fc-11e4-942e-fa163e020fd0 SECONDARY
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid finished success result
------------------------------------ -------- ------- ------
1e8eb7c0-2233-4087-86cb-bedc9e331f1e        1       1      1

state success          when                                                   description
----- ------- ------------- -------------------------------------------------------------
    3       2   1.41527e+09 Triggered by .
    4       2   1.41527e+09                        Executing action (_set_server_status).
    5       2   1.41527e+09                         Executed action (_set_server_status).


$ mysqlfabric group health my_first_fabric
Fabric UUID:  5ca1ab1e-a007-feed-f00d-cab3fe13249e
Time-To-Live: 1

                                uuid is_alive    status is_not_running is_not_configured io_not_running sql_not_running io_error sql_error
------------------------------------ -------- --------- -------------- ----------------- -------------- --------------- -------- ---------
47cf54df-63fc-11e4-942e-fa163e020fd0        1   PRIMARY              0                 0              0               0    False     False
4b7036a9-63fc-11e4-942e-fa163e020fd0        1 SECONDARY              0                 0              0               0    False     False

issue
-----



Σ(ﾟдﾟlll) できたーっ！
PlanetMySQL Voting: Vote UP / Vote DOWN
↧
Search
Viewing all articles
Browse latest Browse all 1081

Trending Articles

モーツァルト ディヴェルティメント 変ホ長調 K.563 の名盤

December 23, 2017, 2:32 am

井上貴博アナウンサー彼女や結婚の噂は？実家や親が話題？人気は？

September 15, 2013, 1:59 am

Ke Aloha Kalikimakaの歌詞を和訳します

January 9, 2014, 6:10 pm

PaliのLepe `Ula`ulaと歌詞の和訳

July 19, 2012, 7:10 pm

2014年6月6日号　三菱東京ＵＦＪ銀行（5月14日付）

June 9, 2014, 1:51 am

LNK2019:未解決の外部シンボル と LNK1120:外部参照 1 が未解決について

October 23, 2005, 9:03 pm

ヴァンパイア・ノーツ　攻略

December 11, 2018, 6:10 am

大阪・泉南イオンで飛び降り自殺とみられる転落事件が発生：ネットで拡散された理由とは

July 15, 2016, 12:05 pm

メールディーラーで受信するアドレスを追加できますか？

July 30, 2019, 1:56 am

Robocopy のエラー (戻り値) について

January 23, 2018, 11:28 pm

林要の結婚や経歴&評判とWikiプロフやLOVOT(ラボット)とグルーブエックス株価は

December 23, 2018, 6:18 am

【極☆寒】「凍った髪」を競い合う『国際ヘア・フリージング・コンテスト』！ 寒〜い写真に身震いしつつ過ぎ行く冬にサヨナラだ!!

March 10, 2015, 2:00 pm

滋賀の部落（同和地区）一覧

March 24, 2010, 6:39 am

【銃刀法違反】吉田総業組長代行 恩田達志容疑者を再逮捕

November 17, 2016, 6:39 pm

和歌山県代表決まる　都道府県対抗中学バレー

September 10, 2013, 2:00 am

大浦街道で重体事故

August 31, 2016, 1:35 am

【世界大学ランキング】 第１位にジュリアード音楽院とウィーン国立音大、日本勢は？

August 23, 2019, 1:32 am

【対策済】「SKYSEA Client View」のアップデートに失敗する問題についてのお知らせ

December 27, 2007, 7:00 am

Lahaina Lunaの歌詞を和訳しました

January 14, 2016, 6:10 pm

画像・写真】ららぽーと横浜で16歳男子高校生が転落死 不審な動き→逃走し警備員に追いかけられ→柵越え飛び降り・12m転落 窃盗・万引き？それとも盗撮？

September 17, 2016, 8:50 am
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>