特定のSQL処理で、GROUP BYなどの集合関数を利用していて、
“Using temporary”,”Using filesort”などが出て処理時間がかかり過ぎたり、
サブクエリーによる結果をJOINしてindexが利用出来無かったりと、
困難な場面に遭遇する事があるかと思います。
基本的には、物理的に変更しても良くて数倍だと思いますので、
アプリケーションやクエリーを工数かけて書き換えて対応するのが良いと思いますが、
なかなか出来ない場合は、可能な範囲でサーバーパラメータを変更したり、
クエリーを若干変更してメモリーテーブルやTEMPORARY TABLEなどでワークテーブルを作成し、
サブクエリーなどの結果を随時集計しIndexを使えるように処理する方法もあるかと思います。
もちろんハードで短期、中期的に解決する為に、力技で対応した場合もありましたが。。。
過去にもMEMORY STORAGE ENGINEについて、書きましたが改めて再確認してみました。
MEMORY ストレージエンジン
その他ソリューション：
Entperprise Edition：Query Rewrite Plugins
検証：MySQL Query Rewrite Plugins
MEMORYストレージエンジンによるワークテーブル
以下、メモリーストレージエンジンを利用した一時テーブル例 (インデックス利用可)
root@localhost [test]> CREATE TABLE T_Work_mem01 (
-> id int(11) DEFAULT NULL,
-> text varchar(100) DEFAULT NULL,
-> KEY idx_T_MEM01_id (id)
-> ) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4;
Query OK, 0 rows affected (0.00 sec)
root@localhost [test]> desc T_Work_mem01;
+-------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+-------+
| id | int(11) | YES | MUL | NULL | |
| text | varchar(100) | YES | | NULL | |
+-------+--------------+------+-----+---------+-------+
2 rows in set (0.00 sec)
root@localhost [test]> insert into T_Work_mem01 select * from T_ONLINE_DDL;
Query OK, 9 rows affected (0.00 sec)
Records: 9 Duplicates: 0 Warnings: 0
root@localhost [test]> explain select * from T_Work_mem01 where id = 8;
+----+-------------+--------------+------+----------------+----------------+---------+-------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------+------+----------------+----------------+---------+-------+------+-------+
| 1 | SIMPLE | T_Work_mem01 | ref | idx_T_MEM01_id | idx_T_MEM01_id | 5 | const | 2 | NULL |
+----+-------------+--------------+------+----------------+----------------+---------+-------+------+-------+
1 row in set (0.00 sec)
root@localhost [test]>
root@localhost [test]> CREATE TABLE T_Work_mem02 (
-> id int(11) DEFAULT NULL,
-> text varchar(100) DEFAULT NULL,
-> INDEX USING HASH (id)
-> ) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4;
Query OK, 0 rows affected (0.00 sec)
root@localhost [test]> desc T_Work_mem02;
+-------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+-------+
| id | int(11) | YES | MUL | NULL | |
| text | varchar(100) | YES | | NULL | |
+-------+--------------+------+-----+---------+-------+
2 rows in set (0.01 sec)
root@localhost [test]> insert into T_Work_mem02 select * from T_ONLINE_DDL;
Query OK, 9 rows affected (0.00 sec)
Records: 9 Duplicates: 0 Warnings: 0
root@localhost [test]> explain select * from T_Work_mem02 where id = 8;
+----+-------------+--------------+------+---------------+------+---------+-------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------+------+---------------+------+---------+-------+------+-------+
| 1 | SIMPLE | T_Work_mem02 | ref | id | id | 5 | const | 2 | NULL |
+----+-------------+--------------+------+---------------+------+---------+-------+------+-------+
1 row in set (0.00 sec)
root@localhost [test]>
メモリーテーブルなので、再起動したらデータは無くなりますが、
ワークテーブルとして一時的に利用するのであれば問題無いかと。
[root@GA01 admin]# /etc/init.d/mysql.server restart Shutting down MySQL.... SUCCESS! Starting MySQL. SUCCESS! [root@GA01 admin]# exit exit [admin@GA01 ~]$ mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.24-enterprise-commercial-advanced-log MySQL Enterprise Server - Advanced Edition (Commercial) Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. root@localhost [(none)]> use test Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed root@localhost [test]> select count(*) from T_Work_mem01; +----------+ | count(*) | +----------+ | 0 | +----------+ 1 row in set (0.00 sec) root@localhost [test]> show tables like 'T_Work%'; +--------------------------+ | Tables_in_test (T_Work%) | +--------------------------+ | T_Work_mem01 | | T_Work_mem02 | +--------------------------+ 2 rows in set (0.00 sec) root@localhost [test]>
※ データは空ですが、テーブル定義は残ります。
参考 (レプリケーション時の注意点等)：
8.3.8 Comparison of B-Tree and Hash Indexes
17.4.1.21 Replication and MEMORY Tables
15.3 The MEMORY Storage Engine
[留意点] MEMORY Tables and Replication
A server’s MEMORY tables become empty when it is shut down and restarted.
If the server is a replication master, its slaves are not aware that these
tables have become empty, so you see out-of-date content if you select data
from the tables on the slaves. To synchronize master and slave MEMORY tables,
when a MEMORY table is used on a master for the first time since it was started,
a DELETE statement is written to the master’s binary log, to empty the table on the slaves also.
The slave still has outdated data in the table during the interval
between the master’s restart and its first use of the table.
To avoid this interval when a direct query to the slave could return stale data,
use the –init-file option to populate the MEMORY table on the master at startup.
今回のデモ環境では,SLAVEにて除外してあります。
TEMPORARYテーブルの利用例(インデックス利用可)
root@localhost [test]> CREATE temporary TABLE T_Work_temp01 (
-> id int(11) DEFAULT NULL,
-> text varchar(100) DEFAULT NULL,
-> KEY idx_T_MEM01_id (id)
-> ) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4;
Query OK, 0 rows affected (0.00 sec)
root@localhost [test]> desc T_Work_temp01;
+-------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+-------+
| id | int(11) | YES | MUL | NULL | |
| text | varchar(100) | YES | | NULL | |
+-------+--------------+------+-----+---------+-------+
2 rows in set (0.00 sec)
root@localhost [test]> insert into T_Work_temp01 select * from T_ONLINE_DDL;
Query OK, 9 rows affected (0.00 sec)
Records: 9 Duplicates: 0 Warnings: 0
root@localhost [test]> explain select * from T_Work_temp01 where id = 8;
+----+-------------+---------------+------+----------------+----------------+---------+-------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------------+------+----------------+----------------+---------+-------+------+-------+
| 1 | SIMPLE | T_Work_temp01 | ref | idx_T_MEM01_id | idx_T_MEM01_id | 5 | const | 2 | NULL |
+----+-------------+---------------+------+----------------+----------------+---------+-------+------+-------+
1 row in set (0.00 sec)
root@localhost [test]> CREATE temporary TABLE T_Work_temp02 (
-> id int(11) DEFAULT NULL,
-> text varchar(100) DEFAULT NULL,
-> INDEX USING HASH (id)
-> ) ENGINE=MEMORY DEFAULT CHARSET=utf8mb4;
Query OK, 0 rows affected (0.01 sec)
root@localhost [test]> desc T_Work_temp02;
+-------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+-------+
| id | int(11) | YES | MUL | NULL | |
| text | varchar(100) | YES | | NULL | |
+-------+--------------+------+-----+---------+-------+
2 rows in set (0.00 sec)
root@localhost [test]> insert into T_Work_temp02 select * from T_ONLINE_DDL;
Query OK, 9 rows affected (0.01 sec)
Records: 9 Duplicates: 0 Warnings: 0
root@localhost [test]> explain select * from T_Work_temp02 where id = 8;
+----+-------------+---------------+------+---------------+------+---------+-------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------------+------+---------------+------+---------+-------+------+-------+
| 1 | SIMPLE | T_Work_temp02 | ref | id | id | 5 | const | 2 | NULL |
+----+-------------+---------------+------+---------------+------+---------+-------+------+-------+
1 row in set (0.00 sec)
TEMPORARYテーブルは、Show Tablesをしても表示されません。
作成したセッションでは、SELECTする事は可能です。
root@localhost [test]> show tables; +--------------------+ | Tables_in_test | +--------------------+ | FW_DEMO | | T_MEM | | T_MEM01 | | T_ONLINE_DDL | | T_ONLY_FULL | | T_UNION01 | | T_UNION02 | | T_Work_mem01 | | T_Work_mem02 | | card_info | | employees | | tbl_partition | | tbl_partition2 | | tbl_partition_year | | users | +--------------------+ 15 rows in set (0.00 sec) root@localhost [test]> select count(*) from T_Work_temp01; +----------+ | count(*) | +----------+ | 9 | +----------+ 1 row in set (0.00 sec) root@localhost [test]> select count(*) from T_Work_temp02; +----------+ | count(*) | +----------+ | 9 | +----------+ 1 row in set (0.00 sec) root@localhost [test]>
オラクル等では、Global Session Temp Tableなどを利用して
他のセッションからもTEMPORARY TABLEにアクセス出来ますが、
こちらは、他のセッションからアクセスする事は出来ません。
[admin@GA02 ~]$ mysql -h 192.168.56.201 -u admin -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 5 Server version: 5.6.24-enterprise-commercial-advanced-log MySQL Enterprise Server - Advanced Edition (Commercial) Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. admin@192.168.56.201 [(none)]> use test Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed admin@192.168.56.201 [test]> select count(*) from T_Work_temp01; ERROR 1146 (42S02): Table 'test.T_Work_temp01' doesn't exist admin@192.168.56.201 [test]>
ログオフ、ログインし直しても、当然無くなっています。
作成した、セッションのみで有効
root@localhost [test]> exit Bye [admin@GA01 ~]$ mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 6 Server version: 5.6.24-enterprise-commercial-advanced-log MySQL Enterprise Server - Advanced Edition (Commercial) Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. root@localhost [(none)]> use test Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed root@localhost [test]> select count(*) from T_Work_temp01; ERROR 1146 (42S02): Table 'test.T_Work_temp01' doesn't exist root@localhost [test]>
その他、参考情報
Materialized Views with MySQL
PlanetMySQL Voting: Vote UP / Vote DOWN