MySQL5.7で実装されたJSONデータ型とJSON関数のレビューになります。
ご利用になる場合は、参考にして頂ければと思います。
ネイティブJSONデータ型 (バイナリ形式)
Insert時のJSON構文バリデーション機能
組み込みJSON関数 (保存、検索、更新、操作)
ドキュメントにインデックス設定し高速アクセス
SQLとの統合を容易にする、新しいインライン構文
utf8mb4の文字セットとutf8mb4_binの照合 「」
サイズはmax_allowed_packetの値で制限 (Default:4MB)
MySQL5.7からは、リレーショナル、スキーマレスを同じ技術スタックで利用可能になっています。
13.16.1 JSON Function Reference
https://dev.mysql.com/doc/refman/5.7/en/json-function-reference.html
参考: Modifying JSON Values in MySQL 5.7
https://planet.mysql.com/entry/?id=5994648
JSONで表現する全てのデータ型をサポート
数値, 文字列, bool(true,false)
オブジェクト {“キー”: “値”}, 配列 [123456, “String”, …]
null
日付(date), 時刻, 日付(datetime), タイムスタンプ, その他
[CONFIRM]> show create table T_JSON_SUPPORT\G *************************** 1. row *************************** Table: T_JSON_SUPPORT Create Table: CREATE TABLE `T_JSON_SUPPORT` ( `id` int(10) NOT NULL AUTO_INCREMENT, `body` json DEFAULT NULL, `type` varchar(20) GENERATED ALWAYS AS (json_type(`body`)) VIRTUAL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=utf8mb4 [CONFIRM]> select * from T_JSON_SUPPORT; +----+---------------------------------------------+----------+ | id | body | type | +----+---------------------------------------------+----------+ | 1 | 123456789 | INTEGER | | 2 | NULL | NULL | | 3 | true | BOOLEAN | | 4 | "abcde" | STRING | | 5 | {“id”: 5, “name”: “オブジェクト”} | OBJECT | | 6 | [-122.42200352825247, 37.80848009696725, 0] | ARRAY | | 7 | "2016-02-29" | DATE | | 8 | "2016-02-29 00:00:00.000000" | DATETIME | +----+---------------------------------------------+----------+
データ型と照合順
4byteなので絵文字を格納可能、またutf8mb4_binなので大文字と小文字を区別します。
[NEW57]> SET @j = JSON_OBJECT('key', 'value'); 1 row in set (0.00 sec) [NEW57]> SELECT @j; +------------------+ | @j | +------------------+ | {"key": "value"} | +------------------+ 1 row in set (0.00 sec) [NEW57]> SELECT CHARSET(@j), COLLATION(@j); +-------------+---------------+ | CHARSET(@j) | COLLATION(@j) | +-------------+---------------+ | utf8mb4 | utf8mb4_bin | +-------------+---------------+ 1 row in set (0.00 sec)
JSONドキュメントと生成列を利用して列を作成し、対象列にINDEXを付けて高速な検索を行う事が可能です
生成列はファンクションインデックスとして利用可能
こちらは、生成列を利用したテーブルの作成例になります。
string1とstring2にデータをINSERTする事で、string1_w_string、string2_w_string、compareのデータは自動生成されます。
[CONFIRM]> show create table T_Character_COLLATE_utf8mb4_bin\G *************************** 1. row *************************** Table: T_Character_COLLATE_utf8mb4_bin Create Table: CREATE TABLE `T_Character_COLLATE_utf8mb4_bin` ( `pid` int(10) unsigned NOT NULL AUTO_INCREMENT, `string1` char(1) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL, `string2` char(1) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL, `string1_w_string` char(8) GENERATED ALWAYS AS (hex(weight_string(`string1`))) VIRTUAL, `string2_w_string` char(8) GENERATED ALWAYS AS (hex(weight_string(`string2`))) VIRTUAL, `compare` char(1) GENERATED ALWAYS AS ((`string1` = `string2`)) VIRTUAL, PRIMARY KEY (`pid`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4 [CONFIRM]> insert into T_Character_COLLATE_utf8mb4_bin(string1,string2) values('A','a'); Query OK, 1 row affected (0.00 sec) [CONFIRM]> insert into T_Character_COLLATE_utf8mb4_bin(string1,string2) values('あ','ぁ'); Query OK, 1 row affected (0.00 sec) [CONFIRM]> insert into T_Character_COLLATE_utf8mb4_bin(string1,string2) values('A','A'); Query OK, 1 row affected (0.00 sec) [CONFIRM]> select * from T_Character_COLLATE_utf8mb4_bin; +-----+---------+---------+------------------+------------------+---------+ | pid | string1 | string2 | string1_w_string | string2_w_string | compare | +-----+---------+---------+------------------+------------------+---------+ | 1 | A | a | 000041 | 000061 | 0 | | 2 | あ | ぁ | 003042 | 003041 | 0 | | 3 | A | A | 000041 | 000041 | 1 | +-----+---------+---------+------------------+------------------+---------+
JSONデータと生成列を利用すると、特定のJSON識別子からデータを抜き出して列を作成し、その列にインデックスを付ける事が可能
例えば、JSONデータをINSERTしてJSONドキュメントにUSER_IDやPRODUCT_IDがあれば、その項目だけを抜き出して列にしてINDEXを付与。
検索する場合は、それらの値を利用して検索するとJSONドキュメントも高速検索が可能になります。
[NEW57]> CREATE TABLE `T_JSON` ( `id` int(11) NOT NULL AUTO_INCREMENT, `feature` json NOT NULL, `feature_type` varchar(30) GENERATED ALWAYS AS (json_unquote(feature->"$.type")) VIRTUAL, `feature_street` varchar(30) GENERATED ALWAYS AS (json_extract(`feature`,‘$.properties.STREET’)) VIRTUAL, PRIMARY KEY (`id`), KEY `idx_feature_street` (`feature_street`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4; [NEW57]> alter table features add index idx_feature_type(`feature_type`); Query OK, 0 rows affected (6.14 sec) Records: 0 Duplicates: 0 Warnings: 0 [NEW57]> explain select distinct(feature_type) from features; +----+-------------+----------+------------+-------+------------------+------------------+---------+------+--------+----------+-------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+----------+------------+-------+------------------+------------------+---------+------+--------+----------+-------------+ | 1 | SIMPLE | features | NULL | index | idx_feature_type | idx_feature_type | 123 | NULL | 199013 | 100.00 | Using index | +----+-------------+----------+------------+-------+------------------+------------------+---------+------+--------+----------+-------------+
【JSON関数の使い方】
(1)JSONドキュメントの情報取得
[NEW57]> SELECT JSON_VALID(body) from T_JSON_DOC where id = 5; +------------------+ | JSON_VALID(body) | +------------------+ | 1 | +------------------+ [NEW57]> SELECT JSON_TYPE(body) from T_JSON_DOC where id = 5; +-----------------+ | JSON_TYPE(body) | +-----------------+ | OBJECT | +-----------------+ [NEW57]> SELECT JSON_KEYS(body) from T_JSON_DOC where id = 5; +---------------------------------------+ | JSON_KEYS(body) | +---------------------------------------+ | ["id", "name", "price", "Conditions"] | +---------------------------------------+ [NEW57]> SELECT JSON_SEARCH(feature,'one','MARKET') AS extract_path FROM features WHERE id = 121254; +-----------------------+ | extract_path | +-----------------------+ | "$.properties.STREET" | +-----------------------+
(2)通常のリレーショナルテーブルから,JSONドキュメントを作成する
Object{}とArray[]関数を利用
/*** リレーショナルテーブル ***/ [NEW57]> SELECT NAME,CountryCode from world.City where CountryCode ='JPN' limit 1; +-------+-------------+ | NAME | CountryCode | +-------+-------------+ | Tokyo | JPN | +-------+-------------+ /*** リレーショナルテーブルからJSONデータを作成 ({})***/ [NEW57]> SELECT JSON_OBJECT('CITY',NAME,'Country',CountryCode) from world.City where CountryCode ='JPN' limit 1; +------------------------------------------------+ | JSON_OBJECT('CITY',NAME,'Country',CountryCode) | +------------------------------------------------+ | {"CITY": "Tokyo", "Country": "JPN"} | +------------------------------------------------+ /*** VIEWにしておくと呼び出しがより楽になります ***/ [NEW57]> CREATE VIEW v_City_json AS -> SELECT JSON_OBJECT('ID', ID, 'name', Name, 'CountryCode', CountryCode, 'District', District,'Population',Population) as doc FROM City where CountryCode = 'JPN'; Query OK, 0 rows affected (0.01 sec) [NEW57]> select * from v_City_json limit 1\G *************************** 1. row *************************** doc: {"ID": 1532, "name": "Tokyo", "District": "Tokyo-to", "Population": 7980230, "CountryCode": "JPN"} 1 row in set (0.01 sec) /*** リレーショナルテーブル ***/ [NEW57]>SELECT NAME,CountryCode from world.City where CountryCode ='JPN' limit 1; +-------+-------------+ | NAME | CountryCode | +-------+-------------+ | Tokyo | JPN | +-------+-------------+ /*** リレーショナルテーブルからJSONデータを作成 ([])***/ [NEW57]> SELECT JSON_ARRAY(NAME,CountryCode) from world.City where CountryCode ='JPN' limit 1; +------------------------------+ | JSON_ARRAY(NAME,CountryCode) | +------------------------------+ | ["Tokyo", "JPN"] | +------------------------------+ /*** リレーショナルテーブル***/ [NEW57]> select * from T_JSON_PLACE; +----+-------------------------------------+----------+-----------+ | id | place | latitude | longitude | +----+-------------------------------------+----------+-----------+ | 1 | 東京都港区北青山2-5-8 | 35.67125 | 139.71864 | | 2 | 大阪府大阪市北区堂2-4-27 | 34.69584 | 135.49291 | +----+-------------------------------------+----------+-----------+ /*** リレーショナルテーブルからJSONデータを作成 ({},[])***/ [NEW57]> select JSON_OBJECT('ID',id,'住所',place,'緯度経度', JSON_ARRAY(latitude,longitude)) from T_JSON_PLACE; +---------------------------------------------------------------------------------------------------+ | JSON_OBJECT('ID',id,'住所',place,'緯度経度', JSON_ARRAY(latitude,longitude)) | +---------------------------------------------------------------------------------------------------+ | {"ID": 1, "住所": "東京都港区北青山2-5-8", "緯度経度": [35.67125, 139.71864]} | | {"ID": 2, "住所": "大阪府大阪市北区堂2-4-27", "緯度経度": [34.69584, 135.49291]} | +---------------------------------------------------------------------------------------------------+
メモ: ST_AsGeoJSONやST_GeomFromGeoJSON等のSpatial GeoJSON Functionsで空間を表すJSONも利用する事が可能
JSONドキュメントの更新処理
/**** JSON_REMOVE (SELECTでの確認とUPDATEによる更新)****/ [NEW57]> select * from T_JSON_DOC; +----+---------------------------------------------------------------------------------------------------+------------+ | id | body | price_only | +----+---------------------------------------------------------------------------------------------------+------------+ | 1 | {"id": 1, "name": "自転車", "price": 10000, "Conditions": ["NEW", 2015, "Excellent"]} | 10000.00 | | 2 | {“id”: 2, “name”: “テレビ”, “price”: 30000, “Conditions”: [“USED”, 2013, “故障”]} | 30000.00 | | 3 | {"id": 3, "name": "冷蔵庫", "price": 17131, "Conditions": ["NEW", 2015]} | 17131.00 | | 4 | {“id”: 4, “name”: “オートバイ”, “price”: 500000, “Conditions”: [“NEW”, 2015]} | 500000.00 | | 5 | {"id": 5, "name": "自転車", "price": 25000, "Quantity": "[1,10]", "Conditions": ["NEW", 2015]} | 25000.00 | +----+---------------------------------------------------------------------------------------------------+------------+ [NEW57]> select JSON_REMOVE(body,"$.Quantity") from T_JSON_DOC where id = 5; +-----------------------------------------------------------------------------+ | JSON_REMOVE(body,"$.Quantity") | +-----------------------------------------------------------------------------+ | {"id": 5, "name": "自転車", "price": 25000, "Conditions": ["NEW", 2015]} | +-----------------------------------------------------------------------------+ [NEW57]> update T_JSON_DOC set body = JSON_REMOVE(body,"$.Quantity") where id = 5; /**** JSON_SET (SELECTでの確認とUPDATEによる更新)****/ [NEW57]> select * from T_JSON_DOC; +----+------------------------------------------------------------------------------+------------+ | id | body | price_only | +----+------------------------------------------------------------------------------+------------+ | 1 | {“id”: 1, “name”: “自転車”, “price”: 10000, “Conditions”: [“NEW”, 2015]} | 10000.00 | | 2 | {"id": 2, "name": "テレビ", "price": 30000, "Conditions": ["USED",2013]} | 30000.00 | | 3 | {"id": 3, "name": "冷蔵庫", "price": 11154, "Conditions": ["NEW", 2015]} | 11154.00 | | 4 | {"id": 4, "name": "冷蔵庫", "price": 50000, "Conditions": ["NEW", 2015]} | 50000.00 | | 5 | {"id": 5, "name": "自転車", "price": 25000, "Conditions": ["NEW", 2015]} | 25000.00 | +----+------------------------------------------------------------------------------+------------+ [NEW57]> SELECT JSON_SET(body,'$.name',"オートバイ", '$.price',500000) from T_JSON_DOC where id = 4; +------------------------------------------------------------------------------------+ | JSON_SET(body,'$.name',"オートバイ", '$.price',500000) | +------------------------------------------------------------------------------------+ | {"id": 4, "name": "オートバイ", "price": 500000, "Conditions": ["NEW", 2015]} | +------------------------------------------------------------------------------------+ [NEW57]> update T_JSON_DOC set body = JSON_SET(body,'$.name',"オートバイ", '$.price',500000) where id = 4; /**** JSON_INSERT (SELECTでの確認とUPDATEによる更新)****/ [NEW57]> select * from T_JSON_DOC; +----+------------------------------------------------------------------------------+------------+ | id | body | price_only | +----+------------------------------------------------------------------------------+------------+ | 1 | {“id”: 1, “name”: “自転車”, “price”: 10000, “Conditions”: [“NEW”, 2015]} | 10000.00 | | 2 | {"id": 2, "name": "テレビ", "price": 30000, "Conditions": ["USED",2013]} | 30000.00 | | 3 | {"id": 3, "name": "冷蔵庫", "price": 11154, "Conditions": ["NEW", 2015]} | 11154.00 | | 4 | {"id": 4, "name": "冷蔵庫", "price": 50000, "Conditions": ["NEW", 2015]} | 50000.00 | | 5 | {"id": 5, "name": "自転車", "price": 25000, "Conditions": ["NEW", 2015]} | 25000.00 | +----+------------------------------------------------------------------------------+------------+ [NEW57]> select JSON_INSERT(body,'$.Quantity','[1,10]') from NEW57.T_JSON_DOC where id = 5; +---------------------------------------------------------------------------------------------------+ | JSON_INSERT(body,'$.Quantity','[1,10]') | +---------------------------------------------------------------------------------------------------+ | {"id": 5, "name": "自転車", "price": 25000, "Quantity": "[1,10]", "Conditions": ["NEW", 2015]} | +---------------------------------------------------------------------------------------------------+ [NEW57]> update NEW57.T_JSON_DOC set body = JSON_INSERT(body,'$.Quantity','[1,10]') where id = 5; /**** JSON_REPLACE (SELECTでの確認とUPDATEによる更新)****/ [NEW57]> select * from T_JSON_DOC; +----+------------------------------------------------------------------------------+------------+ | id | body | price_only | +----+------------------------------------------------------------------------------+------------+ | 1 | {“id”: 1, “name”: “自転車”, “price”: 10000, “Conditions”: [“NEW”, 2015]} | 10000.00 | | 2 | {"id": 2, "name": "テレビ", "price": 30000, "Conditions": ["USED",2013]} | 30000.00 | | 3 | {"id": 3, "name": "冷蔵庫", "price": 11154, "Conditions": ["NEW", 2015]} | 11154.00 | | 4 | {"id": 4, "name": "冷蔵庫", "price": 50000, "Conditions": ["NEW", 2015]} | 50000.00 | | 5 | {"id": 5, "name": "自転車", "price": 25000, "Conditions": ["NEW", 2015]} | 25000.00 | +----+------------------------------------------------------------------------------+------------+ [NEW57]> select JSON_REPLACE(body,"$.price",FLOOR(10000 + (RAND() * 9000))) from T_JSON_DOC where id = 3; +-----------------------------------------------------------------------------+ | JSON_REPLACE(body,"$.price",FLOOR(10000 + (RAND() * 9000))) | +-----------------------------------------------------------------------------+ | {"id": 3, "name": "冷蔵庫", "price": 18359, "Conditions": ["NEW", 2015]} | +-----------------------------------------------------------------------------+ [NEW57]> update T_JSON_DOC set body = JSON_REPLACE(body,"$.price",FLOOR(10000 + (RAND() * 9000))) where id = 3; /**** JSON_ARRAY_INSERT (SELECTでの確認とUPDATEによる更新)****/ [NEW57]> select * from T_JSON_DOC; +----+------------------------------------------------------------------------------+------------+ | id | body | price_only | +----+------------------------------------------------------------------------------+------------+ | 1 | {“id”: 1, “name”: “自転車”, “price”: 10000, “Conditions”: [“NEW”, 2015]} | 10000.00 | | 2 | {"id": 2, "name": "テレビ", "price": 30000, "Conditions": ["USED",2013]} | 30000.00 | | 3 | {"id": 3, "name": "冷蔵庫", "price": 11154, "Conditions": ["NEW", 2015]} | 11154.00 | | 4 | {"id": 4, "name": "冷蔵庫", "price": 50000, "Conditions": ["NEW", 2015]} | 50000.00 | | 5 | {"id": 5, "name": "自転車", "price": 25000, "Conditions": ["NEW", 2015]} | 25000.00 | +----+------------------------------------------------------------------------------+------------+ [NEW57]>SELECT JSON_ARRAY_INSERT(body,'$.Conditions[2]','January') from T_JSON_DOC where id = 5; +----------------------------------------------------------------------------------------+ | JSON_ARRAY_INSERT(body,'$.Conditions[2]','January') | +----------------------------------------------------------------------------------------+ | {"id": 5, "name": "自転車", "price": 25000, "Conditions": ["NEW", 2015, "January"]} | +----------------------------------------------------------------------------------------+ [NEW57]> update T_JSON_DOC set body = JSON_ARRAY_INSERT(body,'$.Conditions[2]','January') where id = 5; /**** JSON_ARRAY_APPEND (SELECTでの確認とUPDATEによる更新)****/ [NEW57]> select * from T_JSON_DOC; +----+------------------------------------------------------------------------------+------------+ | id | body | price_only | +----+------------------------------------------------------------------------------+------------+ | 1 | {“id”: 1, “name”: “自転車”, “price”: 10000, “Conditions”: [“NEW”, 2015]} | 10000.00 | | 2 | {"id": 2, "name": "テレビ", "price": 30000, "Conditions": ["USED",2013]} | 30000.00 | | 3 | {"id": 3, "name": "冷蔵庫", "price": 11154, "Conditions": ["NEW", 2015]} | 11154.00 | | 4 | {"id": 4, "name": "冷蔵庫", "price": 50000, "Conditions": ["NEW", 2015]} | 50000.00 | | 5 | {"id": 5, "name": "自転車", "price": 25000, "Conditions": ["NEW", 2015]} | 25000.00 | +----+------------------------------------------------------------------------------+------------+ [NEW57]> select JSON_ARRAY_APPEND(body,'$.Conditions','故障') from NEW57.T_JSON_DOC where id = 2; +----------------------------------------------------------------------------------------+ | JSON_ARRAY_APPEND(body,'$.Conditions','故障') | +----------------------------------------------------------------------------------------+ | {"id": 2, "name": "テレビ", "price": 30000, "Conditions": ["USED", 2013, "故障"]} | +----------------------------------------------------------------------------------------+ [NEW57]> update T_JSON_DOC set body = JSON_ARRAY_APPEND(body,'$.Conditions','故障') where id = 2; /**** JSON_MERGE (SELECTでの確認とUPDATEによる更新)****/ [NEW57]> select * from T_JSON_DOC; +----+------------------------------------------------------------------------------+------------+ | id | body | price_only | +----+------------------------------------------------------------------------------+------------+ | 1 | {“id”: 1, “name”: “自転車”, “price”: 10000, “Conditions”: [“NEW”, 2015]} | 10000.00 | | 2 | {"id": 2, "name": "テレビ", "price": 30000, "Conditions": ["USED",2013]} | 30000.00 | | 3 | {"id": 3, "name": "冷蔵庫", "price": 11154, "Conditions": ["NEW", 2015]} | 11154.00 | | 4 | {"id": 4, "name": "冷蔵庫", "price": 50000, "Conditions": ["NEW", 2015]} | 50000.00 | | 5 | {"id": 5, "name": "自転車", "price": 25000, "Conditions": ["NEW", 2015]} | 25000.00 | +----+------------------------------------------------------------------------------+------------+ [NEW57]> SELECT JSON_MERGE(body,'{"Conditions":"Excellent"}') from T_JSON_DOC where id = 1; +------------------------------------------------------------------------------------------+ | JSON_MERGE(body,'{"Conditions":"Excellent"}') | +------------------------------------------------------------------------------------------+ | {"id": 1, "name": "自転車", "price": 10000, "Conditions": ["NEW", 2015, "Excellent"]} | +------------------------------------------------------------------------------------------+ [NEW57]> update T_JSON_DOC set body = JSON_MERGE(body,'{"Conditions":"Excellent"}') where id = 1;
補足情報
json_extractと->は同等
`feature_type` varchar(30) GENERATED ALWAYS AS (json_unquote(feature->"$.type")) VIRTUAL `feature_type` varchar(30) GENERATED ALWAYS AS (json_unquote(json_extract(`feature`,'$.type'))) VIRTUAL
メモ:
-> /* JSON_EXTRACT() */
->> /* JSON_UNQUOTE(JSON_EXTRACT()) */
必要に応じてjson_unquoteでQuoteを外して列を作成
`feature_type` varchar(30) GENERATED ALWAYS AS (json_unquote(feature->"$.type")) VIRTUAL `feature_street` varchar(30) GENERATED ALWAYS AS (json_extract(`feature`,'$.properties.STREET')) VIRTUAL [NEW57]> select feature_type,feature_street from T_JSON limit 10,5; +--------------+----------------+ | feature_type | feature_street | +--------------+----------------+ | Feature | "JEFFERSON" | | Feature | "TAYLOR" | | Feature | "BEACH" | | Feature | "BEACH" | | Feature | "JEFFERSON" | +--------------+----------------+
生成列のVirtualとStoredの違いはありますが、Virtualであれば作成時も参照、更新もオンライン処理が可能。
但し、参照時にCPUを利用するのでStoredを選択するかは状況により使い分け。
Storedは実データを含む為、高速だが追加でディスク容量が必用。また、作成時は参照のみが可能になります。
ALTER TABLE T_ONLINE ALGORITHM=INPLACE, ADD feature_type varchar(30) AS (feature->"$.type") VIRTUAL; Query OK, 0 rows affected (0.08 sec) Records: 0 Duplicates: 0 Warnings: 0 ALTER TABLE T_ONLINE ALGORITHM=INPLACE, ADD feature_type varchar(30) AS (feature->"$.type") STORED; ERROR 1845 (0A000): ALGORITHM=INPLACE is not supported for this operation. Try ALGORITHM=COPY. ALTER TABLE T_ONLINE ADD feature_type varchar(30) AS (feature->"$.type") STORED; Query OK, 206560 rows affected (6.17 sec) Records: 206560 Duplicates: 0 Warnings: 0