Elasticsearch data backup and recovery
This paper mainly introduces elasticsearch cluster data backup and recovery, using shared file system, through snapshot backup.
Cluster deployment reference: http://hnr520.blog.51cto.com/4484939/1876467
I. configuration modification
1. The configuration file must add the following parameters
Path.repo: / mnt/backups/es_mybak
II. Data backup
1. Create a backup warehouse
Curl-XPUT 'http://192.168.115.11:9200/_snapshot/EsBackup'-d' {"type": "fs", "settings": {"location": "/ mnt/backups/es_mybak"}'[root@node1 local] # curl-XPUT 'http://192.168.115.11:9200/_snapshot/EsBackup'-d' {> "type": "fs" > "settings": {> "location": "/ mnt/backups/es_mybak" >} >}'{acknowledged ": true}
Give our warehouse a name, in this case it's called EsBackup.
We specify that the type of warehouse should be a shared file system, which uses nfs.
Finally, we provide a mounted device (/ mnt/backups/es_mybak) as the destination address.
Note: the shared file system path must be accessible to all nodes in the cluster.
two。 Query backup warehouse
[root@node1 local] # curl-XGET 'http://192.168.115.11:9200/_snapshot?pretty'{ "EsBackup": {"type": "fs", "settings": {"location": "/ mnt/backups/es_mybak"}}
3. Back up all indexes
Curl-XPUT 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_all'[root@node1 local] # curl-XPUT' http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_all'{"accepted":true}
4. View backup snapshot
Curl-XGET 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_all?pretty'[root@node1 elasticsearch6] # curl-XGET' http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_*?pretty' {"snapshots": [{"snapshot": "snapshot_all", "uuid": "MRTYocxBT4qgwtpu_6jdpg", "version_id": 5000199 "version": "5.0.1", "indices": ["els_student_hnr05", "hnrtest", "els_student_hnr03", "els_student", "els_student_hnr02", "els_student_hnr04", "els_python01", "els_student_hnr01"], "state": "SUCCESS" "start_time": "2017-05-17T10:00:46.860Z", "start_time_in_millis": 1495015246860, "end_time": "2017-05-17T10:00:54.209Z", "end_time_in_millis": 1495015254209, "duration_in_millis": 7349, "failures": [], "shards": {"total": 40 "failed": 0, "successful": 40}}]}
5. Backup partial index
Curl-XPUT 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_els_student_hnr05'-d' {"indices": "els_student_hnr05"}'[root@node1 elasticsearch6] # curl-XPUT 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_els_student_hnr05'-d' {"indices": "els_student_hnr05"}'{"accepted": true}
6. View backup snapshot
Curl-XGET 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_els_student_hnr05?pretty' to view snapshot details You can see the information for each shard curl-XGET 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_els_student_hnr05/_status?pretty'[root@node1 elasticsearch6] # curl-XGET' http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_els_student_hnr05?pretty' {"snapshots": [{"snapshot": "snapshot_els_student_hnr05", "uuid": "hsSkWa63Ss6MJjh0Eo4arA" "version_id": 5000199, "version": "5.0.1", "indices": ["els_student_hnr05"], "state": "SUCCESS", "start_time": "2017-05-18T00:58:35.699Z", "start_time_in_millis": 1495069115699 "end_time": "2017-05-18T00:58:36.396Z", "end_time_in_millis": 1495069116396, "duration_in_millis": 697, "failures": [], "shards": {"total": 5, "failed": 0, "successful": 5}]}
3. Partial index data recovery
1. Query index
Curl-XGET '192.168.115.11:9200/_cat/indices?v'health status index uuid pri rep docs.count docs.deleted store.size pri.store.sizegreen open els_student_hnr05 mkvmTguZS4m1wGKbB8TOEw 5 1 99 0 124.7kb 62.3kbgreen open hnrtest Y-bjuEPLQTGGP88EafDlyg 5 1 00 1. 5kb 795bgreen open els_student_hnr03 _ UHB6P-fRfucH7lEyE-C7g 5 1 99 0 146.7kb 73.3kbgreen open els_student JC69IBEjTji8jU6bxCDE3A 5 1 999 0 495.9kb 247.9kbgreen open els_student_hnr02 k-aLl7tzRUyRfwJs8O0-kA 5 1 99 0 109.8kb 54.9kbgreen open els _ student_hnr04 zAkj7GvfQ-CqmuFMJUTSuA 5 1 99 0 162.8kb 81.4kbgreen open els_python01 wmdREADVQmCX773_JDL46Q 5 100 1.5kb 795bgreen open els_student_hnr01 BJlcIp7mS0ii9nDYj8UDyQ 5 1 100 0 109.2kb 54.6kb
two。 Delete index els_student_hnr05
[root@node1 elasticsearch6] # curl-XDELETE 'http://192.168.115.11:9200/els_student_hnr05'{"acknowledged":true}
3. Restore the index based on the snapshot_els_student_hnr05 snapshot
Curl-XPOST 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_els_student_hnr05/_restore'[root@node1 elasticsearch6] # curl-XPOST' http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_els_student_hnr05/_restore'{"accepted":true}
4. View the index again
[root@node1 elasticsearch6] # curl-XGET '192.168.115.11:9200/_cat/indices?v'health status index uuid pri rep docs.count docs.deleted store.size pri.store.sizegreen open hnrtest Y-bjuEPLQTGGP88EafDlyg 5 1 00 1.5kb 795bgreen open els_student_hnr05 S3AsHa4dRtuXXZ3uIDGvuQ 5 1 99 0 124.7kb 62.3kb
The deleted index els_student_hnr05 has been restored
IV. Complete data recovery
1. Close all nodes, delete all data under the data directory, and restart the cluster
Kill-9 $(ps-ef | grep-v grep | grep elasticsearch | awk'{print $2}')
two。 Restart the cluster
3. Create a backup warehouse
Curl-XPUT 'http://192.168.115.11:9200/_snapshot/EsBackup'-d' {"type": "fs", "settings": {"location": "/ mnt/backups/es_mybak"}'
4. View backup snapshot
Curl-XGET 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_*?pretty'{ "snapshots": [{"snapshot": "snapshot_all", "uuid": "MRTYocxBT4qgwtpu_6jdpg", "version_id": 5000199, "version": "5.0.1", "indices": ["els_student_hnr05", "hnrtest" "els_student_hnr03", "els_student", "els_student_hnr02", "els_student_hnr04", "els_python01", "els_student_hnr01"], "state": "SUCCESS", "start_time": "2017-05-17T10:00:46.860Z", "start_time_in_millis": 1495 15246860 "end_time": "2017-05-17T10:00:54.209Z", "end_time_in_millis": 1495015254209, "duration_in_millis": 7349, "failures": [], "shards": {"total": 40, "failed": 0, "successful": 40}} {"snapshot": "snapshot_els_student_hnr05", "uuid": "hsSkWa63Ss6MJjh0Eo4arA", "version_id": 5000199, "version": "5.0.1", "indices": ["els_student_hnr05"], "state": "SUCCESS", "start_time": "2017-05-18T00:58:35.699Z" "start_time_in_millis": 1495069115699, "end_time": "2017-05-18T00:58:36.396Z", "end_time_in_millis": 1495069116396, "duration_in_millis": 697, "failures": [], "shards": {"total": 5, "failed": 0 "successful": 5}}]}
5. Recover data
Curl-XPOST 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_all/_restore'
6. Query index
[root@node1 elasticsearch6] # curl-XGET '192.168.115.11:9200/_cat/indices?v'health status index uuid pri rep docs.count docs.deleted store.size pri.store.sizegreen open hnrtest LNkwevA2Sr2Jgue1oLRIGA 5 1 00 1.5kb 795bgreen open els_student 6aE8M7KMRN2EdievhDBXcw 5 1 999 0 495.9kb 247.9kbgreen open els_student_hnr04 8hXeVjXYRhS5POdy3hnMuQ 5 1 99 0 162.8kb 81.4kbgreen open els_student_hnr01 afVLqgxpTqii6pNObLC2sA 5 1 100 0 109.2kb 54.6kbgreen open els_student_hnr05 jzHPY_TlSEm-ZEXWz0sz4Q 5 1 99 0 124.7kb 62.3kbgreen open els_ Student_hnr02 f3hqeM1pQkWttZifnCCQDQ 5 1 99 0 109.8kb 54.9kbgreen open els_python01 4_vFwhn0RCi3SZbz2bORCw 5 100 1.5kb 795bgreen open els_student_hnr03 esFyZMwwTaeLf7YrKxyYjQ 5 1 99 0 146.7kb 73.3kb all indexes have been fully restored
Restore a single index from all index snapshots
1. Delete index els_student_hnr05
Curl-XDELETE 'http://192.168.115.11:9200/els_student_hnr05'
two。 Restore the deleted index els_student_hnr05
Curl-XPOST 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_all/_restore'-d' {"indices": "els_student_hnr05", "rename_replacement": "restored_index_$1"}'
VI. Delete a snapshot
Curl-XDELETE 'http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_els_student_hnr05'curl-XDELETE' http://192.168.115.11:9200/_snapshot/EsBackup/snapshot_all'
