How to recover after Ceph Cluster relocation
Editor to share with you how to recover after the relocation of the Ceph cluster. I hope you will get something after reading this article. Let's discuss it together.
The recovery process modifies the cluster IP information in each host
Because the cluster IP has been modified, the cluster IP address information in the configuration file in / etc/hosts is also updated. Each host in the cluster should be updated.
1 "0.57.148.6 ceph01*0.57.148.*7 ceph21*0.57.148.*8 ceph31*0.57.148.*9 ceph41*0.57.148.*0 ceph5mon recovery
The first condition for the recovery of the whole cluster is to restore MON, which is also the need to solve the change of IP. OSD itself is not related to IP. To solve this problem, you can check my previous blog, Ceph, which caused MON exception due to the change of IP address.
It is important to note here that the mon_host information in the ceph.conf file and the network segment information of public network / cluster network are updated.
Mon_initial_members = ceph0,ceph2,ceph3mon_host = your mon ip addrsosd recovery
Since my hard drive does not have the function of automatic mount, there is a manual task here, that is, how to mount the hard disk to the corresponding mount directory.
When I first mounted it, I mounted it sequentially, that is,
/ dev/sdb1-> / var/lib/ceph/osd/ceph-0/dev/sdc1-- > / var/lib/ceph/osd/ceph-1/dev/sdd1-- > / var/lib/ceph/osd/ceph-2/dev/sde1-- > / var/lib/ceph/osd/ceph-3.
Both the drive letter and the mount point are increasing in turn, so you can mount them in turn when you restore them. If you are not sure, you can use the following method: first mount the hard disk and check the number in the whoami to know which mount point the hard disk corresponds to.
After mounting, start the osd process on each host with the / etc/init.d/ceph start command.
Rgw recovery
The modification of the IP address has little impact on the RGW. You only need to change the host option in the configuration file to the actual IP address.
[client.rgw.ceph5] host = your ip addrrgw frontends=civetweb port=80
After modification, just start the rgw process directly.
In this way, the entire cluster can be restored.
Attached
During the recovery process, it is found that the state of the cluster is very unstable, and there are always different osd nodes marked as down. And I didn't find out why. It was later considered whether the policy in the firewall did not add the new IP. It was so after the examination. After modifying the policy, the state changes to active + clean. So far, the cluster has been restored successfully.
The method to remove mon:
Monmap e1: 5 mons at {ceph mon remove dremoved mon.d at 101.71.4.11 root@aaa 101.71.4.12 there are now 4 monitors [root@aaa] # ceph mon remove eremoved mon.e at 101.71.15 There are now 3 monitors finished reading this article. I believe you have a certain understanding of "how to recover after Ceph cluster relocation". If you want to know more about it, you are welcome to follow the industry information channel. Thank you for reading!
