Slow requests for centos7.6 system deployment of ceph services installed on Taishan server
Recently, the company cloud wants to adapt to the Taishan server, so it deploys ceph on the server to run and store. However, when using ceph to upload fast files, there are a large number of slow requests in ceph, which makes it impossible for the ceph cluster to use. However, the above situation does not occur when using object storage and file system, so use the dmesg command on the server to check, and there are some problems:
For this problem, to find information on the Internet, you need to upgrade the hns3 Nic driver in order to solve this problem. Due to
Upgrade tutorial
Edit module configuration echo 'blacklist hns_roce_hw_v2' > > / usr/lib/modprobe.d/dist-blacklist.confecho' balcklist hns_roce' > > / usr/lib/modprobe.d/dist-blacklist.conf
The purpose of this step is to disable roce
two。 Restart
3. Download the network card driver
You need to download it from Huawei's official website.
4. Uninstall the hns3 module
If rmmod hns3# does not have the previous kernel to disable roce, it will cause a panic when downloading the network card module.
5. Execute install.sh in the driver file
6. Install the network card package
Yum-y install NIC-hisi_eth-CentOS7.6-hns3-1.0.1-1-aarch74.rpm# the last step to execute install.sh is to install the rpm package, but in the actual operation, it is found that the NIC-hisi_eth-CentOS7.6-hns3-1.0.1-1-aarch74.rpm package is not installed during the installation process, so it is mentioned separately here.
7. Restart
Conclusion
The Taishan server is not compatible with the hns3 module in the centos7.6 system, which leads to a slow request when the centos7.6 running ceph cluster is deployed on the Taishan server when the rbd command imports the file, which is a bug of cengos7.6 on the Taishan server. After upgrading the network card, there are no errors in the layer 3 and layer 4 packets of ceph.