Record a persistent error caused by a full disk space
Environment: 3.0.7 redis 8-node cluster 4 master 4 slave development test environment
Operation: clear cluster persistence data
# redis-cli-c-p 6383-h 172.31.103.238 after landing
Cluster nodes view node information log in to master node to delete flushall
Hold the error during the execution of one of the nodes:
172.31.103.238 6383 > flushdb
(error) MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.
Online information:
An error occurred while running Redis today with the following error message:
(error) MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.
Redis is configured to hold database snapshots, but it cannot be persisted to the hard disk at this time. The command used to modify the collection data cannot be used. Please check the Redis log for detailed error messages.
Reason:
Force closing Redis snapshots so that they cannot be persisted.
Solution:
After running the config set stop-writes-on-bgsave-error no command, close the configuration item stop-writes-on-bgsave-error to resolve the problem.
Root@ubuntu:/usr/local/redis/bin#. / redis-cli
127.0.0.1 purl 6379 > config set stop-writes-on-bgsave-error no
OK
127.0.1 6379 > lpush myColour "red"
(integer) 1
View your own profile:
Stop-writes-on-bgsave-error yes
Although it is a test environment, it is not modified immediately. Log in to the appropriate node.
Df-h check that root is full.
[root@VM-172-31-103-238 /] # df-h
Filesystem Size Used Avail Use% Mounted on
/ dev/vda3 35G 35G 0100% /
Find out the culprit is the log file 33g made by the developer.
Delete it after confirmation
View after deletion
[root@VM-172-31-103-238 /] # df-h
Filesystem Size Used Avail Use% Mounted on
/ dev/vda3 35G 35G 0100% /
No release.
[root@VM-172-31-103-238 /] # lsof | grep delete
Java 21037 root 1w REG 252 Magi 3 35273113600 395356 / opt/server/logs/catalina.out (deleted)
Java 21037 root 2w REG 252 Magi 3 35273113600 395356 / opt/server/logs/catalina.out (deleted)
[root@VM-172-31-103-238 /] # kill-9 21037
[root@VM-172-31-103-238 /] # df-h
Filesystem Size Used Avail Use% Mounted on
/ dev/vda3 35G 2.0G 32G 6% /
Execute again
[root@VM-172-31-103-73 / cluster] # redis-cli-c-p 6383-h 172.31.103.238
172.31.103.238 6383 > flushall
OK
