Problem of / var Partition full when rsync synchronizes massive files

1, rsync synchronization of large files/var partition full problem

Recently, I always encountered the problem that the/var partition is full, and I can't find which file has inflated the partition.

Use the following command to see the size of the/var directory, which is only 81M.

#du -sh /var

81M /var

But with df command to see the partition situation, found that/var is full..

#df -h

/dev/sdc6 2.0G 2.0G 0 100% /var

The machine is syncing up some images, I'd say roughly 1,2 million images. This is normal at first, but after a few hours of rsync running, /var starts to grow larger and then becomes full. At this time, the rsync process uses only 2MB of memory. I guess because there are too many files, rsync uses the disk of/var partition as a temporary stack when comparing the file list of server and client, and the/var partition is full.

#killall rsync

#df -h

/dev/sdc6 2.0G 109M 1.8G 6% /var

Kill rsync process, free disk space... But...

Summary:

rsync uses temporary files when comparing the list of files on the server and client side, but we can't see where the temporary files are.

In linux, first create a temporary file, and then the process to open the file, and then call unlink delete file, but this time the file is open, the operating system will not really delete it, when the process keeps writing, the partition will slowly fill up, when the process exits, the operating system detects that the file has not been used, will be deleted.

You also don't have to link var to a large partition. When synchronizing large files, you can switch your tasks into several smaller tasks by directory.