UMOUNT NAS error No space left on device and was not found in / proc/mounts
UMOUNT NAS error No space left on device and was not found in / proc/mounts
NAS_IP:/sysidc represents the storage device of the NAS.
[root@mvxl0280 /] # df-h
Filesystem Size Used Avail Use% Mounted on
/ dev/mapper/VolGroup-lv_root 22G 20g 275m 99% /
Tmpfs 7.8G 0 7.8G 0% / dev/shm
/ dev/sda1 485m 37m 423m 9% / boot
/ dev/mapper/vg_01-lv_01 98G 72G 21G 78% / eam_app
NAS_IP:/sysidc 3.0T 2.2T 743G 76% / mnt
# # the result displayed by the mount command without parameters is the same as that recorded in the / etc/mtab file.
[root@mvxl0280 /] # mount
/ dev/mapper/VolGroup-lv_root on / type ext4 (rw)
Proc on / proc type proc (rw)
Sysfs on / sys type sysfs (rw)
Devpts on / dev/pts type devpts (rw,gid=5,mode=620)
Tmpfs on / dev/shm type tmpfs (rw)
/ dev/sda1 on / boot type ext4 (rw)
/ dev/mapper/vg_01-lv_01 on / eam_app type ext4 (rw)
None on / proc/sys/fs/binfmt_misc type binfmt_misc (rw)
Sunrpc on / var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
NAS_IP:/sysidc on / mnt type nfs (rw,addr=NAS_IP)
[root@mvxl0280 /] # cat / proc/mounts
Rootfs / rootfs rw 0 0
Proc / proc proc rw,nosuid,nodev,noexec,relatime 0 0
Sysfs / sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
Devtmpfs / dev devtmpfs rw,nosuid,relatime,size=8157104k,nr_inodes=2039276,mode=755 0 0
Devpts / dev/pts devpts rw,relatime,gid=5,mode=620,ptmxmode=000 00
Tmpfs / dev/shm tmpfs rw,nosuid,nodev,relatime 0 0
/ dev/mapper/VolGroup-lv_root / ext4 rw,relatime,barrier=1,data=ordered 0 0
/ proc/bus/usb / proc/bus/usb usbfs rw,relatime 0 0
/ dev/sda1 / boot ext4 rw,relatime,barrier=1,data=ordered 0 0
/ dev/mapper/vg_01-lv_01 / eam_app ext4 rw,relatime,barrier=1,data=ordered 0 0
None / proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0
Sunrpc / var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
/ etc/auto.misc / misc autofs rw,relatime,fd=7,pgrp=2080,timeout=300,minproto=5,maxproto=5,indirect 00
-hosts / net autofs rw,relatime,fd=13,pgrp=2080,timeout=300,minproto=5,maxproto=5,indirect 00
NAS_IP:/sysidc / mnt nfs rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.16.17.171,mountvers=3,mountport=1234,mountproto=udp,local_lock=none,addr=NAS_IP 00
Umount NAS Storage Tip [No space left on device]
[root@mvxl0280 ~] # umount / mnt
Can't create lock file / etc/mtab~9811: No space left on device (use-n flag to override)
/ mnt was not found in / proc/mounts
The NAS space suddenly becomes smaller, from 3.0T to 22G. In fact, although the mount point of the / mnt is NAS "NAS_IP:/sysidc", it is actually the mount point of the "/" directory.
[root@mvxl0280] # df-h
Filesystem Size Used Avail Use% Mounted on
/ dev/mapper/VolGroup-lv_root 22G 20G 346m 99% /
Tmpfs 7.8G 0 7.8G 0% / dev/shm
/ dev/sda1 485m 37m 423m 9% / boot
/ dev/mapper/vg_01-lv_01 98G 72G 21G 78% / eam_app
NAS_IP:/sysidc 22G 20G 346m 99% / mnt
In fact, after the execution of umount, the NAS storage has actually succeeded in umount, and you can see that it is empty by looking at the / mnt directory.
[root@mvxl0280 ~] # ls-l / mnt
Total 0
It seems that the uninstall is not successful, but the umount with [- f] is executed again at this time, and the error is still reported.
[root@mvxl0280] # umount-f / mnt
/ mnt was not found in / proc/mounts
/ mnt was not found in / proc/mounts
Re-mount NAS storage also prompts [No space left on device]
[root@mvxl0280 /] # mount NAS_IP:/sysidc / mnt
Can't create lock file / etc/mtab~10608: No space left on device (use-n flag to override)
It is said on the Internet that the [- l] parameter of umount can be used, but it has no effect.
[root@mvxl0280 /] # umount-l / mnt/
Can't create lock file / etc/mtab~10512: No space left on device (use-n flag to override)
Can't create lock file / etc/mtab~10513: No space left on device (use-n flag to override)
But mount can see that the NAS storage "NAS_IP:/sysidc" has been mounted under / mnt.
[root@mvxl0280 ~] # mount
/ dev/mapper/VolGroup-lv_root on / type ext4 (rw)
Proc on / proc type proc (rw)
Sysfs on / sys type sysfs (rw)
Devpts on / dev/pts type devpts (rw,gid=5,mode=620)
Tmpfs on / dev/shm type tmpfs (rw)
/ dev/sda1 on / boot type ext4 (rw)
/ dev/mapper/vg_01-lv_01 on / eam_app type ext4 (rw)
None on / proc/sys/fs/binfmt_misc type binfmt_misc (rw)
Sunrpc on / var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
NAS_IP:/sysidc on / mnt type nfs (rw,addr=NAS_IP)
However, the mount point information can not be seen in / proc/ mounts [/ proc/self/mounts soft connection], so an error was reported [/ mnt was not found in / proc/mounts].
[root@mvxl0280 ~] # ls-l / proc/mounts
Lrwxrwxrwx 1 root root 11 Feb 5 09:06 / proc/mounts-> self/mounts
[root@mvxl0280 ~] # file / proc/mounts
/ proc/mounts: symbolic link to `self/mounts'
[root@mvxl0280 ~] # ls-l / proc/self
Lrwxrwxrwx 1 root root 64 Feb 5 08:57 / proc/self-> 9879
[root@mvxl0280 ~] # ls-l / proc/self/mounts
-root root-1 root root 0 Feb 5 09:07 / proc/self/mounts
[root@mvxl0280 ~] # more / proc/self/mounts
Rootfs / rootfs rw 0 0
Proc / proc proc rw,nosuid,nodev,noexec,relatime 0 0
Sysfs / sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
Devtmpfs / dev devtmpfs rw,nosuid,relatime,size=8157104k,nr_inodes=2039276,mode=755 0 0
Devpts / dev/pts devpts rw,relatime,gid=5,mode=620,ptmxmode=000 00
Tmpfs / dev/shm tmpfs rw,nosuid,nodev,relatime 0 0
/ dev/mapper/VolGroup-lv_root / ext4 rw,relatime,barrier=1,data=ordered 0 0
/ proc/bus/usb / proc/bus/usb usbfs rw,relatime 0 0
/ dev/sda1 / boot ext4 rw,relatime,barrier=1,data=ordered 0 0
/ dev/mapper/vg_01-lv_01 / eam_app ext4 rw,relatime,barrier=1,data=ordered 0 0
None / proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0
Sunrpc / var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
/ etc/auto.misc / misc autofs rw,relatime,fd=7,pgrp=2080,timeout=300,minproto=5,maxproto=5,indirect 00
-hosts / net autofs rw,relatime,fd=13,pgrp=2080,timeout=300,minproto=5,maxproto=5,indirect 00
After comparative analysis, it is found that the lack of space in the "/" root directory is caused by a / home/wasadmin/deploy program backup directory at most. After confirming with the developer, the program files in this directory are backed up regularly, but there is no deployment-related regular cleaning script, resulting in space being used up.
[root@mvxl0280 home] # du-sh *
60K monitor
13G wasadmin
/ home/wasadmin is the user's home directory, which uses 13 gigabytes of space, while there are only 2 regular directories under this directory. Among them, client_ffdc is only 204K. So deploy takes up more than the remaining 12G of space.
[root@mvxl0280 home] # cd wasadmin
[root@mvxl0280 wasadmin] # ls
Client_ffdc deploy
[root@mvxl0280 wasadmin] # du-sh client_ffdc/
204K client_ffdc/
After removing some of the backup programs under the / home/wasadmin/deploy directory, the "/" directory has enough space to execute the umount NAS command again and you can execute it successfully.
[wasadmin@mvxl0280] $df-h
Filesystem Size Used Avail Use% Mounted on
/ dev/mapper/VolGroup-lv_root 22G 8.9G 12G 45% /
Tmpfs 7.8G 0 7.8G 0% / dev/shm
/ dev/sda1 485m 37m 423m 9% / boot
/ dev/mapper/vg_01-lv_01 98G 68G 26G 73% / eam_app
NAS_IP:/sysidc 3.0T 2.2T 743G 76% / mnt
[root@mvxl0280 wasadmin] # umount / mnt
[root@mvxl0280 wasadmin] # df-h
Filesystem Size Used Avail Use% Mounted on
/ dev/mapper/VolGroup-lv_root 22G 8.9G 12G 45% /
Tmpfs 7.8G 0 7.8G 0% / dev/shm
/ dev/sda1 485m 37m 423m 9% / boot
/ dev/mapper/vg_01-lv_01 98G 68G 26G 73% / eam_app
