Operational Defect Database
BugZero updated this defect 53 days ago.
Data sources
All data on this page is proprietary to BugZero® or gathered from public sources
VMware | 96990
Utility to list the NFS v3 lock files and their contents in human readable format.
Last update date:
3/27/2024
Affected products:
vSphere ESXi
Affected releases:
8.0
Fixed releases:
No fixed releases provided.
Description:
Symptoms
When one connects an ESX host to a supported NFSv3 server, ESX does not use the traditional Network Lock Manager (NLM) protocol to handle NFS locking. Instead, we create a .lck file for each associated file and populate said file with the hostname in plain text, and also indirectly points to the inode of the file on the NFS export.
Purpose
The purpose of this article is to help you with the usage of the script that provides details about the .lck file in user-readable format.
Cause
There can be times when you are unsure which lock file belongs to which file. A good example of this would be a shared swap directory which may contain multiple swap files from multiple VMs.
Impact / Risks
Manually deleting .lck files should never be done because it can cause random data corruption which may not be immediately visible, or, under the express instruction of a VMware Technical Support Engineer.
Resolution
A script has been added to quickly obtain the .lck file details that are part of NFSv3 datastore(s). The article explains about the different use cases of the script.Use case 1 - List details of all .lck files part of a VM1. SSH into ESXi host.2. Find out the appropriate VM directory.3. For example, the VM is located at /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/ 4. The VM directory contains the following files.[root@esx-host:~] ls /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/ -laitotal 364100751808 drwxr-xr-x 3 root root 4096 Dec 28 10:13 .67157059 drwxr-xr-x 3 root root 25 Dec 28 10:13 ..100772701 -rwxrwxr-x 1 root root 92 Dec 28 10:28 .lck-5cab010600000000100772705 -rwxrwxr-x 1 root root 92 Dec 28 10:28 .lck-5eab010600000000100772704 -rwxrwxr-x 1 root root 92 Dec 28 10:28 .lck-cc59010600000000100751871 -rwxrwxr-x 1 root root 92 Dec 28 10:28 .lck-fe59010600000000100772702 -rw------- 1 root root 33554432 Dec 28 10:13 Ubuntu-test-4c650d76.vswp100751820 -rw------- 1 root root 1048576 Dec 28 10:13 Ubuntu-test-flat.vmdk100772711 -rw------- 1 root root 8684 Dec 28 10:13 Ubuntu-test.nvram100752055 -rw-r--r-- 1 root root 7386 Dec 28 10:13 Ubuntu-test.scoreboard100751860 -rw------- 1 root root 443 Dec 28 10:13 Ubuntu-test.vmdk100751862 -rw-r--r-- 1 root root 0 Dec 28 10:13 Ubuntu-test.vmsd100772712 -rwxr-xr-x 1 root root 1776 Dec 28 10:13 Ubuntu-test.vmx100772700 -rw------- 1 root root 0 Dec 28 10:13 Ubuntu-test.vmx.lck 415014 drwxr-xr-x 2 root root 94 Dec 28 10:13 stats100772703 -rw-r--r-- 1 root root 324993 Dec 28 10:28 vmware.log100751870 -rw------- 1 root root 83886080 Dec 28 10:13 vmx-Ubuntu-test-0be6d08851c29d9cabc7ef29fdb1eb42f6924a6cdd1068719fc49b90d29e0516-1.vswp 5. To list details of all the four .lck files above, the following command can be issued. Please note that the script only takes absolute file path with all links resolved, as input.[root@esx-host:~] python /usr/lib/vmware/vm-support/bin/nfsLockInfo.pyc /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/VM Dir : /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/Local Host : localhost Lock number: 1Lock File : .lck-fe59010600000000Inode : 0x60159fe (100751870)Locked file: vmx-Ubuntu-test-0be6d08851c29d9cabc7ef29fdb1eb42f6924a6cdd1068719fc49b90d29e0516-1.vswpCounter : 37Version : 0xabcd0123LockType : 1 (EXCLUSIVE)Host Name : localhostHost UUID : 658a91df-5ea9-35d7-40b7-02012bfd98d0Non-Shared : 0xffLock Dump :===============================================0000000 2500 0000 0000 0000 2301 cdab ffff ffff0000010 0100 0000 6c6f 6361 6c68 6f73 7400 00000000020 0000 0000 0000 0000 0000 0000 0000 00000000030 0000 0000 658a 91df 5ea9 35d7 40b7 02010000040 2bfd 98d0 0000 0000 0000 0000 0000 00000000050 0000 00ff 0000 0000 0000 0000=============================================== Lock number: 2Lock File : .lck-5cab010600000000Inode : 0x601ab5c (100772700)Locked file: Ubuntu-test.vmx.lckCounter : 36Version : 0xabcd0123LockType : 1 (EXCLUSIVE)Host Name : localhostHost UUID : 658a91df-5ea9-35d7-40b7-02012bfd98d0Non-Shared : 0xffLock Dump :===============================================0000000 2400 0000 0000 0000 2301 cdab ffff ffff0000010 0100 0000 6c6f 6361 6c68 6f73 7400 00000000020 0000 0000 0000 0000 0000 0000 0000 00000000030 0000 0000 658a 91df 5ea9 35d7 40b7 02010000040 2bfd 98d0 0000 0000 0000 0000 0000 00000000050 0000 00ff 0000 0000 0000 0000=============================================== Lock number: 3Lock File : .lck-5eab010600000000Inode : 0x601ab5e (100772702)Locked file: Ubuntu-test-4c650d76.vswpCounter : 35Version : 0xabcd0123LockType : 1 (EXCLUSIVE)Host Name : localhostHost UUID : 658a91df-5ea9-35d7-40b7-02012bfd98d0Non-Shared : 0xffLock Dump :===============================================0000000 2300 0000 0000 0000 2301 cdab ffff ffff0000010 0100 0000 6c6f 6361 6c68 6f73 7400 00000000020 0000 0000 0000 0000 0000 0000 0000 00000000030 0000 0000 658a 91df 5ea9 35d7 40b7 02010000040 2bfd 98d0 0000 0000 0000 0000 0000 00000000050 0000 00ff 0000 0000 0000 0000=============================================== Lock number: 4Lock File : .lck-cc59010600000000Inode : 0x60159cc (100751820)Locked file: Ubuntu-test-flat.vmdkCounter : 34Version : 0xabcd0123LockType : 1 (EXCLUSIVE)Host Name : localhostHost UUID : 658a91df-5ea9-35d7-40b7-02012bfd98d0Non-Shared : 0xffLock Dump :===============================================0000000 2200 0000 0000 0000 2301 cdab ffff ffff0000010 0100 0000 6c6f 6361 6c68 6f73 7400 00000000020 0000 0000 0000 0000 0000 0000 0000 00000000030 0000 0000 658a 91df 5ea9 35d7 40b7 02010000040 2bfd 98d0 0000 0000 0000 0000 0000 00000000050 0000 00ff 0000 0000 0000 0000===============================================6. The script outputs all fields of the .lck file in user readable format, along with hexdump of the contents for verification (if needed). Use case 2 - Find the .lck file of a given parent file1. SSH into ESXi host2. Find out the appropriate VM file3. For example, the VM file is /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/Ubuntu-test-flat.vmdk 4. Use the following command to get the .lck file corresponding to Ubuntu-test-flat.vmdk file. Please note that the script only takes absolute file path with all links resolved, as input.[root@esx-host:~] python /usr/lib/vmware/vm-support/bin/nfsLockInfo.pyc /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/Ubuntu-test-flat.vmdkVM Dir : /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-testLocal Host : localhost Lock number: 1Lock File : .lck-cc59010600000000Inode : 0x60159cc (100751820)Locked file: Ubuntu-test-flat.vmdkCounter : 50Version : 0xabcd0123LockType : 1 (EXCLUSIVE)Host Name : localhostHost UUID : 658a91df-5ea9-35d7-40b7-02012bfd98d0Non-Shared : 0xffLock Dump :===============================================0000000 3200 0000 0000 0000 2301 cdab ffff ffff0000010 0100 0000 6c6f 6361 6c68 6f73 7400 00000000020 0000 0000 0000 0000 0000 0000 0000 00000000030 0000 0000 658a 91df 5ea9 35d7 40b7 02010000040 2bfd 98d0 0000 0000 0000 0000 0000 00000000050 0000 00ff 0000 0000 0000 0000===============================================[root@esx-host:~] python /usr/lib/vmware/vm-support/bin/nfsLockInfo.pyc /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/Ubuntu-test-flat.vmdkVM Dir : /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-testLocal Host : localhost Lock number: 1Lock File : .lck-cc59010600000000Inode : 0x60159cc (100751820)Locked file: Ubuntu-test-flat.vmdkCounter : 50Version : 0xabcd0123LockType : 1 (EXCLUSIVE)Host Name : localhostHost UUID : 658a91df-5ea9-35d7-40b7-02012bfd98d0Non-Shared : 0xffLock Dump :===============================================0000000 3200 0000 0000 0000 2301 cdab ffff ffff0000010 0100 0000 6c6f 6361 6c68 6f73 7400 00000000020 0000 0000 0000 0000 0000 0000 0000 00000000030 0000 0000 658a 91df 5ea9 35d7 40b7 02010000040 2bfd 98d0 0000 0000 0000 0000 0000 00000000050 0000 00ff 0000 0000 0000 0000===============================================
