“…inode table of its ramdisk (root) is full”
by Kwong
So last week a co-worker had an issue with powering up a VM that had died and decided to pull me in to fix the problem. When I tried to vMotion the VM out to another ESXi host, it failed as well. The error was something like “WARNING: VisorFSObj: 1954: Cannot create file /var/run/vmware/tickets/vmtck-XXXXXXX for process hostd-worker because the inode table of its ramdisk (root) is full.”
Note that we are using ESXi 5.1 in our environment. After some googling, I found this article published on 3/11/2013 which describes the problem and a workaround for this.
Note: I strongly recommend reading the official KB to get a deeper understanding of the issue as the below is simply a short summary of it.
Basically, the issue is caused by SNMP trap files filling up in /var/spool/snmp, causing the ESXi host to run out of inodes.
In summary:
To verify the issue is caused by full SNMP trap files:
1. SSH to the host
2. Run ls /var/spool/snmp | wc -l
3. If output exceeds 2000, it is good indication that this is the cause of full inodes.
To delete .trp files and free up inodes
1. cd /var/spool/snmp
2. for i in *.trp; do rm -rf $i; done
Backup and recreate snmp.xml
1. cd /etc/vmware
2. mv snmp.xml snmp.xml.bkup
3. Create a new file called snmp.xml and paste the following in:
<?xml version="1.0" encoding="ISO-8859-1"?>
<config>
<snmpSettings><enable>false</enable><port>161</port><syscontact></syscontact><syslocation></syslocation>
<EnvEventSource>indications</EnvEventSource><communities></communities><loglevel>info</loglevel><authProtocol></authProtocol><privProtocol></privProtocol></snmpSettings>
</config>
Reconfigure SNMP
1. esxcli system snmp set –-enable=true
2. esxcli system snmp get
Again, for more details on this issue I strongly suggest reading up more from the KB article. As far as I know, there is still no permanent fix for this. Hopefully we will see a patch in the near future for this issue.
Disclaimer: The above information are provided for informational/educational purposes only.
stuff about stuff 