General guidelines for preventing inode full issue
 

OS : RHEL 6.5

Last week , one of our main applcation was down for nearly an hour because of inode getting full for /app filesystem.
This filesystem was 70GB in size and it was only 40GB utilized (30GB free). But, because of high number of log files and trace files , the inode has reached it max for this filesystem. We had to delete lots of these unwanted files to fix this.

Two questions :

1 . What will you do if the inode has maxed out for a filesystem and deleting small files does not fix the issue ?

2. Any guidelines to prevent inode getting full issue ? My colleague was saying that if you create the filesystem with higher block size , the inode maxing out issue can be prevented. Is this true?

Googling didn't help much . Hence posting it here.

This is not a very competent opinion, but I believe you have two options:

• If you can extend the filesystem, do that. E.g. it resides on a logical volume, and there is still free space in the volume group. This is a quick fix but wasteful.
• If not, backup, recreate (see point 2) and restore. Or simply copy it to a different filesystem with more inodes (again, point 2)

If this is an ext3 or ext4 filesystem, use the -i option when creating it. See man page. This option allows you to create the right number of inodes if you know the average file size.

Realistically, you need to find a better thing to do with all those log and trace files. For instance, you could put them into a database table . . .

Thank You Bernd.

If you can extend the filesystem, do that. E.g. it resides on a logical volume, and there is still free space in the volume group. This is a quick fix but wasteful.

You mean, number of inodes value will increase dynamically when the filesystem size is increased (LVM) ?

True. But this will have counter effect to the usable space of the filesystem. Greater inode size can contain more metadata, but number of inodes should compliment the number of blocks. I don't know how mission-critical is your data, but if I were in that case, after back-up, I shall try to neutralize or reduce journal sizes.

That's actually a good question. I can't confirm that right now. Perhaps somebody else can.
After increasing the underlying volume, you will also have to resize your filesystem with the resize2fs command (if it's ext[234]).