LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 01-14-2016, 06:26 AM   #1
kathirvel
Member
 
Registered: Jan 2011
Location: Bangalore
Distribution: RHEL,OEL
Posts: 60

Rep: Reputation: 0
/var filesystem is full but unable to find what is being used.


Hi All,

I have /var mounted on system and it is occupied by 92% but unable to find what is used...

DF output:


root@segotl1712# df -Ph | grep /var
/dev/mapper/rootvg-varvol 5.8G 5.0G 501M 92% /var
/dev/mapper/postgres_vg01-soepostgresql 4.8G 12M 4.6G 1% /var/log/soepostgresql
/dev/mapper/postgres_vg01-pgsql_misc 9.8G 23M 9.2G 1% /var/lib/pgsql/misc
/dev/mapper/postgres_vg02-pgsql_xlog 9.8G 599M 8.7G 7% /var/lib/pgsql/xlog
/dev/mapper/postgres_vg03-pgsql_data 197G 5.6G 190G 3% /var/lib/pgsql/data
segote8813-07:/javad_isfsb_1000_q1001n 1.4T 695G 706G 50% /var/lib/pgsql/archive
root@segotl1712#


-------------------

DU ouput.

root@segotl1712# du -sh .* *
845G .
du: cannot access `../proc/23261/task/23261/fd/4': No such file or directory
du: cannot access `../proc/23261/task/23261/fdinfo/4': No such file or directory
du: cannot access `../proc/23261/fd/4': No such file or directory
du: cannot access `../proc/23261/fdinfo/4': No such file or directory
848G ..
4.0K account
257M cache
4.0K crash
4.0K cvs
1.4M db
8.0K empty
4.0K games
845G lib
4.0K local
24K lock
76M log
16K lost+found
0 mail
4.0K nis
4.0K opt
4.0K preserve
232K run
4.2M spool
144K tmp
4.0K yp
root@segotl1712#


==============================

Can anyone help on this issue.

Thanks in advance..


Regards,
Kathirvel.S
 
Old 01-14-2016, 06:37 AM   #2
descendant_command
Senior Member
 
Registered: Mar 2012
Posts: 1,720

Rep: Reputation: 555Reputation: 555Reputation: 555Reputation: 555Reputation: 555Reputation: 555
Well it's only 5.8G.
It's probably under your mountpoints.

Last edited by descendant_command; 01-14-2016 at 06:41 AM.
 
Old 01-14-2016, 06:44 AM   #3
kathirvel
Member
 
Registered: Jan 2011
Location: Bangalore
Distribution: RHEL,OEL
Posts: 60

Original Poster
Rep: Reputation: 0
Could you please elaborate and let me know how to fix this issue?.

Thanks.
 
Old 01-14-2016, 06:55 AM   #4
descendant_command
Senior Member
 
Registered: Mar 2012
Posts: 1,720

Rep: Reputation: 555Reputation: 555Reputation: 555Reputation: 555Reputation: 555Reputation: 555
Unmount these:
Code:
/dev/mapper/postgres_vg01-soepostgresql 4.8G 12M 4.6G 1% /var/log/soepostgresql
/dev/mapper/postgres_vg01-pgsql_misc 9.8G 23M 9.2G 1% /var/lib/pgsql/misc
/dev/mapper/postgres_vg02-pgsql_xlog 9.8G 599M 8.7G 7% /var/lib/pgsql/xlog
/dev/mapper/postgres_vg03-pgsql_data 197G 5.6G 190G 3% /var/lib/pgsql/data
segote8813-07:/javad_isfsb_1000_q1001n 1.4T 695G 706G 50% /var/lib/pgsql/archive
Look in the mountpoints.
 
Old 01-14-2016, 07:16 AM   #5
kathirvel
Member
 
Registered: Jan 2011
Location: Bangalore
Distribution: RHEL,OEL
Posts: 60

Original Poster
Rep: Reputation: 0
It is a production server.. Is there any other way to do without unmounting.
 
Old 01-14-2016, 07:17 AM   #6
DaTore
LQ Newbie
 
Registered: Jan 2016
Location: Gothenburg, Sweden
Distribution: Centos,RHEL,openSuSE,Ubuntu
Posts: 6

Rep: Reputation: Disabled
Quote:
Originally Posted by kathirvel View Post
It is a production server.. Is there any other way to do without unmounting.
Check /lib, it says it has 845G !

Ok, just saw it /var that was full.
It seems you have /var on a separate lvm VolumeGroup: /dev/mapper/rootvg-varvol
The other ones seems like a different VG.
So don't grep for /var in the df command.

Last edited by DaTore; 01-14-2016 at 07:25 AM. Reason: correction
 
Old 01-14-2016, 07:24 AM   #7
descendant_command
Senior Member
 
Registered: Mar 2012
Posts: 1,720

Rep: Reputation: 555Reputation: 555Reputation: 555Reputation: 555Reputation: 555Reputation: 555
Quote:
Originally Posted by kathirvel View Post
It is a production server.. Is there any other way to do without unmounting.
Probably (no clue about lvm) you can mount /dev/mapper/rootvg-varvol somewhere else (as well as where it is already mounted) and get access to the "base" filesystem.
 
Old 01-14-2016, 07:42 AM   #8
DaTore
LQ Newbie
 
Registered: Jan 2016
Location: Gothenburg, Sweden
Distribution: Centos,RHEL,openSuSE,Ubuntu
Posts: 6

Rep: Reputation: Disabled
Quote:
Originally Posted by descendant_command View Post
Probably (no clue about lvm) you can mount /dev/mapper/rootvg-varvol somewhere else (as well as where it is already mounted) and get access to the "base" filesystem.
Well, here rootvg is the Volume Group, to this one you add disks (or more correct partitions)
Inside it youe here have the Logical Volume varvol.
The space dedicated to this varvol is not same as the other Volume Groups.
postgres_vg0x onthe other hand is a different Volume Group using other disks or partitions, not same space.

So look in /var only for used space.
You can drill down like this:
du -s /var/*
du -s /var/dir-with-most-bytes/*

and so on

and do NOT unmount mounpoints for your postgres production database ! It have it's own disk and as you see plenty of room.

Last edited by DaTore; 01-14-2016 at 08:07 AM. Reason: added warning
 
Old 01-14-2016, 08:14 AM   #9
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 13,890

Rep: Reputation: 4487Reputation: 4487Reputation: 4487Reputation: 4487Reputation: 4487Reputation: 4487Reputation: 4487Reputation: 4487Reputation: 4487Reputation: 4487Reputation: 4487
there can be inodes without filenames (for example still opened but already removed logfiles in /var/log). That will not be taken into account by du.
 
Old 01-14-2016, 08:48 AM   #10
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 4,392

Rep: Reputation: 2014Reputation: 2014Reputation: 2014Reputation: 2014Reputation: 2014Reputation: 2014Reputation: 2014Reputation: 2014Reputation: 2014Reputation: 2014Reputation: 2014
To look for large unlinked files:
Code:
lsof /var | grep -i del | less
Scroll through the list and look for anything huge.

To look for files hidden under your active mount points:
Code:
mkdir /tmp/tmpmnt
mount --bind /var /tmp/tmpmnt
du -s /tmp/tmpmnt/log/soepostgresql
du --max-depth=1 /tmp/tmpmnt/lib/pgsql
 
1 members found this post helpful.
Old 01-25-2016, 01:48 PM   #11
kathirvel
Member
 
Registered: Jan 2011
Location: Bangalore
Distribution: RHEL,OEL
Posts: 60

Original Poster
Rep: Reputation: 0
This worked well...Thanks.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Root filesystem is full so unable to mount the second device smilemukul Linux - Newbie 4 03-09-2012 04:53 AM
unable to find a udf filesystem patch, trying to read dvd rom sirius57 SUSE / openSUSE 2 03-13-2009 09:52 PM
hfs: unable to find HFS+ superblock VFS: Can't find ext3 filesystem on dev sdb. macroron Linux - Hardware 3 11-20-2006 09:50 PM
CobaltRAQ - /var is full thus unable to su to root brownmil Linux - Software 0 01-28-2006 05:25 PM
unable to write to var "saying full" but not back_jack Linux - General 1 09-13-2003 09:13 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 05:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration