I have this / root file system 100% full problem. I've I've just installed CENTOS 7. Originally I allowed 20 GB for the root partition. However when running a PHP script that creates text files of around 10MB, the root file system started to fill up very quickly in the order of gigabytes every time I run my PHP script. After just a few runs, I had used the full 20 GB of root file system. I'm using the default CENTOS 7 xfs file system.


I identified this/proc/kcore file that is about 140TB in size. I've seen internet forums that say its just a memory map type file that does not use disk space, however if I add up all the other folders that sit off the root file system, they don't add anywhere near up to 20 GB.

OK so I reinstalled CENTOS 7 again this time with a 100 GB root partition. Yet the same problem occurred with the space of a few hours. Thus now the root file system is 96% full on a 100 GB file system.

The other aspect to this is the SE-LINUX Troubleshooter that reports various things about trying to access certain web folders and write to them etc... Thus I open up the SE-LINUX Trouble Shooter and follow the steps to make the exceptions etc...

Still however every time I run this PHP script the file system gets swamped. I know this script is fine as I've run it on Windows and other RedHat Linux platforms without an issue. Also in the short amount of time it takes to run my script, there's no way it could write tens of gigabytes.

The only other things I can add is that I use a RAM disk to run the script, however even without the RAM disk, I still had the same problem when running this script from the hard disk.

I just don't know the cause, however it's not the script itself which simply decompresses a file can creates a simple text file.

So here I am with this 100GB root partition on a newly installed CENTOS build at 96% full. If I run this script again it could tip it to 100% full and disable my system completely, as before.

Again if I select all folders ( excluding /proc ) sitting off the root file system, they total to around 13.7 GB. The CENTOS 7 "Disk Usage Analyser" application seems to indicate I'm using about 16GB for all file visible systems. /proc folder reads about 140 TB. I admit this is only a GUI inspection of file usage.

I'm looking into the du command to get an idea of disk usage from the command prompt.

# du -a -h -c


Which gives md 99GB full still, so I'm digging a bit deeper. Might also look at the GUI disk usage app.


df - h OUTPUT

[root@localhost test]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 100G 96G 4.5G 96% /
devtmpfs 7.7G 0 7.7G 0% /dev
tmpfs 7.8G 212K 7.8G 1% /dev/shm
tmpfs 7.8G 9.2M 7.7G 1% /run
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/sdb1 1014M 224M 791M 22% /boot
/dev/mapper/centos-home 300G 2.5G 298G 1% /home
tmpfs 1.6G 36K 1.6G 1% /run/user/1000
tmpfs 1.3G 865M 416M 68% /mnt/www/html

MOST BAFFLING

UPDATE! NOW at 100% full for / root file system.

Hi folks,

I found this on another site.

du -xak / >/home/sizes
sort -rn /home/sizes >/home/sorted
less /home/sorted


This pointed me to /var/log/httpd/error_log which was consuming near 100 GB of disk space.

I did an ls-al in there and there is a massive error log file. error_log.

[root@localhost httpd]# ls -al
total 97698712
drwx------. 2 root root 102 Jul 23 04:58 .
drwxr-xr-x. 22 root root 4096 Jul 23 15:10 ..
-rw-r--r--. 1 root root 59896 Jul 23 15:07 access_log
-rw-r--r--. 1 root root 100043407360 Jul 23 15:09 error_log ( MY NOTE: ERROR LOG FILE WAY TOO BIG !!! LEFT ROOT PARTITION 100% FULL ! )
-rw-r--r--. 1 root root 0 Jul 23 04:58 ssl_access_log
-rw-r--r--. 1 root root 3168 Jul 23 13:18 ssl_error_log
-rw-r--r--. 1 root root 0 Jul 23 04:58 ssl_request_log
[root@localhost httpd]#

I'm about to look to see what's inside error_log.

OK there are a whole lot of PHP syntax errors being logged with this version of PHP 5.4. At this stage I've just switched error logging OFF in PHP until I can find time to go back over the code and get things in order. I will probably re-write the code at some stage. Many errors related to variables not being defined prior to use.



QUICK SOLUTION

I've finally managed to delete this /var/log/httpd/error_log file by booting to Linux RESCUE mode.

Then in /etc/php.ini I added the mod;

CHANGED;
log_errors = On

TO;

log_errors = Off

Then restarted the httpd service

This has SOLVED the problem temporarily. If I ever need to log PHP errors, I will need to re-visit this issue and get the PHP source code in proper syntax order. Otherwise possible use NON-STRICT syntax checking for the existing PHP code base, until it's been re-written. i.e. variables not defined errors etc... The /proc folder being 1.4 TB was just an associated observation, that had no impact on this issue at all.

+1 on the detective work.
100GB in indeed huge. logrotate not operate on that host?

Look at /etc/logrotate.d/httpd contents.
I can't tell you what's 'correct' because I have a legacy appliance,
logrotate may in fact be writing, but I gotta say again, 100GB is Huge.

Any hits on
?