[SOLVED] [EMERGENCY] How to free up space on / partition, Centos 7
Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
[EMERGENCY] How to free up space on / partition, Centos 7
I have a / partition of 20GB in size and I am at the point where I am not able to work on fixing a problem that I have because I have run out of space. What are the things that can be deleted. This is a situation where every half of a MB counts...
Thanks.
Last edited by AdultFoundry; 12-18-2016 at 05:07 PM.
This is an EMERGENCY. I am ready to start working on the problem after two days of research, but at this point I am not able to use mysqldump as it gives me a message that the disk is full. I also get messages that the databases are corrupted because of that. I freed up some space and I had 40MB free and 10 seconds from this point, it was showing 0 again. I cant install something very small like "yum install yum-utils", it would not install that as I dont have enough disk space. I cant run something like "yum upgrade" - not enough disk space (with the last two that I mentioned I was trying to delete the old kernels, but they may not be there for deletion anyway).
So basically I am stuck - I cant do nothing because the disk is full or it fills up very fast. This is the problem of /var/lib/mysql/ibdata1 getting large, and for some reason growing very fast (like 1.1G per 24 hours for the last three days).
I stopped the httpd and none of the websites work, and I could have like 1,000 daily traffic. I am turning off the pc now and I am going to sleep as I am too tired for working on this. I will come back tomorrow in the morning, and it would be great if I could get some free space on there. I am not sure what can be deleted. Most of the things are probably done by now, but it did not make any difference... This is at the point when one thing starts messing up other things, and it may be getting worse and worse. If I get the free space for some time I should be able to fix the hosting plan.
related to the DB on /
20 gig for / is about normal
are you using a LVM ?????
the default setting for installing CentOS and the best option for a server
or
did you manually set up partitions
( NOT a good option for a server)
first WHAT is eating up space???
the error logs ?
the database it's self ?
what
how was the database installed ?
is it the version in the cent repos ? or a different one ?
Not for ANYONE HERE it isn't....we volunteer our time...this is not a help desk where you can demand/expect 'emergency' help.
Quote:
I am ready to start working on the problem after two days of research, but at this point I am not able to use mysqldump as it gives me a message that the disk is full.
So dump to another disk/partition...what's the problem? Specify a different file name
Quote:
I also get messages that the databases are corrupted because of that. I freed up some space and I had 40MB free and 10 seconds from this point, it was showing 0 again. I cant install something very small like "yum install yum-utils", it would not install that as I dont have enough disk space. I cant run something like "yum upgrade" - not enough disk space (with the last two that I mentioned I was trying to delete the old kernels, but they may not be there for deletion anyway).
Right...because you probably have a full /var or /log (or both) partition. Check your log files first...null them out. If necessary, pull the network connection.
Quote:
So basically I am stuck - I cant do nothing because the disk is full or it fills up very fast. This is the problem of /var/lib/mysql/ibdata1 getting large, and for some reason growing very fast (like 1.1G per 24 hours for the last three days).
..which may be totally normal, depending on your server/needs/users/access.
Quote:
I stopped the httpd and none of the websites work, and I could have like 1,000 daily traffic. I am turning off the pc now and I am going to sleep as I am too tired for working on this. I will come back tomorrow in the morning, and it would be great if I could get some free space on there. I am not sure what can be deleted. Most of the things are probably done by now, but it did not make any difference... This is at the point when one thing starts messing up other things, and it may be getting worse and worse. If I get the free space for some time I should be able to fix the hosting plan.
Too tired?? Must not be TOO much of an 'emergency' if you can just turn it off and go to bed.
The issue is fixed, as far as this part. I moved the entire /var/lib/mysql to /home/mysql and I got it working from there, so there is no issue with no disk space on the /. If somebody will be reading this, there may be a need of adding the new location of the /var/lib/mysql/mysql.sock to php.ini files (two separate locations there in Centos 7 that I use). I read several articles about the moving on the Net and it has not been working until I made this change.
I am not able to use mysqldump as it gives me a message that the disk is full.
Dump from remote?
Code:
# mysqldump -h <host> -u<user> -p <pass> db > file.sql
from your local machine?
will dump it locally, provided you have grant access in the db, or use root privs, provided that hasn't been secured.
This leaves you with only the partitioning issue.
Sticking mysql on /home is temporary, right?
This is already resolved. I am actually working on the dumps and I have some issues with these. I am able to fix it with mysqlcheck and myisamchk so far. Basically the issue was /var/lib/mysql/ibdata1 (and possibly other files there) growing too large. I have something like partition A - / and partition B (lvm) - /home. I was thinking about shrinking the B and adding it to the A at /var/lib/mysql but this may not be the best way to go. If anybody experiences an issue like this, I recommend moving the whole /var/lib/mysql to something like /home/mysql (in my situation) where you have more disk space. Shrinking LVM volumes may be a risky thing even if all the commands are good, for this reason a backup is needed. It took me full two days to prepare for everything, and I could have solved everything in less than an hour or so with just moving the /var/lib/mysql and this is what I ended up doing anyway. I could have mount 100GB on the old /var/lib/mysql, this could run out of space again, and I would need to come back to it again. What I did should be a better option in the long run anyway. On Centos 7, the new location of this folder needs to be added to /etc/php.ini file. This is not described in most of the articles on the Internet. It was not working for me until I've done that (httpd error_log was showing that the website scripts were not able to connect to the database).
Thanks for sharing your fix for the issue. I would mention that Centos 7 uses XFS and it is NOT possible at this point to shrink an XFS LVM filesystem that I am aware of.
Shrinking LVM volumes may be a risky thing even if all the commands are good, for this reason a backup is needed. It took me full two days to prepare for everything, and I could have solved everything in less than an hour or so with just moving the /var/lib/mysql and this is what I ended up doing anyway.
@Habitual - I am prepared and able to move the database and enlarge the /var/lib/mysql in the original location if this will be a needed thing. It is more important for me to work on a website, instead of doing something that could be good but it is not a necessary thing. I have a full understanding of the situation and I can come back to it this would be a needed thing. I back up to Windows 7 with Cygwin, once per month or less often. I know that this is not the best but better than spending another $42 per month for another hosting plan that I would need. If I will be able to expand what I am working on now, I will set up a Linux back up machine and the whole method and the system will be more advanced. I am able and capable of setting up anything that is needed with the necessary research and work. For now this is good enough.
Why moving /var/lib/mysql to /home/mysql is bad, what are the reasons? I am asking because I dont have the knowledge about that.
@szboardstretcher - I've been reading that Centos 7 uses XFS, but ext4 is what I have there.
Last edited by AdultFoundry; 12-20-2016 at 10:56 AM.
I have a zabbix db and it blew up to like 100G and, Oh Hells Noe!
Historical data, yer fired.
export to file.sql
InnnoDB the tables
restart dbserver
import file.sql
and "All of a sudden" we're at 10G...
Warning: /var/lib/mysql/ibdata1 always grows, never shrinks.
Whatever you do, or don't, burn that into your noodle.
I'll ask again. Is the "move" to /home/mysql temporary, or are you going to research implementing LVMs in the near future?
I hope that you'd not relax since the "Emergency" is over thanks to the work you did.
I guess your "good enough" statement bothered me.
I will get over it.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.