[EMERGENCY] How to free up space on / partition, Centos 7
 

I have a / partition of 20GB in size and I am at the point where I am not able to work on fixing a problem that I have because I have run out of space. What are the things that can be deleted. This is a situation where every half of a MB counts...

Thanks.

This is an EMERGENCY. I am ready to start working on the problem after two days of research, but at this point I am not able to use mysqldump as it gives me a message that the disk is full. I also get messages that the databases are corrupted because of that. I freed up some space and I had 40MB free and 10 seconds from this point, it was showing 0 again. I cant install something very small like "yum install yum-utils", it would not install that as I dont have enough disk space. I cant run something like "yum upgrade" - not enough disk space (with the last two that I mentioned I was trying to delete the old kernels, but they may not be there for deletion anyway).

So basically I am stuck - I cant do nothing because the disk is full or it fills up very fast. This is the problem of /var/lib/mysql/ibdata1 getting large, and for some reason growing very fast (like 1.1G per 24 hours for the last three days).

I stopped the httpd and none of the websites work, and I could have like 1,000 daily traffic. I am turning off the pc now and I am going to sleep as I am too tired for working on this. I will come back tomorrow in the morning, and it would be great if I could get some free space on there. I am not sure what can be deleted. Most of the things are probably done by now, but it did not make any difference... This is at the point when one thing starts messing up other things, and it may be getting worse and worse. If I get the free space for some time I should be able to fix the hosting plan.

i am guessing that this is related to your OTHER post on this
http://www.linuxquestions.org/questi...on-4175595653/

related to the DB on /
20 gig for / is about normal

are you using a LVM ?????
the default setting for installing CentOS and the best option for a server
or
did you manually set up partitions
( NOT a good option for a server)


first WHAT is eating up space???
the error logs ?
the database it's self ?
what


how was the database installed ?
is it the version in the cent repos ? or a different one ?



also please read the moderators post on your other thread
post #6
http://www.linuxquestions.org/questi...3/#post5643681

Not for ANYONE HERE it isn't....we volunteer our time...this is not a help desk where you can demand/expect 'emergency' help.
So dump to another disk/partition...what's the problem? Specify a different file name
Right...because you probably have a full /var or /log (or both) partition. Check your log files first...null them out. If necessary, pull the network connection.
..which may be totally normal, depending on your server/needs/users/access.
Too tired?? Must not be TOO much of an 'emergency' if you can just turn it off and go to bed.

So find the thing that's causing your database to fill up and stop that?
Most likely it's some error repeating as fast as it can.

list the software you're using?
list the programs you're using that have database access?

We get you're frustrated. Can't do much if we don't know much beyond that though.

You might have some really big log files. You can use the 'find' command to look for files over a certain size.

The issue is fixed, as far as this part. I moved the entire /var/lib/mysql to /home/mysql and I got it working from there, so there is no issue with no disk space on the /. If somebody will be reading this, there may be a need of adding the new location of the /var/lib/mysql/mysql.sock to php.ini files (two separate locations there in Centos 7 that I use). I read several articles about the moving on the Net and it has not been working until I made this change.

Dump from remote?
from your local machine?
will dump it locally, provided you have grant access in the db, or use root privs, provided that hasn't been secured.
This leaves you with only the partitioning issue.
Sticking mysql on /home is temporary, right?

This is already resolved. I am actually working on the dumps and I have some issues with these. I am able to fix it with mysqlcheck and myisamchk so far. Basically the issue was /var/lib/mysql/ibdata1 (and possibly other files there) growing too large. I have something like partition A - / and partition B (lvm) - /home. I was thinking about shrinking the B and adding it to the A at /var/lib/mysql but this may not be the best way to go. If anybody experiences an issue like this, I recommend moving the whole /var/lib/mysql to something like /home/mysql (in my situation) where you have more disk space. Shrinking LVM volumes may be a risky thing even if all the commands are good, for this reason a backup is needed. It took me full two days to prepare for everything, and I could have solved everything in less than an hour or so with just moving the /var/lib/mysql and this is what I ended up doing anyway. I could have mount 100GB on the old /var/lib/mysql, this could run out of space again, and I would need to come back to it again. What I did should be a better option in the long run anyway. On Centos 7, the new location of this folder needs to be added to /etc/php.ini file. This is not described in most of the articles on the Internet. It was not working for me until I've done that (httpd error_log was showing that the website scripts were not able to connect to the database).

Good for you.
Repeat yourself some more.
I'm kind of tired asking and not getting any answers, so I guess we'll see you on the next "EMERGENCY"?

I suspect your google.fu doesn't work too well either as https://www.google.com/search?q=ibdata1%20too%20large clearly talks about how to address the topic.

And you're not asking the right question, IMO,
Why does ibdata1 grow so large?

You're welcome.

Thanks for sharing your fix for the issue. I would mention that Centos 7 uses XFS and it is NOT possible at this point to shrink an XFS LVM filesystem that I am aware of.

You should either now be running LVM, or you should start using it at once.

Were backups in the 2 days you "prepared"?
Simple, Secure Backups for Linux with rsync

LVMs is recommended.
No backups is on you.
Moving to /home/mysql is a hack IMO, and calling it "solved" is going to bite you on the ass.

I told you in the beginning this wasn't going to be "easy".

But no backups? shame.

@Habitual - I am prepared and able to move the database and enlarge the /var/lib/mysql in the original location if this will be a needed thing. It is more important for me to work on a website, instead of doing something that could be good but it is not a necessary thing. I have a full understanding of the situation and I can come back to it this would be a needed thing. I back up to Windows 7 with Cygwin, once per month or less often. I know that this is not the best but better than spending another $42 per month for another hosting plan that I would need. If I will be able to expand what I am working on now, I will set up a Linux back up machine and the whole method and the system will be more advanced. I am able and capable of setting up anything that is needed with the necessary research and work. For now this is good enough.

Why moving /var/lib/mysql to /home/mysql is bad, what are the reasons? I am asking because I dont have the knowledge about that.

@szboardstretcher - I've been reading that Centos 7 uses XFS, but ext4 is what I have there.

I didn't say 'bad' but I do consider it the "second best" suggestion.
So while you're keeping an eye on /home/mysql/ibdata1
Have a look at https://www.centos.org/docs/5/html/C...ger/index.html

I have a zabbix db and it blew up to like 100G and, Oh Hells Noe!
Historical data, yer fired.
export to file.sql
InnnoDB the tables
restart dbserver
import file.sql
and "All of a sudden" we're at 10G...

Whatever you do, or don't, burn that into your noodle.

I'll ask again. Is the "move" to /home/mysql temporary, or are you going to research implementing LVMs in the near future?
I hope that you'd not relax since the "Emergency" is over thanks to the work you did.

I guess your "good enough" statement bothered me.
I will get over it.

Good Job btw moving mysql.