LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-18-2016, 04:55 PM   #1
AdultFoundry
Member
 
Registered: Jun 2015
Posts: 282

Rep: Reputation: Disabled
[EMERGENCY] How to free up space on / partition, Centos 7


I have a / partition of 20GB in size and I am at the point where I am not able to work on fixing a problem that I have because I have run out of space. What are the things that can be deleted. This is a situation where every half of a MB counts...

Thanks.

Last edited by AdultFoundry; 12-18-2016 at 05:07 PM.
 
Old 12-18-2016, 05:07 PM   #2
AdultFoundry
Member
 
Registered: Jun 2015
Posts: 282

Original Poster
Rep: Reputation: Disabled
This is an EMERGENCY. I am ready to start working on the problem after two days of research, but at this point I am not able to use mysqldump as it gives me a message that the disk is full. I also get messages that the databases are corrupted because of that. I freed up some space and I had 40MB free and 10 seconds from this point, it was showing 0 again. I cant install something very small like "yum install yum-utils", it would not install that as I dont have enough disk space. I cant run something like "yum upgrade" - not enough disk space (with the last two that I mentioned I was trying to delete the old kernels, but they may not be there for deletion anyway).

So basically I am stuck - I cant do nothing because the disk is full or it fills up very fast. This is the problem of /var/lib/mysql/ibdata1 getting large, and for some reason growing very fast (like 1.1G per 24 hours for the last three days).

I stopped the httpd and none of the websites work, and I could have like 1,000 daily traffic. I am turning off the pc now and I am going to sleep as I am too tired for working on this. I will come back tomorrow in the morning, and it would be great if I could get some free space on there. I am not sure what can be deleted. Most of the things are probably done by now, but it did not make any difference... This is at the point when one thing starts messing up other things, and it may be getting worse and worse. If I get the free space for some time I should be able to fix the hosting plan.
 
Old 12-18-2016, 05:49 PM   #3
John VV
LQ Muse
 
Registered: Aug 2005
Location: A2 area Mi.
Posts: 17,602

Rep: Reputation: 2648Reputation: 2648Reputation: 2648Reputation: 2648Reputation: 2648Reputation: 2648Reputation: 2648Reputation: 2648Reputation: 2648Reputation: 2648Reputation: 2648
i am guessing that this is related to your OTHER post on this
http://www.linuxquestions.org/questi...on-4175595653/

related to the DB on /
20 gig for / is about normal

are you using a LVM ?????
the default setting for installing CentOS and the best option for a server
or
did you manually set up partitions
( NOT a good option for a server)


first WHAT is eating up space???
the error logs ?
the database it's self ?
what


how was the database installed ?
is it the version in the cent repos ? or a different one ?



also please read the moderators post on your other thread
post #6
http://www.linuxquestions.org/questi...3/#post5643681

Last edited by John VV; 12-18-2016 at 05:53 PM.
 
Old 12-18-2016, 08:58 PM   #4
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 25,818

Rep: Reputation: 7747Reputation: 7747Reputation: 7747Reputation: 7747Reputation: 7747Reputation: 7747Reputation: 7747Reputation: 7747Reputation: 7747Reputation: 7747Reputation: 7747
Quote:
Originally Posted by AdultFoundry View Post
This is an EMERGENCY.
Not for ANYONE HERE it isn't....we volunteer our time...this is not a help desk where you can demand/expect 'emergency' help.
Quote:
I am ready to start working on the problem after two days of research, but at this point I am not able to use mysqldump as it gives me a message that the disk is full.
So dump to another disk/partition...what's the problem? Specify a different file name
Quote:
I also get messages that the databases are corrupted because of that. I freed up some space and I had 40MB free and 10 seconds from this point, it was showing 0 again. I cant install something very small like "yum install yum-utils", it would not install that as I dont have enough disk space. I cant run something like "yum upgrade" - not enough disk space (with the last two that I mentioned I was trying to delete the old kernels, but they may not be there for deletion anyway).
Right...because you probably have a full /var or /log (or both) partition. Check your log files first...null them out. If necessary, pull the network connection.
Quote:
So basically I am stuck - I cant do nothing because the disk is full or it fills up very fast. This is the problem of /var/lib/mysql/ibdata1 getting large, and for some reason growing very fast (like 1.1G per 24 hours for the last three days).
..which may be totally normal, depending on your server/needs/users/access.
Quote:
I stopped the httpd and none of the websites work, and I could have like 1,000 daily traffic. I am turning off the pc now and I am going to sleep as I am too tired for working on this. I will come back tomorrow in the morning, and it would be great if I could get some free space on there. I am not sure what can be deleted. Most of the things are probably done by now, but it did not make any difference... This is at the point when one thing starts messing up other things, and it may be getting worse and worse. If I get the free space for some time I should be able to fix the hosting plan.
Too tired?? Must not be TOO much of an 'emergency' if you can just turn it off and go to bed.
 
Old 12-18-2016, 09:42 PM   #5
Sefyir
Member
 
Registered: Mar 2015
Distribution: Linux Mint
Posts: 634

Rep: Reputation: 316Reputation: 316Reputation: 316Reputation: 316
So find the thing that's causing your database to fill up and stop that?
Most likely it's some error repeating as fast as it can.

list the software you're using?
list the programs you're using that have database access?

We get you're frustrated. Can't do much if we don't know much beyond that though.
 
Old 12-18-2016, 10:23 PM   #6
AwesomeMachine
LQ Guru
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,521

Rep: Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015Reputation: 1015
You might have some really big log files. You can use the 'find' command to look for files over a certain size.
 
Old 12-19-2016, 05:04 AM   #7
AdultFoundry
Member
 
Registered: Jun 2015
Posts: 282

Original Poster
Rep: Reputation: Disabled
The issue is fixed, as far as this part. I moved the entire /var/lib/mysql to /home/mysql and I got it working from there, so there is no issue with no disk space on the /. If somebody will be reading this, there may be a need of adding the new location of the /var/lib/mysql/mysql.sock to php.ini files (two separate locations there in Centos 7 that I use). I read several articles about the moving on the Net and it has not been working until I made this change.
 
Old 12-19-2016, 05:42 AM   #8
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by AdultFoundry View Post
I am not able to use mysqldump as it gives me a message that the disk is full.
Dump from remote?
Code:
# mysqldump -h <host> -u<user> -p <pass> db > file.sql
from your local machine?
will dump it locally, provided you have grant access in the db, or use root privs, provided that hasn't been secured.
This leaves you with only the partitioning issue.
Sticking mysql on /home is temporary, right?
 
Old 12-19-2016, 05:55 AM   #9
AdultFoundry
Member
 
Registered: Jun 2015
Posts: 282

Original Poster
Rep: Reputation: Disabled
This is already resolved. I am actually working on the dumps and I have some issues with these. I am able to fix it with mysqlcheck and myisamchk so far. Basically the issue was /var/lib/mysql/ibdata1 (and possibly other files there) growing too large. I have something like partition A - / and partition B (lvm) - /home. I was thinking about shrinking the B and adding it to the A at /var/lib/mysql but this may not be the best way to go. If anybody experiences an issue like this, I recommend moving the whole /var/lib/mysql to something like /home/mysql (in my situation) where you have more disk space. Shrinking LVM volumes may be a risky thing even if all the commands are good, for this reason a backup is needed. It took me full two days to prepare for everything, and I could have solved everything in less than an hour or so with just moving the /var/lib/mysql and this is what I ended up doing anyway. I could have mount 100GB on the old /var/lib/mysql, this could run out of space again, and I would need to come back to it again. What I did should be a better option in the long run anyway. On Centos 7, the new location of this folder needs to be added to /etc/php.ini file. This is not described in most of the articles on the Internet. It was not working for me until I've done that (httpd error_log was showing that the website scripts were not able to connect to the database).
 
Old 12-19-2016, 11:43 AM   #10
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Good for you.
Repeat yourself some more.
I'm kind of tired asking and not getting any answers, so I guess we'll see you on the next "EMERGENCY"?

I suspect your google.fu doesn't work too well either as https://www.google.com/search?q=ibdata1%20too%20large clearly talks about how to address the topic.

And you're not asking the right question, IMO,
Why does ibdata1 grow so large?

You're welcome.

Last edited by Habitual; 12-19-2016 at 11:50 AM.
 
1 members found this post helpful.
Old 12-19-2016, 11:47 AM   #11
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,278

Rep: Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693
Thanks for sharing your fix for the issue. I would mention that Centos 7 uses XFS and it is NOT possible at this point to shrink an XFS LVM filesystem that I am aware of.
 
Old 12-19-2016, 01:08 PM   #12
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,247
Blog Entries: 4

Rep: Reputation: 3777Reputation: 3777Reputation: 3777Reputation: 3777Reputation: 3777Reputation: 3777Reputation: 3777Reputation: 3777Reputation: 3777Reputation: 3777Reputation: 3777
You should either now be running LVM, or you should start using it at once.
 
1 members found this post helpful.
Old 12-20-2016, 08:12 AM   #13
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by AdultFoundry View Post
Shrinking LVM volumes may be a risky thing even if all the commands are good, for this reason a backup is needed. It took me full two days to prepare for everything, and I could have solved everything in less than an hour or so with just moving the /var/lib/mysql and this is what I ended up doing anyway.
Were backups in the 2 days you "prepared"?
Simple, Secure Backups for Linux with rsync

LVMs is recommended.
No backups is on you.
Moving to /home/mysql is a hack IMO, and calling it "solved" is going to bite you on the ass.

I told you in the beginning this wasn't going to be "easy".

But no backups? shame.
 
Old 12-20-2016, 10:24 AM   #14
AdultFoundry
Member
 
Registered: Jun 2015
Posts: 282

Original Poster
Rep: Reputation: Disabled
@Habitual - I am prepared and able to move the database and enlarge the /var/lib/mysql in the original location if this will be a needed thing. It is more important for me to work on a website, instead of doing something that could be good but it is not a necessary thing. I have a full understanding of the situation and I can come back to it this would be a needed thing. I back up to Windows 7 with Cygwin, once per month or less often. I know that this is not the best but better than spending another $42 per month for another hosting plan that I would need. If I will be able to expand what I am working on now, I will set up a Linux back up machine and the whole method and the system will be more advanced. I am able and capable of setting up anything that is needed with the necessary research and work. For now this is good enough.

Why moving /var/lib/mysql to /home/mysql is bad, what are the reasons? I am asking because I dont have the knowledge about that.

@szboardstretcher - I've been reading that Centos 7 uses XFS, but ext4 is what I have there.

Last edited by AdultFoundry; 12-20-2016 at 10:56 AM.
 
Old 12-20-2016, 03:30 PM   #15
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by AdultFoundry View Post
I am able and capable of setting up anything that is needed with the necessary research and work. For now this is good enough.

Why moving /var/lib/mysql to /home/mysql is bad, what are the reasons? I am asking because I dont have the knowledge about that.
I didn't say 'bad' but I do consider it the "second best" suggestion.
So while you're keeping an eye on /home/mysql/ibdata1
Have a look at https://www.centos.org/docs/5/html/C...ger/index.html

I have a zabbix db and it blew up to like 100G and, Oh Hells Noe!
Historical data, yer fired.
export to file.sql
InnnoDB the tables
restart dbserver
import file.sql
and "All of a sudden" we're at 10G...
Warning: /var/lib/mysql/ibdata1 always grows, never shrinks.

Whatever you do, or don't, burn that into your noodle.

I'll ask again. Is the "move" to /home/mysql temporary, or are you going to research implementing LVMs in the near future?
I hope that you'd not relax since the "Emergency" is over thanks to the work you did.

I guess your "good enough" statement bothered me.
I will get over it.

Good Job btw moving mysql.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
partition space and disk space usage and free don't match nsic Linux - Newbie 3 02-22-2013 12:57 PM
Partition Free space RH sgoldman Red Hat 2 04-04-2010 04:11 PM
can't partition free space from shrunken partition john_b Linux - Newbie 7 01-14-2006 04:43 AM
Partition Free Space - Move /home to the new partition gregkise Linux - General 5 12-16-2003 10:19 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 04:18 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration