root partition too small! help!
 

Hi, following is the partitions of one file server:

>fdisk -l
Disk /dev/sda: 72.7 GB, 72795291648 bytes
255 heads, 63 sectors/track, 8850 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 65 522081 83 Linux
/dev/sda2 66 702 5116702+ 83 Linux
/dev/sda3 703 833 1052257+ 82 Linux swap
/dev/sda4 834 8850 64396552+ f W95 Ext'd (LBA)
/dev/sda5 834 8850 64396521 83 Linux

>df -aT
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/sda1 ext3 505604 428969 50531 90% /
none proc 0 0 0 - /proc
none sysfs 0 0 0 - /sys
none devpts 0 0 0 - /dev/pts
usbfs usbfs 0 0 0 - /proc/bus/usb
/dev/sda5 ext3 63384708 13490980 46673904 23% /home/pv-chem-file
none tmpfs 517280 0 517280 0% /dev/shm
/dev/sda2 ext3 5036316 1329044 3451440 28% /usr
none binfmt_misc 0 0 0 - /proc/sys/fs/binfmt_misc
none proc 0 0 0 - /var/named/chroot/proc
sunrpc rpc_pipefs 0 0 0 - /var/lib/nfs/rpc_pipefs
nfsd nfsd 0 0 0 - /proc/fs/nfsd


Now it seems that the root partition has been used more than 95%! Terrible! It's not partitioned by me but others, all such directories as /var, /tmp, /home share this 500M space! What can I do now to solve this space problem?

I have two choices:

1. Use fdisk to re-partition the hard disk, maybe let sda2 give 1.5G to sda1? How to do it without the need to reinstall Linux?

2. Move all contents of /var to /usr/var, then delete /var, and create a symbolic link "var" linking to "/usr/var", is this feasible?

My system is Fedora:
Linux version 2.6.12-1.1376_FC3smp (bhcompile@tweety.build.redhat.com)
(gcc version 3.4.4 20050721 (Red Hat 3.4.4-2)) #1 SMP Fri Aug 26 23:50:33
EDT 2005

Because it's out department file server I need to be very careful. Anyone would like to help me?

Thanks!

I'd suggest leave it alone. There shouldn't be anything growing on the root partition. 10% free is a little low but one can live with that.

Thanks! But /var is under root partition and it increases every day.

I'd definitely relocate /var, preferably to its own partition if you have any space available. Otherwise, I'd suggest one of two things:

1. As you suggested, create it on another partition and link to it.

2. Set up logrotate to rotate log files frequently with backups going to another partition, which could be a link under /var or /var/log pointing to the other location.

One other thing I just read about seems similar to the link, but is a feature of mount. The --bind option allows you to specify a particular directory and mount it on a mount point elsewhere. The contents of the directory (and any subdirs) will be seen from either location, similar to a link, but you will not see a link when you look at the mount point. It will show up with the mount command.

I don't know all the ramifications of using mount --bind, other than that any other filesystems that might be mounted within this tree won't show up on the bound mount point. If you needed them, you could use --rbind instead. As far as which is preferable between a symbolic link and the mount --bind, I'm not sure. I know some processes may have difficulty with links, where the mount --bind probably won't be an issue. RTFM and test it a bunch first.

I just tried mounting a dir onto a mount point I created in my homedir and it worked nicely, but I didn't do anything besides look at the files. Hopefully this will work for you.

If your logs don't grow too fast, the logrotate option may work well for you. If you get LOTS of log traffic and/or mail or print jobs, you might want to move /var to another location.

If you can afford to, I'd consider shrinking the /dev/sda5 partition by a gig or two, then create a /var partition to mount normally, that way it won't be sharing space with anything else, preventing problems. You REALLY, REALLY don't want to run out of space on your root partition. Bad things happen.

If you do have to share space, use logrotate and compress the backups. Text files (like logs) can be compressed to a fraction of their original size. If you have to share, I'd share where the most space is (/dev/sda5).

Oh, I noticed you have only one drive there. If you throw another drive in, you can put /var there and have no more problems. If there's physical space, that is...

-Alan

Thanks a lot, Alan!

I read ur suggestions, it's very nice. My understanding is, we can create link or mount --bind to relocate /var directory to other partition but we are not sure we won't have any problems in the future, right? I cannot afford such an uncertainty because it's not my machine but a department file server.

So it will be safest if I use fdisk to split sda5 into 2 parts and mount them to /home/pv-chem-file and /var separately, right?

Can you tell me if my understanding is correct or not?

You shouldn't have any problems with it functioning correctly in either case, but if you were doing backups, or running scripts against those directories, some utilities won't follow symbolic links, so I think the mount --bind would perhaps be preferable. The downside is that you're still sharing the space with whatever else is on the partition.

If you can afford to, I'd go with shrinking sda5 and create an sda6 to mount as /var. Just make sure you back up everything on sda5 first, just to be safe!

Rather than using fdisk, which would definitely destroy data on sda5, you might try something like qparted to resize sda5; I think it can resize it without disturbing the data, but you really want a backup just to be safe. Be sure to fsck before and after resizing, and make sure your backup is someplace safe, too. Then create the sda6 and mount as /var. Make sure you copy all of /var to the new partition first, and delete the contents from /var on the root partition before mounting the new /var, or all the space in the old /var will still be used, but you won't be able to get to it to remove it.

I guess this means that you will want to do the copy/delete/mount while booted from a recovery disk or something like that (Knoppix comes to mind...). You'll want to edit the /etc/fstab to mount /var while booted from the recovery disk so that it will mount properly on boot from the HD.

I must admit that I've not tried this myself, but I'm fairly certain it should work fine. I'll cross my fingers for you!

-Alan

By the way, a Knoppix Live CD (I use version 3.6, haven't tried 4.0 as I don't have a DVD burner) will have all the utilities you need, as long as you can boot from the CD. Qparted is there, and it will recognize almost all hardware I've thrown at it, though again, I must admit that I haven't had a SCSI box to boot it on. I'm pretty sure it should work for you, though.

-Alan

Oops, it's Qtparted.

-Alan

It's so kind of u, Alan!

You said "The downside is that you're still sharing the space with whatever else is on the partition." But I'm not clear what the problem will be if I let /var share the same space with something else? Anything fatal will happen? Otherwise I prefer to use mount --bind coz it's the simplest.

Log files are notorious for growing a lot bigger than you expect. The problem is that they can use up all the space that was intended for something else.

If this happens on your root partition, your box will crash, so you really want to get the logs to a different partition. If you put them on a partition with, say, an application that is also writing its own logs or data or both, you could run out of space and crash the application or lose data.

If you know that you have LOTS more space than you need, it may be OK to move /var to the other partition, but if you want to be sure that you don't affect anything else on the server, it's better for /var to be on its own partition.

-Alan

I would suggest creating a separate partition for the /tmp directory. Since this is a server setup, you will want to have it mounted with the "noexec" option since it is world writable.
This may bring the % usage down a little bit.

The /var directory usually has its own partition on server setups. A hacker may try to flood the logs on your machine in order to fill up the available space in the root partition. If the /var/log directory isn't on the root partition, this can prevent a crash.

qparted
 

Do make sure you back up your data and keep it away from the tool if you're just learning it. I just made the most boneheaded decision in my life regarding computers. I too had a full os partition and wanted to increase it's size as the disk is huge. Problem for me is that I also had all of my data on one of the other disks. So I had to delete the extended partition to increase the size of the os partition. Worked like a champ. EXCEPT, for the fact that my other disk now has no data showing up on it. I'll figure something out and perhaps hire an expert as all of my invoice data is on the problem sata disk The IRS won't believe me, so I've got to get it back. Still not sure what happened. I umounted the drives so that shouldn't have been a problem-so I still am not sure what screwed it up-but word to the wise-be careful with your data. At least it wasn't one of my customer's data! In 15 years of computing this is the first disaster of this proportion for me. GRRRRR!

HI, Alan:

This morning I checked the disk again. After the latest yum update, 98% of /dev/sda1 (root partition)
has been used. Only 13M space left. Among all the directories, I found /lib used nearly 369M of all
500M root partition. Maybe I need to relocate /lib?

/var only used 45M, /home used only 3K, /boot 18M, /etc 11M, /bin 5M, /sbin 13.4M, /tmp 41K. So I
think /lib is the problem.

In /lib/modules, I found following dirctories
2.6.10-1.766_FC3smp 2.6.10-1.770_FC3smp 2.6.12-1.1372_FC3smp 2.6.12-1.1376_FC3smp
2.6.12-1.1378_FC3smp 2.6.12-1.1381_FC3smp

Can I remove some of them? Are they all used by the system or some are out of date?

removed