[SOLVED] Slackware hard disk usage growing and need help since not finding it in threads.

Originally Posted by bamunds 2) Why doesn't Slackware store non-standard binaries under /usr/local and is there a way for me to get it to do so in the future for both slackbuilds and Slackware packages, like AlienBob's chromium? I use pkgtools, slackpkg+, sbopkg mostly, which should allow me to keep track of what was added that isn't standard, right?

It actually does store them in /usr/local, if you compile programs in the once standard Linux way from source: configure, make, make install. Now we have package managers...

You have two options:
1- use a package manager, track them, upgrade them easily = /usr will be used.
2- compile from source; package managers won't track them = /usr/local will be used.

Edit: I do not use package managers but maybe there's a way to pass an option so that they will use /usr/local.

Originally Posted by vdemuth With respect, that is just an assumption on your part. You might be right, you might not be.

Yes, I know it was, that was the whole point. It was an assumption based on the order in which he'd listed them, which I posted to counter your assumption that removing /tmp would allow the rootfs to be extended, which as far as I can tell didn't seem to be based on anything at all.

Originally Posted by vdemuth So as I said, the guy asked for help, so maybe instead of critcism as per Reaper, or telling him how you wouldn't have chosen that particular disk layout, how about something constructive the guy can work with rather than just "Sometimes you just find that you've painted yourself into a corner and its actually going to be less effort to start-over" That may be the case, but do you learn from it?

Firstly, I didn't say I wouldn't have chosen that layout. The part of my post talking about layout choices was directed towards Reaper (whom I was disagreeing with) and Richard (whom I was agreeing with, and whom was also disagreeing with Reaper and supporting the choice the OP made to have multiple separate filesystems).

Secondly, suggesting someone start over when its likely the most expedient choice is IMHO constructive.

As I've already said above, my approach in this situation would be to do a full file level backup, start over with a new partition layout (whatever that layout may be doesn't really matter as long as it addresses the lack of space in '/'), restore the files, and then fix the bootloader using a chroot from the install-cd/dvd.


@bamunds
Apologies for causing a distraction in this thread with this pointless bickering (On which I've now said all I'm going to). Hopefully you found my previous posts more constructive that vdemth, even if it wasn't what you wanted to hear.

Originally Posted by bamunds But my main question is "what is using up all the space on "/" ..."

I don't have multilib or WINE installed, but I do have an otherwise fully expanded Slackware system and my /usr directory is currently about 8GB. I generally allow for 20GB on my / partitions.

If it were my system, I would temporarily save the contents of /usr/local, /tmp, /opt, and /var; then delete sda4 through sda8; then grow the /home partition to 32GB; and then create a new 12GB partition dedicated to /usr. Once this is done, the saved contents could be moved back to the / partition (well, /usr/local stuff would go into the /usr partition).

Using a quota based partition scheme is not something just anyone should use without consideration.

Let me explain first, and if you still disagree, then you may drop the blade on my neck.

I'm not saying it's right or wrong, but I am saying that for a general usage system for someone who isn't or might not be comfortable and/or familiar with quotas and disk space management for a GNU/Linux system, especially since the person might be new to GNU/Linux based operating systems, or even new Slackware for that matter, using a more simplified partition layout, such as the one I posted as an example, rather than an advanced layout like Richard's, might be a better choice and a good start.

Originally Posted by vdemuth That may be the case, but do you learn from it?

In *my* case, I learned to love LVM (which is pretty easy to do) and to not allocate everything from the physical volumes into logical volumes so that I've always got some slack. (Which is kinda good, since it *is* Slackware.) Oh, and try to pick filesystems that you can grow while they are mounted.

Honestly, that's the easier than the other stuff in the long run. IMO, anyways.

Case in point:

# pvs
  PV         VG      Fmt  Attr PSize   PFree  
  /dev/md1   mdgroup lvm2 a--  136.09g 224.00m
  /dev/md2   mdgroup lvm2 a--  929.81g  84.91g

I've still got ~85G of unallocated space that I can give to any of my logical volumes. When it's time to put in new disks, I add them to the volume group and pvmove stuff out of the old disk(s) that I am replacing. I haven't had to do that on my laptop yet, but I've done it on both of my main servers at home.

First thank you to everyone for responding. I did layout my partitions as sequenced in my OP, I thought I had explained the extended partition volumes. What I'm most concerned about is the growing usage of "/"! If I had used a single partition as suggested by some, I would never have found that "/" was growing for some unknown reason, and /tmp needed CLEANUP also. This is clearly why partitions, even on a small disk, are helpful they let you catch culprits before your whole disk is gone. My /home is a primary and not extended partition so that I can reach it in a recovery mode, where an extened partion is more complicated to mount and recover. The help I'm primarily asking for is: How do I find what is causing "/" to grow with so few extra applications installed?
@dives, I'm going to have to readup on using alias since it didn't work on my system from root account.
@yenn, the command you provided doesn't work either from root account.

@GazL, you assumption was correct on the layout of partitions.
@ReaperX7, I understand your suggestion, but as stated above, if I hadn't used this layout I'd be in the dark that something is advancing undo usage of the disk.
@RichardCranium, thanks for the suggestions. As for LVM I didn't use it because of the extra steps in a disk Recovery mode with LVM and the need to share files in a MS Windows workgroup temporarily. Is there some reading you can suggest that would help me understand how to recover LVM if there was a crash or lost file and do MS Windows workgroup file sharing from LVM? I've read the preliminary documents of when to use LVM and it didn't seem to fit my situation, maybe I'm wrong.
@saulgoode, thanks for your disk size marks and suggestion on how to re-partition.

I plan to correct future slackbuilds scripts to install to /usr/local, since they are local builds and that is what the standards say should be done. My secondary question really is why did Slackware maintainers, the most UNIX like Linux, not supporting the disk standards destinations? This isn't a criticism of PV but of the guys at Slackbuilds and the slackpkg maintainers, and the sbopkg utility developers who are setting the slackpkg installs with DESTDIR of /usr and not /usr/local, since the packages are not binaries but local builds. When I build from source, with config & make && make install I use /opt as specified in the standards. Apache OpenOffice installs correctly to /opt. Let's get Slackware back to full standards support and be proud to say Slackware the most UNIX like GNU/Linux.
@arsivci0, my second question should have been about local built packages not binaries. For example where do installpkg, upgradepkg, and pkgtools install the added packages which aren't part of the "full standard install"? I'm more questioning the slackpkg, slackbuilds, sbopkg package utilities that seem to break the standard.
@55020, thanks for the commands, but since /var/log/packages tracks all the added programs, then if I'm wanting a clean-system that keeps the extra applications in separate locations, not mixed into the full standard system, the files would best be placed in /usr/local or /opt. IMHO the FSH isn't bonehead with the reasons.

But my primary concern is not the standards, it is why is "/" growing with so few extra applications installed above the full Slackware 14.1 install? I'll try some of the helpful suggested commands to find the culprits, but how to I pinpoint if there are leaches or cache's being left behind by XFCE, KDE or e17? I tried Bleachbit, is there another or better utility to identify culprits? Did the upgrade leave behind large unused chunks, where would I find them? When I use FileLight I see parts of /usr that have no real files designated in the next graphical ring of files, why? I just find it hard to believe that the few extra packages that I've installed (listed in my OP) caused an additional 4 GB to be used. Isn't that a reasonable surprise and desire to investigate?

Oh if anyone wants to make suggestions on the other secondary questions of my OP I'd love to hear your thoughts and reasons on those also. I really do appreciate the help and directions on things to try.

Check out your LOG files. I had a problem once that I had a .log file that grew into the GB's filling up my root drive.

Originally Posted by bamunds ... setting the slackpkg installs with DESTDIR of /usr and not /usr/local, since the packages are not binaries but local builds. When I build from source, with config & make && make install I use /opt as specified in the standards.

slackpkg is a tool for managing official Slackware packages. I suspect you mean that you plan to modify the "third-party" SlackBuilds available from sites such as slackbuilds.org

In such SlackBuilds, it is not the DESTDIR shell variable that you should be changing to /usr/local or /opt. DESTDIR typically specifies a temporary location to place the files which will later get tarred up into a Slackware package. The actual installation target directories are usually specified with the ./configure switches --prefix and --exec-prefix (other switches are available for finer tuning of various other installation targets).

Given that you're using separate /var /tmp and so on, I wouldn't be expecting / to be growing like you say it is.


For analysing filesystem growth I prefer to use:
du -Sxb $mountpoint >/tmp/usage.out
That will give you a list of space used per directory(in bytes).

Then you can use any number of the standard tools to find what's going on...

# sort by size.
sort -n /tmp/usage.out

# compare 2 days with diff to see what's changed
diff -u0 <( sort -k2 /tmp/usage.day1 ) <( sort -k2 /tmp/usage.day2 )

diff is a bit quick and dirty, but it gets the job done.

... and so on.

Originally Posted by bamunds @RichardCranium, thanks for the suggestions. As for LVM I didn't use it because of the extra steps in a disk Recovery mode with LVM and the need to share files in a MS Windows workgroup temporarily. Is there some reading you can suggest that would help me understand how to recover LVM if there was a crash or lost file and do MS Windows workgroup file sharing from LVM? I've read the preliminary documents of when to use LVM and it didn't seem to fit my situation, maybe I'm wrong.

I don't see how file sharing relates to LVM unless there is a removable drive that is involved. Could you explain a little bit?

There's not much to recover from LVM in the case of a crash in that the information about the logical volumes is written into areas in the disk partition that are normally untouched. As far as the operating system is concerned, a logical volume is just another block device just like a regular partition as far as any file systems are concerned.

If for some reason you cannot reboot your system (like forgetting to re-run lilo or something), it's only painful when your root partition is really a logical volume, but it isn't that bad. It's been a while since I've done it, so not only is the following from memory but it's from a fairly old memory.

1. Boot from the installation DVD/CD as if you were going to install.
2. Run the command "vgscan --mknodes --ignorelockingfailure"
3. Run the command "vgchange -ay --ignorelockingfailure"
4. Run the command "mkdir -p /mnt/repair"
5. Mount your root directory's logical volume on /mnt/repair
6. Run the command "mount -bind /proc /mnt/repair/proc"
7. Run the command "mount -bind /sys /mnt/repair/sys"
8. Run the command "mount -bind /dev /mnt/repair/dev"
9. Look in /mnt/repair/etc/fstab and hand-mount the stuff that you find in there to /mnt/repair/<fstab_entry> (minus any /proc, /sys, or /dev entries)
10. Run the command "chroot /mnt/repair"

At this point, your environment pretty much looks like it would if you had booted normally. You can fix lilo problems, re-create your initrd, or whatever. I wouldn't run long-term in this state.

When you are done, run "exit" until you are out of the chroot and then reboot.

So what does all of that do? Well, you use the installation DVD to get you a running system (which is really what the installer gives you). You use that system to run the commands to scan your disks for LVM related information (the vgscan and vgchange commands). You make a mount point to mount the file systems that you'll need after you chroot (the mkdir command) and mount your root logical volume there (I can't give a command since I'd have to know the file system used and the logical volume name). Your root logical volume will have /etc in it, so you can go look in etc/fstab to get the rest of the mount points that would be set up during normal boot. You just have to mount them within the /mnt/repair directory tree instead of /. The "mount -bind" stuff ensures that the stuff the kernel figured out about your system during boot will be available after you chroot. The chroot command essentially makes /mnt/repair the same thing as / in your current shell environment.

@bamunds, I understand your frustration with how you feel the packages should be installed vs how they are installed. But personally, I feel that /usr is a better choice for the packages from Slackbuilds. The way Slackware is built and upgraded should leave all installed packages intact unless Patrick puts out a package to replace the one you installed. This was the primary reason the FHS said to install to /usr/local. If you just do the standard ./configure, make, make install, it installs all files to /usr/local by default without a way to remove them (unless you keep the source and it supports make uninstall). Slackware has no way of knowing what was installed when it is installed via make install, so if it was installed to /usr, a new slackware package could easily replace files and cause issues. When you don't have a package for the software, Slackware doesn't know if it has additional files it needs to remove, can't monitor the /etc directory for new conf files, etc. Installing the software via a package makes slackware fully aware of what files were added and what would need to be done to upgrade that package.

Looking at the note for the /usr/local entry on FHS, it states

Software placed in / or /usr may be overwritten by system upgrades (though we recommend that distributions do not overwrite data in /etc under these circumstances). For this reason, local software must not be placed outside of /usr/local without good reason.

While you may not feel this is a good reason, the slackbuilds.org maintainers obviously do feel that way. Each slackbuild is tested with any required dependencies (which are also required to be on slackbuilds.org) to make sure that there are no issues. Slackware's package management is great. It knows exactly what was installed and what needs to be removed/upgraded when the time comes to replace that software. The files in /usr won't be overwritten during system upgrades unless that specific package is being upgraded with an official one.

If you look at the Slackwiki document on creating a slackbuild, it specifically states

There are three configure options I always set: --prefix=/usr --sysconfdir=/etc --localstatedir=/var This makes configuration files go to /etc, state files (such as log files) go to /var, and the rest goes to /usr. That's the usual Slackware way, but it's your system, so you can certainly install everything in /usr/local or some other location. See the Unix Filesystem Hierarchy Standard [[2]] for more information on "correct" locations of various filetypes.

The text for this page was originally created by Florian Mueller but had "Substantial cleanup and enhancement by Robby Workman" according to the page's history. So Robby Workman, one of the major contributors to Slackware, either added this text or felt it was ok being there.

Either way, it's your machine and you're certainly entitled to have the software installed how you'd like it installed. It may just take passing/editing some variables when building the software

=================================

As to your actual problem of determining what space is being used up, you can simply use du -hd 1 in your / (root) directory and it will display the total size of the directories under root in human readable form (shortens things to MB/GB when possible). Then, when you find a directory you think is larger than it should be, you can cd into that directory and run the same command again. You can use this process as much as you want.

However, with something like this, I usually prefer a GUI. I personally use Krusader (slackbuild available) and in it, it has a disk usage utility that will scan everything and show you bar graphs for the directories. I would suggest starting krusader in root mode (should be an option in the KDE menu, or you can select it in one of the dropdowns on the program when started as a normal user). This makes sure that it can scan all directories, even ones only visible to root.

Find can be usefull too

# find files in a directory with size > 100 MB
find /home/mydir -type f -size +100M

I've always managed with / and swap. KISS, no problems. Of course, if I was doing a server only set-up, it would probably be different.

Thanks again for the suggestions to finding what is making "/" fill. I'll try more of them tonight and see what I can determine, then come-back and report here.

@bassmadrigal, thank you for the proper code I would need to modify to get /usr/local for slackbuilds to be correctly installed as I would like to see. I appreciate and know that Slackware package tools are unique and that supposedly, nothing will be overwritten. I agree that the official repository packages should be installed in /usr, since they are packages for Slackware prepackaged for direct installation to the Slackware standards using standard Slackware tools. Yet I was surprised /usr/local wasn't used by the slackbuilds maintainers. I'm wondering if anyone knows what the "good reason" was for breaking the UNIX standard, thus giving others ammunition to say Slackware isn't the most UNIX like Linux? I've also gone to the SlackWiki and read more about the package system and the package tools. Do you know if Slackroll is using /usr or /usr/local?

@RichardCranium and samgoode, thanks for the LVM tutorial on recovery and correction about third-party slackbuilds. I'll check the SlackWiki for more about LVM. I'll consider it if I decide to do a re-install after further investigation on what's causing the "/" growth.