Disappearing file space - deletions don't seem to free space

I'm running Kubuntu 14.04 and have a 4TB drive mounted in my home folder, and it is filled. I have been deleting files using Krusader (by selecting the folders, right clicking and selecting delete -> delete -> delete all. It has worked fine since I installed the OS and never had a problem with space freeing up.

I started noticing when I downloaded a file (via FTP - filezilla) I'd get an error about no disk space. I'd see that I needed to download maybe 3GB of data so I'd clear out maybe 6-10GB of data using the above process. I download the 3GB and then next time I need to download another 3GB and it fails, no disk space. I haven't downloaded anything else, nothing saves there except my FTP downloads, no other transfers, nothing.

I thought this was a fluke, but it has happened at least 20 times, and every time I delete, I check all the folders properties for their size and keep track of the size and it seems like there is about 50%, maybe SLIGHTLY less, disk space missing for each of these processes. It is almost as if the files are copied 2x to the hard drive.

I looked in the drive mount/folder for the trash or recycle folder (which sometimes I have one, sometimes not, IDK why), and it is basically empty, maybe 100KB in a few files.

This started the day I started using Filezilla, which is the only thing that has changed, where I went from downloading via HTTP link to using an FTP que to make it easier. I remember somethimg similar happening years ago when using FTP and I think it was due to my delete method (which sent to recycle/trash) but they aren't showing up there, and it is clearing out the space initially.

So does anyone have any ideas as to what might be happening or where I could look to figure this out?

Post the output of

df -h

Filesystem Size Used Avail Use% Mounted on
udev 16G 12K 16G 1% /dev
tmpfs 3.2G 1.8M 3.2G 1% /run
/dev/sda1 37G 25G 11G 71% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 16G 570M 16G 4% /run/shm
none 100M 32K 100M 1% /run/user
/dev/sda6 131G 83G 42G 67% /home
/dev/sda2 29G 27G 414M 99% /home/user/ssd/30ext4
/dev/sdb1 3.6T 3.4T 8.0K 100% /home/user/Storage
/dev/sdc1 58G 48G 11G 82% /media/user/M64-1


I know that the drive is full, but I continually empty about 6-7GB of files, then download a single 3GB file, then the next 3GB file it says full. I repeat the process and it happens again. It seems like it is either duplicating the file on download or something. I've deleted over 120GB of files and only downloaded less than 60GB of files and it still says it is full.

I've looked in filezilla if it may be somehow storing data on my /storage drive along with data in the download folder, but it doesn't have any folder options in the settings, so I would think that any of the cache files would be within the home/user/.filezilla folder, which is on my main hard drive not my separate storage drive.

When posting the output from terminal commands, put it in between [code] [/code] tags. It preserves formatting and makes it MUCH easier to read.

Anyhow, as I'm sure you've noticed

/dev/sda6 131G 83G 42G 67% /home
/dev/sda2 29G 27G 414M 99% /home/user/ssd/30ext4
/dev/sdb1 3.6T 3.4T 8.0K 100% /home/user/Storage
/dev/sdc1 58G 48G 11G 82% /media/user/M64-1


are all rather full.

I would run something like

du -a /home | sort -n -r |head -n 50

on each of these mountpoints to find out what is using so much space. This will display the 50 largest. You can increase or decrease the number of results as you see fit.

If you suspect it is somehow duplicating downloads you can search for duplicates with the command-line utility fdupes

Something odd is going on as well. I just unplugged and pulled my drives (5 3.5" drives) and cleaned them off of dust. I hadn't had 4 of them plugged in in over a year and when I plugged them back in they spun up and auto-mounted. When I plugged my 4TB drive in, nothing happened. I typed sudo mount /dev/sdb1 /home/user/Storage and it said "device does not exist" I did it about 4 times and it wouldn't mount. It was spun up and running so IDK what was going on.

I unplugged the drive and moved to a different SATA port, and I got a notice like a USB drive had been plugged in, so I selected "open with dolphin" and by doing that it mounted in the proper place. IDK why it wouldn't mount with the command line. By mounting this way it did change to a sde1 instead, which seems normal as it was a different SATA port on the MOBO.

IDK is this has anything to do with my issue or not, I thought the drive had died when it wouldn't mount.

By default 5% is reserved on a ext2/3/4 (I assume ext4) which means about 180GB on a 3.6T filesystem. On a data drive you can safely set reserved space to 1% or even zero. You need to delete enough files to go below 95% used in order for the system to not say no disk space. In addition if you delete files that are currently open the used space will remain the same until the process using the file is stopped.

Reserved space is supposed to reduce fragmentation as well as allow root to login in case the filesystem becomes full.

tune2fs -m 0 /dev/sdb1

would set the reserved space on sdb1 to 0%

Well I found that there is something wrong with the drive. I cleared up 100GB and the free space was gone after downloading 10GB of data. I rebooted and the drive doesnt' re-appear and no Fdisk -l or anything makes it show up, I've tried other machines, other SATA ports, everything. Also, when I did fdisk before rebooting, it said something about he drive "not having an ending" or something like that. It seemed the partition was messed up and I know it worked fine when it was originally installed. Could disconnecting the power cable or SATA cable cause this? It is an EXT4 partition (single large partition). I need to see if i can save the data on this somehow, so I have is set off to the side while I decide what to do.

Thanks for the help you guys provided to see if I could get things working. It's difficult to diagnose things like this as they are often fairly rare and not a go-to option when troubleshooting.

the circuit board on the drive possibly went bad. if you have another drive EXACTLY the same model and size, you can swap the board and see it that's it. also. https://community.spiceworks.com/top...-a-motherboard. i'm saying that since you weren't getting bad sector errors first. does the drive spin up? let it get cold and then retry it.

Originally Posted by whansard the circuit board on the drive possibly went bad. if you have another drive EXACTLY the same model and size, you can swap the board and see it that's it. also. https://community.spiceworks.com/top...-a-motherboard. i'm saying that since you weren't getting bad sector errors first. does the drive spin up? let it get cold and then retry it.

Yeah, I have 3 others Idetnical drives that I thought about swapping the controller card on. I'm waiting till I completely re-install my OS on my system and then going to try to check the drive again. I've read the cold thing is a bunch of BS but some people swear by it. I've also heard that it can be water vapor inside (from a very small hole in the seals) which makes things work better. I was just waiting on troubleshooting until a little later.

On a side note, 3.5 years w/o a re-install of my OS! Used to have to do it 2-3x a year with Windows (XP up to 7), so that is a good note. Some things have broken over time (like monotir alignment, it is wonky from one boot to another...)

Thanks for the suggestion bud!!

All the *buntus used to remove files to a hidden .Trash directory, they weren't actually removed from your disk until you emptied the .Trash directory. Just might be your problem.

inodes?

df -hi

Originally Posted by fatmac All the *buntus used to remove files to a hidden .Trash directory, they weren't actually removed from your disk until you emptied the .Trash directory. Just might be your problem.

First thing I checked. That is what was happening years ago when I was deleting the files through my FTP program but since then I've figured out how to skip that whole process and do a permenant delete.

When I did an Fdisk -l I found that it said something about the drive not having "an ending" or wasn't terminated correctly. It seemed that the partition got messed up and with the drive completely full, I'm thinking that files that were near the "end" of the partition are going to the /null file. When I unplugged the drive (cleaned system of dust) and put it back in, the drive was no longer recognized in any manner. It spun up, but didn't show anywhere, so I think it is a physcially bad drive. Time for some of the old "rescue hard drive" tricks once the new one comes in. I really dont' want' to swap the controller board as I don't think that is what is causing it, I suspect something internal. I do have 2 other identical functioining drives but I'm worried that removing the board from one of those may cause damage on them.

I may copy all the data from a good drive to the new drive, then take the controller card from the drive I just copied and try it on the bad drive, this way, if it messes something up, I don't loose data on the older good drive.

While I'm not a huge fan of SSD's, the one good thing I can say about them is when they go, they seem to do it all at once and there isn't as much data lost (my avg size is 240-250GB vs 4TB) but I have good luck with quality Western Digital mechanical drives.

Thanks for the help guys!

Well as I said the drive stopped showing up anywhere. Not in BIOS posting and not in Fdisk. I did run GSmartControl which is the gui for smarttools I think. I am seeing a drive here that is listed as "unknown model" and no information about the drive comes up but it does show it as /dev/sdg and trying to fdisk /dev/sdg says "unable to read /dev/sdg: Input/Output error". Also when I select the drive in the Gui and look at the drive info it says it is 660.8 PB when it should be 4000GB or 4TB, so the ~661 Pentabytes is a little bit off.

I'm guessing that this must be a bad controller board? I took the board off and looked at all the connections and they were fine. I tested all the pin connectors (the 30 pin connector and the 4 pin motor controller) and tested them with a continuity tester and the contacts were good. I did lightly sand all the contacts with 2000grit sand paper, and it did make the contacts better, needed less pressure to get the probes to detect continuity - but I don't think that was an issue either way.

Is there any potential of damaging another controller board if I swap one from a good drive with this non-working drive?