LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   rm can't delete 999999 files in one directory (https://www.linuxquestions.org/questions/linux-software-2/rm-cant-delete-999999-files-in-one-directory-168324/)

yelo 04-09-2004 07:46 PM

rm can't delete 999999 files in one directory
 
I have a funny problem. I am working on an application which takes a large file and splits it up into smaller ones(like for floppy disks, etc), just with a few special features, so a normal file splitter just won't do the job for me. anyway, I messed up, and created exactly 999,999 empty files into a directory with other stuff in it. the files are all of the form test.test_split00001, test.test_split 000002...test.test_split999999. so, I think ok, just an rm -f test.test*, right? WRONG. I get this error message:

bash: /usr/bin/rm: Argument list too long

the first thing I think is that's funny! I've finally found out how to make the rm command fail :). but, how would I go about deleteing all those files now? I tried opening the folder with rox, but once it hit 700mb of used memory, during the loading of the folder(I only have 256 physical + 1 gb swap, so it was major swappage), I gave up, and used good old xkill to make it leave...

However, now it's not so funny...and I need help, which is why I turned to linuxquestions.org for advice. I really don't want to leave them all there.

dishawjp 04-09-2004 08:05 PM

Just a thought,

How about trying:
rm -f test.test_split0*

then:
rm -f test.test_split1*

and etc.

The rm command may have trouble deleting 1,000,000 files at a time, but maybe can handle 100,000. Another possibility might be to delete the entire directory they are in if they are the only files in the directory using the "rm -rf" command.

Never had this problem myself. Good luck.

Jim Dishaw

yelo 04-09-2004 08:15 PM

Quote:

Originally posted by dishawjp
Just a thought,

How about trying:
rm -f test.test_split0*

then:
rm -f test.test_split1*

and etc.

The rm command may have trouble deleting 1,000,000 files at a time, but maybe can handle 100,000. Another possibility might be to delete the entire directory they are in if they are the only files in the directory using the "rm -rf" command.

Never had this problem myself. Good luck.

Jim Dishaw

Well, it didn't work :( - same error, and the idea about deleting the whole directory wouldn't work for me, as I actually have many files in there.

I think I could probably do something with piping the output of 'find' to some command, but, I wouldn't know which that command would be, as I don't think rm can handle input that way.

raxxor 04-09-2004 08:34 PM

Try to do...

Code:

for X in `ls test.test*` ; do rm -f $X; done
or verbose :)
Code:

for X in `ls test.test*` ; do rm -f $X; echo Removing $X; done
I had a problem similar to this and the above worked...

yelo 04-09-2004 09:31 PM

Well, that comes back:

bash: /usr/bin/ls: Argument list too long

:(

So, to make this semi-educational, my problem is that I'm running into the 4096-byte limit on the input buffer, right? Correct me if I'm wrong

Any other suggestions?

mikshaw 04-09-2004 09:53 PM

I don't think you'd need to do the ls thingy.
you could instead do:
for x in ./test.test_split* ; do rm -f $x; done

raxxor 04-09-2004 09:55 PM

My bad, heh

Actually I didn't use ls:

Code:

for X in test.test* ; do rm -f $X; done
Try that, I just made 1,000,000 files and tested it out ;)

mikshaw 04-09-2004 10:01 PM

That's even better...don't know why I added the "./" and "_split"

yelo 04-09-2004 10:30 PM

well, it's slowly cranking away at about 90/second...bash scripting isn't quite speedy...

Thanks for your help, and may my files bite the dust!

rogerdahl 04-10-2004 01:58 PM

xargs is your friend.

$ ls test.test* | xargs rm

Roger Dahl

mikshaw 04-10-2004 02:00 PM

Quote:

Well, that comes back:

bash: /usr/bin/ls: Argument list too long


All times are GMT -5. The time now is 07:33 AM.