LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 04-09-2004, 07:46 PM   #1
yelo
Member
 
Registered: Oct 2003
Distribution: Debian Unstable
Posts: 41

Rep: Reputation: 15
rm can't delete 999999 files in one directory


I have a funny problem. I am working on an application which takes a large file and splits it up into smaller ones(like for floppy disks, etc), just with a few special features, so a normal file splitter just won't do the job for me. anyway, I messed up, and created exactly 999,999 empty files into a directory with other stuff in it. the files are all of the form test.test_split00001, test.test_split 000002...test.test_split999999. so, I think ok, just an rm -f test.test*, right? WRONG. I get this error message:

bash: /usr/bin/rm: Argument list too long

the first thing I think is that's funny! I've finally found out how to make the rm command fail . but, how would I go about deleteing all those files now? I tried opening the folder with rox, but once it hit 700mb of used memory, during the loading of the folder(I only have 256 physical + 1 gb swap, so it was major swappage), I gave up, and used good old xkill to make it leave...

However, now it's not so funny...and I need help, which is why I turned to linuxquestions.org for advice. I really don't want to leave them all there.

Last edited by yelo; 04-09-2004 at 07:48 PM.
 
Old 04-09-2004, 08:05 PM   #2
dishawjp
Member
 
Registered: Dec 2003
Location: Central New York
Distribution: RH/Fedora/SUSE
Posts: 64

Rep: Reputation: 15
Just a thought,

How about trying:
rm -f test.test_split0*

then:
rm -f test.test_split1*

and etc.

The rm command may have trouble deleting 1,000,000 files at a time, but maybe can handle 100,000. Another possibility might be to delete the entire directory they are in if they are the only files in the directory using the "rm -rf" command.

Never had this problem myself. Good luck.

Jim Dishaw
 
Old 04-09-2004, 08:15 PM   #3
yelo
Member
 
Registered: Oct 2003
Distribution: Debian Unstable
Posts: 41

Original Poster
Rep: Reputation: 15
Quote:
Originally posted by dishawjp
Just a thought,

How about trying:
rm -f test.test_split0*

then:
rm -f test.test_split1*

and etc.

The rm command may have trouble deleting 1,000,000 files at a time, but maybe can handle 100,000. Another possibility might be to delete the entire directory they are in if they are the only files in the directory using the "rm -rf" command.

Never had this problem myself. Good luck.

Jim Dishaw
Well, it didn't work - same error, and the idea about deleting the whole directory wouldn't work for me, as I actually have many files in there.

I think I could probably do something with piping the output of 'find' to some command, but, I wouldn't know which that command would be, as I don't think rm can handle input that way.
 
Old 04-09-2004, 08:34 PM   #4
raxxor
LQ Newbie
 
Registered: Jul 2003
Distribution: Fedora Core 2
Posts: 12

Rep: Reputation: 0
Try to do...

Code:
for X in `ls test.test*` ; do rm -f $X; done
or verbose
Code:
for X in `ls test.test*` ; do rm -f $X; echo Removing $X; done
I had a problem similar to this and the above worked...
 
Old 04-09-2004, 09:31 PM   #5
yelo
Member
 
Registered: Oct 2003
Distribution: Debian Unstable
Posts: 41

Original Poster
Rep: Reputation: 15
Well, that comes back:

bash: /usr/bin/ls: Argument list too long



So, to make this semi-educational, my problem is that I'm running into the 4096-byte limit on the input buffer, right? Correct me if I'm wrong

Any other suggestions?
 
Old 04-09-2004, 09:53 PM   #6
mikshaw
LQ Addict
 
Registered: Dec 2003
Location: Maine, USA
Distribution: Slackware/SuSE/DSL
Posts: 1,320

Rep: Reputation: 45
I don't think you'd need to do the ls thingy.
you could instead do:
for x in ./test.test_split* ; do rm -f $x; done
 
Old 04-09-2004, 09:55 PM   #7
raxxor
LQ Newbie
 
Registered: Jul 2003
Distribution: Fedora Core 2
Posts: 12

Rep: Reputation: 0
My bad, heh

Actually I didn't use ls:

Code:
 for X in test.test* ; do rm -f $X; done
Try that, I just made 1,000,000 files and tested it out
 
Old 04-09-2004, 10:01 PM   #8
mikshaw
LQ Addict
 
Registered: Dec 2003
Location: Maine, USA
Distribution: Slackware/SuSE/DSL
Posts: 1,320

Rep: Reputation: 45
That's even better...don't know why I added the "./" and "_split"
 
Old 04-09-2004, 10:30 PM   #9
yelo
Member
 
Registered: Oct 2003
Distribution: Debian Unstable
Posts: 41

Original Poster
Rep: Reputation: 15
well, it's slowly cranking away at about 90/second...bash scripting isn't quite speedy...

Thanks for your help, and may my files bite the dust!
 
Old 04-10-2004, 01:58 PM   #10
rogerdahl
LQ Newbie
 
Registered: Apr 2004
Posts: 1

Rep: Reputation: 0
xargs is your friend.

$ ls test.test* | xargs rm

Roger Dahl
 
Old 04-10-2004, 02:00 PM   #11
mikshaw
LQ Addict
 
Registered: Dec 2003
Location: Maine, USA
Distribution: Slackware/SuSE/DSL
Posts: 1,320

Rep: Reputation: 45
Quote:
Well, that comes back:

bash: /usr/bin/ls: Argument list too long
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to delete files with 0k file size in a directory justgiver Linux - Newbie 4 01-28-2008 04:56 AM
shell script: delete all directories named directory.# except directory.N brian0918 Programming 3 07-13-2005 06:54 PM
Auto Delete oldest files @ratex when directory is 98% full jmanjohn61 Linux - Software 1 04-05-2005 03:44 PM
How do you delete a directory? R4z0r Linux - Newbie 1 12-25-2002 11:16 AM
How can I delete a directory and its files? OrganicX Linux - General 6 09-01-2002 04:25 AM


All times are GMT -5. The time now is 07:15 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration