LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 11-12-2007, 03:13 AM   #1
ust
Senior Member
 
Registered: Mar 2003
Location: fasdf
Distribution: Debian / Suse /RHEL
Posts: 1,130

Rep: Reputation: 31
Argument list too long


I want to chmod the file , if I use full path :/tmp/db_rec/*.txt" , it pop the message "Argument list too long" , but if use chmod 666 ./*.txt , it is OK , can advise what is wrong ? if I want to use full path , what can I do ? thx

/bin/chmod 666 /tmp/db_rec/*.txt
bash: /bin/chmod: Argument list too long

cd /tmp/db_rec
chmod 666 ./*.txt
 
Old 11-12-2007, 03:46 AM   #2
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1982Reputation: 1982Reputation: 1982Reputation: 1982Reputation: 1982Reputation: 1982Reputation: 1982Reputation: 1982Reputation: 1982Reputation: 1982Reputation: 1982
Including the full path you have reached the limit of the input buffer size. To workaround this you can do, for example
Code:
find /tmp/db_rec -name "*.txt" -print0 | xargs -0 chmod 666
 
Old 11-12-2007, 03:57 AM   #3
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400
Hi,

Nothing is wrong, this is how bash works and reacts if you encounter one of the internal limits (the length of the argument list in this case).

The reason why chmod 666 /tmp/db_rec/*.txt does not work and chmod 666 *.txt does has to do with the 'extra' /tmp/db_rec part (for every .txt).

Example:

$ ls /tmp/*txt

/tmp/foo.txt /tmp/bar.txt

$ cd /tmp
$ ls *txt

foo.txt bar.txt

As you can see the first example takes up 25 chars and the second 'only' 13.

How to circumvent this:

Use find and/or xargs

find /tmp/db_rec -name "*txt" -exec chmod 666 {} \;

or (supposed to be faster with a long argument list):

find /tmp/db_rec -name "*txt" | xargs chmod 666

Hope this helps.
 
Old 11-12-2007, 09:32 AM   #4
nx5000
Senior Member
 
Registered: Sep 2005
Location: Out
Posts: 3,307

Rep: Reputation: 57
Thumbs up

I agree with the answers, I simply want to add an information:

Great change for linux:

Quote:
Originally Posted by http://wiki.kernelnewbies.org/Linux_2_6_23
2.5. Variable argument length

From a Slashdot interview to Rob Pike: I didn't use Unix at all, really, from about 1990 until 2002, when I joined Google. (I worked entirely on Plan 9, which I still believe does a pretty good job of solving those fundamental problems.) I was surprised when I came back to Unix how many of even the little things that were annoying in 1990 continue to annoy today. In 1975, when the argument vector had to live in a 512-byte-block, the 6th Edition system would often complain, 'arg list too long'. But today, when machines have gigabytes of memory, I still see that silly message far too often. The argument list is now limited somewhere north of 100K on the Linux machines I use at work, but come on people, dynamic memory allocation is a done deal!

While Linux is not Plan 9, in 2.6.23 Linux is adding variable argument length. Theoretically you shouldn't hit frequently "argument list too long" errors again, but this patch also limits the maximum argument length to 25% of the maximum stack limit (ulimit -s).
Code: (commit)
 
Old 10-25-2009, 05:16 AM   #5
omisosiru
LQ Newbie
 
Registered: Oct 2009
Posts: 2

Rep: Reputation: 0
Made the script to resolve the problem

# cat chmod_many.sh
if [ $# -ne 3 ]; then
echo First argument : a command you want to conduct.
echo Second argument : a privilege which you want to change the files to.
echo Third argument : a directory where you want to change privileges of all files.
echo Example1: $0 chown you:you /home/you/something
echo Example2: $0 chmod 666 /home/you/something
exit
fi


cd $3
ls > /tmp/chmod_many.sh.tmp
while read line
do
echo $1 $2 $line
$1 $2 $line
done < /tmp/chmod_many.sh.tmp

rm -f /tmp/chmod_many.sh.tmp
 
Old 10-25-2009, 06:51 AM   #6
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400
@omisosiru: Your script will only work if there are no special characters (a space to name just one) in the filenames......

Change these 2 lines:

echo $1 $2 $line
$1 $2 $line

to

echo "$1" "$2" "$line"
"$1" "$2" "$line"


You also need to add the hashbang (#!/bin/bash) at the start of the file and the temporary file is not needed.

Here's a rewrite of your script:
Code:
#!/bin/bash
if [ $# -ne 3 ]
then
  echo First argument : a command you want to conduct.
  echo Second argument : a privilege which you want to change the files to.
  echo Third argument : a directory where you want to change privileges of all files.
  echo Example1: $0 chown you:you /home/you/something
  echo Example2: $0 chmod 666 /home/you/something
  exit 1
fi

cd $3
ls | while read line
do
  echo "$1" "$2" "$line"
  "$1" "$2" "$line"
done
What if there are subdirs in the target (although not part of the OP's initial question)?

Just trying to give you a better understanding of scripting.......
 
Old 10-25-2009, 07:07 AM   #7
i92guboj
Gentoo support team
 
Registered: May 2008
Location: Lucena, Córdoba (Spain)
Distribution: Gentoo
Posts: 4,083

Rep: Reputation: 404Reputation: 404Reputation: 404Reputation: 404Reputation: 404
Why you shouldn't be parsing the ls output like that.

Either use "for file in /patch/to/*" or if you are not using bash use the "find" command, preferably with -print0
 
Old 10-25-2009, 09:27 AM   #8
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400
@i92guboj: I agree. But I don't know what level of scripting omisosiru has (I do have an idea, but you know what they say about assuming).

Personally I think a script for this isn't needed, the find solutions posted in post #2 and #3 are the way to go.
 
Old 10-25-2009, 10:03 AM   #9
i92guboj
Gentoo support team
 
Registered: May 2008
Location: Lucena, Córdoba (Spain)
Distribution: Gentoo
Posts: 4,083

Rep: Reputation: 404Reputation: 404Reputation: 404Reputation: 404Reputation: 404
Yes. To tell the truth, I don't really understand the point of the script at all, unless it's for training or didactic purposes.

About the level of scripting... well, bad practices are bad practices. Having no knowledge is no excuse for that. If I have to learn something, I'd rather learn it the right way from the start, instead of learning a bad thing and then having to cure the disease and re-learn the correct one.
 
Old 10-25-2009, 10:12 AM   #10
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400
I have to agree with that, on both points.....
 
Old 10-26-2009, 10:02 AM   #11
omisosiru
LQ Newbie
 
Registered: Oct 2009
Posts: 2

Rep: Reputation: 0
Ah yeah both of you are right, I made a mistake.

# find . -name *
-bash: /usr/bin/find: Argument list too long

I thought find can't be used for too many files in one directory, but actually it can with this way:
# find . -name "*txt"

As you see, my level is low and we should use find and xargs in this situation.
 
Old 10-26-2009, 10:16 AM   #12
i92guboj
Gentoo support team
 
Registered: May 2008
Location: Lucena, Córdoba (Spain)
Distribution: Gentoo
Posts: 4,083

Rep: Reputation: 404Reputation: 404Reputation: 404Reputation: 404Reputation: 404
Quote:
Originally Posted by omisosiru View Post
Ah yeah both of you are right, I made a mistake.

# find . -name *
-bash: /usr/bin/find: Argument list too long

I thought find can't be used for too many files in one directory, but actually it can with this way:
# find . -name "*txt"

As you see, my level is low and we should use find and xargs in this situation.
There's no problem with the number of files, but with teh way that bash makes the substitutions and how they are given to find.

When you do this:

Code:
find . -name *
The real command is actually this:

Code:
find . -name <a list with all the files in current dir>
The asterisk doesn't reach the find tool, so find doesn't get a chance to handle it. To do that, you have to correctly quote the arguments, or at least escape the special character, so bash ignores it and it passes it to whatever command in in place without making any expansion. Either of these will work in this case:

Code:
find . -name \*
find . -name '*'
find . -name "*"
However, find has an even easier syntax for that:

Code:
find .
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
cp lots of files argument list too long dtra Linux - Software 4 07-07-2005 09:14 AM
argument list to long for linux commands supersucker Linux - Newbie 1 05-10-2005 07:36 AM
-bash: /bin/mv: Argument list too long balanagireddy Linux - General 2 07-27-2004 11:19 PM
Argument list too long trutnev Linux - General 3 04-22-2004 04:32 PM
/bin/rm: Argument list too long dragon49 Linux - Software 1 09-02-2003 12:27 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 01:21 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration