LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-27-2013, 05:35 PM   #16
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,837

Rep: Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981

Yeah, ok. I see your point. Don't depend on it if the array is following an external command. But at least echo and other shell built-ins seem safe.

Edit: I just tried it again with file and stat, and the limit for my system is somewhere between 50 and 70k files. Much higher than I expected. And indeed, built-ins aren't affected, even up to 100k.

Last edited by David the H.; 01-27-2013 at 05:48 PM.
 
Old 01-27-2013, 07:59 PM   #17
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
Quote:
Originally Posted by jpollard View Post
It isn't the size of the data array, it is the number of parameters allowed for a command. echo, being built in just might be able to bypass that, but in the general case, you are limited. That is why the loops using echo -n work on any data, not just small data.
Excuse me but this expands to only one argument though:
Code:
"${lines[*]}"
So generally it's safe. It's actually about how much data "" could represent (which I think should be quite sufficient).
 
Old 01-28-2013, 02:49 AM   #18
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,714

Rep: Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280
Even one argument has limits - there is only so much that can be passed between processes this way.
Code:
   Limits on size of arguments and environment
       Most  UNIX  implementations  impose some limit on the total size of the
       command-line argument (argv) and environment (envp) strings that may be
       passed to a new program.  POSIX.1 allows an implementation to advertise
       this limit using the ARG_MAX constant (either defined in <limits.h>  or
       available at run time using the call sysconf(_SC_ARG_MAX)).

       On  Linux prior to kernel 2.6.23, the memory used to store the environ‐
       ment and argument strings was limited to 32 pages (defined by the  ker‐
       nel  constant  MAX_ARG_PAGES).  On architectures with a 4-kB page size,
       this yields a maximum size of 128 kB.

       On kernel 2.6.23 and later, most architectures  support  a  size  limit
       derived  from  the  soft RLIMIT_STACK resource limit (see getrlimit(2))
       that is in force at the time of the execve() call.  (Architectures with
       no  memory  management  unit are excepted: they maintain the limit that
       was in effect before kernel 2.6.23.)  This change  allows  programs  to
       have  a much larger argument and/or environment list.  For these archi‐
       tectures, the total size is limited to 1/4 of the allowed  stack  size.
       (Imposing  the  1/4-limit  ensures that the new program always has some
       stack space.)  Since Linux 2.6.25, the kernel  places  a  floor  of  32
       pages  on  this size limit, so that, even when RLIMIT_STACK is set very
       low, applications are guaranteed to have at least as much argument  and
       environment  space  as was provided by Linux 2.6.23 and earlier.  (This
       guarantee was not provided in Linux 2.6.23 and 2.6.24.)   Additionally,
       the  limit per string is 32 pages (the kernel constant MAX_ARG_STRLEN),
       and the maximum number of strings is 0x7FFFFFFF.
(from the manpage of execve)

Usually quite large by default...

But 32 pages is only 131072 bytes for a single string.

ARG_MAX for posix is limited to 4096 (not a detailed hunt though) and ARG_MAX for linux is 131072 (the 32 page limit, though there is a following note in sys/param.h that the value from the kernel headers is wrong)
 
Old 01-28-2013, 03:00 AM   #19
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
You're not passing an argument to a command but to a builtin though so that won't apply.
 
Old 01-28-2013, 05:32 AM   #20
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,714

Rep: Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280Reputation: 1280
Quote:
Originally Posted by konsolebox View Post
You're not passing an argument to a command but to a builtin though so that won't apply.
Depends on the shell. Bash, true it doesn't apply. But there is the possibility of aliasing causing a real process to be invoked (/bin/echo does exist for that purpose) and when that happens, things will fail.

Yes, you can do it, just not a portable construct.
 
Old 01-28-2013, 06:16 AM   #21
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
Quote:
Originally Posted by jpollard View Post
Depends on the shell. Bash, true it doesn't apply. But there is the possibility of aliasing causing a real process to be invoked (/bin/echo does exist for that purpose) and when that happens, things will fail.
It's very rare but if aliases or functions are expected we could just use builtin to be safe:
Code:
builtin echo "${AVAR[@]}"
Quote:
Yes, you can do it, just not a portable construct.
I'm not sure if other shells support arrays and that arrays inside a pair of double quotes are also expanded that way so portability is not really a thing that would make a difference I think. We still check other things when porting a code to another shell anyway.

And like I said what matters really is how much data "" could represent. The amount of data presentable could obviously be doubted and will be questioned based on implementation. Still I find it in most cases sufficient. Anyhow the problem about multiple arguments certainly won't apply.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
BASH: Read entire file line in for loop clinton Programming 16 04-18-2013 01:06 PM
vim flags a for loop error depending on file extension in the 'for' line? Dyspeptic Curmudgeon Linux - General 2 09-11-2012 09:49 AM
How can I read a file line by line and add it to a loop in another file? astroumut Linux - Newbie 7 08-24-2009 05:37 AM
shell script loop though text file:-1 line at a time. knockout_artist Linux - Newbie 2 05-04-2008 07:58 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 04:57 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration