[SOLVED] bash - print in a file on the same first line in a loop

David the H. · 01-27-2013, 04:35 PM

Yeah, ok. I see your point. Don't depend on it if the array is following an external command. But at least echo and other shell built-ins seem safe.

Edit: I just tried it again with file and stat, and the limit for my system is somewhere between 50 and 70k files. Much higher than I expected. And indeed, built-ins aren't affected, even up to 100k.

konsolebox · 01-27-2013, 06:59 PM

Quote:

Originally Posted by jpollard

It isn't the size of the data array, it is the number of parameters allowed for a command. echo, being built in just might be able to bypass that, but in the general case, you are limited. That is why the loops using echo -n work on any data, not just small data.

Excuse me but this expands to only one argument though:

Code:

"${lines[*]}"

So generally it's safe. It's actually about how much data "" could represent (which I think should be quite sufficient).

jpollard · 01-28-2013, 01:49 AM

Even one argument has limits - there is only so much that can be passed between processes this way.

Code:

   Limits on size of arguments and environment
       Most  UNIX  implementations  impose some limit on the total size of the
       command-line argument (argv) and environment (envp) strings that may be
       passed to a new program.  POSIX.1 allows an implementation to advertise
       this limit using the ARG_MAX constant (either defined in <limits.h>  or
       available at run time using the call sysconf(_SC_ARG_MAX)).

       On  Linux prior to kernel 2.6.23, the memory used to store the environ‐
       ment and argument strings was limited to 32 pages (defined by the  ker‐
       nel  constant  MAX_ARG_PAGES).  On architectures with a 4-kB page size,
       this yields a maximum size of 128 kB.

       On kernel 2.6.23 and later, most architectures  support  a  size  limit
       derived  from  the  soft RLIMIT_STACK resource limit (see getrlimit(2))
       that is in force at the time of the execve() call.  (Architectures with
       no  memory  management  unit are excepted: they maintain the limit that
       was in effect before kernel 2.6.23.)  This change  allows  programs  to
       have  a much larger argument and/or environment list.  For these archi‐
       tectures, the total size is limited to 1/4 of the allowed  stack  size.
       (Imposing  the  1/4-limit  ensures that the new program always has some
       stack space.)  Since Linux 2.6.25, the kernel  places  a  floor  of  32
       pages  on  this size limit, so that, even when RLIMIT_STACK is set very
       low, applications are guaranteed to have at least as much argument  and
       environment  space  as was provided by Linux 2.6.23 and earlier.  (This
       guarantee was not provided in Linux 2.6.23 and 2.6.24.)   Additionally,
       the  limit per string is 32 pages (the kernel constant MAX_ARG_STRLEN),
       and the maximum number of strings is 0x7FFFFFFF.

(from the manpage of execve)

Usually quite large by default...

But 32 pages is only 131072 bytes for a single string.

ARG_MAX for posix is limited to 4096 (not a detailed hunt though) and ARG_MAX for linux is 131072 (the 32 page limit, though there is a following note in sys/param.h that the value from the kernel headers is wrong)

konsolebox · 01-28-2013, 02:00 AM

You're not passing an argument to a command but to a builtin though so that won't apply.

jpollard · 01-28-2013, 04:32 AM

Quote:

Originally Posted by konsolebox

You're not passing an argument to a command but to a builtin though so that won't apply.

Depends on the shell. Bash, true it doesn't apply. But there is the possibility of aliasing causing a real process to be invoked (/bin/echo does exist for that purpose) and when that happens, things will fail.

Yes, you can do it, just not a portable construct.

konsolebox · 01-28-2013, 05:16 AM

Quote:

Originally Posted by jpollard

Depends on the shell. Bash, true it doesn't apply. But there is the possibility of aliasing causing a real process to be invoked (/bin/echo does exist for that purpose) and when that happens, things will fail.

It's very rare but if aliases or functions are expected we could just use builtin to be safe:

Code:

builtin echo "${AVAR[@]}"

Quote:

Yes, you can do it, just not a portable construct.

I'm not sure if other shells support arrays and that arrays inside a pair of double quotes are also expanded that way so portability is not really a thing that would make a difference I think. We still check other things when porting a code to another shell anyway.

And like I said what matters really is how much data "" could represent. The amount of data presentable could obviously be doubted and will be questioned based on implementation. Still I find it in most cases sufficient. Anyhow the problem about multiple arguments certainly won't apply.