BASH-Adding array element: Naming issue using array[${#array[*]}]=5

calvarado777 · 07-24-2013, 02:16 PM

I am writing a bash script and according to interwebs, when trying to add an element to an array, the syntax is as follows:

array[${#array[*]}]=5

In my case, the array name changes depending on which loop iteration we are on... and I use variable affCtr as my counter to determine which array I will be appending to.

So it should technically look something like this:

core${affCtr}[${#core${affCtr}[*]}]=$tempID

The problem is highlighted-- when I introduce that additional '$' since I want to use a variable there, I somehow lose the meaning of what the '*' should be.

Do you guys have any idea on what might work?
I've tried various combinations of parentheses, back ticks, etc. and nothing seems to work...

Thanks in advance for any help!

rigor · 07-24-2013, 06:46 PM

Hi calvarado777!

Maybe this will help.

Code:

#! /bin/bash

# Create indexed arrays, arrays with numeric subscripts.
# All subscripts from zero to N-1 do *not* have to be used.
declare  -a  indexed_array1  indexed_array2 ;

# Create an associate array, an array with arbitrary strings as subscripts.
declare  -A  associative_array ;


# Add elements to each array.

indexed_array1[ 29 ]="this is the element with index 29.  Index numbers start at zero." ;
indexed_array2[ 13 ]="this is the element with index 13." ;

associative_array[ "fourty five" ]="this is the element with subscript 'fourty five'." ;


# Display the values of the added array elements.

echo ${indexed_array1[ 29 ]} ;
echo ${indexed_array2[ 13 ]} ;

echo ${associative_array[ "fourty five" ]} ;



# Add an element to an indexed array, in different ways.
indexed_array2[ 0 ]="I am the value of index 0." ;
echo "The number '"${#indexed_array2[*]}"' is the number of elements in the array, *_not_necessarily_* the number of last element." ;
indexed_array2[ ${#indexed_array2[*]} ]="I am the element with index 2 of the array." ;
affCtr="indexed_array2" ;
cmd="$affCtr[1]='This is the element with index 1 of the array.'" ;
echo $cmd ;
eval $cmd ;


# Display the values in an indexed array.
echo "Index 0:"  ${indexed_array2[0]} ;

echo "Index 1:"  ${indexed_array2[1]} ;

echo "Index 2:"  ${indexed_array2[2]} ;

echo "Index 13:"  ${indexed_array2[13]} ;

grail · 07-24-2013, 07:13 PM

Actually the issue being faced is that you cannot use a variable name as the array name. You could potentially use eval but for many reasons I try to avoid it like the plague if possible.

So looking at your problem I would suggest using a case statement based on your variable 'affCtr' and then append to the appropriate array as required.

David the H. · 07-25-2013, 08:56 AM

What grail said.

An array is really just a buffed up variable and has the same general limitations in naming and setting:

How can I use variable variables (indirect variables, pointers, references) or associative arrays?
http://mywiki.wooledge.org/BashFAQ/006

What you're apparently wanting to do is create a multi-dimensional array, which isn't available in bash. But you can generally simulate one with an associative array and a two-part index string.

Code:

declare -A myarray
ind_1="1st_list"
ind_2="2nd_list"

myarray["$ind_1:0"]='first entry in list one'
myarray["$ind_1:1"]='second entry in list one'

myarray["$ind_2:0"]='first entry in list two'
myarray["$ind_2:1"]='second entry in list two'

Since an associative array index can be any text string, you can create whatever system you want to specify elements. You could even store the index strings in another array (regular or associative) and use that to track them.

There's a (usually) minor problem in that associative array indexes don't sort naturally, but that can usually be overcome if necessary.

If you explain your ultimate purpose in more detail then we can probably give you more explicit advice.

PS: Please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques. Thanks.

konsolebox · 07-25-2013, 10:27 PM

Using case statements would add complexity and using other tricks besides eval for indirect addressing could sometimes lead to unexpected reinterpretation of data like losing other lines after one. At least the complexity of it is more dangerous or evil compared to eval and one would wonder why people would go for the trouble just for the sake of keeping their pride with declaring eval as evil. So simply use eval:

Code:

eval "core${affCtr}[\${#core${affCtr}[*]}]=\$tempID"

The simple rule is just imagine how command appear after it is reinterpreted, and placing it in one single string argument within one "" is a start. For beginners, you can use echo instead of eval to know if you're doing the right thing. Consider also that variables may contain spaces and other values.

Code:

echo "core${affCtr}[\${#core${affCtr}[*]}]=\$tempID"

When Bash 4.3 is realeased, you could expect declare -n to be useful. I tried to suggest having another builtin command like setvalue which would have been a general command for assigning values to normal variables, array, and associative arrays but too bad, looks like it was rejected. And the danger of using it was just similar to declare -n for having wrong variable names as argument.

grail · 07-26-2013, 08:20 AM

No pride here ... once bitten twice shy ... after having an rm take out a third of a system before I was able to control out of it was enough for me. So my general rule of thumb is why invite the danger if a safer and easier way exists.

I do agree that in this case there will hope fully be no unwarranted issues.

David the H. · 07-26-2013, 09:28 AM

Using eval cannot "sometimes lead to unexpected reinterpretation of data" as well? Huh?

But as I've explained before, I don't consider eval to be "evil", just dangerous, non-transparent, and tricky for inexperienced people to use safely, and that it shouldn't be recommended as a solution except when absolutely necessary. I also believe that modern shells have enough features in place to make its use increasingly unnecessary. Too many people seem too eager to pull it out at every opportunity, the proverbial sledgehammer approach to scripting.

(Ok, so maybe I do sometimes go too far in the other direction, but better safe than sorry.)

I also think that indirect variable creation has many similar transparency and complexity (and even security) problems, and should be avoided too when possible. For example, you have to ensure that the sub-variables you're using to create the final variable name (e.g. $affCtr) only contains valid name characters in it.

If you do decide indirect referencing is the way to go, then at least bash does provide several other ways to create them besides eval (read, declare and printf, as addressed in my previous link), which at the least don't require as much effort to sanitize (syntax escaping, etc) before using.

Then, even if you get them set safely, correctly referencing them later can still be a job in itself.

I agree that if and when bash gets -n indirect referencing it will be easier and safer, but for now, associative arrays and even things like parsing with case statements are to my mind much safer and more transparent than anything you can do with eval or referencing.

konsolebox · 07-26-2013, 10:43 AM

Quote:

Originally Posted by David the H.

Using eval cannot "sometimes lead to unexpected reinterpretation of data" as well? Huh?

If you didn't understand it well, the solution of eval as provided above for indirect addressing would not lead to reinterpretation of data, unlike on read which could cut data on the first line, and printf which could reinterpret escape sequences and other stuffs. (Corrected, read below.) The added parameter-commands like IFS=xyz and make it even more complicated as well which if one beginner would look after each of those commands, eval would be expected as easiest to learn with lesser unexpected results.

Quote:

But as I've explained before, I don't consider eval to be "evil", just dangerous, non-transparent, and tricky for inexperienced people to use safely, and that it shouldn't be recommended as a solution except when absolutely necessary.

And I find you believe that choosing more complex solutions which in most cases require bloated number of lines are far better than learning how to properly use eval.

Quote:

I also believe that modern shells have enough features in place to make its use increasingly unnecessary.

I'm sorry but it's still really not yet enough. Everything are still workarounds. One solution could not fit as a general form for every situation.

Quote:

Too many people seem too eager to pull it out at every opportunity, the proverbial sledgehammer approach to scripting.

Aren't those that try to avoid eval and use other workaround solutions like that?

Quote:

(Ok, so maybe I do sometimes go too far in the other direction, but better safe than sorry.)

And that applies only to conservatives.

Quote:

I also think that indirect variable creation has many similar transparency and complexity (and even security) problems, and should be avoided too when possible.

And in most cases require larger more bloated old-school redundant monolithic codes which are impractical for new age scripting.

Quote:

For example, you have to ensure that the sub-variables you're using to create the final variable name (e.g. $affCtr) only contains valid name characters in it.

So would that apply in read, printf, and even with the new declare -n.

Quote:

If you do decide indirect referencing is the way to go, then at least bash does provide several other ways to create them besides eval (read, declare and printf, as addressed in my previous link), which at the least don't require as much effort to sanitize (syntax escaping, etc) before using.

Then, even if you get them set safely, correctly referencing them later can still be a job in itself.

I agree that if and when bash gets -n indirect referencing it will be easier and safer, but for now, associative arrays and even things like parsing with case statements are to my mind much safer and more transparent than anything you can do with eval or referencing.

They may appear safer to those who are afraid to use the eval command but every other solution besides eval is not in its general form. Eval could be used it any form of assignments in which other commands has limits. IFS='' read -rd '' VAR <<< "$XYZ"appends an extra line (\n) on the end of the value. printf -v is good for single-value assignments but is only available to Bash 3.1 or newer and is not even useful for array assignments until 4.2. Neither of both could be used to assign multiple values on an array like eval "$AVAR=(\"\${VALUES[@]}\"). Eval could be used in any form. Even the new declare -n could only be applied inside functions. Note that each of them would fail if a wrong variable name is passed so one could tell that there isn't much difference to the original solutions of eval compared to other quirks. It's only about learning how to use it. If I am to pick a solution why would pick one that's different for every variation. That would make my code extremely messy, even within a group of scripts solving different problems if not just on one.

And associative arrays on the other hand look dirty workarounds in which values which should have been placed on single variables are now indexed through hashes in one associative variable like a group.

PTrenholme · 07-26-2013, 09:48 PM

If you're still interested, here's a "eval"ed sample script:

Code:

$ cat example
#!/bin/bash
#
# Function to append $2 to the end of the indexed array $1
#
function append_to()
{
  local __size
  eval '__size=${#'${1}'[*]}'
  eval "${1}[${__size}]=${2}"
}
######################################
#
# Test program
#
######################################
for ((affCtr=0; affCtr<5; ++affCtr))
do
  declare -a core${affCtr}
  for ((tempID=0;tempID<$((affCtr + 1));++tempID))
  do
    append_to "core${affCtr}" "${tempID}"
  done
done

for __c in ${!core*}
do
  echo ""
  eval '__n=${#'${__c}'[*]}'
  echo "${__c} has ${__n} elements."
  for ((__i=0; __i < ${__n}; ++__i))
  do
    __s="$(printf '${%s[%s]}' ${__c} ${__i})"
    eval "__val=${__s}"
    printf "  %s = %s\n" ${__s}  ${__val}
  done
done

$ # Test program output

$ bash example

core0 has 1 elements.
  ${core0[0]} = 0

core1 has 2 elements.
  ${core1[0]} = 0
  ${core1[1]} = 1

core2 has 3 elements.
  ${core2[0]} = 0
  ${core2[1]} = 1
  ${core2[2]} = 2

core3 has 4 elements.
  ${core3[0]} = 0
  ${core3[1]} = 1
  ${core3[2]} = 2
  ${core3[3]} = 3

core4 has 5 elements.
  ${core4[0]} = 0
  ${core4[1]} = 1
  ${core4[2]} = 2
  ${core4[3]} = 3
  ${core4[4]} = 4

<edit>
Oh. note the use of single quotes the the eval argument to suppress inappropriate parsing.
</edit>