Bash: Error with eval when array name contains a hyphen

Johnburrell · 05-04-2012, 12:51 PM

I have 2 lines in a file (testfile) like so -

apache=( apache apache- apache-ant )
apache-ant=( apache apache- apache-ant )

I'm trying to get hold of the arrays using a bash script like this-

Code:

sec=testfile
IFS='
'
afile=( $( < $sec ) )
for f  in ${afile[@]}
do
    PkgName="${f%=*}"
    eval "$f"
    eval altnames="(\${$PkgName[@]})"
done

The first line in the file works fine and altnames contains the 3 names but the second line gives-

eval: line 9: syntax error near unexpected token `apache'

It doesn't like the hyphen in the array name apache-ant. How can I get eval to accept the hyphen?

Thanks

jb.

millgates · 05-04-2012, 01:20 PM

please don't use eval. It makes bash cry.

Code:

eval "$f"

why don't you just source the file?

Code:

eval altnames="(\${$PkgName[@]})"

why don't you use an associative array or something?

Quote:

Originally Posted by Johnburrell

It doesn't like the hyphen in the array name apache-ant. How can I get eval to accept the hyphen?

eval does not really care. It's just that variables in bash cannot contain '-'.

Johnburrell · 05-04-2012, 02:26 PM

In answer to your questions -

I eval $f because I need two passes.
On the first pass $f becomes apache=( apache apache- apache-ant )
and on the second pass the array apache gets set up.
With eval altnames, on the first pass $PkgName gets expanded to apache
and on the second pass altnames=${apache[@]} so it contains the array values.
I'm sure there are other ways to do it, but I don't know how.

With regard to the original question, it seems that I can't use hyphens in variable names.
That's a bit of a pain because I want to match the names of Linux packages, a lot of which contain hyphens.

jb.

millgates · 05-04-2012, 03:13 PM

Ok, how about something like this?

Code:

#!/bin/bash

sec=testfile
declare -A altnames
declare -a packages

while read -r line; do
	if [[ "$line" =~ .*=.* ]]; then
		PkgName="${line%=*}"
		packages+=("$PkgName")
		line="${line#*(}"
		altnames[$PkgName]="${line%)*}"
	fi
done < "$sec"

In the end, the list of packages read will be in array packages and array altnames will contain strings containing the lists of alternatives. This is assuming that no package name contains spaces.

David the H. · 05-04-2012, 03:45 PM

eval is one of the most abused, and least secure, features in scripting. And in modern shells it's nearly always completely unnecessary. Most people who do so are usually trying to set up some kind of indirect or dynamic variable referencing, which can almost always be done more cleanly with an array of some kind.

How can I use variable variables (indirect variables, pointers, references) or associative arrays?
http://mywiki.wooledge.org/BashFAQ/006

As you have discovered, one of the big problems with trying to create dynamic variable names is that variable names can only contain letters, numbers, and the underscore, and cannot begin with a number (P.S. only the name has this restriction, not the contents). This can be a difficult restriction to overcome when trying to work with arbitrary text. That's what makes associative arrays so useful, as you can use any arbitrary text string as the index to an array element, even one stored in another variable or array.

And read here to see what the stink is with eval:

Eval command and security issues
http://mywiki.wooledge.org/BashFAQ/048

The short form: never use eval on any line that has variables with unknown content or commands that produce unknown output. If you don't know exactly what the eval'd command is doing at all times, you shouldn't be using it at all.

Edit: BTW, you should really take another look at millgate's first suggestion and just source the file (after fixing the contents to contain properly-defined variable names). Sourcing means incorporating an external file into your script, as if it were written there directly, and so all commands and variable settings in it will be executed at the position of the sourcing.

The source command is written simply as ".":

Code:

if [[ -r testfile ]]; then
	. testfile
else
	echo "testfile not found or readable."
	exit 1
fi

echo "${apache[@]}"
echo "${apache_ant[@]}"

Johnburrell · 05-04-2012, 04:25 PM

Alright, I'll take your advice and get rid of the evals and use associative arrays.

Thanks for your help.

jb.

Johnburrell · 05-09-2012, 12:33 PM

I thought I should post my eventual solution.
In the end I didn't use associative arrays. It's not necessary.
I want the 3 names in the () put in arrays. I did it like this:

Code:

  ic=0
  while read n1 n2 n3 n4
  do
    name1[ic]=$n2
    name2[ic]=$n3
    name3[ic]=$( echo $n4 | awk 'sub("..$", "")' ) # strip last 2 chars
    ((ic++))
  done < testfile

The awk bit removes the last 2 chars from name3 because I wanted apache-ant not apache-ant )

I think this is quite an elegant solution and removes the evals, which people seem to think are a significant security risk. This might help someone looking to do a similar thing.

jb.

millgates · 05-09-2012, 12:50 PM

Quote:

Originally Posted by Johnburrell

I thought I should post my eventual solution.
In the end I didn't use associative arrays. It's not necessary.
I want the 3 names in the () put in arrays. I did it like this:

Code:

  ic=0
  while read n1 n2 n3 n4
  do
    name1[ic]=$n2
    name2[ic]=$n3
    name3[ic]=$( echo $n4 | awk 'sub("..$", "")' ) # strip last 2 chars
    ((ic++))
  done < testfile

The awk bit removes the last 2 chars from name3 because I wanted apache-ant not apache-ant )

I think this is quite an elegant solution and removes the evals, which people seem to think are a significant security risk. This might help someone looking to do a similar thing.

jb.

That's assuming
1) there will allways be 3 elements in each assignment
2) there will be spaces after the first and before the last parenthesis

also,

Code:

name3[ic]=$( echo $n4 | awk 'sub("..$", "")' )

could be done without awk like this:

Code:

name3[ic]="${n4%??}"

lastly, you can just append to arrays

Code:

name1+=("$n2")

saving you the need for the ic variable.

Johnburrell · 05-09-2012, 03:07 PM

Thanks, yes it's better to get rid of the awk. Quicker to let bash do it.

I have one set of files, all of which have the above format with spaces and 3 names.
However, I have another set which are like this:
packagedep=( dep1 dep2 dep3 dep4 ) which define a package's dependencies. Depn can have n=1 or up to may be 20. To read these into an array I do:

Code:

  while read depline
  do
        ic=0
        ndeps=($depline)
        nlen=${#ndeps[@]}
        for (( i=1; i<((nlen-1)); i++ ))
        do
          tdep[ic]=${ndeps[$i]}
          ((ic++))
        done
           ... test whether dependencies are met ...
  done < dependencyFile

I use the counter just because I prefer it that way and I have to maintain the code! BTW, bash doesn't seem to care if one uses ic or $ic in the tdep array counter. But the code works well and is fast - and no trimming needed.

jb.

David the H. · 05-10-2012, 04:38 PM

Quote:

Originally Posted by Johnburrell

I use the counter just because I prefer it that way and I have to maintain the code! BTW, bash doesn't seem to care if one uses ic or $ic in the tdep array counter.

This is because the index field for regular arrays operates in an arithmetic context. Since the only valid characters in arithmetic contexts are digits and math operators, any straight alphanumeric strings are automatically assumed to be variables. Only complex variable expansions need the full "$" form.

So you can do whatever arithmetic operations you want inside array indexes, as well as increment variables directly, when working in a loop.

(Associative arrays do not operate in an arithmetic context, BTW, so $ is necessary when using them.)

Speaking of arithmetic, you also don't need to use one ((..)) inside another. At most you may need to use a single set of parens for grouping expressions. See the bash man page ARITHMETIC EVALUATION section for details on operator precedence.

Finally, read can also populate an array directly with the -a option, allowing you to eliminate one more step.

So you can change your above code to this:

Code:

ic=0
while read -a ndeps; do

	for (( i=1 ; i < ${#ndeps[@]} -1 ; i++ )); do

		tdep[ic++]=${ndeps[i]}

	done

done < dependencyFile

I still think it would probably be easier to just source the files directly and use the array setting code already-available in the file. All this data parsing seems none-too-robust to me.

Johnburrell · 05-11-2012, 12:44 PM

I haven't sourced the files because I don't know how to get hold of the arrays that the sourcing will set up.

Take an example. Suppose the sourcing will set up an array:

libsoupdep=( glib-networking libxml2 )

In the script the package names are in a variable PkgName.
So I'll have ${PkgName}dep as the name of the dependency array.
I don't know how to get hold of libsoupdep when all I know is ${PkgName}dep. That's why I read the file so I can compare the variable holding the libsoupdep line with ${PkgName}dep.

I could do the indirect substitution using eval, but that is back to where I started :-)