Bash array Add function example using indirect array reference as function argument

bobywelsh · 06-20-2010, 07:52 PM

Hello, 1st post

I looked on the net for such function or example and didin't find anything, thus after having made one i guess it would be legitimate to drop it to see what others thinks of it.

Mabey there is nicer way's, i wouldn't mind seeing them.

#!/bin/bash

addelementtoarray()
{

local arrayname=$1
local element=$2

#Evaluate reference-it localy..
eval arrayname=\${${arrayname}[@]}

#Apprend element argument to array
arrayname[${#arrayname[@]}]=$element

#Echo resulting array back
echo "${arrayname[@]}"

}

myarr=()

myarr[${#myarr[@]}]="a"
myarr[${#myarr[@]}]="ab"
myarr[${#myarr[@]}]="abc"
myarr[${#myarr[@]}]="abcd"
myarr[${#myarr[@]}]="z"

echo -e "Original printing Array :\n ${myarr[@]} \n\n"

myarr=(`addelementtoarray "myarr" "vaga1"`)
myarr=(`addelementtoarray "myarr" "vaga2"`)
myarr=(`addelementtoarray "myarr" "vaga3"`)

echo -e "Modified printing Array :\n ${myarr[@]} \n\n"

konsolebox · 06-22-2010, 05:10 AM

Here's my version:

Code:

# void addelementtoarray (string <array_name>, string <element>)
#
function addelementtoarray {
    eval "$1[\${#$1[@]}]=\$2"
}

array=()
addelementtoarray array "element with spaces"

bobywelsh · 06-22-2010, 11:46 AM

Nice but its virtually overwrite the existing array every time dosen't it.

Thus each call would need to be something like:
addelementtoarray array "${array[@]} elementX elementY elementZ"

David the H. · 06-22-2010, 03:19 PM

I came across this collection of scripts, but I haven't tried them yet.

http://github.com/cxreg/cxregs-bash-tools

The bash_arrays functions look pretty handy, and it includes a "push" function.

By the way, you might want to ensure that your functions work with bash v.4's new associative arrays.

bobywelsh · 06-22-2010, 04:51 PM

Interesting link thanks!

I guess a small switch with bash version and an extra argument
would fix it for bash 4 associative array.

I am currently working with bash 4 but i guess it wouldn't be to big of an issue to fix.

By the way, the reason why i dont directly modify my array
is to make it possible to assign the result to an other array.

This is interesting.

konsolebox · 06-22-2010, 11:40 PM

Quote:

Originally Posted by bobywelsh

Nice but its virtually overwrite the existing array every time dosen't it.

No 'array=()' is just the init part and all the function does is append.

Quote:

Thus each call would need to be something like:
addelementtoarray array "${array[@]} elementX elementY elementZ"

If you want to add multiple elements at once this can be done by:

Code:

# void addelementtoarray (string <array_name>, string <element>, ...)
#
function addelementtoarray {
    local R=$1 A
    shift

    for A; do
        eval "$R[\${#$R[@]}]=\$A"
    done

    # Or a one liner but more runtime expensive with large arrays since all elements will always expand.
    # In this method also, all element IDs are reset starting from 0.
    # Maybe this is also what you need since the IDs here does not need to be sorted.  A problem may occur on the former if an ID exist that is higher than the number of elements.
    # Only that it resets all IDs everytime.
    # eval "$1=(\"\${$1[@]}\" \"${2:@}\")"
}

array=()
addelementtoarray array a "b " "c d"
addelementtoarray array 1 2 3 4

addelementtoarray different_array 1 2 3 4   # works safely with method 2

bobywelsh · 06-27-2010, 09:12 PM

Well my point was not about array=() but with the way you need to call the function to actually add an element to the array.

Anyways i think the best an cleanest way comes for the link "David the H." Posted.

konsolebox · 07-01-2010, 02:14 AM

Thanks to this thread I remembered my TODO about compiling my array manipulation scripts into a single package. It almost took me a whole day writing/revising the code but I'm glad and I think it's worth it. You might also find this useful so I thought that I should also make a post of it.

Code:

#!/usr/bin/env bash


# ----------------------------------------------------------------------

# array.sh
#
# Provides functions for common manipulation of arrays in bash.
#
# Functions currently included are:
#
# array_push(), array_pop(), array_shift(), array_unshift(),
# array_get(), array_get_first(), array_get_last(), array_get_all(),
# array_set(), array_reset() and array_copy()
#
# This script requires bash versions 3.0 or newer.
#
# Author: konsolebox
# Copyright Free / Public Domain
# June 30, 2010
#
# Version: 0.1a

# Notes:
#
# * It's still not yet clear to me how multiple elements should be added
#   with the function array_unshift() so I haven't included support for
#   it yet.  Perhaps I need to take a look on how it is implemented in
#   some languages like Perl.

# ----------------------------------------------------------------------


if ! [[ BASH_VERSINFO -ge 3 ]]; then
	echo "array.sh: this script requires bash version 3.0 or newer."
	exit 1
fi


#### CONFIGURATION VARIABLES ####

# Change these default values to fit your usage standards.  These values
# can also be overridden by setting them before calling this script.  If
# you want to have these values customized after the script is called,
# you can create another version of this script that creates everything
# inside a setup function; you will then call the function after you set
# the values:
#
# ----
#
# function array_setup {
#     ....
#         function array_push {
#     ....
#     unset -f array_setup
# }
#
# ----
#
# include array.sh
#
# ARRAY_RESETINDICES_PUSH=true
# ....
# array_setup
#

: ${ARRAY_RESETINDICES_PUSH:=false}
: ${ARRAY_RESETINDICES_POP:=false}
: ${ARRAY_RESETINDICES_SHIFT:=false}
: ${ARRAY_RESETINDICES_UNSHIFT:=false}


#### FUNCTIONS ####

# void array_push (avar <array>, virtual <element_value>, ...)
#
# Pushes or appends one or more elements to an array.
#
if [[ $ARRAY_RESETINDICES_PUSH = true ]]; then
	function array_push {
		eval "shift; $1=(\"\${$1[@]}\" \"\$@\")"  ## probably faster than "${2:@}"?
	}
elif [[ BASH_VERSINFO -ge 4 || (BASH_VERSINFO -eq 3 && BASH_VERSINFO[1] -ge 1) ]]; then
	function array_push {
		eval "shift; $1+=(\"\$@\")"
	}
else
	function array_push {
		eval "
			local -a __T=(\"\${!$1[@]}\")
			local __A __I=\${__T[@]: -1:1}
			shift
			for __A; do
				$1[++__I]=\$__A
			done
		"

		# This function should be simpler if it accepts only a single
		# element:
		# eval "local -a __T=(\"\${!$1[@]}\"); $1[\${__T[@]: -1:1} + 1]=\$2"
	}
fi


# bool array_pop (avar <src>, [var <dest>])
#
# Pops an element from an array.  If <dest> is specified, the value will
# be set to it.  If a value can be popped (array is not empty), this
# function will return a true status code.
#
if [[ $ARRAY_RESETINDICES_POP = true ]]; then
	function array_pop {
		eval "
			[[ \"\${#$1[@]}\" -gt 0 ]] || return
			${2:+"$2=\${$1[@]: -1:1}"}
			$1=(\"\${$1[@]:0:\${#$1[@]} - 1}\")
		"
	}
else
	function array_pop {
		eval "
			[[ \"\${#$1[@]}\" -gt 0 ]] || return
			${2:+"$2=\${$1[@]: -1:1}"}
			local -a __T=(\"\${!$1[@]}\")
			unset \"$1[\${__T[@]: -1:1}]\"
		"
	}
fi


# bool array_shift (avar <array>, [var <dest>])
#
# Shifts or removes the first element in an array.
#
# If ARRAY_RESETINDICES_SHIFT is set to "true", faster method will be
# used and all indices will reset in a continuous fashion starting from
# 0; if set to "false", the index of the next first element will
# decrement to 0 and all other elements will follow with the same number
# of steps that the new first element has decremented.
#
# This function will return true status code if a value can be shifted
# out from the array.
#
if [[ $ARRAY_RESETINDICES_SHIFT = true ]]; then
	function array_shift {
		eval "
			case \"\${#$1[@]}\" in
			0)
				return 1
				;;
			1)
				${2:+"$2=\${$1[*]}"}
				$1=()
				;;
			*)
				${2:+"$2=\${$1[@]:0:1}"}
				local -a __T=(\"\${!$1[@]}\")
				$1=(\"\${$1[@]:\${__T[1]}}\")
				;;
			esac
		"
	}
else
	function array_shift {
		eval "
			case \"\${#$1[@]}\" in
			0)
				return 1
				;;
			1)
				${2:+"$2=\${$1[*]}"}
				$1=()
				;;
			*)
				${2:+"$2=\${$1[@]:0:1}"}
				local -a __T=(\"\${!$1[@]}\")
				unset $1\\[__T\\]
				local -i __I __D=__T[1]
				for __I in \"\${__T[@]:1}\"; do
					$1[__I - __D]=\${$1[__I]}
					unset $1\\[__I\\]
				done
				;;
			esac
		"
	}
fi


# void array_unshift (avar <array>, virtual <element_value>)
#
# Inserts a value as the new first element in an array.
#
# If ARRAY_RESETINDICES_UNSHIFT is set to "true", a simpler method will
# be used and all indices will reset starting from 0; if set to "false",
# all of the array's current elements will have their indices increment
# by 1.
#
if [[ $ARRAY_RESETINDICES_UNSHIFT = true ]]; then
	function array_unshift {
		eval "$1=(\"\$2\" \"\${$1[@]}\")"
	}
else
	function array_unshift {
		eval "
			local -a __T=(\"\${!$1[@]}\")
			local -i __I __J
			for (( __I = \${#__T[@]}; __I--; )); do
				__J=__T[__I]
				$1[__J + 1]=\${$1[__J]}
				unset $1\\[__J\\]
			done
			$1[0]=\$2
		"
	}
fi


# void array_get (avar <src>, var <dest>, uint <index>)
#
# Gets the value of an element in an array.
#
function array_get {
	eval "$2=\${$1[$3]}"
}


# bool array_get_first (avar <src>, var <dest>)
#
# Gets the value from the first element of an array.  Function will
# return true if source array is not empty.
#
function array_get_first {
	eval "[[ \"\${#$1[@]}\" -gt 0 ]] && $2=\${$1[@]:0:1}"
}


# bool array_get_last (avar <src>, var <dest>)
#
# Gets the value from the last element of an array.  Function will
# return true if source array is not empty.
#
function array_get_last {
	eval "[[ \"\${#$1[@]}\" -gt 0 ]] && $2=\${$1[@]: -1:1}"
}


# bool array_get_all (avar <src>, avar <dest>)
#
# Gets all the elements of an array.  Function will return true if
# source array is not empty.
#
function array_get_all {
	eval "[[ \"\${#$1[@]}\" -gt 0 ]] && $2=(\"\${$1[@]}\")"
}


# void array_set (avar <array>, int <index>, virtual <element_value>)
#
# Sets an element to an array.
#
function array_set {
	eval "$1[$2]=\$3"
}


# void array_reset (avar <array>, [virtual <element_value>, ...])
#
# Clears an array or resets it to optional elements.
#
function array_reset {
	eval "shift; $1=(\"\$@\")"
}


# bool array_copy (avar <src>, avar <dest>)
#
# Copies a whole array including index (key) struture.
#
# For a faster method that does not copy key structure, see
# array_get_all().
#
# This function will return true status code even if the source array
# is empty.  It may only return false if other problem occurs like for
# example if source or destination array is not an indexed array
# variable or if the two array variables are not compatible.
#
function array_copy {
	eval "$2=() && local -i __I && for __I in \${!$1[@]}; do $2[__I]=\${$1[__I]}; done"

	# I hope AVAR=() does not reset variable's attributes.  I've been
	# wondering if I should use 'unset AVAR\[*\]' instead.  The latter
	# version probably is a bit slower though since it's a builtin call.
}

The functions are there already tested.

By the way I also took a glance of the link that David the H. has posted but the code seems copyrighted so I considered not to look further. By this you might find the code and/or implementation of this script different from the other. This code is about 97% original (since 3% may be part of other sources). Most of it just came from my original project.

Still this code is also not yet final. I plan to make a real post of it some time when it's already mature or complete.

cxreg · 07-02-2010, 03:43 PM

konsolebox, the code on my github repository is distributed as either GPL or Artistic License. I've updated the code to more clearly and accurately reflect that. Thanks for pointing out that this was not obvious.

Any suggestions or improvements are welcome

The repo also contains some other tools, and I've written a post about them here if you'd like to read more:

http://cxreg.com/blag/2010/06/bash-a...ing-magic.html

konsolebox · 07-05-2010, 04:22 AM

Hello cxreg. Thanks for stopping by the thread and also for the info. Actually, I also would love to look at your codes knowing that I may also know some new tricks to manipulating arrays but since I always want to release my codes in copyright free form, I have no choice but not base any parts of them to any sources that are bound by GPL and similar licenses. Of course I can always freely look at other codes that have similar permissions to mine like 7-zip's source codes which is now distributed in public domain method.

GPL as a summary does not allow downgrading any permission if your not the original author of a code to a copyright free level... So a code that original started as GPL is bound to be GPL or GPL-like forever. That is the very reason I tend to create all my codes originally (as much as possible if not totally since sometimes it can't be really avoided; e.g. when studying/debugging source codes/visiting new sites/etc.). (Edit: The Artistic License may be less "holding" than GPL but it still have limitations. The modifier can still also find lesser freedom from it.)

But anyways if you like you can also base any improvements you can make to your codes from my codes above since it's copyright free. Perhaps it's also safe if things are done that way.

So that's all. Please feel free to improve your codes with mine just in case you also find them helpful.

Cheers.

konsolebox · 07-05-2010, 04:44 AM

This is a sample code I used to test the effectivity of the script. I also included the output. I considered posting this since maybe you'll also find it useful like knowing how it's implemented and finding the difference.

Code:

#!/usr/bin/env bash

function common {
	echo --------------------
	echo "array_push A X"
	A=(0 1 2 3 4)
	echo "before: $(set | grep ^A=)"
	array_push A X
	echo "after:  $(set | grep ^A=)"

	echo --------------------
	echo "array_push A X Y Z"
	A=(0 1 2 3 4)
	unset A\[1\] A\[3\]
	echo "before: $(set | grep ^A=)"
	array_push A X Y Z
	echo "after:  $(set | grep ^A=)"

	echo --------------------
	echo "array_pop A"
	A=(0 1 2 3 X)
	unset A\[1\] A\[3\]
	echo "before: $(set | grep ^A=)"
	array_pop A
	echo "after:  $(set | grep ^A=)"

	echo --------------------
	echo "array_pop A C"
	A=(0 1 2 X 4) C=\?
	unset A\[1\] A\[4\]
	echo "before: $(set | grep ^A=), $(set | grep ^C=)"
	array_pop A C
	echo "after:  $(set | grep ^A=), $(set | grep ^C=)"

	echo --------------------
	echo "array_shift A"
	A=(X 1 2)
	echo "before: $(set | grep ^A=)"
	array_shift A
	echo "after:  $(set | grep ^A=)"

	echo --------------------
	echo "array_shift A C"
	A=(0 X 2 3 4 5 6 7) C=\?
	unset A\[0\] A\[3\] A\[5\] A\[6\]
	echo "before: $(set | grep ^A=), $(set | grep ^C=)"
	array_shift A C
	echo "after:  $(set | grep ^A=), $(set | grep ^C=)"

	echo --------------------
	echo "array_unshift A X"
	A=(0 1 2 3 4)
	unset A\[1\] A\[3\]
	echo "before: $(set | grep ^A=)"
	array_unshift A X
	echo "after:  $(set | grep ^A=)"

	# echo --------------------
	# echo "array_unshift A X Y Z"
	# A=(0 1 2 3 4)
	# unset A\[1\] A\[3\]
	# echo "before: $(set | grep ^A=)"
	# array_unshift A X Y Z
	# echo "after:  $(set | grep ^A=)"

	echo --------------------
	echo
}

echo "----------------------------------------
ARRAY_RESETINDICES_PUSH=false
ARRAY_RESETINDICES_POP=false
ARRAY_RESETINDICES_SHIFT=false
ARRAY_RESETINDICES_UNSHIFT=false
----------------------------------------
"

ARRAY_RESETINDICES_PUSH=false
ARRAY_RESETINDICES_POP=false
ARRAY_RESETINDICES_SHIFT=false
ARRAY_RESETINDICES_UNSHIFT=false

source array.sh

common

echo "----------------------------------------
ARRAY_RESETINDICES_PUSH=true
ARRAY_RESETINDICES_POP=true
ARRAY_RESETINDICES_SHIFT=true
ARRAY_RESETINDICES_UNSHIFT=true
----------------------------------------
"

ARRAY_RESETINDICES_PUSH=true
ARRAY_RESETINDICES_POP=true
ARRAY_RESETINDICES_SHIFT=true
ARRAY_RESETINDICES_UNSHIFT=true

source array.sh

common

echo "----------------------------------------
----------------------------------------
"

echo --------------------
echo "array_get A C 2"
A=(0 1 X 3 4) C=\?
unset A\[1\] A\[3\]
echo "before: $(set | grep ^A=), $(set | grep ^C=)"
array_get A C 2
echo "after:  $(set | grep ^C=)"

echo --------------------
echo "array_get_first A C"
A=(0 X 2 3 4) C=\?
unset A\[0\] A\[3\]
echo "before: $(set | grep ^A=), $(set | grep ^C=)"
array_get_first A C
echo "after:  $(set | grep ^C=)"

echo --------------------
echo "array_get_last A C"
A=(0 1 2 X 4) C=\?
unset A\[1\] A\[4\]
echo "before: $(set | grep ^A=), $(set | grep ^C=)"
array_get_last A C
echo "after:  $(set | grep ^C=)"

echo --------------------
echo "array_get_all A B"
A=(0 1 2 3 4) B=()
unset A\[1\] A\[3\]
echo "before: $(set | grep ^A=), $(set | grep ^B=)"
array_get_all A B
echo "after:  $(set | grep ^B=)"

echo --------------------
echo "array_set A 1 X"
A=(0 1 2)
echo "before: $(set | grep -e ^A=)"
array_set A 1 X
echo "after:  $(set | grep -e ^A=)"

echo --------------------
echo "array_reset A"
A=(0 1 2)
echo "before: $(set | grep -e ^A=)"
array_reset A
echo "after:  $(set | grep -e ^A=)"

echo --------------------
echo "array_reset A X Y Z"
A=(0 1 2)
echo "before: $(set | grep -e ^A=)"
array_reset A X Y Z
echo "after:  $(set | grep -e ^A=)"

echo --------------------
echo "array_copy A B"
A=(0 1 2 3 4) B=()
unset A\[1\] A\[3\]
echo "before: $(set | grep ^A=), $(set | grep ^B=)"
array_copy A B
echo "after:  $(set | grep ^B=)"

echo --------------------

And here's a sample output (same with many versions of bash like 3.0, 3.1, 3.2 and perhaps some of 4.*)

Code:

----------------------------------------
ARRAY_RESETINDICES_PUSH=false
ARRAY_RESETINDICES_POP=false
ARRAY_RESETINDICES_SHIFT=false
ARRAY_RESETINDICES_UNSHIFT=false
----------------------------------------

--------------------
array_push A X
before: A=([0]="0" [1]="1" [2]="2" [3]="3" [4]="4")
after:  A=([0]="0" [1]="1" [2]="2" [3]="3" [4]="4" [5]="X")
--------------------
array_push A X Y Z
before: A=([0]="0" [2]="2" [4]="4")
after:  A=([0]="0" [2]="2" [4]="4" [5]="X" [6]="Y" [7]="Z")
--------------------
array_pop A
before: A=([0]="0" [2]="2" [4]="X")
after:  A=([0]="0" [2]="2")
--------------------
array_pop A C
before: A=([0]="0" [2]="2" [3]="X"), C='?'
after:  A=([0]="0" [2]="2"), C=X
--------------------
array_shift A
before: A=([0]="X" [1]="1" [2]="2")
after:  A=([0]="1" [1]="2")
--------------------
array_shift A C
before: A=([1]="X" [2]="2" [4]="4" [7]="7"), C='?'
after:  A=([0]="2" [2]="4" [5]="7"), C=X
--------------------
array_unshift A X
before: A=([0]="0" [2]="2" [4]="4")
after:  A=([0]="X" [1]="0" [3]="2" [5]="4")
--------------------

----------------------------------------
ARRAY_RESETINDICES_PUSH=true
ARRAY_RESETINDICES_POP=true
ARRAY_RESETINDICES_SHIFT=true
ARRAY_RESETINDICES_UNSHIFT=true
----------------------------------------

--------------------
array_push A X
before: A=([0]="0" [1]="1" [2]="2" [3]="3" [4]="4")
after:  A=([0]="0" [1]="1" [2]="2" [3]="3" [4]="4" [5]="X")
--------------------
array_push A X Y Z
before: A=([0]="0" [2]="2" [4]="4")
after:  A=([0]="0" [1]="2" [2]="4" [3]="X" [4]="Y" [5]="Z")
--------------------
array_pop A
before: A=([0]="0" [2]="2" [4]="X")
after:  A=([0]="0" [1]="2")
--------------------
array_pop A C
before: A=([0]="0" [2]="2" [3]="X"), C='?'
after:  A=([0]="0" [1]="2"), C=X
--------------------
array_shift A
before: A=([0]="X" [1]="1" [2]="2")
after:  A=([0]="1" [1]="2")
--------------------
array_shift A C
before: A=([1]="X" [2]="2" [4]="4" [7]="7"), C='?'
after:  A=([0]="2" [1]="4" [2]="7"), C=X
--------------------
array_unshift A X
before: A=([0]="0" [2]="2" [4]="4")
after:  A=([0]="X" [1]="0" [2]="2" [3]="4")
--------------------

----------------------------------------
----------------------------------------

--------------------
array_get A C 2
before: A=([0]="0" [2]="X" [4]="4"), C='?'
after:  C=X
--------------------
array_get_first A C
before: A=([1]="X" [2]="2" [4]="4"), C='?'
after:  C=X
--------------------
array_get_last A C
before: A=([0]="0" [2]="2" [3]="X"), C='?'
after:  C=X
--------------------
array_get_all A B
before: A=([0]="0" [2]="2" [4]="4"), B=()
after:  B=([0]="0" [1]="2" [2]="4")
--------------------
array_set A 1 X
before: A=([0]="0" [1]="1" [2]="2")
after:  A=([0]="0" [1]="X" [2]="2")
--------------------
array_reset A
before: A=([0]="0" [1]="1" [2]="2")
after:  A=()
--------------------
array_reset A X Y Z
before: A=([0]="0" [1]="1" [2]="2")
after:  A=([0]="X" [1]="Y" [2]="Z")
--------------------
array_copy A B
before: A=([0]="0" [2]="2" [4]="4"), B=()
after:  B=([0]="0" [2]="2" [4]="4")
--------------------