Insert a bash array into a file with sed

eamesj · 05-12-2013, 10:09 AM

Hi,

I'm looking to build an array and insert it into file at a certain location, below the line "# list of items".

user input:

Code:

%Enter list of items, type "q" to finish.
123
456
789

updated file:

Code:

#! /bin/bash
# list of items
item[0]='123'
item[1]='456'
item[2]='798'

for this im building an array and have been trying to use sed to insert each line below the "# list of items" line...

Code:

echo -e "Enter list of items, type \"q\" to finish."
while read -r input ; do
   [[ $input == q ]] && break
   array+=( "$input" )
done

for index in ${!array[*]}; do
   sed -e '/# list of items/{x;/./b;x;h;a\item'[$index]"=\'"${array[$index]}"\'" -e '}' /inpath/infile >> /outpath/outfile
done

but the redirect is simply appending the outflie to be a multiple of the infile for each element of the array rather than adding the individual lines of the array to the same file.

Code:

#! /bin/bash
# list of items
#! /bin/bash
# list of items
item[0]='123'
#! /bin/bash
# list of items
item[1]='456'
#! /bin/bash
# list of items
item[2]='789'

How can i have bash read and output to the same file every loop? If there are alternative ways of doing this without sed, that'd be interesting too.

thanks.

danielbmartin · 05-12-2013, 02:00 PM

With this InFile ...

Code:

This is a header of some sort.
This is another header.
Ignore this line.
# list of items
This is a trailer of some sort.
This is another trailer.
Ignore this line too.

... this brute-force code ...

Code:

echo -e "Enter list of items, type \"q\" to finish."
j=0
sed -n '1,/# list of items/p' $InFile >$OutFile
while read input 
  do
    [[ $input == q ]] && break
    array[++j]=$input
    echo $input >> $OutFile
  done
sed -n '/# list of items/,$p' $InFile  \
|sed '1d' >>$OutFile
 
echo; echo "OutFile ..."; cat $OutFile

... produced this result ...

Code:

Enter list of items, type "q" to finish.
item 1
item 2
item 3
q

OutFile ...
This is a header of some sort.
This is another header.
Ignore this line.
# list of items
item 1
item 2
item 3
This is a trailer of some sort.
This is another trailer.
Ignore this line too.

I don't understand the value of using a bash array but that's what you asked for.

Daniel B. Martin

David the H. · 05-12-2013, 06:02 PM

You should consider the text insertion as a single operation. Save it to a new variable and use that.

I also suggest using ed instead of sed. It can write the updated contents straight to either the original file or a new one. And sed's 'a'ppend command seems to be having a hard time handling the embedded newlines in the variable.

Code:

echo -e "Enter list of items, type \"q\" to finish."
while read -rp "('q' to exit): "; do
    [[ $REPLY == q ]] && break
    input+=( "$REPLY" )
done

insert=$( for i in "${!input[@]}"; do
              printf 'item[%d]="%s"\n' "$i" "${input[i]}"
          done )

printf '%s\n' '/# list/a' "$insert" '.' 'w outfile.txt' | ed -s infile.txt

How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)

konsolebox · 05-12-2013, 10:18 PM

You could juse set | grep ^arrayname= to get the values that could be stored on the file. After that read the file with source. But of course if you really intend to have the format on your own, then there's a better way. But first do you want to update a file or create a new one? And what other values/contents could be there within the file?

eamesj · 05-13-2013, 08:47 AM

Thanks for the replies, I think David's solution is how i'll have to proceed. I've tried throwing the string into the the original file but have not managed to do this yet from online sources.

Code:

ed -s infile.txt <<< printf '%s\n' '/# list/a' "$insert" '.'

I will need to perform a global replace which may require slashes. sed wont do this without some help that i dont want to have to have on the front end (i.e. i dont want to ask the user to type \/), can ed do this? or is there a way for sed to accept a / on face value?

The coding below may look strange; the purpose of all of this is to update the very same script that you are using (opening up the updated file and shutting down the original). As such i have some variables at the top of the script that i will replace and write to a new version of the script.

Code:

name=/users/fred
read -p "Enter the directory:" newname
    oldname=$name
    if [[ ${newname} == "" ]]; then
       newname=${oldname}
    fi

sed 's/'${oldname}'/'${newname}'/g' file_v1.txt > file_v2.txt

thanks

---------- Post added 05-13-13 at 08:47 AM ----------

Thanks for the replies, I think David's solution is how i'll have to proceed. I've tried throwing the string into the the original file but have not managed to do this yet from online sources.

Code:

ed -s infile.txt <<< printf '%s\n' '/# list/a' "$insert" '.'

I will need to perform a global replace which may require slashes. sed wont do this without some help that i dont want to have to have on the front end (i.e. i dont want to ask the user to type \/), can ed do this? or is there a way for sed to accept a / on face value?

The coding below may look strange; the purpose of all of this is to update the very same script that you are using (opening up the updated file and shutting down the original). As such i have some variables at the top of the script that i will replace and write to a new version of the script.

Code:

name=/users/fred
read -p "Enter the directory:" newname
    oldname=$name
    if [[ ${newname} == "" ]]; then
       newname=${oldname}
    fi

sed 's/'${oldname}'/'${newname}'/g' file_v1.txt > file_v2.txt

thanks

konsolebox · 05-14-2013, 01:29 AM

There could be less confusing and cleaner solutions in which you won't need to use sed only if you'd explain what you really intend to do. Again do you want to update a file or create a new one? And what other values/contents could be there within the file? If already there are contents after the header would those contents be replaced as well, or would an entry be inserted after the header line? (as it shows that your sed would insert entries after it). Does creating a new file or updating a new one give a difference to the output? How about the other contents beside the lists? Where could they be placed? Do you just intend to run the output as a script or just a list, or a script that has the form of a list in human readable form? Or do you just need the array data to be recalled at runtime? This could be solved in one shot but only if all details are mentioned.

eamesj · 05-14-2013, 03:25 AM

There are a series of variables at the start of a large bash script that could be any set of alphanumeric and special characters and an array list of any length. What i am trying to do is create a submenu that will let the user edit those values.

This will mean that we are overwriting the same script we are using

I have 2 problems really
1. I am creating some temporary files, it would be cleaner to use 'ed' or similar to modify the same file
2. I am trying to replace one variable with another and so i cannot easily use sed for replacing variables that contain special characters - The user would have to type \/users\/fred.... which is not really an option so i need an alternative to sed or a way of coding sed to read special characters as they are.

Code:

#! /bin/bash
# main script
#############################
# define variables
name=fred
path=/users/fred/freds_path
# list of items - called upon later in the main script
item[0]='123'
item[1]='456'
item[2]='798'
#############################
# main program
 
do lots of things...

# modify menu 
read -p "Do you want to edit the setup parameters? [y/n]:" yesno
if [[ ${yesno} == "y" || ${yesno} == "Y"  ]]; then
 read -p "Enter the new name:" newname
 oldname=$name
 if [[ ${newname} == "" ]]; then
   newname=${oldname}
 fi
 read -p "Enter the path:" newpath
   oldpath=$path
 if [[ ${newpath} == "" ]]; then
   newpath=${oldpath}
 fi

# build an array for the list
 echo -e "Enter a list of items:" 
 echo -e "Type \"q\" to finish."
 while read -rp "('q' to exit): "; do
   [[ $REPLY == q ]] && break
   input+=( "$REPLY" )
 done

# build temporary file without the list of items
 grep -v 'item[' main_script.bsh > main_script_tmp.bsh

# replace variables in the main script 
 read -p "Do you want to update these parameters? [y/n]:" yesno
 if [[ ${yesno} == "y" || ${yesno} == "Y"  ]]; then
   sed 's/'${oldname}'/'${newname}'/g' main_script_tmp.bsh |\
   sed 's/'${oldpath}'/'${newpath}'/g' > main_script_tmp2.bsh

# Insert array into main script below "# list" line
 insert=$( for i in "${!input[@]}"; do
   printf '\titem[%d]=%s\n' "$i" \'${input[i]}\'
 done )
 printf '%s\n' '/# list/a' "$insert" '.' 'w main_script.bsh' | ed -s main_script_tmp2.bsh

# open the updated script, cleanup and exit
 xterm -fn 7x13 -b 5 -geom 120x50+50+50 -display $DISPLAY -e main_script.bsh &
 rm -f main_script_tmp.bsh main_script_tmp2.bsh 
 exit;

elif [[ ${yesno} != "y" || ${yesno} != "Y"  ]]; then
  menu;
fi

exit;

I hope this is a bit clearer (or at least that i havent made things more confusing

). The code is part of a much larger bash script so re-writing in another language like perl is not really an option unless we call on a second script.

konsolebox · 05-14-2013, 06:27 AM

I think it would be better if you just save and load those lists to another file and the format of that file would be at your own option rather than placing those values on the main file. Editing the main file while it's running is not a good idea and could confuse the shell if the parser/interpreter is implemented line-based and not buffer based. This could work though if you use functions to all your procedures and make the last line of your script a function call to the main function, but I still find that not a good idea.

You could simply do something like this as an example. A better code would make reading the list-file safer but this is the basic.

main file

Code:

#!/bin/bash

# load items from list file
. list-file.bash

# do some things that could modify the list
...

# save the list back to file. perhaps next time make use of a data changed flag (variable) to signal changes and place this on a conditional block to check if data is needed to be saved back to file or not.
{
    echo "# list of items - called upon later in the main script"
    for I in "${!ITEM[@]}"; do
        VALUE=${ITEM[I]}
        VALUE=${VALUE//\\/\\\\}
        VALUE=${VALUE//\"/\\\"}
        VALUE=${VALUE//\$/\\\$}
        VALUE=${VALUE//\`/\\\`}
        echo "ITEM[$I]=\"$VALUE\""
    done
} > list-file.bash

list file

Code:

# list of items - called upon later in the main script
ITEM[0]='123'
ITEM[1]='456'
ITEM[2]='798'