wtaicken

[Log in to get rid of this advertisement]

Hi, I'm trying to develop a script which will append a block of text to a series of text files in a number of directories. The appended text also has to have written within a line the relevant directory name. This could work by substituting for other text in the line.

I'm sure this is easy to do, but I'm struggling with the syntax for using sed within a script

Any assistance gratefully received

Will

colucix

Hi and welcome to LQ! Just a thought: you have to establish a rule to list all the files you want to modify. Use the find command to list all these files, loop over them, extract the directory name using the dirname command and echo a string redirecting with >> (append). Pseudo-code:
Just pay attention to file names containing blank spaces. Anyway, show us what you've tried so far: it should be easier to get help.

wtaicken

Hi Colucix

This is what I have generated so far, cobbled together from various sources on the web.......

I was experimenting by specifying a directory as an argument when calling the script, cd'ing to the directory, and then trying to append "input text" to the end of each file in the directory. But it was only appending the test text to the last file in the directory, and writing to the screen. How do I get it to append to all files and write to those original files?

if [ -z $1 ]; then
echo "Needs directory as argument."
exit 1
elif [ -f "$1" ]; then
dirname="$1"
fi
cd $1
list="$(ls ARCH*.dat)"
echo $list
for i in "$list";do sed '$a input text' $i; done

colucix

Ok. I will correct your script and will suggest alternatives. A working version can be:
To test if $1 is a directory, use the -d test, not -f. The former checks if the name is a directory, the latter checks if the name is a regular file.

Moreover, when you embed $list in double quotes, the list of files is interpreted as a single string with file names separated by space (this is a side effect of the shell expansion). Hence the for loop is executed only once and the resulting sed command is something like:
for this reason you append a string to the content of the last file only. Removing the double quotes gives the result you expect. You can also use the command substitution directly in the for loop, avoiding the assignment to the variable list:
In addition, to actually modify the content of the file you have to use the -i option of sed.

The sed command is ok, since it appends (a) the string (input text) to the end ($) of the file. Indeed the special character $ in a sed command matches the last line in the file. However this works only if the file is not empty, that is if there is at least one line in the file.

To keep it simple you can use echo instead of sed
This will echo the string and append it to the file. The double redirection (>>) means append.

Finally I suggest some good reading about shell scripting and sed (and also awk for future reference).
Bash Guide for beginners
Advanced Bash-Scripting Guide
Sed - An Introduction and Tutorial
Gawk: Effective AWK Programing
Have a nice scripting time!

wtaicken

colucix
Hi again

Tried that, and yep it works a treat, cheers

Next step, trying to substitute some of the appended text for the dirname.
I've tried some of the online resources, but they don't have any specific examples

I basically want to change part of the appended text e.g. "nnnnn" for the dirname "$1" How can I do that in the body of the text to append? Tried the following which doesn't work:

#!/bin/bash

# script to append text to each file in a directory. The file wildcard & dir are specified # by user.


if [ -z $1 ]; then
echo "Needs directory as argument."
exit 1
elif [ -d "$1" ]; then
dirname="$1"
fi
cd $1
list="$(ls ARCH*.test)"
echo $list
for i in $list
do
sed -i '$a\
retrieve from ./REF_INTS/nnnn_testy.REFINT\
begin\
partial k-effective\
store as ./REF_INTS/nnnn_testy.REFINT\
begin' $i | sed -i 's/nnnn/"$1"/'
done

colucix

You can insert a command substitution in the sed command using quotes like this:
That is you close the single quote, put the command substitution $(...) and re-open the single quotes to terminate the sed command. The command can be dirname $i or better readlink -f $i, but it depends on the result you want to obtain. I'm afraid that if you use dirname, the result will be always the current directory "." (dot) since you change to the directory at the beginning of the script and build the list of files that haven't got the path in their name.

wtaicken

Ok, will try that.

BTW, how do you insert code inside the box in the message? Obviously a neater way of creating the post

colucix

Of course. If you go to advanced mode (see the "Go Advanced" button at the bottom of the Quick Reply box, below) you can simply highlight some text and press the hash (#) button. Or explicitly put CODE tags around the text. For example if you write

[code]#!/bin/bash
echo Hello World\![/code]

it will result in

ghostdog74

no need for ls
another:

wtaicken

Ok, thanks
I now find I need to substitute another character in the text to append. I need to read through the original input file ($i), and search for the last occurrence of a word from a look up list. Once that word is found, I need to take the last number from the line where the word occurs, and substitute that for the character "x" in the text to append.

I guess I should now use grep?

I also need to extract a character from the file name, and substitute that for a character "y" in the text to append.

E.g The filename will be something like *****_test2_*****. I need to extract the "2" from the name. Guess that can be done in a similar way?

Guess I should now do the appending last, after modifying the text to append