using sed to insert lines with special characters

disorderly · 04-10-2006, 02:35 PM

i'm looking to insert a line in multiple files, and then change their file extension. i've found many examples of using sed to do this but can't get it exactly right.

i need to put the line:

PHP Code:



<?php include("/home/functions.php");validateUser($_SERVER['PHP_SELF']);?>

on line 1 of many pages, *.html, *.htm. in many directories.

i keep getting the error:
bash: syntax error near unexpected token 'include("'

i've come up with this:
cat example.html | sed '1s/^/<?php include("/home/functions.php");validateUser($_SERVER['PHP_SELF']);?>'

i think it's because of the space or something and could really use some guidance
thanks,
disorderly

disorderly · 04-10-2006, 02:59 PM

ok i got it to work for 1 file! i used:
myheading='<?php include("/home/functions.php");validateUser($_SERVER['PHP_SELF']);?>'
cat top.html | sed '1s/^/$myheading/'

any idea how i can repeat this process for multiple files and then change their file extension to *.php?

thanks again,
disorderly

jschiwal · 04-10-2006, 03:49 PM

If they are all in the same directory you can use a for loop in bash:
myheading='<?php include("/home/functions.php");validateUser($_SERVER['PHP_SELF']);?>'
for file in *.htm *.html; do
sed '1s/^/$myheading/' "$file"
mv "${file}" "${file%.*}php"
done

jiml8 · 04-10-2006, 07:40 PM

I believe that your original problem was that you need to escape the quotes and the slashes in order to tell sed that you really want these inserted in the string. sed was taking them as control characters.

viz:

Code:

cat example.html | sed '1s/^/<?php include(\"\/home\/functions.php\");validateUser($_SERVER[\'PHP_SELF\']);?>

disorderly · 04-11-2006, 09:52 AM

jiml8, thank you for your assistance. i am still getting the error - bash: syntax error near unexpected token `;'
how can i enter a sed statement that contains the special character ";" when sed takes it as a command or something?

jschiwal, thank you as well! that is the second half! as soon as i can get this frustrating sed statement to work with a ";", i'll try it!

thanks again,
disorderly

disorderly · 04-11-2006, 01:02 PM

thanks to both of your help i've gotten this almost working except the last file in the directory always ends up blank

Code:

#!/bin/bash

for file in *.html; do
cat $file | sed '1s/^/\<\?php include(\"\/home\/functions.php\");validateUser($_SERVER[\"PHP_SELF\"])\?\>\
/' > $file
mv "${file}" "${file%.*}.php"
done

my goals is to have this working recursively so i can execute this in the top level folder and let it do it's work on the 9,000+ files that need that PHP header. any idea why the last file always ends up with 0 bytes?

*note - this drove me crazy but it was easy enough to fix: if anyone is trying to insert a newline character after their sed insertion, just put a "\", then hit return. heck if i didn't know then there are others...

thanks,
disorderly

disorderly · 04-11-2006, 01:22 PM

nevermind it was the server just being a jerk - the script works fine. i just have to get it to work recursively in multiple directories - now can i do this?
thanks,
disorderly

disorderly · 04-11-2006, 01:49 PM

i haven't found anyway to recursively affect directories so now i'm trying this with:

Code:

find /home/*.htm -exec sed '1s/^/\<\?php include(\"\/home\/functions.php\");validateUser($_SERVER[\"PHP_SELF\"])\?\>/' {} \;

it almost works but won't write to the files! in many examples i've seen people use the -i switch, but it doesn't work on this POS server; i.e. i need to specify the output file and i'm stuck

dive · 04-11-2006, 01:52 PM

Something like this should work. It checks if a file is a directory and then cd's into it it and calls itself again, and then does a cd ..

Code:

for i in *
do
if [ -d "$i" ] # if * is a directory
then
cd "$i" # descend into the directory
for y in *
do

... your code here

if [ -d "$y" ] # if this is also a directory, call this program again
then
cd "$y"
html2php; # this is the name of your program, must be in your PATH or use full path in command
cd ..
fi
done
cd ..
fi
done

disorderly · 04-11-2006, 02:21 PM

ahh, i didn't know i could use nested loops in shell scripting - thank you for reading this post dive, i'll give that a try.
incidently my find script above is very flaky - it deletes my files as often as it writes them correctly, dunno why but a warning for anyone that is going to use it..

dive · 04-11-2006, 03:01 PM

*small edit

It may be the mv line that does this. You could try adding -v option, or maybe even use cp -v to get an idea of filenames are being used

disorderly · 04-11-2006, 03:51 PM

howdy dive
i've used your code and come up with this

Code:

for i in *
do
        if [ -d "$i" ] # if * is a directory
        then
                cd "$i" # descend into the directory

                for y in *
                do

                        if [ -f "$y" && `grep '*.html' $y]
                        then
                                cat $y | sed '1s/^/insertedLineHere/' > $y
                                # mv "${file}" "${file%.*}.php"
                                cat $y >> /homefiles.txt #for testing
                        fi

                        if [ -d "$y" ] # if this is also a directory, call this program again
                        then
                                cd "$y"
                                programName.sh; # this is the name of your program
                                cd ..
                        fi
                done
        cd ..
        fi
done

but i'm pretty sure this line is wrong:

Code:

if [ -f "$y" && `grep '*.html' $y]

i'm not sure how to make sure that that the only files effected are *.html, *.htm.

dive · 04-11-2006, 04:22 PM

if [ 'grep .htm $y' ]

should work. Don't need to test $y with -f flag since if grep is true then it must be a regular file. You don't have any dirs named with .html I take it?

disorderly · 04-11-2006, 07:04 PM

thanks again for reading this dive! yes you can assume none of the directories are named 'html', or 'htm'.
aha i understand the logic in dropping the -f comparison when using grep, but the script is taking in PDF and *.word files as well. isn't grep for searching inside files rather than determining a file's extension?

dive · 04-11-2006, 08:29 PM

Sorry been a long day...

if echo "$y" | grep ".htm"
then
...

This way grep will search whatever is piped to it. But if you have pdf files containing the sequence .htm you will need some other test