LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-05-2009, 06:09 AM   #1
thierryd_uk
LQ Newbie
 
Registered: Oct 2005
Posts: 3

Rep: Reputation: 0
sed problems when run in $( ... )


Hi all,

I'm trying to write a script that will setup a new website on my development box - one that automatically creates its Subversion repository, checks out a local working copy, creates folders for the apache scripts and log files, and updates the apache config. It's all going swimmingly, except for the last bit - automatically updating the apache config. The "template" apache config for each new website is stored in a file with dummy entries where the real values should go (so for instance WEBSITE_NAME will be replaced by a display name for the website), and I'm trying to use sed to replace these values in my script.

Because of what the script is doing, I've added a "debug" mode that echo's the commands to be run, rather than actually running them (so I don't mess up my system), so the commands are run from a variable (for instance $( $cmd )). This bit seems to work fine, except when I try and use sed to update the config file, in which case I get the following error that I haven't been able to resolve after several hours of Googling:

sed: -e expression #1, char 1: unknown command: `''

I've managed to whittle the error down to the following:

Code:
# Create the sed command
cmd="sed 's/WEBSITE_NAME/Test\ With\ Spaces/g' /svn/svn-admin/httpd-conf-skel"

# Try running in a sub-shell
output="$( $cmd )" # Tried both like this...
output="`$cmd`"    # ... and in backticks, and both give the above error

# Try piping through bash
output="$( echo "$cmd" | bash )"
echo "$output"     # This works!

# Try running manually
sed 's/WEBSITE_NAME/Test\ With\ Spaces/g' /svn/svn-admin/httpd-conf-skel
                   # This works too!
Naturally removing the spaces and the single quotes from $cmd makes all the little errors go away, but that's not exactly practical in the real world. I've tried quoting the s/// part of the sed command in both single quotes (as above) and double quotes, and the only difference is that the error complains about double quotes instead. So, is there some kind of extra variable substitution/interpolation that occurs when I run it this way, as the command itself works fine when run manually? Or is there something else wrong that is blatantly obvious to everyone but me?

I'm running an up-to-date Fedora 11, with sed version 4.1.5 and bash 4.0.16(1)-release.

Thanks!

PS. For the record, /svn/svn-admin/httpd-conf-skel contains
Code:
# WEBSITE_NAME
<VirtualHost *:80>
    DocumentRoot /var/www/html/WWW_DIR/htdocs
    ServerName SERVER_NAME
    ErrorLog logs/LOG_DIR/error_log
    CustomLog logs/LOG_DIR/access_log common
</VirtualHost>
 
Old 07-05-2009, 06:52 AM   #2
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
The problem is the way the shell quotes the string in $cmd. If you use bash -x you can see it:
Code:
$ cat test.sh
#!/bin/bash
cmd="sed 's/WEBSITE_NAME/Test\ With\ Spaces/g' /svn/svn-admin/httpd-conf-skel"

# Try running in a sub-shell
output="$( $cmd )"
$ bash -x test.sh
+ cmd='sed '\''s/WEBSITE_NAME/Test\ With\ Spaces/g'\'' /svn/svn-admin/httpd-conf-skel'
++ sed ''\''s/WEBSITE_NAME/Test\' 'With\' 'Spaces/g'\''' /svn/svn-admin/httpd-conf-skel
sed: -e expression #1, char 1: unknown command: `''
+ output=
As you can see the extra single quotes at the beginning of the sed command trigger this error. I have tried different combinations of double and single quotes, but I couldn't run it without errors. However, why not keep it simple and do the following?
Code:
$ cat test.sh
#!/bin/bash
output=$(sed 's/WEBSITE_NAME/Test With Spaces/g' /svn/svn-admin/httpd-conf-skel)
$ bash -x test.sh
++ sed 's/WEBSITE_NAME/Test With Spaces/g' /svn/svn-admin/httpd-conf-skel
+ output='# Test With Spaces
<VirtualHost *:80>
    DocumentRoot /var/www/html/WWW_DIR/htdocs
    ServerName SERVER_NAME
    ErrorLog logs/LOG_DIR/error_log
    CustomLog logs/LOG_DIR/access_log common
</VirtualHost>'
 
Old 07-05-2009, 07:20 AM   #3
thierryd_uk
LQ Newbie
 
Registered: Oct 2005
Posts: 3

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by colucix View Post
If you use bash -x you can see it
That tip looks like it might save me lots of time in the future, so thanks for that!

The reason I'm not running the command directly in a $( ... ) is that there's multiple commands in the script, each of which is run through a common function that checks to see if the debug mode has been set - if it has, it simply echo's the command to STDOUT, otherwise its tries running it.

The "fuller" command that is causing a problem in my script is this:

Code:
# Display command if debugging, otherise run
function check_for_debug() {
  # Display, if debugging
  if [ "x$debug" = "x1" ]
  then
    echo "$cmd"

  # We're not debugging, so run
  else
    output="$( $cmd )"
  fi
}

# State debug mode
debug=1

# Run the commands
cmd="sed -ia 's/WEBSITE_NAME/Test\ With\ Spaces/g' /svn/svn-admin/httpd-conf-skel"
check_for_debug
cmd="sed -ib 's/WWW_DIR/test/g' /svn/svn-admin/httpd-conf-skela"
check_for_debug
cmd="sed -ic 's/SERVER_NAME/test.localdomain/g' /svn/svn-admin/httpd-conf-skelab"
check_for_debug
cmd="sed -id 's/LOG_DIR/test/g' /svn/svn-admin/httpd-conf-skelabc"
check_for_debug
instead of having to duplicate each of the commands in the echo... not to mention the unnecessary extra code that's required

Code:
# State debug mode
debug=1

# Run the commands
if [ "x$debug" = "x1" ]
then
  echo "sed -ia 's/WEBSITE_NAME/Test\ With\ Spaces/g' /svn/svn-admin/httpd-conf-skel"
else
  output=$( sed -ia 's/WEBSITE_NAME/Test\ With\ Spaces/g' /svn/svn-admin/httpd-conf-skel )
fi

if [ "x$debug" = "x1" ]
then
  echo "sed -ib 's/WWW_DIR/test/g' /svn/svn-admin/httpd-conf-skela"
else
  output=$( sed -ib 's/WWW_DIR/test/g' /svn/svn-admin/httpd-conf-skela )
fi

if [ "x$debug" = "x1" ]
then
  echo "sed -ic 's/SERVER_NAME/test.localdomain/g' /svn/svn-admin/httpd-conf-skelab"
else
  output=$( sed -ic 's/SERVER_NAME/test.localdomain/g' /svn/svn-admin/httpd-conf-skelab )
fi

if [ "x$debug" = "x1" ]
then
  echo "sed -id 's/LOG_DIR/test/g' /svn/svn-admin/httpd-conf-skelabc"
else
  output=$( sed -id 's/LOG_DIR/test/g' /svn/svn-admin/httpd-conf-skelabc )
fi
Or am I trying to do too much with bash?

Thanks again.
 
Old 07-05-2009, 08:14 AM   #4
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
Use eval to reverse the bash quoting, that is let bash assign the command using quotes and other stuff, then use eval to evaluate what the bash has written in the cmd string. You should not have any problem anymore.
Code:
output="$(eval $cmd)"
 
Old 07-05-2009, 11:57 AM   #5
thierryd_uk
LQ Newbie
 
Registered: Oct 2005
Posts: 3

Original Poster
Rep: Reputation: 0
Works perfectly, thanks for your help!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Need Links for the SED/Awk to run in window. dina3e General 1 01-21-2009 09:23 PM
Problems with sed maker10 Programming 3 06-27-2008 02:39 PM
Could not run SED command mmunir Other *NIX 3 05-05-2008 07:07 AM
SED has problems with Ram rabby Linux - Software 5 02-28-2005 07:46 PM
sed problems icisyd Linux - Software 2 11-19-2004 07:17 PM


All times are GMT -5. The time now is 10:20 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration