LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices



Reply
 
Search this Thread
Old 05-26-2012, 11:55 PM   #1
samasat
LQ Newbie
 
Registered: Nov 2011
Location: Austin
Distribution: CentOS 6.5
Posts: 14

Rep: Reputation: Disabled
sed gives :sed: -e expression #1, char 1: unknown command: `''


I am attempting to automatically replace a output name string in a batchfile prior to its execution to describe settings explicitely in output. However, I am not able to to do that
My Code:
Code:
inFile="$WORK/${SimName}.in"
replaceCmd="sed 's/sim.dat/${SimName}.dat/g' $inFile"
echo $replaceCmd
$replaceCmd
And the output tracked by echo'ing is:
Code:
sed 's/sim.dat/sim0525_PerfCentOS_U032_MAT04_THRD024.dat/g' /home/ssr/Desktop/PerfCentOS/sim0525_PerfCentOS_U032_MAT04_THRD024/sim0525_PerfCentOS_U032_MAT04_THRD024.in
sed: -e expression #1, char 1: unknown command: `''
Any suggestions ?
Thanks,

Last edited by samasat; 05-27-2012 at 01:27 PM. Reason: Formatting correction - mistakely used "Quotes" replaced by "Code" as per David's suggession
 
Old 05-27-2012, 12:11 AM   #2
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 2,129

Rep: Reputation: Disabled
It seems to be choking on the quote character. Have you tried using escaped double quotes (\") instead of single quotes around the sed expression?
 
Old 05-27-2012, 03:50 AM   #3
vonbiber
Member
 
Registered: Apr 2009
Distribution: slackware
Posts: 299

Rep: Reputation: 50
Quote:
Originally Posted by samasat View Post
inFile="$WORK/${SimName}.in"
replaceCmd="sed 's/sim.dat/${SimName}.dat/g' $inFile"
echo $replaceCmd
$replaceCmd
you're trying to replace 'sim.dat' with the value of $SimName.dat in $inFile
try this
Code:
cmd="sed 's?sim\.dat?$SimName.dat?g' $inFile"
eval "$cmd"

Last edited by vonbiber; 05-27-2012 at 03:52 AM.
 
Old 05-27-2012, 09:23 AM   #4
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
Please use ***[code][/code] tags*** around your code and data, to preserve formatting and to improve readability. Please do not use quote tags, colors, or other fancy formatting.


Oh, hells no! Never, and I mean NEVER, use eval for something as trivial as this. Indeed, you should only very rarely need it at all. It can be quite a security risk, and is almost never really necessary in general scripting.

Eval command and security issues
http://mywiki.wooledge.org/BashFAQ/048


In addition, variables are for storing data, not code. The spaces, quotes, and other special characters cease to be special when in a variable (unless the expansion is unquoted, then some of them still have an effect, but likely not the one you want; see my next point), and in any case leads to unclear, obfuscated code flow. If you need repeated use of the same command, don't put it inside a variable, use a function.

I'm trying to put a command in a variable, but the complex cases always fail!
http://mywiki.wooledge.org/BashFAQ/050


A second problem here is that if the sed command is surrounded by single quotes, the variable inside them does not get expanded. No quotes is even worse, as the shell attempts to perform word-splitting and glob expansion after the substitution.

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes


Here's what you should be doing:

Code:
replaceCmd(){

	sed "s/sim.dat/$1.dat/g" "$2"

}

inFile="$work/$SimName.in"

replaceCmd "$SimName" "$inFile"

BTW, since environment variables are generally all upper-case, it's good practice to keep your own user variables in lower-case or mixed-case to help differentiate them.

Also, please don't use the full brace-surounded "${var}" form unless absolutely necessary. It does nothing useful, and only adds extra typing and clutter to your code.

Last edited by David the H.; 05-27-2012 at 09:29 AM. Reason: Darn that text duplicating bug! + some rewording
 
1 members found this post helpful.
Old 05-27-2012, 01:35 PM   #5
samasat
LQ Newbie
 
Registered: Nov 2011
Location: Austin
Distribution: CentOS 6.5
Posts: 14

Original Poster
Rep: Reputation: Disabled
\" around sed expression did not help

Quote:
Originally Posted by Ser Olmy View Post
It seems to be choking on the quote character. Have you tried using escaped double quotes (\") instead of single quotes around the sed expression?
Thanks for your suggestion Ser. I did not make any difference in error message overall.

I used following
Code:
replaceCmd="sed \"s/sim.dat/${SimName}.dat/g\" $inFile"
echo $replaceCmd
$replaceCmd
Output was:
Code:
sed "s/sim.dat/sim0502_PerfCentOS_U038_MAT01_THRD01.dat/g" /home/ssr/Desktop/TrashIt_BashTest/sim0502_PerfCentOS_U038_MAT01_THRD01/sim0502_PerfCentOS_U038_MAT01_THRD01.in
sed: -e expression #1, char 1: unknown command: `"'
 
Old 05-27-2012, 02:10 PM   #6
samasat
LQ Newbie
 
Registered: Nov 2011
Location: Austin
Distribution: CentOS 6.5
Posts: 14

Original Poster
Rep: Reputation: Disabled
making sed command a function worked

Thanks David. Your help did work.
Only minor correction I had to make was that sed line I was using earlier was not correct. My objective was to replace the string inside the file and save the changes. When I used the old approach it just threw it corrected on screen but changes in file were not saved.
Then I tried this one which also did not work. It may be my ignorance that redirection does not work with sed:
Code:
replaceCmd(){
	sed "s/sim.dat/$1.dat/g" "$2" > "$2"
}
Finally, after googling a bit I found '-i' switch that saved file changes as given below:
Code:
replaceCmd(){
	sed -i "s/sim.dat/$1.dat/g" "$2"
}
I just inserted this function in my code to replace old lines. However, I was wondering if there is any standard/conventional location of such additional function in the code e.g. in C we declare it right at the top or through header file ...

Also, I do agree with approach from your post and the link that:
Quote:
variables are for storing data, not code
and also regarding security risks involved in eval. My only addition is that it also depends on where the code is going to be implemented. e.g. if I am using this code for my research, probably I would think that security is not a concern for me as long as task gets done since anyway my work is going to go open-source. Also, there is a fact that out of 100 things or approaches may be 5-6 do work. I guess, in that case, the ease to try out things quickly gains more importance prior to arriving at final professional level code for finalized approach. There definitely one has to be cautious and perfect and should implement your suggestion.

Thanks David, Ser and vonbiber. I appreciate your help and time !
Thanks,

Last edited by samasat; 05-27-2012 at 02:58 PM. Reason: Added correction that I need to use '-i' in sed command
 
Old 05-27-2012, 07:14 PM   #7
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.6, Centos 5.10
Posts: 16,324

Rep: Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041
Quote:
I just inserted this function in my code to replace old lines. However, I was wondering if there is any standard/conventional location of such additional function in the code e.g. in C we declare it right at the top or through header file ...
The shells read strictly top to bottom, so the fn defn has to come before its called in the 'main' as you can't reference a fn you haven't already 'read' (parsed).

This is in contrast to C or Perl, where the 'compiler/linker' reads the entire src file before trying to action anything.
Shell is interpreted ie parsed+actioned as it is read.

HTH

Last edited by chrism01; 05-28-2012 at 06:56 PM. Reason: typo
 
Old 05-28-2012, 11:34 AM   #8
samasat
LQ Newbie
 
Registered: Nov 2011
Location: Austin
Distribution: CentOS 6.5
Posts: 14

Original Poster
Rep: Reputation: Disabled
Thanks Chrism01 !
 
Old 05-28-2012, 11:55 AM   #9
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
Code:
sed "s/sim.dat/$1.dat/g" "$2" > "$2"
And this doesn't work because the ">" redirection is parsed, and the input file is overwritten, before the command is launched. So you end up with an empty file for input. You have to go through a temporary file, or use the -i option, which actually does the same thing, just in the background.


Incidentally, I've recently been learning how to use ed, and you might consider using that instead. As a real text editor, it can operate on the file directly.

It might also be a good idea to make your function more general in scope, so you can continue to use it even with different starting and ending strings.

Code:
# function: replaceCmd
# globally replace one string with another using ed.
# $1 is the filename, $2 is the string to match
# $3 is the replacement string.  strings are treated as regexes.
# In addition, any forward slashes in the strings must be
# escaped with backslashes, due to the delimiter.

replaceCmd() {

	printf '%s\n' "g/$2/ s/$2/$3/g" 'w' | ed -s "$1"

}

replaceCmd "$inFile" "sim.dat" "$SimName.dat"
How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)


Finally, re your eval comment. While you are certainly correct that you don't need to worry so much about security in a limited, single-user situation, I'd still recommend against it.

First of all, security isn't just about unauthorized malicious users; it also includes authorized users making mistakes. Don't underestimate the possibility of a malformed string accidentally forming an unwanted command or having other unintended side-effects.

Second, laziness and sloppiness are also risks to beware of. If you're lax in coding when it's safe, then you aren't training yourself to be careful when it isn't. Over-dependence on quick 'n easy, just-get-it-done solutions means you aren't really learning how to use the better alternatives available.

eval is the nuclear weapon of the scripting world; it's only meant to be brought out as a last resort when nothing else works. And since it can nearly always be avoided with just a little bit of work and know-how, any use of it, without a damn good reason and without knowing exactly what you're doing, is really a bad use of it, IMO.

Last edited by David the H.; 05-28-2012 at 11:57 AM. Reason: minor code mistake
 
1 members found this post helpful.
Old 05-28-2012, 02:31 PM   #10
Roken
Member
 
Registered: Oct 2011
Location: Bolton, UK
Distribution: Arch local, Debian on VPS
Posts: 252

Rep: Reputation: 40
I can second (or should that be third) comments about using eval. I'm a reasonably experienced and competent bash scripter, but yesterday I managed to completely trash a system configuration file with the use of eval and a string that didn't end up quite how I intended.

Two lessons learned:

i. Don't use eval
ii. Never test live on system configurations lol

As it happens, it was a fairly trivial matter to re-write the config file - this time. Other times I may not be so lucky.
 
1 members found this post helpful.
Old 06-09-2012, 06:31 PM   #11
samasat
LQ Newbie
 
Registered: Nov 2011
Location: Austin
Distribution: CentOS 6.5
Posts: 14

Original Poster
Rep: Reputation: Disabled
Smile Thanks !

Thanks David and Roken for your advice. I will keep that in mind !

Also, thanks David for the idea of making it generalized. I hope to apply same while using sed as well. I am yet to try 'ed'.

UPDATE:
I have checked that both sed and ed work equally good.

Code:
replaceCmd_sed(){ # $1 -> OldStr,  $2 -> NewStr , $3 -> Name of file to edit
	sed -i "s/$1/$2/g" "$3"
}

replaceCmd_ed(){ # $1 -> OldStr,  $2 -> NewStr , $3 -> Name of file to edit
	printf '%s\n' "g/$1/ s/$1/$2/g" 'w' | ed -s "$3"
}

Last edited by samasat; 06-10-2012 at 07:59 PM. Reason: Updating about successful use of ed and sed
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] sed help to run sed command against multiple different file names bkone Programming 2 04-16-2012 01:27 PM
sed and regexp matching (GNU sed version 4.2.1) Ashkhan Programming 8 02-27-2012 10:12 AM
[SOLVED] head -n -1 (sed -n -e '1,$-1p' is incorrect) sed question, as stupid as it could be. kaz2100 Programming 1 12-27-2011 10:18 PM
[SOLVED] sed 's/Tb05.5K5.100/Tb229/' alone but doesn't work in sed file w/ other expressions Radha.jg Programming 6 03-03-2011 08:59 AM
Insert character into a line with sed? & variables in sed? jago25_98 Programming 5 03-11-2004 07:12 AM


All times are GMT -5. The time now is 05:50 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration