Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
|
05-26-2012, 11:55 PM
|
#1
|
LQ Newbie
Registered: Nov 2011
Location: Bangalore
Distribution: Ubuntu/Cygwin
Posts: 29
Rep:
|
sed gives :sed: -e expression #1, char 1: unknown command: `''
I am attempting to automatically replace a output name string in a batchfile prior to its execution to describe settings explicitely in output. However, I am not able to to do that
My Code:
Code:
inFile="$WORK/${SimName}.in"
replaceCmd="sed 's/sim.dat/${SimName}.dat/g' $inFile"
echo $replaceCmd
$replaceCmd
And the output tracked by echo'ing is:
Code:
sed 's/sim.dat/sim0525_PerfCentOS_U032_MAT04_THRD024.dat/g' /home/ssr/Desktop/PerfCentOS/sim0525_PerfCentOS_U032_MAT04_THRD024/sim0525_PerfCentOS_U032_MAT04_THRD024.in
sed: -e expression #1, char 1: unknown command: `''
Any suggestions ?
Thanks,
Last edited by samasat; 05-27-2012 at 01:27 PM.
Reason: Formatting correction - mistakely used "Quotes" replaced by "Code" as per David's suggession
|
|
|
05-27-2012, 12:11 AM
|
#2
|
Senior Member
Registered: Jan 2012
Distribution: Slackware
Posts: 3,348
Rep:
|
It seems to be choking on the quote character. Have you tried using escaped double quotes (\") instead of single quotes around the sed expression?
|
|
|
05-27-2012, 03:50 AM
|
#3
|
Member
Registered: Apr 2009
Distribution: slackware64-15.0
Posts: 546
Rep:
|
Quote:
Originally Posted by samasat
inFile="$WORK/${SimName}.in"
replaceCmd="sed 's/sim.dat/${SimName}.dat/g' $inFile"
echo $replaceCmd
$replaceCmd
|
you're trying to replace 'sim.dat' with the value of $SimName.dat in $inFile
try this
Code:
cmd="sed 's?sim\.dat?$SimName.dat?g' $inFile"
eval "$cmd"
Last edited by vonbiber; 05-27-2012 at 03:52 AM.
|
|
|
05-27-2012, 09:23 AM
|
#4
|
Bash Guru
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852
|
Please use *** [code][/code] tags*** around your code and data, to preserve formatting and to improve readability. Please do not use quote tags, colors, or other fancy formatting.
Oh, hells no! Never, and I mean NEVER, use eval for something as trivial as this. Indeed, you should only very rarely need it at all. It can be quite a security risk, and is almost never really necessary in general scripting.
Eval command and security issues
http://mywiki.wooledge.org/BashFAQ/048
In addition, variables are for storing data, not code. The spaces, quotes, and other special characters cease to be special when in a variable (unless the expansion is unquoted, then some of them still have an effect, but likely not the one you want; see my next point), and in any case leads to unclear, obfuscated code flow. If you need repeated use of the same command, don't put it inside a variable, use a function.
I'm trying to put a command in a variable, but the complex cases always fail!
http://mywiki.wooledge.org/BashFAQ/050
A second problem here is that if the sed command is surrounded by single quotes, the variable inside them does not get expanded. No quotes is even worse, as the shell attempts to perform word-splitting and glob expansion after the substitution.
http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes
Here's what you should be doing:
Code:
replaceCmd(){
sed "s/sim.dat/$1.dat/g" "$2"
}
inFile="$work/$SimName.in"
replaceCmd "$SimName" "$inFile"
BTW, since environment variables are generally all upper-case, it's good practice to keep your own user variables in lower-case or mixed-case to help differentiate them.
Also, please don't use the full brace-surounded " ${var}" form unless absolutely necessary. It does nothing useful, and only adds extra typing and clutter to your code.
Last edited by David the H.; 05-27-2012 at 09:29 AM.
Reason: Darn that text duplicating bug! + some rewording
|
|
1 members found this post helpful.
|
05-27-2012, 01:35 PM
|
#5
|
LQ Newbie
Registered: Nov 2011
Location: Bangalore
Distribution: Ubuntu/Cygwin
Posts: 29
Original Poster
Rep:
|
\" around sed expression did not help
Quote:
Originally Posted by Ser Olmy
It seems to be choking on the quote character. Have you tried using escaped double quotes (\") instead of single quotes around the sed expression?
|
Thanks for your suggestion Ser. I did not make any difference in error message overall.
I used following
Code:
replaceCmd="sed \"s/sim.dat/${SimName}.dat/g\" $inFile"
echo $replaceCmd
$replaceCmd
Output was:
Code:
sed "s/sim.dat/sim0502_PerfCentOS_U038_MAT01_THRD01.dat/g" /home/ssr/Desktop/TrashIt_BashTest/sim0502_PerfCentOS_U038_MAT01_THRD01/sim0502_PerfCentOS_U038_MAT01_THRD01.in
sed: -e expression #1, char 1: unknown command: `"'
|
|
|
05-27-2012, 02:10 PM
|
#6
|
LQ Newbie
Registered: Nov 2011
Location: Bangalore
Distribution: Ubuntu/Cygwin
Posts: 29
Original Poster
Rep:
|
making sed command a function worked
Thanks David. Your help did work.
Only minor correction I had to make was that sed line I was using earlier was not correct. My objective was to replace the string inside the file and save the changes. When I used the old approach it just threw it corrected on screen but changes in file were not saved.
Then I tried this one which also did not work. It may be my ignorance that redirection does not work with sed:
Code:
replaceCmd(){
sed "s/sim.dat/$1.dat/g" "$2" > "$2"
}
Finally, after googling a bit I found '-i' switch that saved file changes as given below:
Code:
replaceCmd(){
sed -i "s/sim.dat/$1.dat/g" "$2"
}
I just inserted this function in my code to replace old lines. However, I was wondering if there is any standard/conventional location of such additional function in the code e.g. in C we declare it right at the top or through header file ...
Also, I do agree with approach from your post and the link that:
Quote:
variables are for storing data, not code
|
and also regarding security risks involved in eval. My only addition is that it also depends on where the code is going to be implemented. e.g. if I am using this code for my research, probably I would think that security is not a concern for me as long as task gets done since anyway my work is going to go open-source. Also, there is a fact that out of 100 things or approaches may be 5-6 do work. I guess, in that case, the ease to try out things quickly gains more importance prior to arriving at final professional level code for finalized approach. There definitely one has to be cautious and perfect and should implement your suggestion.
Thanks David, Ser and vonbiber. I appreciate your help and time !
Thanks,
Last edited by samasat; 05-27-2012 at 02:58 PM.
Reason: Added correction that I need to use '-i' in sed command
|
|
|
05-27-2012, 07:14 PM
|
#7
|
LQ Guru
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,415
|
Quote:
I just inserted this function in my code to replace old lines. However, I was wondering if there is any standard/conventional location of such additional function in the code e.g. in C we declare it right at the top or through header file ...
|
The shells read strictly top to bottom, so the fn defn has to come before its called in the 'main' as you can't reference a fn you haven't already 'read' (parsed).
This is in contrast to C or Perl, where the 'compiler/linker' reads the entire src file before trying to action anything.
Shell is interpreted ie parsed+actioned as it is read.
HTH
Last edited by chrism01; 05-28-2012 at 06:56 PM.
Reason: typo
|
|
|
05-28-2012, 11:34 AM
|
#8
|
LQ Newbie
Registered: Nov 2011
Location: Bangalore
Distribution: Ubuntu/Cygwin
Posts: 29
Original Poster
Rep:
|
Thanks Chrism01 !
|
|
|
05-28-2012, 11:55 AM
|
#9
|
Bash Guru
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852
|
Code:
sed "s/sim.dat/$1.dat/g" "$2" > "$2"
And this doesn't work because the " >" redirection is parsed, and the input file is overwritten, before the command is launched. So you end up with an empty file for input. You have to go through a temporary file, or use the -i option, which actually does the same thing, just in the background.
Incidentally, I've recently been learning how to use ed, and you might consider using that instead. As a real text editor, it can operate on the file directly.
It might also be a good idea to make your function more general in scope, so you can continue to use it even with different starting and ending strings.
Code:
# function: replaceCmd
# globally replace one string with another using ed.
# $1 is the filename, $2 is the string to match
# $3 is the replacement string. strings are treated as regexes.
# In addition, any forward slashes in the strings must be
# escaped with backslashes, due to the delimiter.
replaceCmd() {
printf '%s\n' "g/$2/ s/$2/$3/g" 'w' | ed -s "$1"
}
replaceCmd "$inFile" "sim.dat" "$SimName.dat"
How to use ed:
http://wiki.bash-hackers.org/howto/edit-ed
http://snap.nlc.dcccd.edu/learn/nlc/ed.html
(also read the info page)
Finally, re your eval comment. While you are certainly correct that you don't need to worry so much about security in a limited, single-user situation, I'd still recommend against it.
First of all, security isn't just about unauthorized malicious users; it also includes authorized users making mistakes. Don't underestimate the possibility of a malformed string accidentally forming an unwanted command or having other unintended side-effects.
Second, laziness and sloppiness are also risks to beware of. If you're lax in coding when it's safe, then you aren't training yourself to be careful when it isn't. Over-dependence on quick 'n easy, just-get-it-done solutions means you aren't really learning how to use the better alternatives available.
eval is the nuclear weapon of the scripting world; it's only meant to be brought out as a last resort when nothing else works. And since it can nearly always be avoided with just a little bit of work and know-how, any use of it, without a damn good reason and without knowing exactly what you're doing, is really a bad use of it, IMO.
Last edited by David the H.; 05-28-2012 at 11:57 AM.
Reason: minor code mistake
|
|
1 members found this post helpful.
|
05-28-2012, 02:31 PM
|
#10
|
Member
Registered: Oct 2011
Location: Warrington, UK
Distribution: Arch local, Debian on VPS, several RPIs.
Posts: 300
Rep:
|
I can second (or should that be third) comments about using eval. I'm a reasonably experienced and competent bash scripter, but yesterday I managed to completely trash a system configuration file with the use of eval and a string that didn't end up quite how I intended.
Two lessons learned:
i. Don't use eval
ii. Never test live on system configurations lol
As it happens, it was a fairly trivial matter to re-write the config file - this time. Other times I may not be so lucky.
|
|
1 members found this post helpful.
|
06-09-2012, 06:31 PM
|
#11
|
LQ Newbie
Registered: Nov 2011
Location: Bangalore
Distribution: Ubuntu/Cygwin
Posts: 29
Original Poster
Rep:
|
Thanks !
Thanks David and Roken for your advice. I will keep that in mind !
Also, thanks David for the idea of making it generalized. I hope to apply same while using sed as well. I am yet to try 'ed'.
UPDATE:
I have checked that both sed and ed work equally good.
Code:
replaceCmd_sed(){ # $1 -> OldStr, $2 -> NewStr , $3 -> Name of file to edit
sed -i "s/$1/$2/g" "$3"
}
replaceCmd_ed(){ # $1 -> OldStr, $2 -> NewStr , $3 -> Name of file to edit
printf '%s\n' "g/$1/ s/$1/$2/g" 'w' | ed -s "$3"
}
Last edited by samasat; 06-10-2012 at 07:59 PM.
Reason: Updating about successful use of ed and sed
|
|
|
All times are GMT -5. The time now is 03:06 PM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|