LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 06-18-2011, 10:59 PM   #1
ctrlbrk
LQ Newbie
 
Registered: Jun 2011
Posts: 11

Rep: Reputation: Disabled
Setting variable in bash with sed


I am killing myself with this, please someone come to the rescue...

Code:
#!/bin/sh
IFILE=$@
set OFILE=`sed 's/.\{4\}'$IFILE'/'`
echo input file is... $IFILE
echo output file is $OFILE
#nice ffmpeg -i "$INFILE" -f matroska -acodec copy -vcodec libx264 -preset slow -crf 20 -threads 0 "$OFILE-new.mkv"
Line 3 is the problem (set OFILE...). I've tried a million variations, I always get a syntax error.

I am trying to remove the 4 trailing chars from $@ and save that as OFILE.

An example:
Code:
# ~/encode.sh STAR\ TREK\ ENTERPRISE\ -\ S01\ E14\ -\ SHADOWS\ OF\ P\'JEM\ 720P\ DD5.1\ x264\ MMI.mkv
sed: -e expression #1, char 12: unterminated `s' command
input file is... STAR TREK ENTERPRISE - S01 E14 - SHADOWS OF P'JEM 720P DD5.1 x264 MMI.mkv
output file is
Thanks in advance.
 
Old 06-18-2011, 11:13 PM   #2
binary_pearl
Member
 
Registered: Jul 2007
Location: Chicago Illinois
Distribution: SLES 10 SP2/SP3, SLES 11 SP1, OpenSUSE, Sabayon, Gentoo, Fedora 14, RHEL 3/4/5/6
Posts: 98
Blog Entries: 1

Rep: Reputation: 23
I can't really test it out right now, but try using using a backslash on the internal single quotes between $IFILE:

set OFILE=`sed 's/.\{4\}\'$IFILE\'/'`

--Shaun
 
Old 06-19-2011, 12:13 AM   #3
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,689

Rep: Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987
Were you looking for something like:
Code:
IFILE="$1"
OFILE="${IFILE%.*}"
 
1 members found this post helpful.
Old 06-19-2011, 01:07 AM   #4
ctrlbrk
LQ Newbie
 
Registered: Jun 2011
Posts: 11

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by grail View Post
Were you looking for something like:
Code:
IFILE="$1"
OFILE="${IFILE%.*}"
Yes, exactly that.

Thx
 
Old 06-19-2011, 03:00 AM   #5
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,689

Rep: Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987
Please mark as SOLVED when you have a solution.
 
Old 06-19-2011, 05:44 AM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
As shown, parameter substitution is generally a better option when it comes to simple manipulation of variable strings like this, and particularly when stripping out paths or extensions from a basename.

But this line has quite a few issues I want to comment on:
Code:
set OFILE=`sed 's/.\{4\}'$IFILE'/'`
1. In most shells the set command isn't used to define a variable, but to set the $@ positional parameters from inside the script, and to manipulate certain shell options. Do you perhaps want the declare/typeset command instead? But in truth variables don't usually need to be declared at all before use.

2. $(..) is highly recommended over `..`.

3. There's no input string for sed to work on. You want to echo the filename into sed, not use it as part of the expression.

4. The s/// substitution command is unterminated.

5. You can avoid using backslash escapes on the brackets with the -r "regex" option.

6. When the input can contain "/" slashes, such as with file paths, you can replace the regular substitution delimiters with another ascii character, e.g. sed 's|foo|bar|'. This would let you input files without actually being in the directory it's sitting in.

7. When using a variable inside a longer string, the quoting can usually be simplified by using a single set of double-quotes, instead of using the close+insert variable+reopen pattern. e.g. use sed "s/$var/foo/" instead of sed '/'$var'/foo/'. But in this case you don't really want to put the variable inside the expression at all so the point is moot.

See here for a detailed rundown on using the two types of quotes:
http://www.tldp.org/LDP/abs/html/quoting.html

8. You need to anchor the regex to the end of the line so that it strips the last part off. As it is, it looks like it would strip off the first 4 characters. If it weren't already completely broken, that is.

Taking all this into account, a much better command would look like this:
Code:
OFILE=$( echo "$IFILE" | sed -r 's|.{4}$||' )

Second, let's talk about how you're using the input parameter. Your script does this:
Code:
~/encode.sh STAR\ TREK\ ENTERPRISE\ -\ S01\ E14\ -\ SHADOWS\ OF\ P\'JEM\ 720P\ DD5.1\ x264\ MMI.mkv
...and the script reads it with "$@".

This is very sloppy. There are much better ways to handle filenames that include spaces and other reserved characters. See here:
http://mywiki.wooledge.org/BashFAQ/020

I've found that the most convenient overall option is to set your IFS to newline-only (IFS=$'\n') at the start of your script, then no word breaking will take place on whole-line strings.

But also note that backslash escaping on the command-line turns off the word-breaking meaning of spaces, so the filename in the command above is seen as only a single $1 parameter anyway.

P.S. The single quote-mark in that name is a very bad thing. It will break a lot of scripts and commands, and will always be a headache to work with in any case. Experienced *nix users learn quickly to avoid filenames that include things like that. I suggest renaming your files to remove reserved shell characters like that ASAP. I would even recommend removing the spaces as well. Experienced users usually use underscores instead of spaces.

Check out the detox program for an easy way to clean up filenames.
http://detox.sourceforge.net/

Finally, about the shebang:
Code:
#!/bin/sh
Using /bin/sh means that the script will run in a limited posix-compatible mode, and many shell-specific features will be disabled. This is only really needed when the script needs to be portable to systems that don't have bash (or the shell you use) installed. Unless you really need that, and I suspect you don't, I suggest you use #!/bin/bash instead (or whatever shell you're using) and take advantage of the full power of your shell.
 
1 members found this post helpful.
Old 06-19-2011, 06:00 AM   #7
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,563
Blog Entries: 29

Rep: Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179
All good stuff and nicely set out, David the H

But ...
Quote:
Originally Posted by David the H. View Post
P.S. The single quote-mark in that name is a very bad thing. It will break a lot of scripts and commands, and will always be a headache to work with in any case. Experienced *nix users learn quickly to avoid filenames that include things like that. I suggest renaming your files to remove reserved shell characters like that ASAP. I would even recommend removing the spaces as well. Experienced users usually use underscores instead of spaces.
That would certainly make life easier for script writers but I disagree for three reasons:
  1. "If it can go wrong, it will go wrong" and the principle of defensive programming. Linux allows all characters except / in pathname components and unusual ones do arise by accident or user freedom to name files as they choose. Hence it is prudent to write scripts that do not break for all possible pathnames. It's not as easy but techniques can be developed and once developed used over.
  2. Information Systems exist to support their users and I see my job as a professional is to ease their work, not to place unnecessary restrictions on them.
  3. As a user I like the legibility that comes from using any character I like in pathname components
 
Old 06-19-2011, 06:34 AM   #8
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
Oh, I agree, and I always try to write my scripts "defensively" like that. But I still sterilize my filenames too, since I haven't personally written every program that exists on my system . Good practices should be mentioned for both sides of the equation.

I also brought it up mostly because the OP is clearly still a beginner and not yet well-versed on just how to write scripts that can handle poorly-formed input. It's faster in the short term, and will lead to fewer problems long-term, if he gets into the habit of cleanly naming his files now. If he starts out with a very strict naming routine, then over time he should gain the experience to decide for himself what exceptions he wants to make, and the amount of hassle he's willing to accept in the process.

I personally tend to avoid all reserved and non-ascii characters in my filenames except for simple parentheses. I've found I do occasionally do need a grouping/commenting pattern of some kind, particularly for my music collection. I also keep everything lowercase-only when possible, again with the exception of my music files, which are capitalized.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Bash; awk or sed output to variable: how keep newline at end of each output line porphyry5 Programming 3 06-10-2011 06:50 PM
[BASH] Hangman Game: changing a character within a variable without use of sed or awk uzi85 Linux - Newbie 1 03-27-2009 02:41 AM
bash - setting a persistent variable that remains after the script exits CJ Chitwood Programming 9 01-24-2009 08:48 PM
Bash, using variable in sed fur Programming 3 11-12-2005 08:41 AM
sed doesn't accept $variable in bash script chii-chan Programming 6 05-28-2005 08:07 AM


All times are GMT -5. The time now is 06:10 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration