LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 10-29-2009, 01:12 PM   #1
PB0711
Member
 
Registered: Aug 2004
Location: London, UK
Distribution: Ubuntu 10.10, ubuntu 11.04, suse 9.2, OSX
Posts: 259

Rep: Reputation: 30
Bash scripting using EOF to exec external programs


Dear all,

As sad as it is I've always strayed away from bash scripting, I used Perl or even c-shell. However, I'm trying to set up a program on my university's cluster system so now I have to. I want to run a program called 'R' and run some commands that will depend upon the arguments in the script. In c-shelll I know I can do a redirect using >>EOF like syntax, can I do this is bash and if so what am I doing wrong?

Code:
#PBS -l walltime=6:00:00
#PBS -l mem=8000mb
#PBS -l ncpus=1

##module load R/2.9.1 
##module load netcdf/3.2.6
##module load intel-suite/10.1

mzFiles=$(find . -iname '*.mz*' -print)
netCDFfiles=$(find . -iname '*.*cdf' -print) ##find all the files we need

fileString=$(echo $mzFiles $netCDFfiles) ##paste them together
filesArr=(`echo $foo | tr ' ' ' '`) ##make it into an array
date=$(date +%d%b) # get the date for unique file making

for f in ${filesArr[@]}
do
	if "$method" == "centWave"
	then
		qsub singlejob.sh $f $method $peakwidth
	else
		qsub singlejob.sh $f $method
	fi
	##This hasn't been written yet
	##This line will submit each file to the queing system
done

##Then we run R to paste everything together on one node.
##Check that the DIR are the same
/apps/R/2.9.1/bin/R --vanilla > $HOME/myReport$date.out <<EOF
.libPaths("$HOME/library")
library(xcms)
rdfiles <- list.files(path="$TMPDIR/", recursive = TRUE, pattern="*.Rd")
EOF
There's more code to this but it's not the point. At the moment when I run the script it says "line 42" all of this code ": bad substitution"
I'm guessing that means that I can't do this. So hows the best way to do this? I've googled and read quite a few bash scripting tutorials but I havn't really seen an answer to this yet.

any help welcomed. thx
 
Old 10-29-2009, 03:57 PM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
By EOF, are you referring to here documents?

But "bad substitution" sounds more like a sed error or something to me.

BTW, what's this supposed to do? It doesn't look right to me. I don't think you want $foo, at the very least, and the tr command doesn't change anything.
Code:
filesArr=(`echo $foo | tr ' ' ' '`)
 
Old 10-29-2009, 04:10 PM   #3
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,546
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
You need double quotes around fileString=$(echo $mzFiles $netCDFfiles) like
Code:
fileString="$(echo $mzFiles $netCDFfiles)" ##paste them together
If that doesn't work please post with line numbers.
 
Old 10-29-2009, 07:52 PM   #4
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Quote:
Originally Posted by David the H. View Post
BTW, what's this supposed to do? It doesn't look right to me. I don't think you want $foo, at the very least, and the tr command doesn't change anything.
Code:
filesArr=(`echo $foo | tr ' ' ' '`)
I thought that too, but if you count characters the replacement could well be a tab.
Kevin Barry
 
Old 11-04-2009, 06:34 AM   #5
PB0711
Member
 
Registered: Aug 2004
Location: London, UK
Distribution: Ubuntu 10.10, ubuntu 11.04, suse 9.2, OSX
Posts: 259

Original Poster
Rep: Reputation: 30
Hello all,

Thank you for the replys. I've looked again at this and I think as you rightly point out that my $foo isn't doing anything because I forgot to translate between my testing and my script opps
However, I'm having a problem now with good old line spaces. I want $fileArr to be an array of $fileString, hence using the 'tr'. Is there another way to do make a list of files into an array
Code:
filesArr=(`echo $fileString | tr '' ' '`)
box:SDS user $ echo ${filesArr[3]}
./SDS1/046
box:SDS user$ echo $fileString
./SDS1/025 Sample 1.mzXML ./SDS1/046 Sample 1.mzXML ./SDS1/081 Sample 1.mzXML ./SDS2/030 Sample 2.mzXML ./SDS2/066 Sample 2.mzXML ./SDS2/079 Sample 2.mzXML ./SDS3/035 Sample 3.mzXML ./SDS3/044 Sample 3.mzXML ./SDS3/097 Sample 3.mzXML ./SDS4/018 Sample 4.mzXML ./SDS4/040 Sample 4.mzXML ./SDS4/071 Sample 4.mzXML ./SDS5/020 Sample 5.mzXML ./SDS5/057 Sample 5.mzXML ./SDS5/072 Sample 5.mzXML
I also tried something like this
Code:
IFS=!
box:SDS user$ mzFiles=$(find . -iname '*.mz*' -print $f!)
-bash: !: event not found
box:SDS user$ mzFiles=$(`find . -iname '*.mz*' -print $f!`)
-bash: !`: event not found
$ mzFiles=$(" find . -iname '*.mz*' -print $f! ")
-bash:  find . -iname '*.mz*' -print ! : command not found
 
Old 11-04-2009, 11:07 AM   #6
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,546
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
If $fileString contains a space-separated list of files with names containing embedded spaces then there is no way to identify the individual files. If every file begins with ./ then it can be done but is not easy in shell script. Maybe better to take a step back and tell us how $fileString is generated; it may be easier to get the file names into an array at that point. You may find Greg's Wiki helpful.
 
Old 11-04-2009, 01:16 PM   #7
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
Probably the most convenient way when working with spaces in strings, especially file names, is to change the internal field separator so that it ignores them. Of course, it depends on how you've generated the strings in the first place. If you have a variable containing a list of names, and you change the IFS to newline, for example, then you just need to do is make sure that the individual entries are separated by newlines.

Code:
oldIFS=$IFS  # Save the original values in case you need to reset them later.
             # Default: space,tab, newline.

IFS='
'            # Change the separator to newline only.

<your code>

IFS=$oldIFS  # To reset the value, if necessary.
You could also change the IFS to something else, such as a comma or colon, if you're using such characters as delimiters in your input.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell Scripting Question (mv via find/-exec) ddenton Linux - General 3 11-21-2008 08:10 AM
bash scripting help want to execute external function on current item. Benanzo Linux - Software 3 02-03-2007 02:45 PM
Bash Scripting: Editing external files trek413 Linux - Software 1 11-02-2006 04:11 PM
Bash Scripting and Chroot/Exec yoshiznit123 Linux - General 5 12-14-2005 08:20 PM
bash scripting - referring to external arguments into loops linsson Linux - General 2 07-23-2004 12:24 PM


All times are GMT -5. The time now is 11:06 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration