LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 06-28-2020, 10:35 AM   #1
centguy
Member
 
Registered: Feb 2008
Posts: 468
Blog Entries: 1

Rep: Reputation: 38
printing error in a bash script using awk formatting. What is the real cause?


I just do not understand the output of this simple script.
What is going on?

The jid was printed correctly. But it has a wrong value in the awk line.

Quote:
[centos69]$ cat bash-nscc-vasp-phonon-use-list
#!/bin/bash

for jid in 0010 0011 0012 0013
do
echo $jid
cmd=$(echo | awk '{printf("qsub -N batch%s -o batch%s.o -e batch%04d.e nscc-vasp-phonon.pbs",'$jid','$jid','$jid')}' )
echo $cmd
done



[centos69]$ ./bash-nscc-vasp-phonon-use-list
0010
qsub -N batch8 -o batch8.o -e batch0008.e nscc-vasp-phonon.pbs
0011
qsub -N batch9 -o batch9.o -e batch0009.e nscc-vasp-phonon.pbs
0012
qsub -N batch10 -o batch10.o -e batch0010.e nscc-vasp-phonon.pbs
0013
qsub -N batch11 -o batch11.o -e batch0011.e nscc-vasp-phonon.pbs
[centos69]$
 
Old 06-28-2020, 10:45 AM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,663

Rep: Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777
I think you need to add "" if you want to specify strings (something like this):
Code:
non.pbs","'$jid'","'$jid'","'$jid'")}' )
but printf is available in shell, do not need to use awk for this.
 
Old 06-28-2020, 10:49 AM   #3
centguy
Member
 
Registered: Feb 2008
Posts: 468

Original Poster
Blog Entries: 1

Rep: Reputation: 38
for jid in 0010 0011 0012 0013

replaced by

for jid in 10 11 12 13

print the correct output. But that is not what I want.

um...
 
Old 06-28-2020, 10:52 AM   #4
centguy
Member
 
Registered: Feb 2008
Posts: 468

Original Poster
Blog Entries: 1

Rep: Reputation: 38
pan64: I think I have tried that and it seems that is not the problem. I think printf cannot handle number starts with 0.
 
Old 06-28-2020, 11:13 AM   #5
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,663

Rep: Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777
you need to handle them as strings.
Code:
 for jid in 0010 0011 0012 0013; do echo $jid; done
0010
0011
0012
0013
Code:
 $ for jid in 0010 0011 0012 0013; do echo $jid; cmd=$(echo | awk '{printf("qsub -N batch%s -o batch%s.o -e batch%04d.e nscc-vasp-phonon.pbs","'$jid'","'$jid'","'$jid'")}' ); echo $cmd; done
0010
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
0011
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
0012
qsub -N batch0012 -o batch0012.o -e batch0012.e nscc-vasp-phonon.pbs
0013
qsub -N batch0013 -o batch0013.o -e batch0013.e nscc-vasp-phonon.pbs
I do not really understand what is your problem?
Again almost the same printf works without awk.
Code:
for jid in 0010 0011 0012 0013; do echo $jid; printf "qsub -N batch%s -o batch%s.o -e batch%04d.e nscc-vasp-phonon.pbs\n" $jid $jid $jid; done
 
Old 06-28-2020, 12:12 PM   #6
centguy
Member
 
Registered: Feb 2008
Posts: 468

Original Poster
Blog Entries: 1

Rep: Reputation: 38
OK. Thanks a lot pan64!

Realize there is a world of difference between the double quote outside
versus single quote outside.



Quote:
$cat bash-s

#!/bin/bash

for jid in 0010 0011 0012 0013
do
echo $jid

#wrong if single quote is outside
cmd=$(echo | awk '{printf("qsub -N batch%s -o batch%s.o -e batch%s.e nscc-vasp-phonon.pbs",'"$jid"','"$jid"','"$jid"')}' )
echo $cmd

#linuxquestions/pan64, work if double quote is outside
cmd=$(echo | awk '{printf("qsub -N batch%s -o batch%s.o -e batch%s.e nscc-vasp-phonon.pbs","'$jid'","'$jid'","'$jid'")}' )
echo $cmd

# treat it as number, works if double quote is outside
cmd=$(echo | awk '{printf("qsub -N batch%04d -o batch%04d.o -e batch%04d.e nscc-vasp-phonon.pbs","'$jid'","'$jid'","'$jid'")}' )
echo $cmd

# if we insist to put single quote outside, then change octal to decimal
# see https://stackoverflow.com/questions/...o-number-error
cmd=$(echo | awk '{printf("qsub -N batch%04d -o batch%04d.o -e batch%04d.e nscc-vasp-phonon.pbs",'"$((10#$jid))"','"$((10#$jid))"','"$((10#$jid))"')}' )
echo $cmd
done
Output

Quote:
0010
qsub -N batch8 -o batch8.o -e batch8.e nscc-vasp-phonon.pbs
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
0011
qsub -N batch9 -o batch9.o -e batch9.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
0012
qsub -N batch10 -o batch10.o -e batch10.e nscc-vasp-phonon.pbs
qsub -N batch0012 -o batch0012.o -e batch0012.e nscc-vasp-phonon.pbs
qsub -N batch0012 -o batch0012.o -e batch0012.e nscc-vasp-phonon.pbs
qsub -N batch0012 -o batch0012.o -e batch0012.e nscc-vasp-phonon.pbs
0013
qsub -N batch11 -o batch11.o -e batch11.e nscc-vasp-phonon.pbs
qsub -N batch0013 -o batch0013.o -e batch0013.e nscc-vasp-phonon.pbs
qsub -N batch0013 -o batch0013.o -e batch0013.e nscc-vasp-phonon.pbs
qsub -N batch0013 -o batch0013.o -e batch0013.e nscc-vasp-phonon.pbs

Last edited by centguy; 06-28-2020 at 12:15 PM.
 
Old 06-28-2020, 12:41 PM   #7
shruggy
Member
 
Registered: Mar 2020
Posts: 994

Rep: Reputation: Disabled
You marked the thread solved, but I think there are easier ways to achieve what you're trying to. TBH, I don't quite understand what you're after. E.g. the output you got can be generated with
Code:
#!/bin/bash
for jid in 10 11 12 13
do
  printf -vpjid batch%04d $jid
  echo "qsub -N $pjid -o $pjid.o -e $pjid.e nscc-vasp-phonon.pbs"
done

Last edited by shruggy; 06-28-2020 at 12:51 PM.
 
Old 06-28-2020, 10:28 PM   #8
centguy
Member
 
Registered: Feb 2008
Posts: 468

Original Poster
Blog Entries: 1

Rep: Reputation: 38
shruggy: It is solved.
The job is not only to echo to the screen but to form a command line to issue to submit jobs in a systematic manner, while taking care of
numbers that start from zero that can cause confusion. Once the script is working, I just make "echo $cmd" to become "$cmd".

Last edited by centguy; 06-28-2020 at 10:37 PM.
 
Old 06-28-2020, 10:36 PM   #9
centguy
Member
 
Registered: Feb 2008
Posts: 468

Original Poster
Blog Entries: 1

Rep: Reputation: 38
I see:

Putting 00 back to the line to confirm it is working, I have

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vpjid batch%04d $jid
echo "qsub -N $pjid -o $pjid.o -e $pjid.e nscc-vasp-phonon.pbs"
done

qsub -N batch0008 -o batch0008.o -e batch0008.e nscc-vasp-phonon.pbs
qsub -N batch0009 -o batch0009.o -e batch0009.e nscc-vasp-phonon.pbs
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
Output:



I learned printf -v.

Thanks.
 
Old 06-28-2020, 10:48 PM   #10
centguy
Member
 
Registered: Feb 2008
Posts: 468

Original Poster
Blog Entries: 1

Rep: Reputation: 38
The third alternative:

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vjobname batch%04d $jid
cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
echo $cmd
done
This removes the reliance on awkward combination of echo, awk, printf that unexpected octal number interpretation creeps in.
 
Old 06-29-2020, 02:20 AM   #11
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,663

Rep: Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777
this looks much better. And now you can try:
Code:
for jid in {10..13}
...
You need to check in your last input:
spedicifed: for jid in 0010 0011 0012 0013 but batch0008 is the first printed line. Is this what you want?
(numbers beginning with an extra 0 taken as octal, not decimal).
 
1 members found this post helpful.
Old 06-30-2020, 12:07 AM   #12
centguy
Member
 
Registered: Feb 2008
Posts: 468

Original Poster
Blog Entries: 1

Rep: Reputation: 38
pan64:

My bad. The third alternative
should not be

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vjobname batch%04d $jid
cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
echo $cmd
done
which gives a wrong output

Quote:
qsub -N batch0008 -o batch0008.o -e batch0008.e nscc-vasp-phonon.pbs
qsub -N batch0009 -o batch0009.o -e batch0009.e nscc-vasp-phonon.pbs
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
The correct one is
Quote:
#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vjobname batch%s $jid
cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
echo $cmd
done
This gives the desired correct output
Quote:
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
qsub -N batch0012 -o batch0012.o -e batch0012.e nscc-vasp-phonon.pbs
qsub -N batch0013 -o batch0013.o -e batch0013.e nscc-vasp-phonon.pbs
Another correct variant is

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vjobname batch%04d $((10#$jid))
cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
echo $cmd
done
One final remark:
The following will NOT work.

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vjobname batch%04d "'$jid'"
cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
echo $cmd
done

which gives a strange output:

Quote:
qsub -N batch0048 -o batch0048.o -e batch0048.e nscc-vasp-phonon.pbs
qsub -N batch0048 -o batch0048.o -e batch0048.e nscc-vasp-phonon.pbs
qsub -N batch0048 -o batch0048.o -e batch0048.e nscc-vasp-phonon.pbs
qsub -N batch0048 -o batch0048.o -e batch0048.e nscc-vasp-phonon.pbs

Last edited by centguy; 06-30-2020 at 12:20 AM.
 
Old 06-30-2020, 01:43 AM   #13
shruggy
Member
 
Registered: Mar 2020
Posts: 994

Rep: Reputation: Disabled
Quote:
Originally Posted by centguy View Post
which gives a strange output
48 is the ASCII code for "0". When you quote an argument to printf (or even just precede it with a quote), printf will treat it as a string and take the numeric ASCII value of the first character after the quote.
Code:
$ printf %d\\n \'0
48
$ printf %d\\n \"A
65
From the Bash Reference Manual:
Quote:
Arguments to non-string format specifiers are treated as C language constants, except that a leading plus or minus sign is allowed, and if the leading character is a single or double quote, the value is the ASCII value of the following character.
From the POSIX standard:
Quote:
The argument operands shall be treated as strings if the corresponding conversion specifier is b, c, or s, and shall be evaluated as if by the strtod() function if the corresponding conversion specifier is a, A, e, E, f, F, g, or G. Otherwise, they shall be evaluated as unsuffixed C integer constants, as described by the ISO C standard, with the following extensions:
  • A leading <plus-sign> or <hyphen-minus> shall be allowed.
  • If the leading character is a single-quote or double-quote, the value shall be the numeric value in the underlying codeset of the character following the single-quote or double-quote.
  • Suffixed integer constants may be allowed.
BTW, why are you including leading zeros in the job ID value? printf can do this for you:
Code:
#!/bin/bash
for jid in 10 11 12 13
do
  printf -vjobname batch%04d $jid
  cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
  echo $cmd
done

Last edited by shruggy; 06-30-2020 at 06:39 AM.
 
Old 06-30-2020, 09:49 AM   #14
centguy
Member
 
Registered: Feb 2008
Posts: 468

Original Poster
Blog Entries: 1

Rep: Reputation: 38
<< BTW, why are you including leading zeros in the job ID value? printf can do this for you:

Answer: I do not want my directories to be named, say, 1, 2, 3, ..., 100
and ls will show confusing sequential listing.

Say I have 0001, 0002, ... 0276

and I easily generate these zero padded 276 numbers in a file called indexfile.

Then I simply use

for jid in $(cat indexfile)

I can cd $jid and do any task I need such as submit a script through qsub

and jobs are submitted with the correct ID in the PBS queueing system.

Please see my first post.

Last edited by centguy; 06-30-2020 at 09:51 AM.
 
Old 06-30-2020, 10:13 AM   #15
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,663

Rep: Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777Reputation: 4777
in that case you need to use that as string, not as integer. That will keep the original line (in the indexfile).
Additionally:
Code:
while read -r jid
do
    jobname="batch$jid"
....
done < indexfile
is more efficient.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Once again... awk.. awk... awk shivaa Linux - Newbie 13 12-31-2012 04:56 AM
[SOLVED] call awk from bash script behaves differently to awk from CLI = missing newlines titanium_geek Programming 4 05-26-2011 09:06 PM
real, real, real, basic computer for my Mom shengchieh General 3 12-18-2005 12:02 AM
Real Programmers Real People Real CS Students nakkaya General 5 07-04-2003 02:46 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 05:25 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration