LinuxQuestions.org - [SOLVED] printing error in a bash script using awk formatting. What is the real cause?

Page 1 of 2

Show 50 post(s) from this thread on one page

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - printing error in a bash script using awk formatting. What is the real cause? (https://www.linuxquestions.org/questions/programming-9/printing-error-in-a-bash-script-using-awk-formatting-what-is-the-real-cause-4175677783/)

centguy

06-28-2020 10:35 AM

printing error in a bash script using awk formatting. What is the real cause?

I just do not understand the output of this simple script.
What is going on?

The jid was printed correctly. But it has a wrong value in the awk line.

Quote:

[centos69]$ cat bash-nscc-vasp-phonon-use-list
#!/bin/bash

for jid in 0010 0011 0012 0013
do
echo $jid
cmd=$(echo | awk '{printf("qsub -N batch%s -o batch%s.o -e batch%04d.e nscc-vasp-phonon.pbs",'$jid','$jid','$jid')}' )
echo $cmd
done

[centos69]$ ./bash-nscc-vasp-phonon-use-list
0010
qsub -N batch8 -o batch8.o -e batch0008.e nscc-vasp-phonon.pbs
0011
qsub -N batch9 -o batch9.o -e batch0009.e nscc-vasp-phonon.pbs
0012
qsub -N batch10 -o batch10.o -e batch0010.e nscc-vasp-phonon.pbs
0013
qsub -N batch11 -o batch11.o -e batch0011.e nscc-vasp-phonon.pbs
[centos69]$

pan64

06-28-2020 10:45 AM

I think you need to add "" if you want to specify strings (something like this):

Code:

non.pbs","'$jid'","'$jid'","'$jid'")}' )

but printf is available in shell, do not need to use awk for this.

centguy

06-28-2020 10:49 AM

for jid in 0010 0011 0012 0013

replaced by

for jid in 10 11 12 13

print the correct output. But that is not what I want.

um...

centguy

06-28-2020 10:52 AM

pan64: I think I have tried that and it seems that is not the problem. I think printf cannot handle number starts with 0.

pan64

06-28-2020 11:13 AM

you need to handle them as strings.

Code:

 for jid in 0010 0011 0012 0013; do echo $jid; done

0010

0011

0012

0013

Code:

 $ for jid in 0010 0011 0012 0013; do echo $jid; cmd=$(echo | awk '{printf("qsub -N batch%s -o batch%s.o -e batch%04d.e nscc-vasp-phonon.pbs","'$jid'","'$jid'","'$jid'")}' ); echo $cmd; done

0010

qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs

0011

qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs

0012

qsub -N batch0012 -o batch0012.o -e batch0012.e nscc-vasp-phonon.pbs

0013

qsub -N batch0013 -o batch0013.o -e batch0013.e nscc-vasp-phonon.pbs

I do not really understand what is your problem?
Again almost the same printf works without awk.

Code:

for jid in 0010 0011 0012 0013; do echo $jid; printf "qsub -N batch%s -o batch%s.o -e batch%04d.e nscc-vasp-phonon.pbs\n" $jid $jid $jid; done

centguy

06-28-2020 12:12 PM

OK. Thanks a lot pan64!

Realize there is a world of difference between the double quote outside
versus single quote outside.

Quote:

$cat bash-s

#!/bin/bash

for jid in 0010 0011 0012 0013
do
echo $jid

#wrong if single quote is outside
cmd=$(echo | awk '{printf("qsub -N batch%s -o batch%s.o -e batch%s.e nscc-vasp-phonon.pbs",'"$jid"','"$jid"','"$jid"')}' )
echo $cmd

#linuxquestions/pan64, work if double quote is outside
cmd=$(echo | awk '{printf("qsub -N batch%s -o batch%s.o -e batch%s.e nscc-vasp-phonon.pbs","'$jid'","'$jid'","'$jid'")}' )
echo $cmd

# treat it as number, works if double quote is outside
cmd=$(echo | awk '{printf("qsub -N batch%04d -o batch%04d.o -e batch%04d.e nscc-vasp-phonon.pbs","'$jid'","'$jid'","'$jid'")}' )
echo $cmd

# if we insist to put single quote outside, then change octal to decimal
# see https://stackoverflow.com/questions/...o-number-error
cmd=$(echo | awk '{printf("qsub -N batch%04d -o batch%04d.o -e batch%04d.e nscc-vasp-phonon.pbs",'"$((10#$jid))"','"$((10#$jid))"','"$((10#$jid))"')}' )
echo $cmd
done

Output

Quote:

0010
qsub -N batch8 -o batch8.o -e batch8.e nscc-vasp-phonon.pbs
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
0011
qsub -N batch9 -o batch9.o -e batch9.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
0012
qsub -N batch10 -o batch10.o -e batch10.e nscc-vasp-phonon.pbs
qsub -N batch0012 -o batch0012.o -e batch0012.e nscc-vasp-phonon.pbs
qsub -N batch0012 -o batch0012.o -e batch0012.e nscc-vasp-phonon.pbs
qsub -N batch0012 -o batch0012.o -e batch0012.e nscc-vasp-phonon.pbs
0013
qsub -N batch11 -o batch11.o -e batch11.e nscc-vasp-phonon.pbs
qsub -N batch0013 -o batch0013.o -e batch0013.e nscc-vasp-phonon.pbs
qsub -N batch0013 -o batch0013.o -e batch0013.e nscc-vasp-phonon.pbs
qsub -N batch0013 -o batch0013.o -e batch0013.e nscc-vasp-phonon.pbs

shruggy

06-28-2020 12:41 PM

You marked the thread solved, but I think there are easier ways to achieve what you're trying to. TBH, I don't quite understand what you're after. E.g. the output you got can be generated with

Code:

#!/bin/bash

for jid in 10 11 12 13

do

  printf -vpjid batch%04d $jid

  echo "qsub -N $pjid -o $pjid.o -e $pjid.e nscc-vasp-phonon.pbs"

done

centguy

06-28-2020 10:28 PM

shruggy: It is solved.
The job is not only to echo to the screen but to form a command line to issue to submit jobs in a systematic manner, while taking care of
numbers that start from zero that can cause confusion. Once the script is working, I just make "echo $cmd" to become "$cmd".

centguy

06-28-2020 10:36 PM

I see:

Putting 00 back to the line to confirm it is working, I have

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vpjid batch%04d $jid
echo "qsub -N $pjid -o $pjid.o -e $pjid.e nscc-vasp-phonon.pbs"
done

qsub -N batch0008 -o batch0008.o -e batch0008.e nscc-vasp-phonon.pbs
qsub -N batch0009 -o batch0009.o -e batch0009.e nscc-vasp-phonon.pbs
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
Output:

I learned printf -v.

Thanks.

centguy

06-28-2020 10:48 PM

The third alternative:

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vjobname batch%04d $jid
cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
echo $cmd
done

This removes the reliance on awkward combination of echo, awk, printf that unexpected octal number interpretation creeps in.

pan64

06-29-2020 02:20 AM

this looks much better. And now you can try:

Code:

for jid in {10..13}

...

You need to check in your last input:
spedicifed: for jid in 0010 0011 0012 0013 but batch0008 is the first printed line. Is this what you want?
(numbers beginning with an extra 0 taken as octal, not decimal).

centguy

06-30-2020 12:07 AM

pan64:

My bad. The third alternative
should not be

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vjobname batch%04d $jid
cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
echo $cmd
done

which gives a wrong output

Quote:

qsub -N batch0008 -o batch0008.o -e batch0008.e nscc-vasp-phonon.pbs
qsub -N batch0009 -o batch0009.o -e batch0009.e nscc-vasp-phonon.pbs
qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs

The correct one is

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vjobname batch%s $jid
cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
echo $cmd
done

This gives the desired correct output

Quote:

qsub -N batch0010 -o batch0010.o -e batch0010.e nscc-vasp-phonon.pbs
qsub -N batch0011 -o batch0011.o -e batch0011.e nscc-vasp-phonon.pbs
qsub -N batch0012 -o batch0012.o -e batch0012.e nscc-vasp-phonon.pbs
qsub -N batch0013 -o batch0013.o -e batch0013.e nscc-vasp-phonon.pbs

Another correct variant is

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vjobname batch%04d $((10#$jid))
cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
echo $cmd
done

One final remark:
The following will NOT work.

Quote:

#!/bin/bash
for jid in 0010 0011 0012 0013
do
printf -vjobname batch%04d "'$jid'"
cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"
echo $cmd
done

which gives a strange output:

Quote:

qsub -N batch0048 -o batch0048.o -e batch0048.e nscc-vasp-phonon.pbs
qsub -N batch0048 -o batch0048.o -e batch0048.e nscc-vasp-phonon.pbs
qsub -N batch0048 -o batch0048.o -e batch0048.e nscc-vasp-phonon.pbs
qsub -N batch0048 -o batch0048.o -e batch0048.e nscc-vasp-phonon.pbs

shruggy

06-30-2020 01:43 AM

Quote:

Originally Posted by centguy (Post 6139534)

which gives a strange output

48 is the ASCII code for "0". When you quote an argument to printf (or even just precede it with a quote), printf will treat it as a string and take the numeric ASCII value of the first character after the quote.

Code:

$ printf %d\\n \'0

48

$ printf %d\\n \"A

65

From the Bash Reference Manual:

Quote:

Arguments to non-string format specifiers are treated as C language constants, except that a leading plus or minus sign is allowed, and if the leading character is a single or double quote, the value is the ASCII value of the following character.

From the POSIX standard:

Quote:

The argument operands shall be treated as strings if the corresponding conversion specifier is b, c, or s, and shall be evaluated as if by the strtod() function if the corresponding conversion specifier is a, A, e, E, f, F, g, or G. Otherwise, they shall be evaluated as unsuffixed C integer constants, as described by the ISO C standard, with the following extensions:

A leading <plus-sign> or <hyphen-minus> shall be allowed.
If the leading character is a single-quote or double-quote, the value shall be the numeric value in the underlying codeset of the character following the single-quote or double-quote.
Suffixed integer constants may be allowed.

BTW, why are you including leading zeros in the job ID value? printf can do this for you:

Code:

#!/bin/bash

for jid in 10 11 12 13

do

  printf -vjobname batch%04d $jid

  cmd="qsub -N $jobname -o $jobname.o -e $jobname.e nscc-vasp-phonon.pbs"

  echo $cmd

done

centguy

06-30-2020 09:49 AM

<< BTW, why are you including leading zeros in the job ID value? printf can do this for you:

Answer: I do not want my directories to be named, say, 1, 2, 3, ..., 100
and ls will show confusing sequential listing.

Say I have 0001, 0002, ... 0276

and I easily generate these zero padded 276 numbers in a file called indexfile.

Then I simply use

for jid in $(cat indexfile)

I can cd $jid and do any task I need such as submit a script through qsub

and jobs are submitted with the correct ID in the PBS queueing system.

Please see my first post.

pan64

06-30-2020 10:13 AM

in that case you need to use that as string, not as integer. That will keep the original line (in the indexfile).
Additionally:

Code:

while read -r jid

do

    jobname="batch$jid"

....

done < indexfile

is more efficient.

All times are GMT -5. The time now is 02:09 AM.

Page 1 of 2

Show 50 post(s) from this thread on one page