[SOLVED] how do i batch execute multiple line commands

atjurhs · 10-20-2017, 01:40 PM

hi guys,

i need to run the same python executable command on multiple input files, each having a different input path, and then write the output files to a different output path.

then i change a setting within the python code and re-run it on the same set of input files.

the python executable is coded to know what input file it needs in the input directory and it automatically dumps out a set of files in the output directory.

for example, with the executable having the first setting value, i run each line command by hand:

Code:

py27 exe.py -r /input_path1 -o /output_path1
py27 exe.py -r /input_path2 -o /output_path2
py27 exe.py -r /input_path3 -o /output_path3
py27 exe.py -r /input_path4 -o /output_path4
py27 exe.py -r /input_path5 -o /output_path5
etc...

then i look at the output files, and then change the executable's setting to a different value and re-run each line command by hand:

Code:

py27 exe.py -r /input_path1 -o /output_path1
py27 exe.py -r /input_path2 -o /output_path2
py27 exe.py -r /input_path3 -o /output_path3
py27 exe.py -r /input_path4 -o /output_path4
py27 exe.py -r /input_path5 -o /output_path5
etc...

this process is very tedious.

what i'd like to do is to copy all the line commands into a single text file and then run an bash or AWK command (or something similar) on that text file to "batch" process the line commands, then i can change the setting's value in the python code, and re-run the batch.

i suspect this batch processing can be done inside the python code, only i'm not the owner of the code and it's waaaay outside my ability to write python, but i do understand a bit of bash and AWK

so i'm thinking something very crude like...

Code:

#!/bin/sh
ListOfLineCommands.txt | { while read each line ; do
     py27 exe.py )
     done
}

can anybody help me? i'd really appreciate it!

thanks!

Todd

michaelk · 10-20-2017, 02:01 PM

One quick script.

Code:

#!/bin/bash

while read in out
do
  py27 exe.py -r $in -o $out
done < "data-file"

and data-file contains

Code:

/input_path1 /output_path1
/input_path2 /output_path2
/input_path3 /output_path3
/input_path4 /output_path4
/input_path5 /output_path5

atjurhs · 10-20-2017, 02:11 PM

hi michaelk, and thanks!

i was just thinking about maybe a for loop before you replied

something like

Code:

#!/bin/bash
for i in ListOfLineCommands.txt; do
    echo py27 exe.py
    $i
done

BUT...

what is meant by your "while read in out"? don't while loops generally look like

Code:

while read line

or

Code:

while [some sort of a $count]

can you tell i'm a total hack at this

michaelk · 10-20-2017, 02:22 PM

Not necessarily.

The while loop is reading the contents of a text file. done < "data-file"

The read command (read in out) is requesting two strings which is assigned to the variables in and out.

Shadow_7 · 10-20-2017, 02:23 PM

If you're lazy just extract the lines from .bash_history into a .sh file.

$ history | grep -i exe.py | tee myscript.sh
$ chmod +x myscript.sh
$ nano myscript.sh

Add a header

Code:

#!/bin/bash

Add a footer

Code:

exit 0

And delete any lines you didn't want to do / duplicates / false positives and whatnot.

atjurhs · 10-20-2017, 02:47 PM

michaelk, it worked, cool! and thank you!

i'll remember how to do this for later on

Todd