LinuxQuestions.org - Simultaneously execute 8 instances of a program until all input files are processed?

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - Simultaneously execute 8 instances of a program until all input files are processed? (https://www.linuxquestions.org/questions/linux-newbie-8/simultaneously-execute-8-instances-of-a-program-until-all-input-files-are-processed-834818/)

Simultaneously execute 8 instances of a program until all input files are processed?

Hi all,

I have a machine with 8 cores. I have a directory of input files that I need to process using a non-parallelized program. I would like to write a script that works its way through a list of commands executing 8 commans (i.e., instances of this program; each with specific flags having to do with the current input file) until the list of commands has been exhausted. Is there an easy way to do this?

Code:

hamstrsearch_local-hmmer3.pl -protein -refspec=lotgi_2713 -hmmset=lophotrochozoa_hmmer3 -sequence_file=Aplysia_californica.fas -taxon=ACAL

Thanks!
Kevin

I removed my reply because it was not useful for the OP.

Hopefully that little rant makes SharpyWarpy feel better ...

Now to the matter at hand.
And yes the requirements are a little vague. But assuming you mean 8 instances per input file, something as basic as this should suffice

Code:

#! /bin/sh

for file in /some/directory/*

  do

    program -f flags1 ${file} &

    program -f flags2 ${file} &

    program -f flags3 ${file} &

    program -f flags4 ${file} &

    program -f flags5 ${file} &

    program -f flags6 ${file} &

    program -f flags7 ${file} &

    program -f flags8 ${file} &

  done

That will (immediately) spawn 8 per infile, so you could wind up with quite a few. Niceties might be to add a sleep after number 8, or count the instances and limit the number of concurrent instances, ...

Thanks! This is extremely helpful!

(and in retrospect I realize that my question wasn't entirely clear but I think you got my meaning)

Kevin