GNU parallel aws cli

aidylewis · 07-17-2018, 06:38 AM

Hello,

I have a bash script and I just need to parallelise an aws cli command with X number of jobs.

$ seq 10 | parallel aws sqs send-message --queue-url "<queue-url>" --message-body "aws_sqs"

But I am being told my aws command is incorrect. Runs OK without parallel

Many Thanks

Aidy

scasey · 07-18-2018, 12:01 AM

From man parallel

Code:

Synopsis

parallel [options] [command] -- [argument ...]
parallel [options] -- [command ...]

so there needs to be two hyphens (--) between either the command and its arguments or between parallel and the command.

Review the man page to figure out which applies. My guess would be the first one.

Code:

$ seq 10 | parallel aws -- sqs send-message --queue-url "<queue-url>" --message-body "aws_sqs"

aidylewis · 07-18-2018, 07:36 AM

Both -- infixes, give aws unknown options. Does I need to operate on a list?

scasey · 07-18-2018, 08:11 AM

Here's a link to the man page

Maybe -n option...I don't know.