for loop with 2 variables in list

Entropy1024 · 02-08-2018, 07:28 AM

I need to write a script to scp files from a few different remote servers as a backup.

ie if I have 3 machines at 1.1.1.1 1.1.1.4 & 1.1.1.7 and they all have the same port number I could use:

Code:

for ADDRESS in 1.1.1.1; 1.1.1.4; 1.1.1.7
do
 scp -P 22 me@$ADDRESS /blah...
done

However each machine has a different port number, say 3,7 & 10. Is there a way to have a second set of variables for the port number?

Code:

for ADDRESS in 1.1.1.1; 1.1.1.4; 1.1.1.7 PORT in 3; 7; 10
do
 scp -P $PORT me@$ADDRESS /blah...
done

Many thanks for any help

michaelk · 02-08-2018, 07:53 AM

While there are several ways to do it in a loop I would use a local ssh config file i.e ~/.ssh/config.

Code:

host server1
     hostname 1.1.1.1
     user username
     port 3
host server2
    ...
host server3
    ...

Code:

for name in server1; server2; server3
do
 scp $name:/blah  blah
done

keefaz · 02-08-2018, 07:54 AM

You could use bash associative array

Code:

declare -A hosts

hosts["1.1.1.1"]=3
hosts["1.1.1.4"]=7
hosts["1.1.1.7"]=10

for ADDRESS in "${!hosts[@]}"
do
  scp -P "${hosts[$ADDRESS]}" me@$ADDRESS /blah...
done

TenTenths · 02-08-2018, 09:21 AM

Could also use something like this:

Code:

#!/bin/bash

for hostport in 1.1.1.1:3 1.1.1.4:7 1.1.1.7:10 ; do

  host=$(echo ${hostport} | awk -F: {'print $1'})
  port=$(echo ${hostport} | awk -F: {'print $2'})

  echo Host ${host}
  echo Port ${port}

done

Personally I like the solution keefaz proposes though!

Entropy1024 · 02-08-2018, 10:01 AM

Quote:

Originally Posted by michaelk

While there are several ways to do it in a loop I would use a local ssh config file i.e ~/.ssh/config.

Code:

host server1
     hostname 1.1.1.1
     user username
     port 3
host server2
    ...
host server3
    ...

Code:

for name in server1; server2; server3
do
 scp $name:/blah  blah
done

Many thanks, that worked a treat.

Thanks for all the other solutions also. Very interesting.

TenTenths · 02-08-2018, 10:11 AM

Quote:

Originally Posted by Entropy1024

Many thanks, that worked a treat.

Thanks for all the other solutions also. Very interesting.

My issue with michaelk's solution is that it's "hidden" from anyone else that goes back to the script and has to try and work out WHY the connections are being made on non-standard ports, or to add another server on a non-standard port. Of course you COULD put in the script notes that it uses the ssh/config file.

The other solutions are more readable and make it easier for anyone looking at the script (and yourself in a couple years time!) to work out how it's being done. They are also "transportable" to other servers without having to remember to alter your ssh config file.

michaelk · 02-08-2018, 10:19 AM

True and as stated you could add comments. The other benefit of using a config file is that provides options defined per host. If you have several servers with many/different options you do not have to remember them nor have to type them on the command line. It works for ssh, scp, sftp and rsync.

scasey · 02-08-2018, 02:12 PM

I prefer to use the ~/.ssh/config file myself, but to TenTenths point, every time I need to make a change or configure a new server, I have to remember that's how it works.

It can also provide different options for different users, just to be more confusing.

MadeInGermany · 02-09-2018, 08:35 AM

A while read loop can break the input line into words

Code:

while read addr port <&3
do
  scp -P "$port" me@"$addr" /blah...
done 3<<EOT
1.1.1.1 3
1.1.1.4 7
1.1.1.7 10
EOT

The data is in a classic here document.
bash can take a here string, that maybe has a smaller footprint (no temp file):

Code:

#!/bin/bash
while read addr port <&3
do
  scp -P "$port" me@"$addr" /blah...
done 3<<< "\
1.1.1.1 3
1.1.1.4 7
1.1.1.7 10"

The extra file descriptor 3 makes the while loop robust against commands that read from stdin, like ssh (without -n). Certainly not needed here.
A custom input field separator can be set and/or a further field can be added, as follows:

Code:

while IFS=":" read addr port user <&3
do
  scp -P "$port" "$user@$addr" /blah...
done 3<<EOT
1.1.1.1:3:me
1.1.1.4:7:me
1.1.1.7:10:you
EOT