[SOLVED] Help with bash/expect

Honest Abe · 09-20-2018, 05:48 AM

Hi Folks,

I have made a bash script which ssh's to a number of servers (IPs are read from a text file) and adds users with a specified username and userid. The user details are also read from a text file. So I am basically running nested for loops (once for Host IP and once for each line in user_list).

The script is working as expected, only letdown being the password prompts. For each IP the password is prompted twice (first time for ssh and second time for sudo).

I would very much like to use an expect script to automate the credentials part. I have no prior experience with expect and I could not find examples of nested loop or string manipulation (when read from a file) in expect.

Here's my working bash script [some details have been omitted]

Code:

#!/bin/bash

SSH_USER="username"

HOST_IPs=/path/to/ip_list

users=/path/to/user_list


for ip in $(cat $HOST_IPs)

do

        for i in $(cat $users)

        do

        #echo $i
        USERS=`echo $i|awk -F: '{print $1}'`
        USRID=`echo $i|awk -F: '{print $2}'`
#echo $USERS

        echo "adding $USERS with ID $USRID to $ip"

        ssh $SSH_USER@$ip  " sudo  /usr/sbin/useradd -M -u $USRID -c 'description' -G secondary_GRP $USERS ;sudo chage -M -1 $USERS "
###
                if [ $? -eq 0 ]; then
                        echo "User $USERS has been created successfully on $ip"
                else
                        echo "Please check the log or verify it manually."
                fi
        done
done

Could I get some pointers on --
A. How may I use nested for loops in expect ?
B. How may I use string manipulation from file ? [my user details are kept on a text file in USERNAME:USERID format. I would like to know if I may use something like awk -F: '{print $1/$2}' FILENAME]
C. Is there any way I can call an expect script to this position (###) and provide the passwords ?

Note:
1. I am not looking for a solution with sshpass /ssh keypair.
2. My environment is a CentOS 6.9 VM. Openssh -> 5.3p1-123.el6_9, bash -> 4.1.2-48.el6, expect -> 5.44.1.15-5.el6_4

michaelk · 09-20-2018, 12:26 PM

expect is an extension of Tcl, you can use any of its built in commands as well as calling an external command like awk.

Tcl does have a split function.

You can use nested loops in Tcl similar to any other language.

I have not tried using sudo in an expect script. You can write your entire script in Tcl/expect, you can execute via expect -c "cmds" or use a heredoc as bash.

Some documentation.
https://www.tcl.tk/doc/
https://www.tcl.tk/man/expect5.31/expect.1.html

MadeInGermany · 09-20-2018, 02:32 PM

There are some mistakes in the script:

The for loops take a list; they are not good in reading lines. A space in the user_list can spoil everything.
A while-read loop is much better.

The exit status of ssh is not related to the exit status of the remotely run commands.

The following proposal also tries to combine the remote comands into one ssh command; would be more sufficient.

Code:

#!/bin/bash

SSH_USER="username"

HOST_IPs=host_ips

users=user_list

while read ip
do

    # construct the remote script
    while IFS=":" read USR USRID others
    do

        echo "
echo 'adding $USR with ID $USRID to $ip'
sudo /usr/sbin/useradd -M -u '$USRID' -c 'description' -G secondary_GRP '$USR' &&
sudo chage -M -1 '$USR' &&
echo 'User $USR has been created successfully on $ip' ||
echo 'Please check the log or verify it manually.'
"
###
    done < $users |
    # and feed the output into a remote /bin/sh
    ssh -x $SSH_USER@$ip /bin/sh
done < $HOST_IPs

You can give the remote script a better shape with

Code:

echo 'adding $USR with ID $USRID to $ip'
if
  sudo /usr/sbin/useradd -M -u '$USRID' -c 'description' -G secondary_GRP '$USR' &&
  sudo chage -M -1 '$USR'
then
  echo 'User $USR has been created successfully on $ip'
else
  echo 'Please check the log or verify it manually.'
fi

But be aware that this is all within an

Code:

echo "
"

on the calling system, so the $variables are substituted, but you may not use further " signs unless you \escape them.
--
Regarding the password setting,
you should use the chpasswd command, that reads it from stdin by default. Like this:

Code:

echo "user:password" | chpasswd

Honest Abe · 09-20-2018, 11:57 PM

Thanks for your attention.

@MadeInGermany,

I actually do have a sed snippet to handle leading/trailing whitespaces {sed -e 's/^[ \t]*//' -e 's/[ \t]*$//'}. I could not post the whole script for workplace reasons, but I provided a bare-bone version to show my efforts.
I shall try out your recommendation with while loop.

@michaelk,

Thanks for the links, but documentation and manual pages were the first thing where I checked. Hope there were more examples for expect (much like nmcli-examples, wishful thinking

).

michaelk · 09-21-2018, 03:35 PM

Here is a quick expect heredoc that you can run within bash that shows how to login via ssh and run multiple commands using sudo.

Code:

#!/bin/bash
host=myhost
user=myuser

expect  << EOF
spawn ssh $user@$host
expect "*password: "
send "myuser_password\r"
expect "$ "
send "sudo bash -c 'ls -l /var'; whoami\r"
expect "*$user:"
send "myuser_password\r"
expect "$ "
EOF

As one ssh command. The first password is to login via ssh and the second is the sudo password.

Code:

host=myhost
user=myuser

expect  << EOF
spawn ssh -t $user@$host "sudo bash -c 'ls -l /var'; whoami"
expect "*password: "
send "mypassword\r"
expect "*$user:"
send "mypassword\r"
expect "$ "
EOF

Do not indent the heredoc.

Honest Abe · 09-22-2018, 05:09 AM

Thanks @michaelk, This was very helpful. I have added the EOF block with minor modifications to the ### section and it now works flawlessly (albeit with a time delay, so I guess I have to use the set timeout).

Few links for future visitors to this thread in hopes that they find the resources useful -

link 1
link 2
link 3
link 4

Honest Abe · 10-01-2018, 06:23 AM

Stuck again on passing bash non-command directives via expect.
I am trying to send something similar to -

Code:

#! /bin/bash
SERVER=/test/ab/serverlist
USER=someuser

#login to each server -
for ip in $(cat $SERVER)
do
/usr/bin/expect <<iHERESTRING
set timeout 100
spawn ssh -o StrictHostKeyChecking=no -t $USER@$ip
expect "*password: "
send "Mypassword\r"
expect "$ "
# taking backups of files--
send "sudo cp -p /path/to/file1 /path/to/file1.`date +"%d-%m-%Y"`; sudo cp -p /path/to/file1 /path/to/file1.`date +"%d-%m-%Y"`"
expect "*$USER:"
send "Mypassword\r"
expect "$ "
#Generate userlist that may be removed
send " sudo cat /etc/sssd/sssd.conf | grep -i simple_allow_users | fmt -1| grep -Ev \"simple\|=\"|sed -e s/,//g>/home/someuser/user_list \r"
expect "$ "
#remove users
send "for j in \$(cat /home/someuser/user_list); do if ![[ \"\$j\" =~ ^(AAAAAAA|BBBBBBB|CCCCCCC) ]] && [[ \$j != n* ]] ; then echo "deleting \$j" ; fi; done;/r"
iHERESTRING
done

I have tried the highlighted part this way as well -

Code:

send "for j in \$(sudo grep -iP simple_allow_users /etc/sssd/sssd.conf | fmt -1 | grep -Ev "simple|="| sed -e s/,//g); do if ![[ "\$j" =~ ^(AAAAAAA|BBBBBBB|CCCCCCC) ]] && [[ \$j != n* ]] ; then echo "deleting \$j" ; fi; done;/r"
expect "$ "

The backup files are created successfully but I am getting the following error code - (similar for both attempts)

Code:

extra characters after close-quote
    while executing
"send " for j in $(cat /home/someuser/user_list); do if ![[ \"$j\" =~ ^

I had hoped that this was a syntactical error and I escaped double quotes using a backslash, but that doesn't help. (Either I am not not escaping enough characters, or the error is not known to me.)

Code:

send "for j in \$(cat /home/someuser/user_list); do if ![[ \"\$j\" =~ ^(AAAAAAA|BBBBBBB|CCCCCCC) ]] && [[ \$j != n* ]] ; then echo \"deleting \$j\" ; fi; done;/r"

May I get a pointer as to how I may use this for loop with expect ?

Environments -
A. 2.6.32-696.10.3.el6.x86_64/bash 4.1.2(1)/expect 5.44.1.15-5 (running from)
B. 3.10.0-862.11.6.el7.x86_64/bash 4.2.46(2) (target machine)

NevemTeve · 10-05-2018, 02:33 AM

Please don't do this... there is newusers(8) for adding multiple users