LinuxQuestions.org - [SOLVED] Help with bash/expect

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Help with bash/expect (https://www.linuxquestions.org/questions/programming-9/help-with-bash-expect-4175638790/)

Help with bash/expect

Hi Folks,

I have made a bash script which ssh's to a number of servers (IPs are read from a text file) and adds users with a specified username and userid. The user details are also read from a text file. So I am basically running nested for loops (once for Host IP and once for each line in user_list).

The script is working as expected, only letdown being the password prompts. For each IP the password is prompted twice (first time for ssh and second time for sudo).

I would very much like to use an expect script to automate the credentials part. I have no prior experience with expect and I could not find examples of nested loop or string manipulation (when read from a file) in expect.

Here's my working bash script [some details have been omitted]

Code:

#!/bin/bash



SSH_USER="username"



HOST_IPs=/path/to/ip_list



users=/path/to/user_list





for ip in $(cat $HOST_IPs)



do



        for i in $(cat $users)



        do



        #echo $i

        USERS=`echo $i|awk -F: '{print $1}'`

        USRID=`echo $i|awk -F: '{print $2}'`

#echo $USERS



        echo "adding $USERS with ID $USRID to $ip"



        ssh $SSH_USER@$ip  " sudo  /usr/sbin/useradd -M -u $USRID -c 'description' -G secondary_GRP $USERS ;sudo chage -M -1 $USERS "

###

                if [ $? -eq 0 ]; then

                        echo "User $USERS has been created successfully on $ip"

                else

                        echo "Please check the log or verify it manually."

                fi

        done

done

Could I get some pointers on --
A. How may I use nested for loops in expect ?
B. How may I use string manipulation from file ? [my user details are kept on a text file in USERNAME:USERID format. I would like to know if I may use something like awk -F: '{print $1/$2}' FILENAME]
C. Is there any way I can call an expect script to this position (###) and provide the passwords ?

Note:
1. I am not looking for a solution with sshpass /ssh keypair.
2. My environment is a CentOS 6.9 VM. Openssh -> 5.3p1-123.el6_9, bash -> 4.1.2-48.el6, expect -> 5.44.1.15-5.el6_4

expect is an extension of Tcl, you can use any of its built in commands as well as calling an external command like awk.

Tcl does have a split function.

You can use nested loops in Tcl similar to any other language.

I have not tried using sudo in an expect script. You can write your entire script in Tcl/expect, you can execute via expect -c "cmds" or use a heredoc as bash.

Some documentation.
https://www.tcl.tk/doc/
https://www.tcl.tk/man/expect5.31/expect.1.html

There are some mistakes in the script:

The for loops take a list; they are not good in reading lines. A space in the user_list can spoil everything.
A while-read loop is much better.

The exit status of ssh is not related to the exit status of the remotely run commands.

The following proposal also tries to combine the remote comands into one ssh command; would be more sufficient.

Code:

#!/bin/bash



SSH_USER="username"



HOST_IPs=host_ips



users=user_list



while read ip

do



    # construct the remote script

    while IFS=":" read USR USRID others

    do



        echo "

echo 'adding $USR with ID $USRID to $ip'

sudo /usr/sbin/useradd -M -u '$USRID' -c 'description' -G secondary_GRP '$USR' &&

sudo chage -M -1 '$USR' &&

echo 'User $USR has been created successfully on $ip' ||

echo 'Please check the log or verify it manually.'

"

###

    done < $users |

    # and feed the output into a remote /bin/sh

    ssh -x $SSH_USER@$ip /bin/sh

done < $HOST_IPs

You can give the remote script a better shape with

Code:

echo 'adding $USR with ID $USRID to $ip'

if

  sudo /usr/sbin/useradd -M -u '$USRID' -c 'description' -G secondary_GRP '$USR' &&

  sudo chage -M -1 '$USR'

then

  echo 'User $USR has been created successfully on $ip'

else

  echo 'Please check the log or verify it manually.'

fi

But be aware that this is all within an

Code:

echo "

"

on the calling system, so the $variables are substituted, but you may not use further " signs unless you \escape them.
--
Regarding the password setting,
you should use the chpasswd command, that reads it from stdin by default. Like this:

Code:

echo "user:password" | chpasswd

Thanks for your attention.

@MadeInGermany,

I actually do have a sed snippet to handle leading/trailing whitespaces {sed -e 's/^[ \t]*//' -e 's/[ \t]*$//'}. I could not post the whole script for workplace reasons, but I provided a bare-bone version to show my efforts.
I shall try out your recommendation with while loop. :)

@michaelk,

Thanks for the links, but documentation and manual pages were the first thing where I checked. Hope there were more examples for expect (much like nmcli-examples, wishful thinking :rolleyes:).

Here is a quick expect heredoc that you can run within bash that shows how to login via ssh and run multiple commands using sudo.

Code:

#!/bin/bash

host=myhost

user=myuser



expect  << EOF

spawn ssh $user@$host

expect "*password: "

send "myuser_password\r"

expect "$ "

send "sudo bash -c 'ls -l /var'; whoami\r"

expect "*$user:"

send "myuser_password\r"

expect "$ "

EOF

As one ssh command. The first password is to login via ssh and the second is the sudo password.

Code:

host=myhost

user=myuser



expect  << EOF

spawn ssh -t $user@$host "sudo bash -c 'ls -l /var'; whoami"

expect "*password: "

send "mypassword\r"

expect "*$user:"

send "mypassword\r"

expect "$ "

EOF

Do not indent the heredoc.

Thanks @michaelk, This was very helpful. I have added the EOF block with minor modifications to the ### section and it now works flawlessly (albeit with a time delay, so I guess I have to use the set timeout).

Few links for future visitors to this thread in hopes that they find the resources useful -

link 1
link 2
link 3
link 4

Stuck again on passing bash non-command directives via expect.
I am trying to send something similar to -

Code:

#! /bin/bash

SERVER=/test/ab/serverlist

USER=someuser



#login to each server -

for ip in $(cat $SERVER)

do

/usr/bin/expect <<iHERESTRING

set timeout 100

spawn ssh -o StrictHostKeyChecking=no -t $USER@$ip

expect "*password: "

send "Mypassword\r"

expect "$ "

# taking backups of files--

send "sudo cp -p /path/to/file1 /path/to/file1.`date +"%d-%m-%Y"`; sudo cp -p /path/to/file1 /path/to/file1.`date +"%d-%m-%Y"`"

expect "*$USER:"

send "Mypassword\r"

expect "$ "

#Generate userlist that may be removed

send " sudo cat /etc/sssd/sssd.conf | grep -i simple_allow_users | fmt -1| grep -Ev \"simple\|=\"|sed -e s/,//g>/home/someuser/user_list \r"

expect "$ "

#remove users

send "for j in \$(cat /home/someuser/user_list); do if ![[ \"\$j\" =~ ^(AAAAAAA|BBBBBBB|CCCCCCC) ]] && [[ \$j != n* ]] ; then echo "deleting \$j" ; fi; done;/r"

iHERESTRING

done

I have tried the highlighted part this way as well -

Code:

send "for j in \$(sudo grep -iP simple_allow_users /etc/sssd/sssd.conf | fmt -1 | grep -Ev "simple|="| sed -e s/,//g); do if ![[ "\$j" =~ ^(AAAAAAA|BBBBBBB|CCCCCCC) ]] && [[ \$j != n* ]] ; then echo "deleting \$j" ; fi; done;/r"

expect "$ "

The backup files are created successfully but I am getting the following error code - (similar for both attempts)

Code:

extra characters after close-quote

    while executing

"send " for j in $(cat /home/someuser/user_list); do if ![[ \"$j\" =~ ^

I had hoped that this was a syntactical error and I escaped double quotes using a backslash, but that doesn't help. (Either I am not not escaping enough characters, or the error is not known to me.)

Code:

send "for j in \$(cat /home/someuser/user_list); do if ![[ \"\$j\" =~ ^(AAAAAAA|BBBBBBB|CCCCCCC) ]] && [[ \$j != n* ]] ; then echo \"deleting \$j\" ; fi; done;/r"

May I get a pointer as to how I may use this for loop with expect ?

Environments -
A. 2.6.32-696.10.3.el6.x86_64/bash 4.1.2(1)/expect 5.44.1.15-5 (running from)
B. 3.10.0-862.11.6.el7.x86_64/bash 4.2.46(2) (target machine)

Please don't do this... there is newusers(8) for adding multiple users