[SOLVED] [Python]

jonnybinthemix · 02-08-2017, 08:37 AM

Hey Guys,

In the same vein as my last post, but taking consideration of the forum rules, I'm just going to ask a question.

Is there a module that you know of which can handle SSH connections and pass the information back into a variable within python?

I was playing around with using subversion to just use the ssh command, but I couldn't seem to figure out how to get the data back into a variable.

Code:

#!/usr/bin/python
#
#
import subversion

ssh = subprocess.Popen("ssh root@boot nodestatus | egrep -v "(up|standby1|standby2)" | awk '{ print $2 }'")
    stdout = subprocess.PIPE, stderr=subprocess.STDOUT

ssh,stderr = ssh.communicate()
status = ssh.poll()

print output

Just in testing the above does not work, it won't even print the output. I was thinking along the lines of.. if the above worked, could I just assume that 'output' is a variable in it's own right?

If that's the case, can I use python to manipulate the content? Like in shell scripts I would just use grep, awk and sed to manipulate the content of the returned data from the ssh session.

Any help is appreciated.

Thanks
Jon

CincinnatiKid · 02-08-2017, 08:42 PM

You are not setting the "output" variable to anything. Therefore your last "print output" line is doing nothing. I haven't tested this, but try this.

Code:

#!/usr/bin/python
#
#
import subversion

ssh = subprocess.Popen("ssh root@boot nodestatus | egrep -v "(up|standby1|standby2)" | awk '{ print $2 }'")
    stdout = subprocess.PIPE, stderr=subprocess.STDOUT

ssh,stderr = ssh.communicate()
status = ssh.poll()

output = ssh.stdout.read()

print output

CincinnatiKid · 02-08-2017, 08:45 PM

After looking at it a second time you want to "import subprocess" not "subversion"

Here is an example I wrote using echo instead of ssh since I don't even have ssh installed locally to test your code:

Code:

#!/usr/bin/python

import subprocess

ssh = subprocess.Popen(["echo", "test"], stdout = subprocess.PIPE, stderr=subprocess.STDOUT)

output, err = ssh.communicate()

print output

One thing to note is that you have to separate the arguments of your commands. For example in the above command "echo test" became "echo" "test"

jonnybinthemix · 02-09-2017, 02:43 AM

Ah great, thanks for your help.

Subversion was a typo. My mistake there

I'm going to test some things now and I will update shortly.

Thanks again for your help,

Jon

pan64 · 02-09-2017, 03:01 AM

probably this helps: http://stackoverflow.com/questions/2...esses-by-pipes

jonnybinthemix · 02-09-2017, 03:05 AM

I've got some kind of progress, I can get some response.. but not quite what I'm looking for. However progress is progress

Code:

#!/usr/bin/python

import subprocess

ssh = subprocess.Popen(["ssh", "root@boot", "xtprocadmin| grep -v up"], stderr=subprocess.STDOUT)
output, err = ssh.communicate()

print output

Gives a result..

Code:

   NID    (HEX)    NODENAME     TYPE    STATUS        MODE
   345    0x159  c1-0c2s6n1  compute admindown       batch
   385    0x181  c2-0c0s0n1  service      down       batch
   577    0x241  c3-0c0s0n1  service      down       batch
  1487    0x5cf  c7-0c2s3n3  compute admindown       batch
  1730    0x6c2  c9-0c0s0n2  compute      down       batch
  1731    0x6c3  c9-0c0s0n3  compute admindown       batch
  2497    0x9c1 c13-0c0s0n1  compute      down       batch
  2529    0x9e1 c13-0c0s8n1  compute admindown       batch
  2780    0xadc c14-0c1s7n0  compute admindown       batch
  3205    0xc85 c16-0c2s1n1  compute admindown       batch
  3207    0xc87 c16-0c2s1n3  compute admindown       batch
None

I'm getting "None" at the bottom, which I guess is the output from "print output" - But I thought "print output" would be the statement delivering the actual output, which is indeed displayed.

When I add some extra criteria to the grep command using egrep, things stop working:

Code:

#!/usr/bin/python

import subprocess

ssh = subprocess.Popen(["ssh", "root@boot", "xtprocadmin | egrep -v ("up|c2-0c0s0n1|c3-0c0s0n1")"], stderr=subprocess.STDOUT)
output, err = ssh.communicate()

print output

This give me:

Code:

  File "./down_nodes.py", line 5
    ssh = subprocess.Popen(["ssh", "root@boot", "xtprocadmin | egrep -v ("up|c2-0c0s0n1|c3-0c0s0n1")"], stderr=subprocess.STDOUT)
                                                                           ^
SyntaxError: invalid syntax

I've played with what feels like a million variations of "s, 's, and )s etc, and tried splitting the string up somewhat, but I can't seem to get it to work.

Inevitably I don't actually want the above table to display, I just want to extract column 3 out of it and store in a variable or something, so that I can check some log files against the node names which are down.

Any pointers in the right direction are greatly appreciated

Thanks
Jon

jonnybinthemix · 02-09-2017, 03:07 AM

Quote:

Originally Posted by pan64

probably this helps: http://stackoverflow.com/questions/2...esses-by-pipes

Looks like you posted this while I was typing my below comment. Thank you for this, I will read through it now and see if it helps me get closer to my goal.

Thanks again,
Jon

jonnybinthemix · 02-09-2017, 04:10 AM

Quote:

Originally Posted by pan64

probably this helps: http://stackoverflow.com/questions/2...esses-by-pipes

So, I'm picking up from this that maybe it would be better to use a couple of processes to achieve this?

P1 to run the first command, and P2 to process the output of the command and return the desired results.. or have I picked up the wrong vibe from what you're suggesting?

Maybe something like:

Code:

exclude = '("up|c2-0c0s0n1|c3-0c0s0n1")'
search = subprocess.Popen(["xtprocadmin", "egrep", "-v", "exclude"],
                          stdin=subprocess.PIPE,
                          stdout=subprocess.PIPE)

subprocess.Popen(["ssh", "root@boot"], stdout=search.stdin)
out, err = search.communicate()

I 'sort of' understand what's going on. But I wouldn't say it's overly clear.

Am I wandering the right lines?

Thanks,
Jon

EDIT:

It doesn't work. It's likely formatting and/or syntax. But is my thought process correct?

Code:

Traceback (most recent call last):
  File "./test.py", line 9, in <module>
    stdout=subprocess.PIPE)
  File "/usr/lib64/python2.6/subprocess.py", line 623, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.6/subprocess.py", line 1141, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

pan64 · 02-09-2017, 04:20 AM

yes, right direction, but you need to continue:

Code:

p1 = subprocess.Popen([something here, the first command],
                          stdout=subprocess.PIPE)

p2 = subprocess.Popen([something here, the second command],
                          stdin=p1.stdout,
                          stdout=subprocess.PIPE)

p3 = subprocess.Popen([something here, the third command],
                          stdin=p2.stdout,
                          stdout=subprocess.PIPE)
out, err = p3.communicate()

and actually in your first Popen you need not use ":

search = subprocess.Popen(["xtprocadmin", "egrep", "-v", "exclude"],

because exclude is a variable here, not a string

jonnybinthemix · 02-09-2017, 04:39 AM

Quote:

Originally Posted by pan64

yes, right direction, but you need to continue:

Code:

p1 = subprocess.Popen([something here, the first command],
                          stdout=subprocess.PIPE)

p2 = subprocess.Popen([something here, the second command],
                          stdin=p1.stdout,
                          stdout=subprocess.PIPE)

p3 = subprocess.Popen([something here, the third command],
                          stdin=p2.stdout,
                          stdout=subprocess.PIPE)
out, err = p3.communicate()

and actually in your first Popen you need not use ":

search = subprocess.Popen(["xtprocadmin", "egrep", "-v", "exclude"],

because exclude is a variable here, not a string

I actually did omit the "s when I tested it as I noticed it also.

But, forgive me as I'm not sure I fully understand the process to follow.

You say I should carry on, but which segments of the command I want to run should be split into their own processes?

If I want to run:

Code:

ssh root@boot 'xtprocadmin | egrep -v ("NID|up|c2-0c0s0n1|c3-0c0s0n1")'

Would it look like:

Code:

exclude = '("NID|up|c2-0c0s0n1|c3-0c0s0n1")'

p1 = subprocess.Popen(["ssh root@boot"],
                          stdout=subprocess.PIPE)

p2 = subprocess.Popen([xtprocadmin],
                          stdin=p1.stdout,
                          stdout=subprocess.PIPE)

p3 = subprocess.Popen(["egrep -v", exclude],
                          stdin=p2.stdout,
                          stdout=subprocess.PIPE)
out, err = p3.communicate()

Or, would the commands be reversed?

egrep -- xtprocadmin -- ssh

Or

ssh -- xtprocadmin --egrep

I apologise if I'm slow to pick up the basics here, but I'm trying to make sure I fully understand things. I'd much rather learn how it works, than just get it working if you know what I mean

Thanks
Jon

jonnybinthemix · 02-09-2017, 05:01 AM

I have this:

Code:

#!/usr/bin/python
#
import subprocess

exclude = '("NID|up|c2-0c0s0n1|c3-0c0s0n1")'

p1 = subprocess.Popen(["ssh", "root@boot"],
                        stdout=subprocess.PIPE)

p2 = subprocess.Popen(["xtprocadmin"],
                        stdin=p1.stdout,
                        stdout=subprocess.PIPE)

p2 = subprocess.Popen(["egrep", "-v", exclude),
                        stdin=p2.stdout,
                        stdout=subprocress.PIPE)

out, err = p3.communicate()

It unfortunately does nothing more than:

Quote:

File "./test.py, Line 14
p2 = subprocess.Popen(["egrep", "-v", exclude),
^
SyntaxError: invalid syntax

I'm not too concerned that it doesn't work, as I'm sure I've got things wrong. It'll work when I understand it enough to make it work :-)

In the meantime I'll read more online, but any advice is always welcome

Thanks
Jon

EDIT: I had 2 p2's instead of p3!

Fixed that:

However, it now doesn't like p3:

Quote:

File "./test.py", line 14
p3 subprocess.Popen(["egrep", "-v", exclude),
^
SyntaxError: invalid syntax

jonnybinthemix · 02-09-2017, 05:06 AM

To keep things clean and clear, this is exactly what I have so far:

Code:

#!/usr/bin/python
#
import subprocess

exclude = '("NID|up|c2-0c0s0n1|c3-0c0s0n1")'

p1 = subprocess.Popen(["ssh", "root@boot"],
                        stdout=subprocess.PIPE)

p2 = subprocess.Popen(["xtprocadmin"],
                        stdin=p1.stdout,
                        stdout=subprocess.PIPE)

p3 =  subprocess.Popen(["egrep", "-v", exclude),
                        stdin=p2.stdout,
                        stdout=subprocress.PIPE)

out, err = p3.communicate()

And the output:

Quote:

File "./test.py", line 14
p3 = subprocess.Popen(["egrep", "-v", exclude),
^
SyntaxError: invalid syntax

EDIT: Fixed the rogue space:

Output:

Quote:

File "./test.py", line 14
p3 = subprocess.Popen(["egrep", "-v", exclude),
^
SyntaxError: invalid syntax

I don't know if the extra space before p3 made a difference, but I got rid of it anyway.

EDIT AGAIN: The ^ is actually under the variable name in the output. But pasting into here moves it.

pan64 · 02-09-2017, 05:46 AM

you missed the closing ]

jonnybinthemix · 02-09-2017, 05:59 AM

Quote:

Originally Posted by pan64

you missed the closing ]

Ah yes, I used a ) instead of a ] - Good spot

Fixed that, and how getting this:

Code:

Traceback (most recent call last):
  File "./test.py", line 12, in <module>
    stdout=subprocess.PIPE)
  File "/usr/lib64/python2.6/subprocess.py", line 623, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.6/subprocess.py", line 1141, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

pan64 · 02-09-2017, 06:25 AM

sorry guy, but without seeing your code I cannot say anything. It is in test.py, line 12, or somewhere near...
probably you missed a ) this time, please check your code carefully. You can try something like this: http://infoheap.com/python-lint-online/ to check your code