LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 02-08-2017, 08:37 AM   #1
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Rep: Reputation: Disabled
[Python] - SSH Module?


Hey Guys,

In the same vein as my last post, but taking consideration of the forum rules, I'm just going to ask a question.

Is there a module that you know of which can handle SSH connections and pass the information back into a variable within python?

I was playing around with using subversion to just use the ssh command, but I couldn't seem to figure out how to get the data back into a variable.

Code:
#!/usr/bin/python
#
#
import subversion

ssh = subprocess.Popen("ssh root@boot nodestatus | egrep -v "(up|standby1|standby2)" | awk '{ print $2 }'")
    stdout = subprocess.PIPE, stderr=subprocess.STDOUT

ssh,stderr = ssh.communicate()
status = ssh.poll()

print output
Just in testing the above does not work, it won't even print the output. I was thinking along the lines of.. if the above worked, could I just assume that 'output' is a variable in it's own right?

If that's the case, can I use python to manipulate the content? Like in shell scripts I would just use grep, awk and sed to manipulate the content of the returned data from the ssh session.

Any help is appreciated.

Thanks
Jon
 
Old 02-08-2017, 08:42 PM   #2
CincinnatiKid
Member
 
Registered: Jul 2010
Posts: 454

Rep: Reputation: 47
You are not setting the "output" variable to anything. Therefore your last "print output" line is doing nothing. I haven't tested this, but try this.

Code:
#!/usr/bin/python
#
#
import subversion

ssh = subprocess.Popen("ssh root@boot nodestatus | egrep -v "(up|standby1|standby2)" | awk '{ print $2 }'")
    stdout = subprocess.PIPE, stderr=subprocess.STDOUT

ssh,stderr = ssh.communicate()
status = ssh.poll()

output = ssh.stdout.read()

print output
 
Old 02-08-2017, 08:45 PM   #3
CincinnatiKid
Member
 
Registered: Jul 2010
Posts: 454

Rep: Reputation: 47
After looking at it a second time you want to "import subprocess" not "subversion"

Here is an example I wrote using echo instead of ssh since I don't even have ssh installed locally to test your code:

Code:
#!/usr/bin/python

import subprocess

ssh = subprocess.Popen(["echo", "test"], stdout = subprocess.PIPE, stderr=subprocess.STDOUT)

output, err = ssh.communicate()

print output
One thing to note is that you have to separate the arguments of your commands. For example in the above command "echo test" became "echo" "test"

Last edited by CincinnatiKid; 02-08-2017 at 08:53 PM.
 
Old 02-09-2017, 02:43 AM   #4
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Ah great, thanks for your help.

Subversion was a typo. My mistake there

I'm going to test some things now and I will update shortly.

Thanks again for your help,

Jon
 
Old 02-09-2017, 03:01 AM   #5
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,236

Rep: Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452
probably this helps: http://stackoverflow.com/questions/2...esses-by-pipes
 
Old 02-09-2017, 03:05 AM   #6
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
I've got some kind of progress, I can get some response.. but not quite what I'm looking for. However progress is progress

Code:
#!/usr/bin/python

import subprocess

ssh = subprocess.Popen(["ssh", "root@boot", "xtprocadmin| grep -v up"], stderr=subprocess.STDOUT)
output, err = ssh.communicate()

print output
Gives a result..
Code:
   NID    (HEX)    NODENAME     TYPE    STATUS        MODE
   345    0x159  c1-0c2s6n1  compute admindown       batch
   385    0x181  c2-0c0s0n1  service      down       batch
   577    0x241  c3-0c0s0n1  service      down       batch
  1487    0x5cf  c7-0c2s3n3  compute admindown       batch
  1730    0x6c2  c9-0c0s0n2  compute      down       batch
  1731    0x6c3  c9-0c0s0n3  compute admindown       batch
  2497    0x9c1 c13-0c0s0n1  compute      down       batch
  2529    0x9e1 c13-0c0s8n1  compute admindown       batch
  2780    0xadc c14-0c1s7n0  compute admindown       batch
  3205    0xc85 c16-0c2s1n1  compute admindown       batch
  3207    0xc87 c16-0c2s1n3  compute admindown       batch
None
I'm getting "None" at the bottom, which I guess is the output from "print output" - But I thought "print output" would be the statement delivering the actual output, which is indeed displayed.

When I add some extra criteria to the grep command using egrep, things stop working:

Code:
#!/usr/bin/python

import subprocess

ssh = subprocess.Popen(["ssh", "root@boot", "xtprocadmin | egrep -v ("up|c2-0c0s0n1|c3-0c0s0n1")"], stderr=subprocess.STDOUT)
output, err = ssh.communicate()

print output
This give me:

Code:
  File "./down_nodes.py", line 5
    ssh = subprocess.Popen(["ssh", "root@boot", "xtprocadmin | egrep -v ("up|c2-0c0s0n1|c3-0c0s0n1")"], stderr=subprocess.STDOUT)
                                                                           ^
SyntaxError: invalid syntax
I've played with what feels like a million variations of "s, 's, and )s etc, and tried splitting the string up somewhat, but I can't seem to get it to work.

Inevitably I don't actually want the above table to display, I just want to extract column 3 out of it and store in a variable or something, so that I can check some log files against the node names which are down.

Any pointers in the right direction are greatly appreciated

Thanks
Jon
 
Old 02-09-2017, 03:07 AM   #7
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
Looks like you posted this while I was typing my below comment. Thank you for this, I will read through it now and see if it helps me get closer to my goal.

Thanks again,
Jon
 
Old 02-09-2017, 04:10 AM   #8
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
So, I'm picking up from this that maybe it would be better to use a couple of processes to achieve this?

P1 to run the first command, and P2 to process the output of the command and return the desired results.. or have I picked up the wrong vibe from what you're suggesting?

Maybe something like:

Code:
exclude = '("up|c2-0c0s0n1|c3-0c0s0n1")'
search = subprocess.Popen(["xtprocadmin", "egrep", "-v", "exclude"],
                          stdin=subprocess.PIPE,
                          stdout=subprocess.PIPE)

subprocess.Popen(["ssh", "root@boot"], stdout=search.stdin)
out, err = search.communicate()
I 'sort of' understand what's going on. But I wouldn't say it's overly clear.

Am I wandering the right lines?

Thanks,
Jon

EDIT:

It doesn't work. It's likely formatting and/or syntax. But is my thought process correct?

Code:
Traceback (most recent call last):
  File "./test.py", line 9, in <module>
    stdout=subprocess.PIPE)
  File "/usr/lib64/python2.6/subprocess.py", line 623, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.6/subprocess.py", line 1141, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Last edited by jonnybinthemix; 02-09-2017 at 04:19 AM.
 
Old 02-09-2017, 04:20 AM   #9
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,236

Rep: Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452
yes, right direction, but you need to continue:
Code:
p1 = subprocess.Popen([something here, the first command],
                          stdout=subprocess.PIPE)

p2 = subprocess.Popen([something here, the second command],
                          stdin=p1.stdout,
                          stdout=subprocess.PIPE)

p3 = subprocess.Popen([something here, the third command],
                          stdin=p2.stdout,
                          stdout=subprocess.PIPE)
out, err = p3.communicate()
and actually in your first Popen you need not use ":

search = subprocess.Popen(["xtprocadmin", "egrep", "-v", "exclude"],

because exclude is a variable here, not a string
 
Old 02-09-2017, 04:39 AM   #10
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
yes, right direction, but you need to continue:
Code:
p1 = subprocess.Popen([something here, the first command],
                          stdout=subprocess.PIPE)

p2 = subprocess.Popen([something here, the second command],
                          stdin=p1.stdout,
                          stdout=subprocess.PIPE)

p3 = subprocess.Popen([something here, the third command],
                          stdin=p2.stdout,
                          stdout=subprocess.PIPE)
out, err = p3.communicate()
and actually in your first Popen you need not use ":

search = subprocess.Popen(["xtprocadmin", "egrep", "-v", "exclude"],

because exclude is a variable here, not a string
I actually did omit the "s when I tested it as I noticed it also.

But, forgive me as I'm not sure I fully understand the process to follow.

You say I should carry on, but which segments of the command I want to run should be split into their own processes?

If I want to run:

Code:
ssh root@boot 'xtprocadmin | egrep -v ("NID|up|c2-0c0s0n1|c3-0c0s0n1")'
Would it look like:
Code:
exclude = '("NID|up|c2-0c0s0n1|c3-0c0s0n1")'

p1 = subprocess.Popen(["ssh root@boot"],
                          stdout=subprocess.PIPE)

p2 = subprocess.Popen([xtprocadmin],
                          stdin=p1.stdout,
                          stdout=subprocess.PIPE)

p3 = subprocess.Popen(["egrep -v", exclude],
                          stdin=p2.stdout,
                          stdout=subprocess.PIPE)
out, err = p3.communicate()
Or, would the commands be reversed?

egrep -- xtprocadmin -- ssh

Or

ssh -- xtprocadmin --egrep

I apologise if I'm slow to pick up the basics here, but I'm trying to make sure I fully understand things. I'd much rather learn how it works, than just get it working if you know what I mean

Thanks
Jon

Last edited by jonnybinthemix; 02-09-2017 at 04:52 AM.
 
Old 02-09-2017, 05:01 AM   #11
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
I have this:

Code:
#!/usr/bin/python
#
import subprocess

exclude = '("NID|up|c2-0c0s0n1|c3-0c0s0n1")'

p1 = subprocess.Popen(["ssh", "root@boot"],
                        stdout=subprocess.PIPE)

p2 = subprocess.Popen(["xtprocadmin"],
                        stdin=p1.stdout,
                        stdout=subprocess.PIPE)

p2 = subprocess.Popen(["egrep", "-v", exclude),
                        stdin=p2.stdout,
                        stdout=subprocress.PIPE)

out, err = p3.communicate()
It unfortunately does nothing more than:

Quote:
File "./test.py, Line 14
p2 = subprocess.Popen(["egrep", "-v", exclude),
^
SyntaxError: invalid syntax
I'm not too concerned that it doesn't work, as I'm sure I've got things wrong. It'll work when I understand it enough to make it work :-)

In the meantime I'll read more online, but any advice is always welcome

Thanks
Jon

EDIT: I had 2 p2's instead of p3!

Fixed that:

However, it now doesn't like p3:

Quote:
File "./test.py", line 14
p3 subprocess.Popen(["egrep", "-v", exclude),
^
SyntaxError: invalid syntax

Last edited by jonnybinthemix; 02-09-2017 at 05:04 AM.
 
Old 02-09-2017, 05:06 AM   #12
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
To keep things clean and clear, this is exactly what I have so far:

Code:
#!/usr/bin/python
#
import subprocess

exclude = '("NID|up|c2-0c0s0n1|c3-0c0s0n1")'

p1 = subprocess.Popen(["ssh", "root@boot"],
                        stdout=subprocess.PIPE)

p2 = subprocess.Popen(["xtprocadmin"],
                        stdin=p1.stdout,
                        stdout=subprocess.PIPE)

p3 =  subprocess.Popen(["egrep", "-v", exclude),
                        stdin=p2.stdout,
                        stdout=subprocress.PIPE)

out, err = p3.communicate()
And the output:

Quote:
File "./test.py", line 14
p3 = subprocess.Popen(["egrep", "-v", exclude),
^
SyntaxError: invalid syntax
EDIT: Fixed the rogue space:

Output:
Quote:
File "./test.py", line 14
p3 = subprocess.Popen(["egrep", "-v", exclude),
^
SyntaxError: invalid syntax
I don't know if the extra space before p3 made a difference, but I got rid of it anyway.

EDIT AGAIN: The ^ is actually under the variable name in the output. But pasting into here moves it.

Last edited by jonnybinthemix; 02-09-2017 at 05:08 AM.
 
Old 02-09-2017, 05:46 AM   #13
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,236

Rep: Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452
you missed the closing ]
 
Old 02-09-2017, 05:59 AM   #14
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
you missed the closing ]
Ah yes, I used a ) instead of a ] - Good spot

Fixed that, and how getting this:

Code:
Traceback (most recent call last):
  File "./test.py", line 12, in <module>
    stdout=subprocess.PIPE)
  File "/usr/lib64/python2.6/subprocess.py", line 623, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.6/subprocess.py", line 1141, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
 
Old 02-09-2017, 06:25 AM   #15
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,236

Rep: Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452
sorry guy, but without seeing your code I cannot say anything. It is in test.py, line 12, or somewhere near...
probably you missed a ) this time, please check your code carefully. You can try something like this: http://infoheap.com/python-lint-online/ to check your code
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
no python module base.g? hcgernhardt Linux - Software 2 04-24-2018 09:03 AM
[SOLVED] Using python GNUPG module over SSH guardian1 Linux - Server 2 08-20-2012 08:53 AM
configure: error: The xdg python module is required (pyxdg or python-xdg) Sargalus Linux - Software 7 03-24-2010 07:34 AM
Python: How to use the re module? donnied Programming 2 01-19-2009 12:26 PM
python update - Unable to load GTK2 Python bindings: No module named gtk itpedersen Linux - Software 2 10-03-2008 03:44 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 11:03 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration