LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 02-09-2017, 06:49 AM   #16
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled

Quote:
Originally Posted by pan64 View Post
sorry guy, but without seeing your code I cannot say anything. It is in test.py, line 12, or somewhere near...
probably you missed a ) this time, please check your code carefully. You can try something like this: http://infoheap.com/python-lint-online/ to check your code
Sorry, my code is here:

Code:
#!/usr/bin/python
#
import subprocess

exclude = '("NID|up|c2-0c0s0n1|c3-0c0s0n1")'

p1 = subprocess.Popen(["ssh", "root@boot"],
                        stdout=subprocess.PIPE)

p2 = subprocess.Popen(["xtprocadmin"],
                        stdin=p1.stdout,
                        stdout=subprocess.PIPE)

p3 = subprocess.Popen(["egrep", "-v", exclude],
                        stdin=p2.stdout,
                        stdout=subprocress.PIPE)

out, err = p3.communicate()
I ran it through the checker you mentioned and it passed. Maybe I've made different mistake?

Jon
 
Old 02-09-2017, 07:13 AM   #17
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,236

Rep: Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452
so probably exclude is incorrect. try first a simple expression, like "c2-0c0s0n1" and later try to improve that.
probably:
Code:
exclude = '"(NID|up|c2-0c0s0n1|c3-0c0s0n1)"'
or something similar will work
 
Old 02-09-2017, 07:39 AM   #18
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
I think I may be a step closer to an answer.

I tested playing around with the exclude variable, and ended up trying just "up" instead of a string of excludes.

I looked around a little and I see this comment here: http://stackoverflow.com/questions/1...ll-from-python

Quote:
A slight modification does the trick:

def test_ssh():
from subprocess import call
call('ssh user@1.2.3.4 -i "bla.pem"', shell=True)

As the answer to this question suggests the error suggests the error comes from the inability of ssh to connect to the stdin/out of a process in the background.

With call the fabric task does not end in the background, but I am fine with that as long as as it is not interfering with my stdin/out.
So... I changed my code to the following as a test:

Code:
#!/usr/bin/python
#
import subprocess

exclude = "(NID|up|c2-0c0s0n1|c3-0c0s0n1)"

p1 = subprocess.call(["ssh", "root@boot"],
                        stdout=subprocess.PIPE)

p2 = subprocess.call(["xtprocadmin"],
                        stdin=p1.stdout,
                        stdout=subprocess.PIPE)

p3 = subprocess.call(["egrep", "-v", "up"],
                        stdin=p2.stdout,
                        stdout=subprocress.PIPE)

out, err = p3.communicate()
Unfortunately, something went wrong and it just hung...doing nothing! I had to open another session to the same server, and find the pid to kill.

I'm sensing that learning Python is not going to be as straight forward as I had previously hoped But I'm determined.

Thanks
Jon
 
Old 02-10-2017, 03:14 AM   #19
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
So, I've read this: https://docs.python.org/2/library/subprocess.html

I've watched a bunch of youtube videos. AND, I signed up for a premium account on LinuxAcademy.com, and started doing to the Python challenges on HackerRank.com.

I think I'm getting closer to figuring this out.

One question I have though, which I can't seem to read anywhere is:

What is the benefit from calling 3 processes for one command? Is it not more efficient to put all aspects of one command into a list and process that list with Popen?

So instead of:

Code:
p1 = subprocess.call(["ssh", "root@boot"],
                        stdout=subprocess.PIPE)

p2 = subprocess.call(["xtprocadmin"],
                        stdin=p1.stdout,
                        stdout=subprocess.PIPE)

p3 = subprocess.call(["egrep", "-v", "up"],
                        stdin=p2.stdout,
                        stdout=subprocress.PIPE)

out, err = p3.communicate()
Would this acheive the same result with less processes used:

Code:
p1 = subprocess.call(["ssh", "root@boot", "xtprocadmin", "egrep", "-v" exclude],
                        stdout=subprocess.PIPE)

out, err = p1.communicate()
However, I've raised a question here... which may end up answering my above mentioned question:

How do we handle the fact that any arguments passed to the ssh command, need to be within single or double quotes:

Code:
ssh root@server 'ls -la /tmp | grep foo'
or
ssh root@server "ls -la /tmp | grep $foo"
When breaking down the above command in Python, how to we handle that when all components in the list passed to Popen also need to be within quotes? Can we just escape some of the quotes with ""??

Would this suffice?

Code:
p1 = subprocess.call(["ssh", "root@boot", "\"xtprocadmin", "egrep", "-v" exclude\"],
                        stdout=subprocess.PIPE)

out, err = p1.communicate()
Thanks again,
Jon
 
Old 02-10-2017, 03:37 AM   #20
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,236

Rep: Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452
you can obviously test it, but you need to take into account: xtprocadmin and egrep are two commands, and xtprocadmin is executed on the remote site, grep is executed on the current host. And also you need a pipe between them.if you want to pass a remote command to your ssh, you need to construct the full command into one string and use as a single argument:
Code:
remote_command = "something"
p1 = subprocess.call(["ssh", "root@boot", remote_command],
 
1 members found this post helpful.
Old 02-10-2017, 04:46 AM   #21
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
you can obviously test it, but you need to take into account: xtprocadmin and egrep are two commands, and xtprocadmin is executed on the remote site, grep is executed on the current host. And also you need a pipe between them.if you want to pass a remote command to your ssh, you need to construct the full command into one string and use as a single argument:
Code:
remote_command = "something"
p1 = subprocess.call(["ssh", "root@boot", remote_command],
Ah that's a very good point, I'd not thought about constructing the remote command as a variable. Maybe that will work better.

It doesn't really matter which end the egrep command runs, In fact it's probably cleaner to run the egrep command on the remote side also... In standard shell I guess it just depends where you put the quote mark:

Code:
ssh root@boot 'xtprocadmin | egrep -v "(stuff)"'
or
ssh root@boot 'xtprocadmin' | egrep -v "(stuff)"
...would both acheive the same result, but in this case I always run the 'egrep' command on the remote side, because the output of 'xtprocadmin' on the remote side would be a list of 6912 nodes, whereas the result of the egrep command should never be more than ~20 - and thus less data is being transmitted over the ssh connection, and I would assume a slightly faster and more efficient script. Would you agree?

I'll test a little bit with putting the remote command as a string into a variable and using that within the subprocess.Popen() command.

Thanks
Jon
 
Old 02-10-2017, 04:03 PM   #22
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
I'll not give up! I'm nothing if not persistent!

So, I've got some code that 'kind of' works:

Code:
#!/usr/bin/python
#

from subprocess import *
import sys

host = 'root@boot'
remote_cmd = 'ls -la'
#remote_cmd = 'ssh root@boot "xtprocadmin | grep -v up"'

#p1 = Popen(["ssh", "root@boot", "ls", "-la")]. <-- This works
p1 = Popen(["ssh", "%s" % (host, remote_cmd)], #<--This does not
            shell=False,
            stdout=PIPE,
            stderr=PIPE)

result = p1.stdout.readlines()

if result == []:
    error = p1.stderr.readlines()
    print >>sys.stderr, "ERROR: %s" % error
else:
    print result
It will create an SSH connection, and depending on how I structure the remote commands, it will work.

Quick question though; have I made a mistake with the way I'm calling the variables? You can see what I mean from the code comments..

Thanks
Jon
 
Old 02-10-2017, 04:31 PM   #23
jonnybinthemix
Member
 
Registered: May 2014
Location: Bristol, United Kingdom
Distribution: RHEL 5 & 6
Posts: 169

Original Poster
Rep: Reputation: Disabled
I think I've found another way to achieve what I want. It doesn't directly work, but we're wandering off topic of SSH. So I'm going to mark this topic as SOLVED. I'll open a new post with my other question.

Thanks for your help.

Jon
 
Old 02-11-2017, 03:40 AM   #24
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,236

Rep: Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452Reputation: 5452
if I understand it correctly in this case the variable remote_command is not in use
Code:
p1 = Popen(["ssh", "%s" % (host, remote_cmd)], #<--This does not
 
Old 02-11-2017, 08:30 AM   #25
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,590

Rep: Reputation: 1908Reputation: 1908Reputation: 1908Reputation: 1908Reputation: 1908Reputation: 1908Reputation: 1908Reputation: 1908Reputation: 1908Reputation: 1908Reputation: 1908
I think you want this:
Code:
p1 = Popen(["ssh", host, remote_cmd],
You need to think about how the shell splits arguments, and then use the same split in your program. A quick way to see how arguments are split by the shell is to use printf:

Code:
~$ printf '{%s}\n' ssh root@boot 'xtprocadmin | egrep -v "(stuff)"'
{ssh}
{root@boot}
{xtprocadmin | egrep -v "(stuff)"}
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
no python module base.g? hcgernhardt Linux - Software 2 04-24-2018 09:03 AM
[SOLVED] Using python GNUPG module over SSH guardian1 Linux - Server 2 08-20-2012 08:53 AM
configure: error: The xdg python module is required (pyxdg or python-xdg) Sargalus Linux - Software 7 03-24-2010 07:34 AM
Python: How to use the re module? donnied Programming 2 01-19-2009 12:26 PM
python update - Unable to load GTK2 Python bindings: No module named gtk itpedersen Linux - Software 2 10-03-2008 03:44 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 11:05 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration