LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 07-02-2016, 10:40 PM   #1
trendal
LQ Newbie
 
Registered: Jul 2016
Posts: 10

Rep: Reputation: Disabled
Pipe output of local script, ran on remote machine, to local python script


Background:
Python program adjusting router configurations (OSPF) on WISP main backhaul system due to radio state changes.

Goal:
Run local shell script on remote router, and return the output to the parent Python program.

Problem:
Not understanding how to direct or redirect the output

I can run the shell script stand alone from a terminal and it prints the output to the terminal screen but when I call it from python using subprocess.check_output I get an empty string and no errors.

This is the stand alone command: (there are 3 args passed to route-cost.sh, the target router hostname, the target interface, and the OSPF cost to check/set)

ssh -q myusername@10.1.1.19 "vbash -s" -- < ./route-cost.sh router-hostname eth0 10

The script looks like this:
Code:
#!/bin/vbash

hname=$1
iface=$2
cost=$3
checkcost=" cost $3"

source /opt/vyatta/etc/functions/script-template

if [ `hostname` = "$hname" ]; then
	configure
	currentcost=`show interfaces ethernet $iface ip ospf cost`
	if [ "$currentcost" != "$checkcost" ]; then
		echo "$hname $iface cost set to $cost"
		set interfaces ethernet $iface ip ospf cost $cost
		commit
		save
	else
		echo "$hname $iface cost already set to $cost"
	fi
else
	echo "Hostname doesn't match"
fi
exit
In the python code the call is like this:
Code:
args = ['ssh', '-q', 'myusername@10.1.1.19', 'vbash -s', '--', '<', '. path-to-/route-cost.sh', 'router-hostname', 'eth0', '10']
subprocess.check_ouput(args)
 
Old 07-03-2016, 12:54 AM   #2
trendal
LQ Newbie
 
Registered: Jul 2016
Posts: 10

Original Poster
Rep: Reputation: Disabled
In an effort to move on I regressed to copying the script to the remote routers into persistent storage and running it remotely. One file write won't wear out the flash enough to matter and it gets the job done. Still it bothers me I can't get it done the "right" way.

Now if I ever want to make changes to said script I will have to manipulate all our routers again and make sure they have the updated version as opposed to changing the one single script locally on the management server that should get ran remotely.

I really thing the bug, if it is, is in python's subprocess module. I don't think it is handling all the arguments exactly like an open shell does when I test the command. I don't have the energy to prove that nor the expertise to fix it if I do prove it.
 
Old 07-03-2016, 01:08 AM   #3
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=15, FreeBSD_12{.0|.1}
Posts: 6,258
Blog Entries: 24

Rep: Reputation: 4193Reputation: 4193Reputation: 4193Reputation: 4193Reputation: 4193Reputation: 4193Reputation: 4193Reputation: 4193Reputation: 4193Reputation: 4193Reputation: 4193
Welcome to LQ!

After reading this several times, I have to say that I still do not understand what you are trying to do.

You cannot run a local script on a remote mchine, you must copy it over to the remote machine and run it there with proper ownership and permissions.

Perhaps if you tried again to explain what you are trying to do...?
 
Old 07-03-2016, 01:09 AM   #4
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,258
Blog Entries: 3

Rep: Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713
The first line of the script looks off, it should probably be "#!/bin/bash" instead of "#!/bin/vbash"

If both the local and remote scripts run, you can transfer the data with a pipe.

Code:
 ssh -q myusername@10.1.1.19 "vbash -s" -- | ./route-cost.sh router-hostname eth0 10
That will run the script "vbash" on the remote machine and use its output as input for "route-cost" which will run on the local machine. If you want "vbash" to run locally and "route-cost" to run on the remote, then move the use of "ssh"

Code:
 vbash -s | ssh -q myusername@10.1.1.19 "./route-cost.sh router-hostname eth0 10"
That will run "vbash" locally and send its output over to the remote machine where "route-cost" can use it as input.
 
Old 07-03-2016, 01:25 AM   #5
trendal
LQ Newbie
 
Registered: Jul 2016
Posts: 10

Original Poster
Rep: Reputation: Disabled
Okay, the way I had "vbash -s" there came directly from the router manufactures recommendations for running a local script remotely with arguments.

There is only one script involved here, route-cost.sh, unless you consider invoking vbash as "running" a script.

VBASH

Last edited by trendal; 07-03-2016 at 01:27 AM.
 
Old 07-03-2016, 01:26 AM   #6
trendal
LQ Newbie
 
Registered: Jul 2016
Posts: 10

Original Poster
Rep: Reputation: Disabled
Or more likely I'm not catching onto what you are getting at...
 
Old 07-03-2016, 01:31 AM   #7
trendal
LQ Newbie
 
Registered: Jul 2016
Posts: 10

Original Poster
Rep: Reputation: Disabled
route-cost.sh is stored on the local machine. It needs to be ran by vbash on the remote machine. But the whole thing is being ran from a python program on the same local machine using the subprocess.check_output function.

route-cost.sh runs as it should when called from a terminal on the local machine like I previously mentioned.
Code:
ssh -q myusername@10.1.1.19 "vbash -s" -- < ./route-cost.sh router-hostname eth0 10
But when I try to call that from python is when the failure happens.
 
Old 07-03-2016, 01:39 AM   #8
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,258
Blog Entries: 3

Rep: Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713
You mention in the first post a python script and a shell script. Both should identify the interpreter to be used in the first line of the script. The interpreter is likely to be one of these:
  • /usr/bin/python
  • /usr/bin/perl
  • /bin/sh
  • /bin/bash

It sounds like you are trying to run one script on a remote computer and pass the output to a script on the local computer. Or is it the reverse?

Edit: I now see your latest post. See my earlier one and swap the redirect ( < ) for a pipe ( | ) in your example or try one of the syntaxes given above.

Last edited by Turbocapitalist; 07-03-2016 at 01:41 AM.
 
Old 07-03-2016, 01:48 AM   #9
trendal
LQ Newbie
 
Registered: Jul 2016
Posts: 10

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Turbocapitalist View Post

It sounds like you are trying to run one script on a remote computer and pass the output to a script on the local computer.
Correct, I am trying to run one script on a remote router and pass the output to a python script on the local computer.

I understand the interpreter line, #!/bin/vbash for the script running on the remote router and #!/usr/bin/python for the local python script.

I'll give your examples a try. Thank you very much for the suggestion. Like I said, I had moved on, but I really would like to get this done right. Unless this whole idea fails this should be a long term operation.
 
Old 07-03-2016, 03:01 AM   #10
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,258
Blog Entries: 3

Rep: Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713
There some good but abstract explanations of redirects and pipes, but it's harder to find something basic. The redirects send output to or from a file. So if you were to use a redirect with a remote script, it could go something like this:

Code:
ssh router.example.com "remotescript" > localfile
localscript < localfile
With a pipe, no intermediate file is needed. stdout from one program or script becomes stdin for the next on in the line.

Code:
apropos -w '*'

apropos -w '*' | awk '$2 == "(1)" || $2== "(8)"'

apropos -w '*' | awk '$2 == "(1)" || $2== "(8)"' | sort -u

apropos -w '*' | awk '$2 == "(1)" || $2== "(8)"' | sort -u | less
 
1 members found this post helpful.
Old 07-03-2016, 12:11 PM   #11
trendal
LQ Newbie
 
Registered: Jul 2016
Posts: 10

Original Poster
Rep: Reputation: Disabled
I lack a good understanding of the finer points of piping and redirects. So I find myself programming around those weak areas. So many ways to get things done but usually only a couple nice ways.
 
Old 07-03-2016, 12:28 PM   #12
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,692

Rep: Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274
If I understand it well you want to execute it from python. But the redirection "<" is not part of the command you wanted to execute, therefore the arg list is not ok (in your original post).
I do not know if your command in post #7 works properly (I mean without python), or that line was also incorrect.
 
1 members found this post helpful.
Old 07-03-2016, 12:35 PM   #13
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,258
Blog Entries: 3

Rep: Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713
Quote:
Originally Posted by trendal View Post
I lack a good understanding of the finer points of piping and redirects.
The pipes are more useful in this case, if I understand your set up. Pipes are pretty simple and you can even use them with ssh to get data from a program on one machine to a program on another. See #4 above for going either direction with the data. The pipes always flow from left to right so you have to line up your programs with that in mind.
 
1 members found this post helpful.
Old 07-03-2016, 06:53 PM   #14
trendal
LQ Newbie
 
Registered: Jul 2016
Posts: 10

Original Poster
Rep: Reputation: Disabled
I'm fooling around with it again today. Haven't made any progress yet. I'm thankful for the help but also happy to leave it as is if nobody feels like tagging along here.

One thing that makes it a bit more trippy, I think, is the 3 args that need to be passed to the script when it is run on the remote machine.

In a terminal I tried this:
Code:
cat route-cost.sh | ssh username@10.1.1.19
It runs on the remote machine but of course there is no args so the script properly dumps back the appropriate error.

When I run that same sequence in python:
Code:
subprocess.check_output(["cat", "route-cost.sh", "|", "ssh", "-q", "username@10.1.1.19"])
It returns this error:
Code:
cat: invalid option -- 'q'
So somehow the list getting passed to subprocess.check_output is not being treated the same as when it's ran in a terminal. The "-q" is getting applied to "cat" instead of "ssh".
 
Old 07-04-2016, 12:47 AM   #15
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,692

Rep: Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274
I can only repeat myself. Redirection and pipe are not part of the command, therefore you cannot put | and < and similar into the argument list. That caused the problem you see.
here: http://stackoverflow.com/questions/4...ut-using-popen you will find some basic piping tips
and here: https://jimmyg.org/blog/2009/working...ined-pipelines is a more detailed page
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] SSH remote command: Pipe remote output to local machine? kenneho Linux - Server 6 12-06-2012 01:37 AM
ssh - using variables in call to start remote script from local script babag Linux - Networking 2 06-03-2008 04:50 PM
bash - start remote script from local script? babag Programming 7 04-06-2008 05:46 PM
clamd upgradation in remote mach through local mac by script to be run on local mach Narayandutt Programming 3 11-29-2006 10:00 AM
shell script to compare filese b/w local and remote and delete files from local serve dsids Linux - Networking 9 08-23-2006 07:20 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 04:13 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration