LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 05-24-2005, 01:17 PM   #1
R00ts
Member
 
Registered: Mar 2004
Location: Austin TX, USA
Distribution: Ubuntu 11.10, Fedora 16
Posts: 545

Rep: Reputation: 30
Bash redirection being taken literally by programs


I'm a little stumped over the behavior of one of my bash scripts. I call a series of programs in this script and on the lines I execute those programs I give the input/output redirection ('<' '>' or '2>'), but the programs are interpretting this literally as arguments to the program and are exhibiting incorrect behavior, such as:

1) Printing a message that the argument '<' was not recognized
2) Printing to stdout instead of redirecting
3) Generating seg faults because it did not recieve it's expected input

I echo out the exact line everytime I call the program and it's absolutely correct. Furthermore, when I copy this output line on the command line the redirection works and the program happily runs. Does anyone have a clue as to what may be causing this?


The only thing I can think of is maybe because this script I'm running calls another script multiple times which in turns calls several other programs and scripts, and somewhere somehow the re-direction is getting messed up between the pass calls. (That's 3 levels in the script hierarchy if you weren't counting). (I literally pass "/home/me/bin/prog < input.i > output.o 2> error.e" from the top-level script down to the mid-level, and the mid-level passes it down to the bottom-level program/script.


Thanks in advance for any clues you can give as to what may be going wrong. I've been working on this problem since yesterday evening and I still haven't been able to resolve the issue.
 
Old 05-24-2005, 01:23 PM   #2
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,483

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
This is normal behaviour, if your target programs get the redirection symbols as parameters, they almost always can't do anything with them.

The problem is certainly due to your script quoting the redirections, thus hiding them to the shell, which is the one that should have handled them.
 
Old 05-24-2005, 01:36 PM   #3
R00ts
Member
 
Registered: Mar 2004
Location: Austin TX, USA
Distribution: Ubuntu 11.10, Fedora 16
Posts: 545

Original Poster
Rep: Reputation: 30
Yeah, after thinking about it that sounds right. It's redirecting the output of the mid-level script instead of the low-level program that the mid-level script calls. Hmmm, is there a way to safely pass redirection arguments down through a script? I pass them as a simple quoted string when I send it from the high-level to mid-level script.
 
Old 05-24-2005, 01:48 PM   #4
R00ts
Member
 
Registered: Mar 2004
Location: Austin TX, USA
Distribution: Ubuntu 11.10, Fedora 16
Posts: 545

Original Poster
Rep: Reputation: 30
One fix I thought of would be to create an option in my mid-level script that grabs the name of the file for input, output, and error like so:

Code:
./midscript -i input.i -o output.o -e error.e
And then when I call my bottom level scripts/programs I can give them the redirection plus the file name. However this seems inefficient and would require a significant amount of work for me to re-structure my top level script (which contains about 50 unique calls to the mid-level script) and I'd have to add this functionality in the mid-level script. Does anyone know a way that I can tell my mid-level script "Ok, you got these redirection arguments but don't interpret them, because they aren't for you."? I hope so, but a part of me doubts it. Maybe using a `` or '' string instead of a "" string when I pass the program + redirections from top level to mid-level would do it?
 
Old 05-24-2005, 01:56 PM   #5
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,483

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Can you post a small sample of your scripts, I mean the smaller and simpler you can write that still exhibits the issue ?
 
Old 05-24-2005, 02:19 PM   #6
R00ts
Member
 
Registered: Mar 2004
Location: Austin TX, USA
Distribution: Ubuntu 11.10, Fedora 16
Posts: 545

Original Poster
Rep: Reputation: 30
My scripts are really huge and intricate (hunderds of line each). But I think I found the solution to the problem from someone at work. Basically you put a \ before the re-direction arguments for each level of redirection and that prevents the script from using the re-direction. I'm not sure if I'm implementing the way he told me to, but I wrote a sample script and it seems like it works for my test case. I'm going to try putting it in the main script hiearchy right now and if it's successful, I'll post the solution here.
 
Old 05-24-2005, 05:39 PM   #7
R00ts
Member
 
Registered: Mar 2004
Location: Austin TX, USA
Distribution: Ubuntu 11.10, Fedora 16
Posts: 545

Original Poster
Rep: Reputation: 30
Nope, I couldn't get it to work. I first tried putting one \ in front of each < and > character in passing the script and then using sed in the mid-level script to remove each \ in the argument passed. The command was echoed correctly, but didn't run correctly (same exact problems as before). Then I tried two \\ in front of the < and > and removing them in mid-level, but that didn't work either. Then I tried actually replacing < and > with ? and \ respesctively so they wouldn't get picked up at all and used sed to replace them with < and > in the mid-level script, and again the same problems.


I don't understand why this isn't working. Even in my sample script when I did this the lowest level didn't printed to stdout instead of the output file specified by the redirection. If anyone knows what's going on here I am in dire need of your help.
 
Old 05-24-2005, 06:12 PM   #8
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,483

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
post a test case so we could go further.
 
Old 05-25-2005, 07:39 AM   #9
eddiebaby1023
Member
 
Registered: May 2005
Posts: 378

Rep: Reputation: 33
If you start escaping the redirection operators he shell wil strip the '\'s off at each level. Echoing commands may also cause the shell do to some escape-stripping so can't be relied on for veracity. As others have requested, post some examples so we can give you pertinent advice (or at least explain why what you're doing isn't working).
 
Old 05-25-2005, 10:46 AM   #10
R00ts
Member
 
Registered: Mar 2004
Location: Austin TX, USA
Distribution: Ubuntu 11.10, Fedora 16
Posts: 545

Original Poster
Rep: Reputation: 30
Here's a little demo I whipped up that is a gross simplification of my problem. I call a toplevel script, which calls a mid-level script with redirection arguments not intended for it, which calls a hello world c program that prints to stdout and stderr. Here's the code:

toplevel
Code:
#!/bin/bash

echo "TOP"
./midlevel "\> output.o \2> error.e"
exit 0
midlevel
Code:
#!/bin/bash

echo "MIDDLE"
./hello_world $1
exit 0
hello_world (bottom level program)
Code:
#include <stdio.h>

int main() {
  fprintf(stdout, "Hello, world!\n");
  fprintf(stderr, "Hello, error!\n");
  return 0;
}
So I try to put two \ in front of the redirection args so that the redirection is "reserved" for the hello_world program. Here's the output:

Code:
$ ./toplevel
TOP
MIDDLE
Hello, world!
Hello, error!
Which is obviously incorrect, because "Hello, world!" should go in output.o and "Hello, error!" should go in error.e (neither of these two files were created by running the script). The redirection args are being completely ignored. tried changing the number of backslashes in toplevel to zero, two, three, and four and still got the same result. I'll keep playing with this but I'm running out of ideas on how to make it work. Thanks for your help guys





Ack, just before I hit reply I figured out the problem. I'm passing the redirection args in quotes "" because the way my mid-level script works is it takes the last argument (which is in quotes) as a string containing the program path + binary, it's arguments, and it's redirection arguments. When I take out the quotes the following are printed to output.o and error.e with no \ in front of the operators:

output.o
Code:
MIDDLE
Hello, world!
error.e
Code:
Hello, error!
Using backslashes makes things worse. "Hello, error!" is never printed to error.e (always stdout) and in one case I actually got MIDDLE and "Hello, world!" printed to error.e.
 
Old 05-25-2005, 01:14 PM   #11
R00ts
Member
 
Registered: Mar 2004
Location: Austin TX, USA
Distribution: Ubuntu 11.10, Fedora 16
Posts: 545

Original Poster
Rep: Reputation: 30
Ok, I have plan. It's now obvious that whenever redirection is enclosed in quotes it doesn't get operated on. My mid-level script needs quotes in it's last argument for work and it would be a pain in the ass to have to change that. So, what I think I'll do is in the mid-level script I'll use sed to extract the input, output, and error redirection names (if they exist) and then extract them from the quote. Then, when I make the call to the program, I'll see what redirection it needs and put that in by hand. It may be a little inefficient way of doing things, but I can't be delayed by this problem any longer. I'll post back if it works.
 
Old 05-25-2005, 03:06 PM   #12
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,483

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Try that:
Code:
echo "TOP"
./midlevel "> output.o 2> error.e"
exit 0
Code:
echo "MIDDLE"
eval ./helloworld $1
exit 0
 
Old 05-25-2005, 03:14 PM   #13
R00ts
Member
 
Registered: Mar 2004
Location: Austin TX, USA
Distribution: Ubuntu 11.10, Fedora 16
Posts: 545

Original Poster
Rep: Reputation: 30
You sir, are my hero!!! Everything works great now, and it's just a single word solution! I may have to write a song singing your praise and erecting a few statues in your honor for this.
 
Old 05-26-2005, 12:46 AM   #14
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,483

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Thanks, looking forward for the song ...
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
BASH scripting I/O redirection error blinux1 Programming 13 03-18-2008 08:21 PM
Bash redirection and subshells nx5000 Programming 2 11-03-2005 06:13 AM
bash redirection "$ cat << EOF > file" (how does this work) ninmonkeys Linux - General 1 11-09-2004 03:37 PM
bash output redirection: debian vs freebsd kenners Programming 2 10-07-2004 04:56 PM
BASH scripting: confused about redirection & file descriptors funkymunky Programming 1 06-07-2004 07:47 AM


All times are GMT -5. The time now is 05:55 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration