LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-06-2013, 08:57 AM   #1
h4rri
LQ Newbie
 
Registered: Jul 2013
Posts: 11

Rep: Reputation: Disabled
Using SED with EXPECT to trim a string from buffer


Hi,

I am currently attempting to trim a string using SED within an Expect script.

I am looking to remove either all but the first line or everything after a specific word.

The string is:

Code:
 Resistance:   (T-R) 999999999 (T-S) 2108967 (R-S) 2108967 Ohm

 999999999 Indicates out of range result

 operation completed.
The code I am using is:

Code:
#set the variable and dump the buffer contents in it
set res_var $expect_out(buffer)
exec cat $res_var | sed -e '1!d' > $res_var
send_user $res_var
I have tried encapsulating the 1!d in "'s, {'s and 's and nothing trims the string.

The end result I am trying to achieve is to have everything after 'Ohm' removed thus presenting one single line to the user. I know the SED coding is correct in that it will run from command line on a test file, it's the integration into Expect that's getting me.


Any thoughts?
 
Old 11-06-2013, 09:10 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by h4rri View Post
cat $res_var | sed -e '1!d' > $res_var

I know the SED coding is correct in that it will run from command line on a test file.
The above statement cannot be true. If you use the same file for input and output you'll end up with an empty file.
Code:
$ cat infile
 Resistance:   (T-R) 999999999 (T-S) 2108967 (R-S) 2108967 Ohm

 999999999 Indicates out of range result

 operation completed.

$ cat infile | sed '1!d' > infile
$ cat infile
<no output>
If you have a modern sed version (version 4+) you can use sed's -i flag to do in-place edits:
Code:
sed -i '1!d' infile
If you do not have a modern sed version you need to create a temporary file and move it after the action is done:
Code:
# this also gets rid of the useless use of cat
sed '1!d' infile > outfile
mv outfile infile
 
1 members found this post helpful.
Old 11-06-2013, 10:33 AM   #3
h4rri
LQ Newbie
 
Registered: Jul 2013
Posts: 11

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by druuna View Post
The above statement cannot be true. If you use the same file for input and output you'll end up with an empty file.
Code:
h4rri[~]$ cat test.txt
Resistance:   (T-R) 999999999 (T-S) 2111820 (R-S) 2108967 Ohm


line 4

line 6

h4rri[~]$


h4rri[~]$ cat test.txt | sed -e '1!d' > test.txt

h4rri[~]$ cat test.txt
Resistance:   (T-R) 999999999 (T-S) 2111820 (R-S) 2108967 Ohm
h4rri[~]$
Seems to be working and is performing exactly as I would like it to, it's the integration into Expect that's giving me issues.

Code:
exec cat $res_var | sed {1!d} > test.txt
Gives an empty text file.

Code:
exec cat $res_res | sed {1!d} > $res_var
Gives an unchanged variable output
 
Old 11-06-2013, 11:45 AM   #4
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
There must be something you are not telling us because what you describe will not work on a normal Unix/Linux machine.

The reason being that the shell opens stdout before executing commands. I.e (in short):
1 - the shell sees > test.txt and opens a new file called test.txt (empty),
2 - now the cat/sed commans is executed, which uses an empty test.txt as input.

This is default behaviour.

Which Linux distro or Unix flavor are you using and which shell?
 
1 members found this post helpful.
Old 11-06-2013, 12:20 PM   #5
h4rri
LQ Newbie
 
Registered: Jul 2013
Posts: 11

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by druuna View Post
There must be something you are not telling us because what you describe will not work on a normal Unix/Linux machine.
Thanks for the explanation. I am not deliberately hiding information so I apologise for any missing information. It is configured as a network 'jump host' if that makes any difference?

I am using GNU Bash v2.05a on FreeBSD 4.8 release P13

Thanks
 
Old 11-07-2013, 03:16 AM   #6
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
You mention this in one of your previous replies:
Quote:
Originally Posted by h4rri
Code:
exec cat $res_var | sed {1!d} > test.txt
Gives an empty text file.
This is consistent with my previous replies.

Have you tried this:
Code:
exec sed -i.bak 1!d $res_var
A simple test on my side shows that both of the following work:
Code:
exec sed -i.bak 1!d infile
exec cat infile | sed 1!d > infile.new
The first one is preferred (no useless use of the cat command).

If you remove the .bak part, no backup/copy will be made.
 
1 members found this post helpful.
Old 11-09-2013, 03:39 AM   #7
h4rri
LQ Newbie
 
Registered: Jul 2013
Posts: 11

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by druuna View Post
Have you tried this:
Code:
exec sed -i.bak 1!d $res_var
I have just tried this and get the following:

Code:
sed: 1: " (T-R) 999999999 (T-S)  ...": invalid command code (
    while executing
"exec sed -i 1!d $res_var"
 
Old 11-09-2013, 05:02 AM   #8
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
I'm not sure why all this doesn't work for you.

I did another test and this also works on my side:
Code:
#!/usr/bin/expect

set res_var "infile.txt"
exec sed -i.bak {1!d} $res_var
Example run:
Code:
$ cat infile.txt
 Resistance:   (T-R) 999999999 (T-S) 2108967 (R-S) 2108967 Ohm

 999999999 Indicates out of range result

 operation completed.
$ ./tst.exp
$ cat infile.txt
 Resistance:   (T-R) 999999999 (T-S) 2108967 (R-S) 2108967 Ohm
$ ls -1 infile*
infile.txt
infile.txt.bak
BTW: I'm not too familiar with expect.
 
1 members found this post helpful.
Old 11-09-2013, 02:47 PM   #9
h4rri
LQ Newbie
 
Registered: Jul 2013
Posts: 11

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by druuna View Post
I'm not sure why all this doesn't work for you.
It has always worked on the test machine but as soon as I invoked the commands within the 'spawned session' they failed with various errors. It has just dawned on me (after hours of tinkering) why, the spawned session is to an Actelis DSLAM which doesn't have an install of SED or AWK or anything I can use to manipulate the string / file.

I'm sorry if I have wasted your time.

I now figure I need to 'quit' the spawned telnet session, drop back into the unix host and SED/AWK the file. The problem is, the spawned session is killed:

Code:
send: sending "quit\r" to { exp4 }
quit
Connection closed by foreign host.
expect: read eof
expect: set expect_out(spawn_id) "exp4"
Which is effectively ending the script and not allowing further commands to be run as it expects the to be sent to exp4, the spawned session.

Do you have any pointers on how to reference the spawning (parent) session once I have killed the spawned (child) session?
 
Old 11-10-2013, 07:30 AM   #10
h4rri
LQ Newbie
 
Registered: Jul 2013
Posts: 11

Original Poster
Rep: Reputation: Disabled
Ok, for completeness and to assist anyone who is having a similar issue I have managed to 'work around' the issue by using Tcl based commands so the work is being done by expect rather than passing it to the spawned processes shell:

Create (and open) a test file, dump the contents of the buffer in there:

Code:
set tmpfile "tmp[pid].txt"
set output [open $tmpfile "w"]
     set outcome $expect_out(buffer)
     puts $output $outcome
close $output
Now re-open the file for writing and read the first line into a variable:

Code:
set fp [open $tmpfile "r"]
set file_data [gets $fp]
close $fp
exec rm $tmpfile
This will then allow you to output the first line of the buffer using:

Code:
send_user "$file_data"
Now I know this isn't the cleanest way of doing this, using a file and having to open it twice isn't ideal but I am trying to refine it down to create, write, read and delete in one strike.
 
Old 11-10-2013, 08:58 AM   #11
JJJCR
Senior Member
 
Registered: Apr 2010
Posts: 1,102

Rep: Reputation: 181Reputation: 181
good resource for awk and sed:

http://sage.math.washington.edu/home...ll_sed_awk.pdf
 
Old 11-10-2013, 08:59 AM   #12
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by JJJCR View Post
How would this help with a Tcl / Expect problem?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Sed/awk/cut to pull a repeating string out of a longer string StupidNewbie Programming 3 03-21-2012 02:47 PM
[SOLVED] using sed to trim lines greater than maximum number of characters cxny Linux - Software 10 09-03-2011 02:25 PM
how do i replace a text string in a file with a random string? (with sed etc) steve51184 Linux - Software 16 09-02-2010 12:05 PM
Trying to change String using sed with a string \/home\/user\/Desktop icecoolcorey Programming 10 06-13-2008 12:32 AM
syntax error in string trim and wrong node routing agent in tcl script newbie06 Linux - General 0 02-23-2007 03:00 AM


All times are GMT -5. The time now is 01:10 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration