Using SED with EXPECT to trim a string from buffer

h4rri · 11-06-2013, 07:57 AM

Hi,

I am currently attempting to trim a string using SED within an Expect script.

I am looking to remove either all but the first line or everything after a specific word.

The string is:

Code:

 Resistance:   (T-R) 999999999 (T-S) 2108967 (R-S) 2108967 Ohm

 999999999 Indicates out of range result

 operation completed.

The code I am using is:

Code:

#set the variable and dump the buffer contents in it
set res_var $expect_out(buffer)
exec cat $res_var | sed -e '1!d' > $res_var
send_user $res_var

I have tried encapsulating the 1!d in "'s, {'s and 's and nothing trims the string.

The end result I am trying to achieve is to have everything after 'Ohm' removed thus presenting one single line to the user. I know the SED coding is correct in that it will run from command line on a test file, it's the integration into Expect that's getting me.

Any thoughts?

druuna · 11-06-2013, 08:10 AM

Quote:

Originally Posted by h4rri

cat $res_var | sed -e '1!d' > $res_var

I know the SED coding is correct in that it will run from command line on a test file.

The above statement cannot be true. If you use the same file for input and output you'll end up with an empty file.

Code:

$ cat infile
 Resistance:   (T-R) 999999999 (T-S) 2108967 (R-S) 2108967 Ohm

 999999999 Indicates out of range result

 operation completed.

$ cat infile | sed '1!d' > infile
$ cat infile
<no output>

If you have a modern sed version (version 4+) you can use sed's -i flag to do in-place edits:

Code:

sed -i '1!d' infile

If you do not have a modern sed version you need to create a temporary file and move it after the action is done:

Code:

# this also gets rid of the useless use of cat
sed '1!d' infile > outfile
mv outfile infile

h4rri · 11-06-2013, 09:33 AM

Quote:

Originally Posted by druuna

The above statement cannot be true. If you use the same file for input and output you'll end up with an empty file.

Code:

h4rri[~]$ cat test.txt
Resistance:   (T-R) 999999999 (T-S) 2111820 (R-S) 2108967 Ohm


line 4

line 6

h4rri[~]$


h4rri[~]$ cat test.txt | sed -e '1!d' > test.txt

h4rri[~]$ cat test.txt
Resistance:   (T-R) 999999999 (T-S) 2111820 (R-S) 2108967 Ohm
h4rri[~]$

Seems to be working and is performing exactly as I would like it to, it's the integration into Expect that's giving me issues.

Code:

exec cat $res_var | sed {1!d} > test.txt

Gives an empty text file.

Code:

exec cat $res_res | sed {1!d} > $res_var

Gives an unchanged variable output

druuna · 11-06-2013, 10:45 AM

There must be something you are not telling us because what you describe will not work on a normal Unix/Linux machine.

The reason being that the shell opens stdout before executing commands. I.e (in short):
1 - the shell sees > test.txt and opens a new file called test.txt (empty),
2 - now the cat/sed commans is executed, which uses an empty test.txt as input.

This is default behaviour.

Which Linux distro or Unix flavor are you using and which shell?

h4rri · 11-06-2013, 11:20 AM

Quote:

Originally Posted by druuna

There must be something you are not telling us because what you describe will not work on a normal Unix/Linux machine.

Thanks for the explanation. I am not deliberately hiding information so I apologise for any missing information. It is configured as a network 'jump host' if that makes any difference?

I am using GNU Bash v2.05a on FreeBSD 4.8 release P13

Thanks

druuna · 11-07-2013, 02:16 AM

You mention this in one of your previous replies:

Quote:

Originally Posted by h4rri

Code:

exec cat $res_var | sed {1!d} > test.txt

Gives an empty text file.

This is consistent with my previous replies.

Have you tried this:

Code:

exec sed -i.bak 1!d $res_var

A simple test on my side shows that both of the following work:

Code:

exec sed -i.bak 1!d infile
exec cat infile | sed 1!d > infile.new

The first one is preferred (no useless use of the cat command).

If you remove the .bak part, no backup/copy will be made.

h4rri · 11-09-2013, 02:39 AM

Quote:

Originally Posted by druuna

Have you tried this:

Code:

exec sed -i.bak 1!d $res_var

I have just tried this and get the following:

Code:

sed: 1: " (T-R) 999999999 (T-S)  ...": invalid command code (
    while executing
"exec sed -i 1!d $res_var"

druuna · 11-09-2013, 04:02 AM

I'm not sure why all this doesn't work for you.

I did another test and this also works on my side:

Code:

#!/usr/bin/expect

set res_var "infile.txt"
exec sed -i.bak {1!d} $res_var

Example run:

Code:

$ cat infile.txt
 Resistance:   (T-R) 999999999 (T-S) 2108967 (R-S) 2108967 Ohm

 999999999 Indicates out of range result

 operation completed.
$ ./tst.exp
$ cat infile.txt
 Resistance:   (T-R) 999999999 (T-S) 2108967 (R-S) 2108967 Ohm
$ ls -1 infile*
infile.txt
infile.txt.bak

BTW: I'm not too familiar with expect.

h4rri · 11-09-2013, 01:47 PM

Quote:

Originally Posted by druuna

I'm not sure why all this doesn't work for you.

It has always worked on the test machine but as soon as I invoked the commands within the 'spawned session' they failed with various errors. It has just dawned on me (after hours of tinkering) why, the spawned session is to an Actelis DSLAM which doesn't have an install of SED or AWK or anything I can use to manipulate the string / file.

I'm sorry if I have wasted your time.

I now figure I need to 'quit' the spawned telnet session, drop back into the unix host and SED/AWK the file. The problem is, the spawned session is killed:

Code:

send: sending "quit\r" to { exp4 }
quit
Connection closed by foreign host.
expect: read eof
expect: set expect_out(spawn_id) "exp4"

Which is effectively ending the script and not allowing further commands to be run as it expects the to be sent to exp4, the spawned session.

Do you have any pointers on how to reference the spawning (parent) session once I have killed the spawned (child) session?

h4rri · 11-10-2013, 06:30 AM

Ok, for completeness and to assist anyone who is having a similar issue I have managed to 'work around' the issue by using Tcl based commands so the work is being done by expect rather than passing it to the spawned processes shell:

Create (and open) a test file, dump the contents of the buffer in there:

Code:

set tmpfile "tmp[pid].txt"
set output [open $tmpfile "w"]
     set outcome $expect_out(buffer)
     puts $output $outcome
close $output

Now re-open the file for writing and read the first line into a variable:

Code:

set fp [open $tmpfile "r"]
set file_data [gets $fp]
close $fp
exec rm $tmpfile

This will then allow you to output the first line of the buffer using:

Code:

send_user "$file_data"

Now I know this isn't the cleanest way of doing this, using a file and having to open it twice isn't ideal but I am trying to refine it down to create, write, read and delete in one strike.

JJJCR · 11-10-2013, 07:58 AM

good resource for awk and sed:

http://sage.math.washington.edu/home...ll_sed_awk.pdf

druuna · 11-10-2013, 07:59 AM

Quote:

Originally Posted by JJJCR

good resource for awk and sed:

http://sage.math.washington.edu/home...ll_sed_awk.pdf

How would this help with a Tcl / Expect problem?