[SOLVED] here-document with parameter as limitstring

pawn · 05-30-2011, 06:03 PM

Hello,
I'm trying to write a program, that will take a parameter as the limit string for a here-document.
Also, I want to output any document as the here-document.
For example i want to say:
./hereProgram document.txt EOF
where the first parameter is the file and the second the limitstring.
I'm facing two problems here.
First, when using a parameter as the limitstring, the end is not recognized:
hereProgram.txt:

Code:

cat <<"$@"
Line 1
Line 2
EOF

calling this with:
./hereProgram EOF
will fail because EOF is not the same as $@. this however works:
hereProgram.txt:
cat <<"$@"
Line 1
Line 2
$@

The second problem is, that theres always input expected after calling cat <<EOF. So i can place some text after the command, but not insert a file. This will work

Code:

cat <<EOF
Line 1
EOF

however this will not

Code:

cat <<EOF
cat document.txt

here EOF is located in document.txt.

I hope you can help me with either problem.
thanks a lot

David the H. · 05-30-2011, 06:40 PM

First of all, please use [code][/code] tags around your code, to preserve formatting and to improve readability.

A here string has a very specific design and syntax, and I don't think it's subject to parameter expansion. The whole point of it is to separate out a block of text in the current shell, after all.

For example, your second code example works because it's using the literal delimiting string "$@", not replacing it with the actual parameter.

Also, the ending delimiter must always start at the beginning of a line, with no spaces in front of it. Some shells may allow certain things to follow it, but it's not defined behavior, so the only safe thing is to place it alone on a line.

It seems to be possible to do this using eval though.

Code:

#!/bin/bash

eval cat <<$1

"some stuff"

$1

exit 0

This appears to work with simple here strings at least. I haven't yet figured out if there's any way to quote the delimiter string this way though. Putting some escaped quotes around it doesn't seem to work.

Nominal Animal · 05-30-2011, 07:28 PM

Quote:

Originally Posted by pawn

Hello,
I'm trying to write a program, that will take a parameter as the limit string for a here-document.
Also, I want to output any document as the here-document.
For example i want to say:
./hereProgram document.txt EOF
where the first parameter is the file and the second the limitstring.

I recommend awk instead of a shell script for this.

Code:

awk -v end="EOF" 'BEGIN { RS="(\r\n|\n\r|\r|\n)" ; RT="\n"; FS="[\t\v\f ]+" }
                  (NF==1 && $1==end) { exit(0); }
                  { printf("%s%s", $0, RT) }' document.txt

If you use GNU awk, it will retain the newline convention used in the file; other awks will convert the newline convention to UNIX newlines (LF or "\n" only). The end pattern must be the only thing on that line, but it can be indented with whitespace. You can use any string or parameter instead of EOF, as long as it does not contain newlines (LF "\n" or CR "\r"). (For all command line parameters, use $* and not $@, so that it is considered a single string and not split into multiple command line parameters. For arrays, use ${variable[*]} and not ${variable[@]} for the same reason.)

This awk script first sets RS, the record separator (newline); RT, the default output record separator (GNU awk will always reset it to whatever was used for that record); and FS, field separator (so that the end string can be indented).
If there is only one field on a line, and it matches the end string, the script exits.
Otherwise the record is printed (using the original newline if supported, default newline otherwise).

As awk streams the input files, it only keeps about one record worth of data (or a few kilobytes, whichever is greater) in RAM at a time. It is therefore pretty efficient. It is much faster than using a shell script, too.
I suspect the eval example shown by David the H. reads the entire file into RAM first.

grail · 05-30-2011, 08:38 PM

I am curious as to why the need for a different delimiter for the here document? (or did I miss something?)
The word used is of no particular importance and as far as I know is not able to be used as a reference point elsewhere in your code.

As for redirecting a file into a here document:

Code:

cat <<EOF
$(< file_name)
EOF

pawn · 05-31-2011, 01:55 AM

this is actually part of a task we've been given at school. And since the program has to run in any shell, unfortunately it's not possible to do it with awk. thank you anyway for the suggestion.
right now i'm thinking of building the whole command as a simple string and then just evaluating it afterwards. i'll keep you updated.
thanks

catkin · 05-31-2011, 05:23 AM

Quote:

Originally Posted by pawn

this is actually part of a task we've been given at school

Can you post the relevant part of the task specification?

pawn · 05-31-2011, 09:53 AM

as it turns out, i didn't read the problem very thoroughly, as was pointed out to me today

.
it says to build a program, that will act "like" cat <<EOF with a variable parameter for the limitstring.
so i don't actually have to use an actual here document. All i'm gonna do now is to read every line from a text file and check against the limitstring.
that aside, i'm really surprised about all those fast and devoted answers.
thank you very much