pseudo-files in shell programming?

ocicat · 01-30-2010, 01:52 AM

Perl has the concept of pseudo virtual files where data can be appended to the end of the script, & Perl will treat it as a legitimate source of data:

Code:

$ cat pseudo_file.pl
#!/usr/bin/env perl

use strict;

while (<DATA>) {
    print;
}

__DATA__
1
2
3
$ perl pseudo_file.pl
1
2
3
$ _

I would really like to be able to do something similar in a Bourne shell script where I don't have to manage a collateral text file which is prone to getting lost. Does anyone know if there is a similar construct in shell programming? I would have thought this feature was a remnant that Perl inherited from its shell programming roots, but searching on the Web has revealed nothing of consequence.

Anyone have insight into similar functionality?

Thanks.

macemoneta · 01-30-2010, 03:44 AM

In shell they are called 'here documents':

http://tldp.org/LDP/abs/html/here-docs.html

catkin · 01-30-2010, 03:47 AM

AFAIK no such facility exists. While it could be programmed there would be a fundamental problem in that it is not possible for bash scripts to reliably find the file they are run from as explained here.

macemoneta · 01-30-2010, 03:51 AM

Quote:

Originally Posted by catkin

AFAIK no such facility exists. While it could be programmed there would be a fundamental problem in that it is not possible for bash scripts to reliably find the file they are run from as explained here.

Are you replying to the correct post?

theYinYeti · 01-30-2010, 03:52 AM

You can also write things like this:

Code:

while read line; do
  echo "$line"
done <<<"first line
second line
third line
last line"

theYinYeti · 01-30-2010, 03:56 AM

Or even:

Code:

DATA="1
2
3
$(/get/output/from/another/script.sh)
1
2
3"

while read line; do
  echo "$line"
done <<<"$DATA"

On numerous alternatives, actually, using command substitution…

Yves.

ocicat · 01-30-2010, 02:30 PM

Quote:

Originally Posted by theYinYeti

You can also write things like this...

Thanks, but the <<< redirection operator is Bash-specific.

ocicat · 01-30-2010, 02:37 PM

Quote:

Originally Posted by macemoneta

In shell they are called 'here documents'...

Thanks. I am aware of "here" documents. I was hoping that someone would know the history or lineage where __DATA__ came from (thinking it came from shell programming...), but just as searching didn't reveal anything, it must have started with Perl.

Larry Wall is just one righteous dude.

Thanks everyone for your answers.

catkin · 01-30-2010, 04:44 PM

Quote:

Originally Posted by macemoneta

Are you replying to the correct post?

The OP requirement was "data can be appended to the end of the script"

macemoneta · 01-30-2010, 05:08 PM

Quote:

Originally Posted by ocicat

Thanks. I am aware of "here" documents. I was hoping that someone would know the history or lineage where __DATA__ came from (thinking it came from shell programming...), but just as searching didn't reveal anything, it must have started with Perl.

Larry Wall is just one righteous dude.

Thanks everyone for your answers.

As far as the origin, it goes back much before perl. Mainframe systems, using punched card readers use a process control language (early scripting) called JCL (job control language). Card decks were 'stacked' in the reader: JCL, source, more JCL, data. For example:

Code:

//MYJOB   JOB  CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1),REGION=256K
//S1      EXEC PGM=IEBGENER
//SYSUT1   DD  DATA,DLM=$$
/*******************************************************/
/*                                                     */
/*                      Hello World                    */
/*                                                     */
/*******************************************************/
$$
//SYSUT2   DD  SYSOUT=A
//SYSPRINT DD  SYSOUT=A
//SYSIN    DD  DUMMY
//

The DLM defines the delimiter for the inline data. This has been around since the mid-1960's.

catkin · 01-31-2010, 01:10 AM

Quote:

Originally Posted by macemoneta

As far as the origin, it goes back much before perl. Mainframe systems, using punched card readers use a process control language (early scripting) called JCL (job control language).

Oooo! It made me come over all nostalgic

I remember when ...

JCL is where the dd command came from, both in name and its unconventional (for UNIX) syntax. IIRC DD meant "data definition" ... ?

wje_lq · 01-31-2010, 06:12 PM

Quote:

Originally Posted by catkin

IIRC DD meant "data definition"

That's correct, it did.

In FORTRAN shops, for example, you could save compute time (and therefore money in your research budget) by compiling your program and sending the object deck to the card punch. Then you could run your program over and over with different sets of data (in those days, a file was known as a "data set"). You'd put the punched object deck near the beginning of your batch job preceded by this:

Code:

//LINK.SYSIN DD *

and followed by the all-purpose end of file indicator:

Code:

/*

That end of file indicator could have anything you wanted in columns 3 through 80, so it was customary (at least at UC Irvine) to have lots of cards available which, for the sake of convenience, looked like this:

Code:

/*                                                                            */