script to parse variables to curl script and execute

morphix · 11-21-2007, 12:13 AM

Hi Guys,

I currently have a situation which up until now has been a complete manual copy, paste & URL editing procedure.

I have a text file which contains information in basic form:

111111,21-11-2007,Some Text Seperate By Spaces,No
222222,21-11-2007,More Text Seperate By Spaces,Yes

above is the example of what is in the text file.

What i am trying to do is make a script which reads the text file and puts the above into variables such as 22222 = $NUM, 21-11-2007 = $DATE, Text Spaces = $REASON, Yes = $ENABLED

For the ",Some Text Seperate By Spaces," i need all the spaces to get turned into "+" characters before being put as a variable

After this i need the example below to be run

curl <URL>/index.php?number=$NUM&date=$DATE&reason=$REASON&enabled=$ENABLED

if anyone knows how i can do this i would appreciate it as ive been messing with bash, grep, sed, curl & awk.

Cheers,

Morphix

Four · 11-21-2007, 12:22 AM

Perl script should do the trick

Code:

# read a line (assuming file is opened using open())

$read = <file>;

# get relevent data
$read =~ /(.*?),(.*?),(.*?),(.*)/;

$num = $1;
$data = $2;
$reason = $3;
$reason =~ s/ /+/g;
$enable =$4

open TOCURL, "| curl";
# its case sensitive so the below and above need to match case
print TOCURL "<URL>/index.php?number=$NUM&date=$DATE&reason=$REASON&enabled=$ENABLED
";

close TOCURL;

Four

morphix · 11-21-2007, 04:06 PM

@Four: so i assume this reads line by line from the text file and does the same curl command per line?

Just i am at work atm and unable to test it.

EDIT:

Just tested it, it just errors out

$ perl test.pl
syntax error at test.pl line 16, near "$4

open "
Execution of test.pl aborted due to compilation errors.

theNbomr · 11-21-2007, 05:08 PM

Maybe this is better. Substitute a valid url & curl options.

Code:

#! /usr/bin/perl -w

use strict;
use POSIX ":sys_wait_h";

    while(<>){
        my( $num, $date, $reason, $enabled ) = split /,/, $_;
        $reason =~ s/ /+/g;
        my $kid = fork();
        if( $kid ){
            waitpid( $kid, WNOHANG );
        }
        else{
            exec( "curl", "--help", "http://someurl.net/index.php?number=$num&date=$date&reason=$reason&enabled=$enabled\n" );
        }
    }
    exit( 0 );

Run the script with the data file as a commandline arg.
--- rod.

morphix · 11-21-2007, 05:27 PM

@theNbomr: can't seem to get that to work either :/

chrism01 · 11-21-2007, 05:39 PM

How about telling exactly what you did and what error you got?

theNbomr · 11-21-2007, 05:50 PM

Yah. What chrism01 said. Emphasis on exactly. With error messages and other relevant output pasted.
Did you replace the URL and curl options appropriately, as I mentioned?
--- rod.

morphix · 11-21-2007, 09:04 PM

nevermind, i got it to work, was just me not fully understanding hah.

EDIT:

is there anyway i can have this script perform an if statement on the last variable?

eg. if $enabled = Yes then change text to "Yes+this+is+enabled" else "This+is+disabled"

cheers theNbomr

angrybanana · 11-22-2007, 01:43 AM

Quote:

Originally Posted by morphix

is there anyway i can have this script perform an if statement on the last variable?

eg. if $enabled = Yes then change text to "Yes+this+is+enabled" else "This+is+disabled"

Code:

$enabled = ($enabled eq "Yes" ? "Yes+this+is+enabled" : "This+is+disabled")

archtoad6 · 11-23-2007, 09:41 AM

Quote:

Originally Posted by morphix

. . . if anyone knows how i can do this i would appreciate it as ive been messing with bash, grep, sed, curl & awk.

Assuming you are still interested, in a short

solution, here's the awk:

Code:

awk -F, '{gsub(" ","+",$3);system("curl $URL/index.php?number="$1"\\&date="$2"\\&reason="$3"\\&enabled="$4"")}' $FILE

This assumes that FILE & URL have been set to your desired values. FILE seems to have to be exported, because (AFAIK) the awk system call executes in a sub-shell.
The final double double quote is an artifact of the piece-wise construction of the code & may be unnecessary.

As for your 2nd Q, I am not clear if angrybanana's
code represents the logic you want, or if you want $3/$reason changed regardless of its value. I.E., if $4/$enabled does = "yes" but $3/$reason does NOT = "Yes+this+is+enabled" do you want it to be changed to "This+is+disabled" anyway?

BTW, I find the GNU all-on-1-page version of the The GNU Awk User's Guide aka "Gawk: Effective AWK Programming" to be the most helpful.

Four · 11-23-2007, 12:33 PM

Quote:

Originally Posted by morphix

@Four: so i assume this reads line by line from the text file and does the same curl command per line?

Just i am at work atm and unable to test it.

EDIT:

Just tested it, it just errors out

$ perl test.pl
syntax error at test.pl line 16, near "$4

open "
Execution of test.pl aborted due to compilation errors.

Oops I forgot a semicolon

Heres a more complete version simply run by

perl perlcode.pl file.txt_or_some_file

Code:

# this code is not tested I'm at school right now using windows machine with no perl
# read a line (assuming file is opened using open())
$file = $ARGV[0];
if(!$file){
   print "Please enter a file name\n";
   $file = <>;
}
if(!(-e $file)){
   print "file doesn't exit\n";
}
open (file, "$file") or die ("couldn't open $file\n");
while(<file>){
   $read = $_;

   # get relevent data
   $read =~ /(.*?),(.*?),(.*?),(.*)/;

   $num = $1;
   $data = $2;
   $reason = $3;
   $reason =~ s/ /+/g;
   $enable = $4;

   if($enable eq "yes"){
      # put code to do something here if you want
   }
   open (TOCURL, "| curl") or die("Couldn't open pipe to curl\n");
   # its case sensitive so the below and above need to match case
   print TOCURL "<URL>/index.php?number=$num&date=$date&reason=$reason&enabled=$enable
";

   close TOCURL;
}
close file;

chrism01 · 11-24-2007, 07:38 PM

You need to chomp() the input filename to remove the trailing '\n'.
Also, for cmds like 'open'/'close', perl stores the error code/msg in '$!'. You should add this to the exit msg so you know WHY it failed.

morphix · 11-25-2007, 04:47 PM

theNbomr scripts works upto a point.

when i enter the CURL URL and run the script it gives me back the following errors.

$ perl script.pl script_txt.txt
Bareword found where operator expected at script.pl line 14, near ""curl -s -S -b cookie.txt -d "number"
(Missing operator before number?)
Operator or semicolon missing before &radius at script.pl line 14.
Ambiguous use of & resolved as operator & at script.pl line 14.
Operator or semicolon missing before &email at script.pl line 14.
Ambiguous use of & resolved as operator & at script.pl line 14.
Operator or semicolon missing before &ftp at script.pl line 14.
Ambiguous use of & resolved as operator & at script.pl line 14.
Operator or semicolon missing before &cancel_reason at script.pl line 14.
Ambiguous use of & resolved as operator & at script.pl line 14.
Operator or semicolon missing before &do_cancel_service at script.pl line 14.
Ambiguous use of & resolved as operator & at script.pl line 14.
String found where operator expected at script.pl line 14, near "1" http://url/subdir/page.php""
(Missing operator before " http://url/subdir/page.php"?)
Global symbol "$reason" requires explicit package name at script.pl line 8.
syntax error at script.pl line 14, near ""curl -s -S -b cookie.txt -d "number"
Execution of script.pl aborted due to compilation errors.

------

Also with $enabled checking if = yes or no, all i want it to do is. if $enabled = Yes then change $enabled to "ETC Applies" if $enabled = No then change $enabled to "No ETC"

morphix · 11-25-2007, 04:54 PM

Four: your updated script when running just creates 4-5 empty lines (i am testing output via echo command instead of curl to test).

theNbomr · 11-25-2007, 07:14 PM

Quote:

Originally Posted by morphix

theNbomr scripts works upto a point.

when i enter the CURL URL and run the script it gives me back the following errors.

$ perl script.pl script_txt.txt
Bareword found where operator expected at script.pl line 14, near ""curl -s -S -b cookie.txt -d "number"

Please post the line in question with your edits. I see nothing that prevents it from compiling, and it compiled fine in the form that I posted.
--- rod.