Help reading numbers from file with bash.

kesi · 12-01-2011, 06:55 PM

I'm trying to take numbers from a large text file that consists of a number on each line and then using them in curl to retrieve some files.

What I've been doing is:

FILE=filename.txt
for line in $(cut -d: -f1 $FILE)
do
curl 'http://mydomain.com/pet.php?num={$line}&date=2011 -o fileout.xml
done

The curl line works if I just copy all of the numbers in, comma-delimited, but in this format doesn't seem to grab the info.

Also, if I replace curl with echo, it prints all of the numbers to the screen.
Suggestions? Thanks!

jb_gpk · 12-01-2011, 07:21 PM

it seems to work fine here...

what result do you got? any error message?

the script that you are executing have this singe quote before http? ('http) :P

kesi · 12-01-2011, 07:23 PM

Quote:

Originally Posted by jb_gpk

it seems to work fine here...

what result do you got? any error message?

the script that you are executing have this singe quote before http? ('http) :P

That was a typo. It has a single quote before and after the link. It doesn't seem to be inserting the info because it doesn't pull the page and just overwrites the same file over and over. If I insert the number manually, it works.

kesi · 12-01-2011, 07:27 PM

Quote:

Originally Posted by kesi

That was a typo. It has a single quote before and after the link. It doesn't seem to be inserting the info because it doesn't pull the page and just overwrites the same file over and over. If I insert the number manually, it works.

Basically, there's a default page supplied when no id number is provided and that's the one I get. Also, if I change the output to 2011_#1.xml and have the numbers manually entered, it will name each file like such number=1234 file=2011_1234.xml. In this case, it's named 2011_$line.xml

kesi · 12-01-2011, 07:39 PM

Quote:

Originally Posted by jb_gpk

it seems to work fine here...

what result do you got? any error message?

the script that you are executing have this singe quote before http? ('http) :P

Aha! It does seem to be the quotes causing the variable to not be read. I had them there because some of the link have ampersands but it's easier to just escape them.

Thanks!

jb_gpk · 12-01-2011, 07:43 PM

when you put quotes what is inside them is treated as a "string"

kesi · 12-01-2011, 07:45 PM

Quote:

Originally Posted by jb_gpk

when you put quotes what is inside them is treated as a "string"

Yep. Thanks.

Cameron04 · 12-01-2011, 07:50 PM

Hi, i am a little comfused about what you are talking about.

kesi · 12-01-2011, 07:55 PM

Quote:

Originally Posted by Cameron04

Hi, i am a little comfused about what you are talking about.

What are you confused about?

David the H. · 12-01-2011, 09:00 PM

Please use [code][/code] tags around your code and data, to preserve formatting and to improve readability.

It usually helps to an example of the input text as well, when diagnosing problems.

Also, you shouldn't read lines with a for loop. A while+read loop is safer and more efficient. It also lets you avoid using cut.

Code:

FILE=filename.txt

while IFS=':' read -r line trash; do
	curl "http://mydomain.com/pet.php?num={$line}&date=2011" -o fileout.xml
done <"$FILE"

The trash variable will catch everything in the line after the first column.

Also, using double-quotes around the string escapes most characters, but still treats "$" as special, allowing variables to expand. Single quotes escape everything.

See the quoting section of the bash man page, and here:
http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes