How can I read a file line by line and add it to a loop in another file?

astroumut · 08-20-2009, 11:06 AM

Hi All,

I have a problem for making a for or a while loop to the following example.

I have two files

------------------------
filelist.dat, which contains
-----------------------
dummy1.dat
dummy2.dat
dummy3.dat
...
------------------------------
and
------------------------------
conversion.sh which contains (it is longer but nevermind),
------------------------------
set listcube = dummy1.dat
@ i = 0
while ($i <= 3)
@ j = 0
while ($j <= 3)
echo "Finished ${listcube}_${i}_${j}.dat"
@ j ++
end
@ i ++
end
------------------------------------

My question is, I have a long list of files like dummyx.dat and I just want read one line from filelist.dat and put this line to the the first line of conversion.sh file like
set listcube = dummy1.dat,
then the rest is the same in the conversion.sh
set listcube = dummy2.dat
then the rest is the same in the conversion.sh

How can I make a loop for that.
I hope it is clear.

Thanks for any help.

Umut

catkin · 08-20-2009, 12:46 PM

Which language are you writing in? Your file name ends in .sh but it doesn't look like shell script.

It would be easier to read your code if you put CODE tags around it. That's CDODE and /CODE enclosed in [] brackets. If you go into Advanced edit mode, you can use the # icon to automate code tags.

astroumut · 08-20-2009, 03:22 PM

Hi,
I actually run it from the command window with only ./conversion.sh . So it should be either bash or shell script.

OK, I am writing it how you wanted.

filelist.dat

Code:

dummyfile1.dat
dummyfile2.dat
dummyfile3.dat
...

conversion.sh

Code:

set listcube = dummyfile1.dat
@ i = 0
while ($i <= 3)
@ j = 0
while ($j <= 3)
echo "Finished ${listcube}_${i}_${j}.dat"
@ j ++
end
@ i ++
end

The filenames in the filelist.dat file should be read one by one in the conversion.sh file and put in a loop in it.

In the end, it should do the task like that. But with a loop it is much easier of course.

Code:

set listcube = dummyfile1.dat
@ i = 0
while ($i <= 3)
@ j = 0
while ($j <= 3)
echo "Finished ${listcube}_${i}_${j}.dat"
@ j ++
end
@ i ++

set listcube = dummyfile2.dat
@ i = 0
while ($i <= 3)
@ j = 0
while ($j <= 3)
echo "Finished ${listcube}_${i}_${j}.dat"
@ j ++
end
@ i ++

....
end

But I could not do the loop.
Please help.

Thanks

jschiwal · 08-20-2009, 03:31 PM

Since you are reading the filelist.dat file line by line:
while read filename; do
# do stuff. The variable $filename is used.
done < filelist.dat

You can also use "$(wc -l)" to return the number of lines in the file if you insist on a for loop.

chrism01 · 08-20-2009, 06:18 PM

What are the '@' at the start of the lines?

catkin · 08-21-2009, 06:14 AM

Quote:

Originally Posted by astroumut

conversion.sh

Code:

set listcube = dummyfile1.dat
@ i = 0
while ($i <= 3)
@ j = 0
while ($j <= 3)
echo "Finished ${listcube}_${i}_${j}.dat"
@ j ++
end
@ i ++
end

Ignoring the "@"s pending your answer to chrism01's question ...

Code:

set listcube = dummyfile1.dat

That sets $1 to listcube, $2 to = and $3 to dummyfile1.dat. I guess you wanted to set $listcube to dummyfile1.dat in which case you can use

Code:

listcube=dummyfile1.dat

It is important that there are no spaces either side of the "=".

Code:

while ($i <= 3)

has the effect of running $i <= 3 in a subshell and testing the exit status from the subshell which is always going to be 1 (error, false). In the subshell, bash first processes the redirection operator < and finds it is being asked to redirect input from a file called =. Probably there is no such file and this will result in an error. I guess you want to loop while $i is less than or equal to 3. This will do it

Code:

while [[ "$i" -le 3 ]]
do
    <commands>
done

The double quotes around $i are defensive, in case it is empty, unset or contains only whitespace. The "-le" is an arithmetic comparison operator ("<=" is a string comparison operator). The "do" and "done" define the loop body and are essential after while to complete the "compound command".

It seems you are trying to write shellscript by guesswork rather than studying how to do it and that aint never going to work; shellscript is not intuitive. Better you netsearch for bash shellscript introductions (there are lots out there) and study some examples. The Advanced Bash Scripting guide has lots of examples.

astroumut · 08-24-2009, 03:44 AM

Thanks a lot for the answers. I solved my problem and realized my mistake too.

In the beginning of the conversion.sh file there is

Code:

#!/bin/csh -f

so that is indeed a shell script, and the commands that you were written did not work. But when I changed it to

Code:

#!/bin/bash

then your scripts work but mine did not work. So I tried to find a middle way between. I have no idea what the @'s do, because I took it from the standard conversion file, but it looks like it has a meaning in shell.

Thanks for your answers

colucix · 08-24-2009, 04:37 AM

The @ in csh/tcsh is a kind of set statement to assign the value of an arithmetic expression to a variable. It let you do C-style increment and decrement, as in

Code:

@ var++
@ var--

Regarding your question, reading a file line-by-line in csh is one of the top 10 reasons to not use C shell programming. Take in mind that in csh you don't have a read statement and you can read only from the standard input. If for some reason the standard input changes during the execution of the script, the content of the file is not parsed anymore. In other words you cannot read from standard input as "terminal input" and as "file input" simultaneously.

In any case, here is a version of your script that should work:

Code:

#!/bin/csh -f
set listcube = $<
set i = 0
while ( $listcube != "" )
  @ i++
  echo line number $i is: $listcube
  set listcube = $<
end

the $< notation means "the next line of input": this is the way to read standard input line by line in csh. As you can notice there is no way to redirect the standard input from a file, unless you explicitly run the script using input redirection:

Code:

./conversion.sh < filelist.dat