LinuxQuestions.org - Loop line by line script

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Loop line by line script (https://www.linuxquestions.org/questions/programming-9/loop-line-by-line-script-277869/)

Loop line by line script

Hi there,

Is there a way I can store a line in a text to a variable? For example:

text.txt contains:
this is line one
this is line two
this is line three

If you do:

for i in `cat text.txt`; do echo $i; done

Variable 'i' will contain the words (this, is, line, one, etc...) but not the entire line itself. How would one accomplish this task? Thanks for your help!

-twantrd

Check out the man page for bash, and look up the "read" command. You can do what you want with this:

Code:

while read line ; do

  echo ${line}

done < test.txt

Your basic problem is that you need to use double quotes around the variable. The same problem would occur if you have files with whitespace in the filenames:

for file in ~/mp3dir/*.mp3; do mv ${file} ${file// /_}; done

The above example could have a problem because after ${file} is evaluated, it is expanded into several arguments instead of one for filenames containing whitespace characters. So instead you need to use:

for file in ~/mp3dir/*.mp3; do mv "${file}" "${file// /_}"; done

Quote:

Originally posted by jschiwal
for file in ~/mp3dir/*.mp3; do mv "${file}" "${file// /_}"; done

Wouldn't the for loop get confused even before the mv takes place? I'm not on my linux machine, so I can't test that right now. What I'm thinking is, if the goal is to replace a filename that has a spaces with underscores, then "~/mp3dir/*.mp3" will expand to something like:

~/mp3dir/some song name.mp3
~/mp3dir/some other song name.mp3
etc.

And that would mean the file variable would proceed, in sequence, like this:
~/mp3dir/some
song
name.mp3
~/mp3dir/some
other
song
name.mp3

Again, I'm not on my linux box to test that. Assuming this is supposed to be part of a shell script, the simplest way I've managed to do things like this is to temporarily change the IFS variable from it's default value to only containing newlines.

THank you guys. THat did the trick for me!

-twantrd

In your example filename, "${file}" will expand to

"/home/<username>/mp3dir/some song name.mp3"

Also, the double slashes in "${file// /_}" will replace every space and not just the first one.

There could still be a problem if there are other special characters, such as '!' in the filename. A script saved in ~/bin should be written to convert all whitespace and special characters. Some characters such as '!' will need to be backslashed, i.e. mv "${file}" "${file//\!/_}"

Here is proof:

Code:

jschiwal@linux:~/test> touch "abc def ghi"

jschiwal@linux:~/test> touch "ab cd ef gh"

jschiwal@linux:~/test> for file in a*; do echo "${file}"; done

ab cd ef gh

abc def ghi

jschiwal@linux:~/test> for file in a*; do echo "${file// /_}"; done

ab_cd_ef_gh

abc_def_ghi





jschiwal@linux:~/test> touch 'abc!def'

jschiwal@linux:~/test> touch 'abdef!!'



jschiwal@linux:~/test> for file in *\!*

> do mv "${file}" "${file//\!/_}" -v

> done

`abc!def' -> `abc_def'

`abdef!!' -> `abdef__'

Sounds good to me :)

What I wasn't sure of was how the for loop would treat the * expansion. Your example shows a behavior that goes against what bash does 90% of the time. That, or there's something I'm missing in understanding its behavior. In the for loop, bash would break the list up on spaces. So, given your example, bash must be making an exception when it comes to using the * wildcard. It must be doing something internally because they are filenames. As an example, if I were to do this:

Code:

$ ls /home/someuser/mp3dir > file_list

$ for file in $( cat file_list ) ; do echo ${file} ; done

some

song

name.mp3

Same information, different way of getting it to the shell, but indicates that filename expansion is treated as a special case. It's convenient, but I don't know if I like it (I'm a consistency nut).

Quote:

for file in $( cat file_list )

The results of $( cat file_list ) will be a stream of characters containing spaces and return characters.
Example: 'song one.mp3\nsong two.mp3'. As a result, if the first file name in the list was "song one.mp3", the variable $file would contain "song" because the space would act as a delimiter. If you wanted this to work, you would have to change the IFS variable so it didn't contain a space. However this might result in some commands like 'ls' giving errors. E.g. IFS=\n; for file in `ls *.mp3`; do ... would error out.

Compare that with
for file in <pattern>;
For each iteration, the $file variable is set to the name of a file fitting the pattern. The arguments to the file command are expanded like "song one.mp3" "song two.mp3". The file name may contain spaces. Suppose the value of the variable is "song one.mp3". Then you need double quotes around the variable so the command using it doesn't break it up.

The difference between the two examples are between the use of 'in *.mp3' and $( cat file_list ). This is what actually caused the two different behaviors.

There is a similar difference between using $* and $@.

For general purpose scripts, there would be a lot to consider. Especially since the source of the file may be from some other type of system with different legal characters in filenames. For example, a filename 'test/song.mp3' would be handled as the file song.mp3 in the test/ directory. Also, as linux can mount foreign file-systems, what the legal length of a file name would be an issue. These issues would be present however whatever kind of script or programming language was used.