Script to read line by line from a file
 

Hi guys,

I have a file called test.txt which I would like to read line by line in a while loop and store the contents of every line in a variable, do some processing and again loop back to the next line until I reach at the end of the file(test.txt) through a shell script.

Basically, here is what I am looking for:

Read line of the file
store it in a temp variable
do some operation/testing
loop till the end of the file.





Thanks

That's not the most efficient way to process a text file. Can you post what you want to do with the text in the file? There are plenty of tools which will operate on all of the lines in a file rather than being called line by line (e.g. sed, grep, awk, etc.).

I would like to run the command cat /dev/null on the text (for example #cat /dev/null > file1.txt), since the text on each line is a file on the disk. I would really appreciate if you tell me how to read line by line through a shell script. I have tired the following command but it does not work, that's why I am asking this question
I have spent two entire days to get this working but I was not able to.

Thanks in advance

You need to pipe the read command to read line by line. Something like this:
In every execution of the while loop the $a_line variable will contain the next line.

If I understand you correctly you want to empty all the
files whose name & path are stored in list.txt ...
To achieve that you could just:

cat list.txt|xargs -i cat /dev/null > {}


Cheers,
Tink

Tink, You are exactly right, I want to achieve what you said, also I tried the command you suggested above but it did not work.

Any ideas why?

Kushal

I assume you are trying to create empty files with "cat /dev/null > somefile.txt". Do you realize that if a file already exists will be made empty (i.e. all content deleted) as well? If this is not what you want, or if it doesn't matter, just do:The code above will not truncate already existing files (but it will update the date/time as if they were modified).

If you do want to truncate an existing file:This will cause an error if an empty line exists in "test.txt", also if the last line is empty. To prevent that:

Hey Perfect Circle, thanks a lot. You made my day. You are really perfect. The suggestion you gave worked perfectly for me. Thanks a lot dude.

I wonder why
find /home/user/test -type f -exec cat /dev/null > {} \;
and
cat list.txt|xargs -i cat /dev/null > {}

do not work although they look ok to me..

Isn;t that strange?

Not without more input :}

What does the file look like internally, are the paths
to the files you want to empty absolute?


Cheers,
Tink

The file looks something like this:

They are not OK. Here's why:
This will will search for any file called "/home/user/test". There will be only one file found: /home/user/test.

So you could expect the the file containing the list of file ("list.txt") would be truncate to zero-length! But you were lucky this did not happen, because of another error:

When redirecting to {} , the -exec option does not replace {} with the files found by "find". So /dev/null is catted to a file called "{}". I bet you have a an empty file now in your directory called "{}".

The same problem here with redirecting to {}.

Also, "xargs" will put as many arguments from its standard input (stdin) as possible. So the command actually executed will be something like:...which isn't what you were expecting.

Bummer - it's my bad, the redirect within xargs doesn't work; the
output of xargs is being redirected to {} rather than redirecting the
output to the substituted filename. Sorry to get your hopes up :/

Alternatively create a script
#!/bin/bash
cat /dev/null > $1

and feed the cat to that ... :}

cat list.txt|xargs -i empty {}
works ...



Cheers,
Tink

I've never heard of the "empty" command. And it it's not available on my computer (Debian sarge). What distro do you use?

That's empty ;)


Cheers,
Tink

Right.
I missed that part.
I should have read your post less quickly.

whooo!! That's so much to learn. Thank you guys.

Truly said "When you work with linux, everyday you learn something new"...:D

Nuh, it's my fault ... I never said what to call it :}


And I sure am guilty of that one too often! ;D

Cheers,
Tink

Sorry to ask the question again, actually, I noticed something in Hko's reply which I didn;t before.

Hko, you are right I do see file '{}' in my directory.

But, then why does the command
works fine. Also in man find, they actually do say that {} is replaced by the name of the files found in the directory from which you run the find command.

???

This is because of a limit of the "-exec" option of "find": "find" doesn't do the replacement after a '>'. The redirecting to a file is done before replacing '{}'.

So you could do:But not:There may be a "hack" to do it work around it. But I don't know off any.

I don't think it matters where the redirect is in the command. It's being applied for the 'find' not the command in the exec statement.

The easiest way around this is to dump the exec stuff into a sub-shell. For example:

That works on my system anyways...

Yes. That does make sense.
Thanks for clearing that up.