LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 08-19-2005, 04:39 PM   #1
mijohnst
Member
 
Registered: Nov 2003
Location: Huntsville, AL
Distribution: RHEL, Solaris, OSX, SuSE
Posts: 401

Rep: Reputation: 31
Reading a file and running a command for each line.


Can anyone give me an example of how to run an argument past each line of a file and run a command? For example...

I have a file with these lines in it at /home/mijohnst/filelist.txt

# A list of Files
/home/mijohnst/dir1/testfile1.sh
/home/mijohnst/dir2/testfile2.sh
/home/mijohnst/dir3/testfile3.sh
/home/mijohnst/dir4/testfile4.sh
/home/mijohnst/dir5/testfile5.sh
/home/mijohnst/dir6/testfile6.sh

What command would I run to read each line of this file and run a command? Say that I wanted to copy all of those files into the /tmp directory...

I'm thinking something like "cat /home/mijohnst/filelist.txt | ..." something...

I know this is easy, but I'm really drawing a blank...

Thanks in advance!
 
Old 08-19-2005, 04:53 PM   #2
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,593
Blog Entries: 2

Rep: Reputation: 326Reputation: 326Reputation: 326Reputation: 326
cat /home/mijohnst/filelist.txt | awk '{print "somecommand",$1}' | bash
 
Old 08-19-2005, 05:07 PM   #3
mijohnst
Member
 
Registered: Nov 2003
Location: Huntsville, AL
Distribution: RHEL, Solaris, OSX, SuSE
Posts: 401

Original Poster
Rep: Reputation: 31
Thanks macemoneta! For some reason I don't remember doing this at all...
 
Old 08-19-2005, 05:22 PM   #4
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,593
Blog Entries: 2

Rep: Reputation: 326Reputation: 326Reputation: 326Reputation: 326
There are probably a dozen ways (or more) to accomplish the same thing. I prefer to remember the use of a few commands (grep, cut, paste, sed, awk, tr), and find that I can accomplish almost anything I need with them. There are more elegant ways to do things, but they tend to use "one trick ponies" instead of more flexible tools.
 
Old 08-19-2005, 05:55 PM   #5
mijohnst
Member
 
Registered: Nov 2003
Location: Huntsville, AL
Distribution: RHEL, Solaris, OSX, SuSE
Posts: 401

Original Poster
Rep: Reputation: 31
Well, Linux hasn't been my main OS until recently so I haven't mastered these commands yet. I'm learning as I go thanks to people like you.

I'm trying to understand the flow of the command you just showed me, but I guess that I'm thinking of it wrong.

When I do:

cat /home/mijohnst/filelist.txt | awk '{print "cp /tmp",$1}' | bash

I get an error for each line that says "Omitting /tmp" and nothing is copied... Did I misunderstand what you were trying to tell me?
 
Old 08-19-2005, 09:42 PM   #6
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,593
Blog Entries: 2

Rep: Reputation: 326Reputation: 326Reputation: 326Reputation: 326
The command, as you entered it, is trying to do the following:

cp /tmp /home/mijohnst/dir1/testfile1.sh
...

Leave off the last pipe (" | bash") to see the results before you try to execute.

If you want to copy each of the files to /tmp, then you need to rework the pipe to:

cat /home/mijohnst/filelist.txt | awk '{print "cp",$1,"/tmp"}' | bash

Each entry in the file /home/mijohnst/filelist.txt is passed to awk. It will print "cp" a space (because of the comma) the first space delimited word from the file (for example, "/home/mijohnst/dir1/testfile1.sh") another space and then "/tmp".

Run this instead:

cat /home/mijohnst/filelist.txt | awk '{print "cp",$1,"/tmp"}'

Without the final bash command, the output of awk will simply be displayed at the terminal. Once you have the commands the way you want them, you can send them to bash for execution by adding the pipe to a bash shell " | bash".
 
Old 08-20-2005, 09:42 AM   #7
/bin/bash
Senior Member
 
Registered: Jul 2003
Location: Indiana
Distribution: Mandrake Slackware-current QNX4.25
Posts: 1,802

Rep: Reputation: 46
for i in `cat filelist`;do echo "$i";done

Replace (echo "$i") with the command you want to run.
I use this command all the time to work on multiple files.
 
Old 08-21-2005, 10:20 AM   #8
eddiebaby1023
Member
 
Registered: May 2005
Posts: 378

Rep: Reputation: 33
Quote:
Originally posted by macemoneta
cat /home/mijohnst/filelist.txt | awk '{print "somecommand",$1}' | bash
Why are you guys so obsessed with cat()ing files into filters that take filename arguments?
Code:
awk '{print "somecommand",$1}' /home/mijohnst/filelist.txt | bash
Anyway, if the file only has the commands in it (as posted)
Code:
bash </home/mijohnst/filelist.txt
is the easiest way to run it.
 
Old 08-21-2005, 02:49 PM   #9
/bin/bash
Senior Member
 
Registered: Jul 2003
Location: Indiana
Distribution: Mandrake Slackware-current QNX4.25
Posts: 1,802

Rep: Reputation: 46
Quote:
eddiebaby1023
Anyway, if the file only has the commands in it (as posted)

bash </home/mijohnst/filelist.txt

is the easiest way to run it.
The way I read the question

Quote:
I have a file with these lines in it at /home/mijohnst/filelist.txt

# A list of Files
/home/mijohnst/dir1/testfile1.sh
/home/mijohnst/dir2/testfile2.sh
/home/mijohnst/dir3/testfile3.sh
/home/mijohnst/dir4/testfile4.sh
/home/mijohnst/dir5/testfile5.sh
/home/mijohnst/dir6/testfile6.sh

What command would I run to read each line of this file and run a command? Say that I wanted to copy all of those files into the /tmp directory...
mijohnst doesn't want to run all the scripts, but wants to run a command on each one, such as copy them to the /tmp directory
 
Old 08-22-2005, 11:54 AM   #10
mijohnst
Member
 
Registered: Nov 2003
Location: Huntsville, AL
Distribution: RHEL, Solaris, OSX, SuSE
Posts: 401

Original Poster
Rep: Reputation: 31
That's exactly right /bin/bash.

Thank you guys so much! Your help has provided the exact means of getting my job done and it works!

I need a good linux command example site for newbies so that I don't have to post so many dumb questions. Thanks again all!
 
Old 08-22-2005, 12:11 PM   #11
Matir
Moderator
 
Registered: Nov 2004
Location: San Jose, CA
Distribution: Ubuntu
Posts: 8,507

Rep: Reputation: 116Reputation: 116
I think this is simpler, and allows for comments in the source file:
Code:
egrep -v '^\W*#' /SOME/FILE | xargs -i cp {} /tmp
 
Old 08-22-2005, 06:18 PM   #12
/bin/bash
Senior Member
 
Registered: Jul 2003
Location: Indiana
Distribution: Mandrake Slackware-current QNX4.25
Posts: 1,802

Rep: Reputation: 46
Of course LQ has many HOWTO's and tutorials available. And here are a few more.

Linux-tip.net

SuperMan Pages

Rute User's Tutorial and Exposition

Linux StepByStep.org
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
File reading line by line and compare Goni Linux - Software 14 09-21-2005 12:24 AM
Pop3 Command stream end of file while reading line sesimonsen Linux - Networking 0 05-13-2005 12:29 PM
reading from a file line by line Menestrel Programming 5 12-15-2004 02:17 AM
reading a line at a time in a file topcat Programming 2 12-14-2004 05:49 AM
reading a single line of a text file davi_cabral Linux - Software 1 10-29-2003 12:24 PM


All times are GMT -5. The time now is 01:35 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration