LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-12-2012, 03:26 AM   #1
Trd300
Member
 
Registered: Feb 2012
Posts: 89

Rep: Reputation: Disabled
General: selecting input files and applying a command automatically


Hi !

A very general question.
How could we select 2 input files based on their names and apply any command to them automatically?

I have a directory containing 2,000 files:
Code:
file1_red.tab
file1_red_info.temp
file2_blue.tab
file2_blue_info.temp
file3_green.tab
file3_green_info.temp
...
file1000_black.tab
file1000_black_info.temp
I want to apply automatically a command (let's say the "join" command) to specific pairs of files, both containing the same "number" and the same "color".
Just to avoid writing 1,000 times the same command with different files,
Code:
join file1_red.tab file1_red_info.temp
join file2_blue.tab file2_blue_info.temp
join file3_green.tab file3_green_info.temp
...
join file1000_black.tab file1000_black_info.temp
is there any way to use the names of files to run this join (or another) command?
Maybe using a conditional expression (in a for loop) in this spirit:
Code:
# if ( 2 file names contain the same "number" and "color")
    then (apply join command to them)
I know awk has a built-in variable "FILENAME", but it would mean that all the command we want to apply to these files must be written with awk, which is sometimes complicated .

Last edited by Trd300; 03-12-2012 at 04:20 AM.
 
Old 03-12-2012, 05:06 AM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,243

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Do we know the pattern is always the same?
Example:
Code:
file1_red.tab
file1_red_info.temp
If the 2 pieces in red are always the same (obviously colour and number changing for new pairs), then simply use something like parameter substitution:
Code:
FILE=file1_red.tab

join $FILE ${FILE%.tab}_info.temp
Simple for loop can feed the variable and you may wish to test that the other file exists first.
 
1 members found this post helpful.
Old 03-12-2012, 05:48 AM   #3
Trd300
Member
 
Registered: Feb 2012
Posts: 89

Original Poster
Rep: Reputation: Disabled
When I write:

Code:
for FILE=file*_*.tab
do
    join $FILE ${FILE%.tab}_info.temp
done
or:
Code:
for FILE=file*_*.tab
do
    join ${FILE} ${FILE%.tab}_info.temp
done
in test.sh, and run the file:
Code:
sh test.sh
It returns for both scenario:
Code:
test.sh: line 4: `FILE=f*_*.tab': not a valid identifier
 
Old 03-12-2012, 05:55 AM   #4
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,576
Blog Entries: 31

Rep: Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195
Try for FILE in file*_*.tab

It would be prudent to change the command to echo join $FILE ${FILE%.tab}_info.temp so you can see what it is going to do.

The { } in ${FILE} is not wrong but neither is it necessary.
 
1 members found this post helpful.
Old 03-12-2012, 06:11 AM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,243

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
I think you need to read up a bit on things as your attempt was a guess.

Here are some links to help you:

http://tldp.org/LDP/abs/html/
http://mywiki.wooledge.org/TitleIndex
 
Old 03-12-2012, 06:18 AM   #6
Trd300
Member
 
Registered: Feb 2012
Posts: 89

Original Poster
Rep: Reputation: Disabled
When I try:
Code:
for FILE in file*_*.tab
do
join $FILE ${FILE%.tab}_info.temp > ${FILE%.tab}_final.tab
done
only with 4 files:
Code:
file1_red.tab
file1_red_info.temp
file2_blue.tab
file2_blue_info.temp
I get,
file1_red_final.tab which is correct,

but file2_blue_final.tab contains (with or without "echo"):
Code:
join $FILE ${FILE%.tab}_info.temp > ${FILE%.tab}_final.tab
 
Old 03-12-2012, 06:32 AM   #7
Trd300
Member
 
Registered: Feb 2012
Posts: 89

Original Poster
Rep: Reputation: Disabled
Found it !

Code:
for i in file*_*.tab
do
    j=`basename ${i} .tab`
    join ${i} ${j}_info.temp > ${j}_final.tab
done

Last edited by Trd300; 03-12-2012 at 06:34 AM.
 
Old 03-12-2012, 09:22 AM   #8
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,243

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Well, glad you found your answer although I see no difference between the 2 scripts and they both work as expected for me.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
cp command failing for large files - Input/Output error googs Linux - General 5 08-09-2012 01:28 PM
Evolution not automatically applying filters... xmnemonic Linux - Software 27 07-28-2011 08:36 AM
Applying a script to all the files of the find command flamingo_l Programming 39 10-26-2010 03:48 AM
rsync error: errors selecting input/output files, dirs (code 3) at main.c(323) cucolin@ Linux - Networking 3 11-20-2007 05:56 PM
cp command: how can I make it not automatically overwrite files? hamish Linux - Software 6 11-18-2006 12:30 PM


All times are GMT -5. The time now is 06:07 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration