LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 11-28-2011, 04:58 PM   #1
bonissen
LQ Newbie
 
Registered: Nov 2011
Location: Norway
Distribution: Ubuntu
Posts: 3

Rep: Reputation: Disabled
Batch merging pairs of files in a directory to unique file names


Hi all LQ members !

I have a question regarding two sets of file types that I want to merge contained in a directory; file1-1000.sp and file1-1000.sp2 > file12.merged.
Here is an example how the merged files should look like, from the first pair in the dataset:

[loci]=10 ("from file1.sp")

[populations]=2

[pop]=1
1 18 2 13 5
2 14 2 10 4
3 22 2 12 10
4 21 2 21 0
5 13 2 13 0
6 13 2 9 4
7 21 2 21 0
8 14 2 14 0
9 17 2 17 0
10 14 2 7 7
[pop]=2 (from "file1.sp2)
1 18 2 11 7
2 14 2 8 6
3 17 2 5 12
4 26 2 26 0
5 14 2 14 0
6 11 2 5 6
7 12 2 9 3
8 19 2 19 0
9 26 2 26 0
10 18 2 8 10

The directory can contain thousands of files in pair, and need a unique out name for each file.1-1000.merged, for further analysis in bayescan (http://cmpg.unibe.ch/software/bayescan/).
Any ideas how to make this work ?

Regards bonissen
 
Old 11-28-2011, 05:41 PM   #2
Velotrol
LQ Newbie
 
Registered: Apr 2011
Location: Rio, Brazil
Distribution: Gentoo
Posts: 15

Rep: Reputation: Disabled
Hi,
if I understand correclty, you need something like that:

Code:
#!/bin/sh

for i in {1..1000}
do
   cat file$i.sp > file$i.merged
   cat file$i.sp2 >> file$i.merged
done
 
Old 11-28-2011, 06:58 PM   #3
bonissen
LQ Newbie
 
Registered: Nov 2011
Location: Norway
Distribution: Ubuntu
Posts: 3

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Velotrol View Post
Hi,
if I understand correclty, you need something like that:

Code:
#!/bin/sh

for i in {1..1000}
do
   cat file$i.sp > file$i.merged
   cat file$i.sp2 >> file$i.merged
done
Thanks for the reply Velotrol !

I got it to work with this small modification, this counter {1..1000} did not work for me.

#!/bin/sh

for i in 1 2 3

do
cat $i.sp > file$i.merged
cat $i.sp2 >> file$i.merged
done


Bonissen
 
Old 11-28-2011, 07:49 PM   #4
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,627

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
Try using bash instead of sh and the sequence should work for you.
 
Old 11-28-2011, 10:40 PM   #5
bonissen
LQ Newbie
 
Registered: Nov 2011
Location: Norway
Distribution: Ubuntu
Posts: 3

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by grail View Post
Try using bash instead of sh and the sequence should work for you.
Thanks both !

Now it works nicely.

#!/bin/bash

#script for merging pair of files in a dir

for i in {1..1000}
do
cat $i.sp > file$i.merged
cat $i.sp2 >> file$i.merged
done
 
Old 11-29-2011, 02:57 PM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
You generally shouldn't need two cat commands here.

Code:
for i in {1..1000} ; do
     cat $i.sp $i.sp2 > file$i.merged
done
The only problem might be if your files don't contain final newlines, then you'd have to insert a line-break command between the two, probably with a process substitution.

Code:
for i in {1..1000} ; do
     cat $i.sp <(echo) $i.sp2 > file$i.merged
done
http://wiki.bash-hackers.org/syntax/expansion/brace
http://mywiki.wooledge.org/ProcessSubstitution
http://partmaps.org/era/unix/award.html
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Use Linux script to batch change file names in a folder jiapei100 Linux - General 3 11-09-2009 06:33 AM
Batch script remove old files based on their names and keep on for each version Reda01 Programming 6 05-08-2008 03:35 PM
copying files and give new unique names to each file by using xargs command gnim66 Programming 6 06-22-2005 09:29 PM


All times are GMT -5. The time now is 06:17 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration