LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 08-15-2011, 03:58 AM   #1
niharikaananth
Member
 
Registered: Aug 2011
Posts: 58

Rep: Reputation: Disabled
Merging many files as one


Hi.....I have around 71 files in a folder. I wanted to merge all these files as one. The file's numbers are numerically i.e 1_file.txt, 2_something.txt, 3_someohtername.txt & so on upto 71_lastfile.txt. I ran "cat * > ../notes.txt", but it is not coming serially, I mean if I run any command to merge all of these files as a one file, that the new file content should be started serially i.e 1_*.txt, 2_*.txt, 3_*.txt and so on. So could anybody guide me how can I merge all 71 files as a one file with serially. Otherwise I will have copy-paste single-single file which consume more time.
 
Old 08-15-2011, 04:22 AM   #2
sycamorex
LQ Veteran
 
Registered: Nov 2005
Location: London
Distribution: Slackware64-current
Posts: 5,811
Blog Entries: 1

Rep: Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191
Try >> instead of >
This will append to a file.
 
Old 08-15-2011, 07:05 AM   #3
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,256

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
If I understand correctly the issue will be the order they are being passed to cat. Try using a sort did get them in the correct order and then pass them to cat.
 
Old 08-15-2011, 07:09 AM   #4
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Germany
Distribution: Whatever fits the task best
Posts: 17,130
Blog Entries: 2

Rep: Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825
Assuming that you are using Bash 3.0 or newer:
Code:
for i in {1..71}
  do
     cat $i*.txt >> newfile
  done
 
Old 08-15-2011, 07:13 AM   #5
sycamorex
LQ Veteran
 
Registered: Nov 2005
Location: London
Distribution: Slackware64-current
Posts: 5,811
Blog Entries: 1

Rep: Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191
Quote:
Originally Posted by grail View Post
If I understand correctly the issue will be the order they are being passed to cat. Try using a sort did get them in the correct order and then pass them to cat.
Ooops, I forgot about that.
 
Old 08-15-2011, 02:07 PM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
I'm assuming that the problem you're experiencing is that filename globbing is based on dictionary sorting, so you get sequences like this:

10 11 12... 18 19 1 20 21 ... 69 70 71 8 9

The only ways to get around this are to use sort with the numerical sorting option, some other technique for matching the actual sequence, like TobiSGD offered, or else to rename your files so that they are all zero-padded (or otherwise in alphanumeric order). I generally prefer the last myself, as it solves the problem permanently.

There are several batch renaming utilities out there for cleaning up filenames, and the topic comes up here regularly, so search around a bit. But here's a quick script I just whipped up that can handle simple jobs.
Code:
#!/bin/bash

shopt -s extglob     #needed for zero-stripping below

#loop through the files given to the script
#(you can use a glob, like "*.txt")
#defaults to globbing everything in the PWD
for file in ${@:-$PWD/*} ; do 

     #ignore any files without numbers
     [[ $file != *[0-9]* ]] && continue 

     #break the filename into (prefix)-(number)-(suffix).
     #the substrings are stored in the BASH_REMATCH array
     [[ $file =~ ([^[0-9]*)([0-9]+)(.*) ]]
                                            
     #pad the number (2 digits by default).  strip any existing
     #leading zeroes first, or bash will treat them as octal
     printf -v numpad "%02d" "${BASH_REMATCH[2]##*(0)}"

     #build the new filename
     newfile="${BASH_REMATCH[1]}${numpad}${BASH_REMATCH[3]}"

     #confirm the result. remove the echo to rename
     echo mv "$file" "$newfile"

done

exit 0
This assumes that the names have only a single number sequence in them. It separates the number string from the non-number strings before and after it, and pads it to 2 places (simply change "%02d" if you want more). Then it reassembles the pieces into a new filename. You can give it a list of files, or else it defaults to everything in the present working directory.

Finally, be careful with it. It might have unforeseen side-effects, so I've disabled the actual renaming operation. Don't remove the echo at the end until you've confirmed that it works.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] merging columns from different files leonardo2887 Linux - Newbie 4 10-11-2010 03:28 AM
plz help in merging files PKrishna Linux - Newbie 3 03-27-2008 02:57 AM
Merging ,pdf files satimis Linux - General 6 11-21-2007 09:31 PM
Merging Two Files using C++ ckoniecny Programming 5 09-26-2006 10:00 AM
merging log files help please digitalgravy Linux - Newbie 3 12-10-2003 03:26 PM


All times are GMT -5. The time now is 11:15 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration