Merging log files in the created order
 

Hi...

I want a shell script to merge all files into a single file which are exist in a directory , But the merging should be done in the order of files created.

ex: there are 5 files in a folder i.e
a.txt -- > Created at 7.00 pm
b.txt -- > Created at 7.10 pm
c.txt -- > Created at 7.20 pm
d.txt -- > Created at 7.30 pm
e.txt -- > Created at 7.40 pm

Note:- The files are dynamically comes form the system and we do not know how many files will be there

Now all these files should be merge into a single file in the same order.

Can any one help me on this.

cat a.txt >new.txt
cat b.txt >>new.txt
cat c.txt >>new.txt

The first line creates (or overwrites if it already exists) new.txt with the single redirect.
The following lines with double redirect append to the file new.txt. If run in order they will append in order.

If they are fairly large you might want to put sleep statements in between.

If these files grow large you may wish to delete the originals after doing the append. You DO want to make sure the filesystem into which your putting your new.txt has enough space to handle this operation.

If a cron job is creating each of the txt files you could modify it so it does the append after creating the next file.

If they are in collating sequence, a simple for "i in *" loop should suffice. If not "ls" has options for time ordering.

Hi Santhoo87545 and welcome to LQ.

There are some excellent ideas offered. What you should do at this point is start writing your script.

Please note that LQ members are volunteers and we are here to help, however the customary way of this forum is to enable you to complete these tasks on your own so that you learn from them. Therefore no one here should be just writing a script for you, however we're happy to review what you have and offer suggestions as to what directions to take with it.

For starters, do you have a script language in mind? Such as bash, or other?

And do you have skills and knowledge to write a script?

I think this problem breaks down into two parts: (1) identifying and sorting the input files by time/date (2) concatenating them to a single file (which incidentally you don't end up adding that new file to your concatenation ...)

Please update us with some more information about your script language choice and any attempts you have put forth to trying to solve this problem.

Perhaps just one command
?
I have given another directory for new.txt, so it will not match if the command is repeated.

There is no guarantee *.txt would list the files in order as the OP requested.