LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   merge files in a directory (https://www.linuxquestions.org/questions/linux-newbie-8/merge-files-in-a-directory-4175587925/)

sysmicuser 08-25-2016 02:03 AM
merge files in a directory
 
Hey Guys,

I want to merge all files (Apache Tomcat Access Logs) for a particular date say "Aug 24" to be merged into a single file.

Is there any quick hack for that ?
Code:
[tomcat@localhost logs]$ ls -alrth access_log2016-08-*|grep "Aug 24"
-rw-rw-r--. 1 tomcat        tomcat          16M Aug 24 00:00 access_log2016-08-23.23.log
-rw-rw-r--. 1 tomcat        tomcat        6.8M Aug 24 01:00 access_log2016-08-24.00.log
-rw-rw-r--. 1 tomcat        tomcat        4.7M Aug 24 02:00 access_log2016-08-24.01.log
-rw-rw-r--. 1 tomcat        tomcat          14M Aug 24 03:00 access_log2016-08-24.02.log
-rw-rw-r--. 1 tomcat        tomcat          18M Aug 24 04:00 access_log2016-08-24.03.log
-rw-rw-r--. 1 tomcat        tomcat          15M Aug 24 05:00 access_log2016-08-24.04.log
-rw-rw-r--. 1 tomcat        tomcat        5.6M Aug 24 06:00 access_log2016-08-24.05.log
-rw-rw-r--. 1 tomcat        tomcat        8.9M Aug 24 07:00 access_log2016-08-24.06.log
-rw-rw-r--. 1 tomcat        tomcat          19M Aug 24 08:00 access_log2016-08-24.07.log
-rw-rw-r--. 1 tomcat        tomcat          32M Aug 24 09:00 access_log2016-08-24.08.log
-rw-rw-r--. 1 tomcat        tomcat          45M Aug 24 10:00 access_log2016-08-24.09.log
-rw-rw-r--. 1 tomcat        tomcat          44M Aug 24 11:00 access_log2016-08-24.10.log
-rw-rw-r--. 1 tomcat        tomcat          49M Aug 24 12:00 access_log2016-08-24.11.log
-rw-rw-r--. 1 tomcat        tomcat          51M Aug 24 13:00 access_log2016-08-24.12.log
-rw-rw-r--. 1 tomcat        tomcat          53M Aug 24 14:00 access_log2016-08-24.13.log
-rw-rw-r--. 1 tomcat        tomcat          52M Aug 24 15:00 access_log2016-08-24.14.log
-rw-rw-r--. 1 tomcat        tomcat          84M Aug 24 16:00 access_log2016-08-24.15.log
-rw-rw-r--. 1 tomcat        tomcat          57M Aug 24 17:00 access_log2016-08-24.16.log
-rw-rw-r--. 1 tomcat        tomcat          48M Aug 24 18:00 access_log2016-08-24.17.log
-rw-rw-r--. 1 tomcat        tomcat          37M Aug 24 19:00 access_log2016-08-24.18.log
-rw-rw-r--. 1 tomcat        tomcat          38M Aug 24 20:00 access_log2016-08-24.19.log
-rw-rw-r--. 1 tomcat        tomcat          40M Aug 24 21:00 access_log2016-08-24.20.log
-rw-rw-r--. 1 tomcat        tomcat          37M Aug 24 22:00 access_log2016-08-24.21.log
-rw-rw-r--. 1 tomcat        tomcat          26M Aug 24 23:00 access_log2016-08-24.22.log
[tomcat@localhost logs]$
They should be merged into single file as per date time in ascending order so access_log2016-08-23.23.log then access_log2016-08-24.00.log so on and so forth.

We can do manually like cat individual file but want to explore more smarter way. I heard from my colleague a command called plus to concatenate into single file ,but not 100% sure..

Please assist.

chrism01 08-25-2016 04:36 AM
Code:
#my files
 ls
f1  f2  f3

# contents
cat *
f1-data

f2-data

f3-data

# concat in date order (f1, f2, f3)
for file in $(ls -rt)
do
cat $file>>f4
done

cat f4
f1-data

f2-data

f3-data
:)

malekmustaq 08-25-2016 04:48 AM
Code:
cat file 1 >> file 2
contents of file 1 is appended to file 2.
Check it
Code:
cat file 2
If there are plenty of files you can FOR IN WHILE and DO script it.

grail 08-25-2016 10:23 AM
I would use your existing glob and test the file date information with stat and then cat those desired. You could also try feeding a while loop with find assuming you know how long ago you want to concatenate.

schneidz 08-25-2016 10:30 AM
Code:
cat access_log2016-08-24* > sysmicuser.log
?

keefaz 08-25-2016 11:55 AM
Code:
files=($(ls -alrth access_log2016-08-*|grep "Aug 24" | awk '{print $9}'))
cat "${files[@]}" > access_log2016-08_Aug24_merged.log
edit, I like schneidz simple suggestion though

grail 08-25-2016 12:49 PM
schneidz example is simple but in the example there is a file with 23 at the top which still has a 24th date on it.

Rinndalir 08-25-2016 12:56 PM
@schneidz is the correct answer.

keefaz 08-25-2016 02:44 PM
Quote:
Originally Posted by grail (Post 5595953)
schneidz example is simple but in the example there is a file with 23 at the top which still has a 24th date on it.
Yes with 00:00 modification time, personally I would go away counting it in the previous set (2016-08-23)


All times are GMT -5. The time now is 04:12 PM.