LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 02-08-2013, 10:04 AM   #1
vdamgo
LQ Newbie
 
Registered: Feb 2013
Posts: 11

Rep: Reputation: Disabled
Sum of file sizes generated in particular month and year


Hi,

I am trying to find the sum of file sizes that ran in Jan 2012 from a directory and also group of directories. (folder has 1000's of files generated every month and every year, among all these files, i want to calculate the sum of file sizes in MB generated in Jan 2012, which also has files generated in Jan 2013 etc)

Is there any script to find the sum of filesizes that ran on particular month and year.

please let me know.

Thanks!
 
Old 02-08-2013, 10:08 AM   #2
schneidz
Senior Member
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-19-live-usb/ aix
Posts: 3,842

Rep: Reputation: 590Reputation: 590Reputation: 590Reputation: 590Reputation: 590Reputation: 590
i think the find command should help (-ctime or -newer mite be useful parameters).

what have you tried and where are you stuck ?

also the du command would probably help here.

Last edited by schneidz; 02-08-2013 at 10:11 AM.
 
Old 02-08-2013, 10:13 AM   #3
vdamgo
LQ Newbie
 
Registered: Feb 2013
Posts: 11

Original Poster
Rep: Reputation: Disabled
I tried the below statement

ls -lrt | grep "Jan " | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'

But I have files Jan 2012 and Jan 2013 files in the directory but I just need sum file size of Jan 2013 files only
 
Old 02-08-2013, 10:17 AM   #4
schneidz
Senior Member
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-19-live-usb/ aix
Posts: 3,842

Rep: Reputation: 590Reputation: 590Reputation: 590Reputation: 590Reputation: 590Reputation: 590
Quote:
Originally Posted by vdamgo View Post
I tried the below statement

ls -lrt | grep "Jan " | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'

But I have files Jan 2012 and Jan 2013 files in the directory but I just need sum file size of Jan 2013 files only
its probably not a good idea to parse the output of ls but maybe this will help:
Code:
ls -lrt | grep "Jan.*2012 " | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'
 
Old 02-08-2013, 10:50 AM   #5
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Using find - as previously suggested:
Code:
$ touch -t 201112312359 /tmp/file1
$ touch -t 201201312359 /tmp/file2
$ find . -type f -newer /tmp/file1 -not -newer /tmp/file2 -printf "%s\n" | awk '{sum += $0} END{print sum}'
$ rm /tmp/file1 /tmp/file2
 
1 members found this post helpful.
Old 02-08-2013, 10:51 AM   #6
vdamgo
LQ Newbie
 
Registered: Feb 2013
Posts: 11

Original Poster
Rep: Reputation: Disabled
I think it did not work, I tried as below but it returned 0

[testserver@forfinding-or]/abcd/efg/ehome/karti>ls -lrt
total 4
-rw-rw-r-- 1 pinto pinto 30 Feb 7 14:18 listfiles.txt
-rwxrwxrwx 1 pinto pinto 124 Feb 7 14:54 test_file.sh
-rw-rw-r-- 1 pinto pinto 0 Feb 7 15:05 ls
[testserver@forfinding-or]/abcd/efg/ehome/karti>ls -lrt | grep "Feb.*2013" | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'
0
 
Old 02-08-2013, 10:56 AM   #7
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Quote:
Originally Posted by vdamgo View Post
I think it did not work, I tried as below but it returned 0

[testserver@forfinding-or]/abcd/efg/ehome/karti>ls -lrt
total 4
-rw-rw-r-- 1 pinto pinto 30 Feb 7 14:18 listfiles.txt
-rwxrwxrwx 1 pinto pinto 124 Feb 7 14:54 test_file.sh
-rw-rw-r-- 1 pinto pinto 0 Feb 7 15:05 ls
[testserver@forfinding-or]/abcd/efg/ehome/karti>ls -lrt | grep "Feb.*2013" | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'
0
schneidz gave a warning: it's not a good idea to use the output of ls. The date format may vary and in this case, files more recent than 6 months don't show the year, therefore the grep statement returns nothing. You may use some options of ls to customize the appearance of the timestamp, but I would rely on the find solution, instead.
 
Old 02-08-2013, 10:59 AM   #8
vdamgo
LQ Newbie
 
Registered: Feb 2013
Posts: 11

Original Poster
Rep: Reputation: Disabled
Sorry, Am really poor in linux scripting, Do we need to keep the below statements in script and then execute

$ touch -t 201112312359 /tmp/file1
$ touch -t 201201312359 /tmp/file2
$ find . -type f -newer /tmp/file1 -not -newer /tmp/file2 -printf "%s\n" | awk '{sum += $0} END{print sum}'
$ rm /tmp/file1 /tmp/file2
 
Old 02-08-2013, 11:10 AM   #9
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Quote:
Originally Posted by vdamgo View Post
Sorry, Am really poor in linux scripting, Do we need to keep the below statements in script and then execute

$ touch -t 201112312359 /tmp/file1
$ touch -t 201201312359 /tmp/file2
$ find . -type f -newer /tmp/file1 -not -newer /tmp/file2 -printf "%s\n" | awk '{sum += $0} END{print sum}'
$ rm /tmp/file1 /tmp/file2
Not necessarily. You can run them on the command line. Take in mind that the leading $ sign is the command prompt (you don't have to type it). To test the results of the find command, run it without the -printf statement and without the awk part. Maybe using -ls, that is:
Code:
$ touch -t 201112312359 /tmp/file1
$ touch -t 201201312359 /tmp/file2
$ find . -type f -newer /tmp/file1 -not -newer /tmp/file2 -ls
May I suggest a good reading about the linux command line? Here is it: http://linuxcommand.org/tlcl.php

Last edited by colucix; 02-08-2013 at 11:12 AM.
 
Old 02-08-2013, 11:39 AM   #10
vdamgo
LQ Newbie
 
Registered: Feb 2013
Posts: 11

Original Poster
Rep: Reputation: Disabled
I think we have lost somewhere, here is example as below, I have four files in a directory. two files got generated in feb 2012 and another two files in Feb 2013, now i want to sum up the file size gennerated in feb 2012 so answer should be 85+30=115


[pin@test-omf]/abcd/def>ls -lrt
total 12
-rwxrwxrwx 1 pinot pinot 85 Feb 6 23:59 dummy1.txt (this file is generated in Feb 2012)
-rw-rw-r-- 1 pinot pinot 30 Feb 7 14:18 listfiles.txt (this file is generated in Feb 2012)
-rwxrwxrwx 1 pinot pinot 124 Feb 7 14:54 test_abcd.txt (this file is generated in Feb 2013)
-rwxrwxrwx 1 pinot pinot 86 Feb 8 12:02 dummy2.txt (this file is generated in Feb 2013)

The above is just an example, like wise I have 1000s of files in a directory which are mixed up like some are generated in year feb 2012 and some are generated in feb 2011, some are generated in feb 2013 etc, but am trying to find the sum of file sizes generated in feb 2012.
 
Old 02-08-2013, 12:24 PM   #11
Habitual
Senior Member
 
Registered: Jan 2011
Distribution: Slack14_64_Multilib
Posts: 3,074
Blog Entries: 4

Rep: Reputation: 745Reputation: 745Reputation: 745Reputation: 745Reputation: 745Reputation: 745Reputation: 745
Code:
ls -al | grep " 2012 " | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'
works every where I use it.
 
Old 02-09-2013, 05:37 AM   #12
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Quote:
Originally Posted by vdamgo View Post
I think we have lost somewhere
What's wrong with the touch and find solution? You have just to set the proper dates in the touch commands (my example was based on your initial requirement Jan 2012, changed later to Feb 2012).

Quote:
Originally Posted by vdamgo View Post
-rwxrwxrwx 1 pinot pinot 85 Feb 6 23:59 dummy1.txt (this file is generated in Feb 2012)
-rw-rw-r-- 1 pinot pinot 30 Feb 7 14:18 listfiles.txt (this file is generated in Feb 2012)
I don't think so. They appear to be created on February 2013. You can verify it using the stat command or the --full-time option of ls:
Code:
$ ls -l --full-time dummy1.txt listfiles.txt
$ stat dummy1.txt listfiles.txt
 
2 members found this post helpful.
Old 02-11-2013, 11:47 AM   #13
vdamgo
LQ Newbie
 
Registered: Feb 2013
Posts: 11

Original Poster
Rep: Reputation: Disabled
Thank you.

I think it worked.

$ touch -t 201112312359 /tmp/file1
$ touch -t 201201312359 /tmp/file2
$ find . -type f -newer /tmp/file1 -not -newer /tmp/file2 -printf "%s\n" | awk '{sum += $0} END{print sum}'
$ rm /tmp/file1 /tmp/file2

Can you please give me little explanation like whats is the procedure and what the script is performing.

Appreciate your help
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
LQ-er of the day/week/month/year? alan_ri LQ Suggestions & Feedback 0 01-24-2009 05:11 PM
How to write function to list file (*.log) follow feature time(date, month, year) phanvinhgiap Programming 2 12-30-2008 08:30 PM
SQL Query - sum / group by one year periods smaida Programming 2 03-09-2007 08:10 AM
Date command - month of year, blank padded? menator Programming 3 06-27-2006 06:00 AM
sum total of file sizes allelopath Linux - General 3 01-07-2006 08:30 AM


All times are GMT -5. The time now is 11:38 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration