LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (http://www.linuxquestions.org/questions/linux-general-1/)
-   -   Sum of file sizes generated in particular month and year (http://www.linuxquestions.org/questions/linux-general-1/sum-of-file-sizes-generated-in-particular-month-and-year-4175449183/)

vdamgo 02-08-2013 10:04 AM

Sum of file sizes generated in particular month and year
 
Hi,

I am trying to find the sum of file sizes that ran in Jan 2012 from a directory and also group of directories. (folder has 1000's of files generated every month and every year, among all these files, i want to calculate the sum of file sizes in MB generated in Jan 2012, which also has files generated in Jan 2013 etc)

Is there any script to find the sum of filesizes that ran on particular month and year.

please let me know.

Thanks!

schneidz 02-08-2013 10:08 AM

i think the find command should help (-ctime or -newer mite be useful parameters).

what have you tried and where are you stuck ?

also the du command would probably help here.

vdamgo 02-08-2013 10:13 AM

I tried the below statement

ls -lrt | grep "Jan " | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'

But I have files Jan 2012 and Jan 2013 files in the directory but I just need sum file size of Jan 2013 files only

schneidz 02-08-2013 10:17 AM

Quote:

Originally Posted by vdamgo (Post 4887097)
I tried the below statement

ls -lrt | grep "Jan " | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'

But I have files Jan 2012 and Jan 2013 files in the directory but I just need sum file size of Jan 2013 files only

its probably not a good idea to parse the output of ls but maybe this will help:
Code:

ls -lrt | grep "Jan.*2012 " | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'

colucix 02-08-2013 10:50 AM

Using find - as previously suggested:
Code:

$ touch -t 201112312359 /tmp/file1
$ touch -t 201201312359 /tmp/file2
$ find . -type f -newer /tmp/file1 -not -newer /tmp/file2 -printf "%s\n" | awk '{sum += $0} END{print sum}'
$ rm /tmp/file1 /tmp/file2


vdamgo 02-08-2013 10:51 AM

I think it did not work, I tried as below but it returned 0

[testserver@forfinding-or]/abcd/efg/ehome/karti>ls -lrt
total 4
-rw-rw-r-- 1 pinto pinto 30 Feb 7 14:18 listfiles.txt
-rwxrwxrwx 1 pinto pinto 124 Feb 7 14:54 test_file.sh
-rw-rw-r-- 1 pinto pinto 0 Feb 7 15:05 ls
[testserver@forfinding-or]/abcd/efg/ehome/karti>ls -lrt | grep "Feb.*2013" | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'
0

colucix 02-08-2013 10:56 AM

Quote:

Originally Posted by vdamgo (Post 4887125)
I think it did not work, I tried as below but it returned 0

[testserver@forfinding-or]/abcd/efg/ehome/karti>ls -lrt
total 4
-rw-rw-r-- 1 pinto pinto 30 Feb 7 14:18 listfiles.txt
-rwxrwxrwx 1 pinto pinto 124 Feb 7 14:54 test_file.sh
-rw-rw-r-- 1 pinto pinto 0 Feb 7 15:05 ls
[testserver@forfinding-or]/abcd/efg/ehome/karti>ls -lrt | grep "Feb.*2013" | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'
0

schneidz gave a warning: it's not a good idea to use the output of ls. The date format may vary and in this case, files more recent than 6 months don't show the year, therefore the grep statement returns nothing. You may use some options of ls to customize the appearance of the timestamp, but I would rely on the find solution, instead.

vdamgo 02-08-2013 10:59 AM

Sorry, Am really poor in linux scripting, Do we need to keep the below statements in script and then execute

$ touch -t 201112312359 /tmp/file1
$ touch -t 201201312359 /tmp/file2
$ find . -type f -newer /tmp/file1 -not -newer /tmp/file2 -printf "%s\n" | awk '{sum += $0} END{print sum}'
$ rm /tmp/file1 /tmp/file2

colucix 02-08-2013 11:10 AM

Quote:

Originally Posted by vdamgo (Post 4887129)
Sorry, Am really poor in linux scripting, Do we need to keep the below statements in script and then execute

$ touch -t 201112312359 /tmp/file1
$ touch -t 201201312359 /tmp/file2
$ find . -type f -newer /tmp/file1 -not -newer /tmp/file2 -printf "%s\n" | awk '{sum += $0} END{print sum}'
$ rm /tmp/file1 /tmp/file2

Not necessarily. You can run them on the command line. Take in mind that the leading $ sign is the command prompt (you don't have to type it). To test the results of the find command, run it without the -printf statement and without the awk part. Maybe using -ls, that is:
Code:

$ touch -t 201112312359 /tmp/file1
$ touch -t 201201312359 /tmp/file2
$ find . -type f -newer /tmp/file1 -not -newer /tmp/file2 -ls

May I suggest a good reading about the linux command line? Here is it: http://linuxcommand.org/tlcl.php

vdamgo 02-08-2013 11:39 AM

I think we have lost somewhere, here is example as below, I have four files in a directory. two files got generated in feb 2012 and another two files in Feb 2013, now i want to sum up the file size gennerated in feb 2012 so answer should be 85+30=115


[pin@test-omf]/abcd/def>ls -lrt
total 12
-rwxrwxrwx 1 pinot pinot 85 Feb 6 23:59 dummy1.txt (this file is generated in Feb 2012)
-rw-rw-r-- 1 pinot pinot 30 Feb 7 14:18 listfiles.txt (this file is generated in Feb 2012)
-rwxrwxrwx 1 pinot pinot 124 Feb 7 14:54 test_abcd.txt (this file is generated in Feb 2013)
-rwxrwxrwx 1 pinot pinot 86 Feb 8 12:02 dummy2.txt (this file is generated in Feb 2013)

The above is just an example, like wise I have 1000s of files in a directory which are mixed up like some are generated in year feb 2012 and some are generated in feb 2011, some are generated in feb 2013 etc, but am trying to find the sum of file sizes generated in feb 2012.

Habitual 02-08-2013 12:24 PM

Code:

ls -al | grep " 2012 " | awk -F" " 'BEGIN { sum=0} {sum+=$5} END {print sum}'
works every where I use it.

colucix 02-09-2013 05:37 AM

Quote:

Originally Posted by vdamgo (Post 4887150)
I think we have lost somewhere

What's wrong with the touch and find solution? You have just to set the proper dates in the touch commands (my example was based on your initial requirement Jan 2012, changed later to Feb 2012).

Quote:

Originally Posted by vdamgo (Post 4887150)
-rwxrwxrwx 1 pinot pinot 85 Feb 6 23:59 dummy1.txt (this file is generated in Feb 2012)
-rw-rw-r-- 1 pinot pinot 30 Feb 7 14:18 listfiles.txt (this file is generated in Feb 2012)

I don't think so. They appear to be created on February 2013. You can verify it using the stat command or the --full-time option of ls:
Code:

$ ls -l --full-time dummy1.txt listfiles.txt
$ stat dummy1.txt listfiles.txt


vdamgo 02-11-2013 11:47 AM

Thank you.

I think it worked.

$ touch -t 201112312359 /tmp/file1
$ touch -t 201201312359 /tmp/file2
$ find . -type f -newer /tmp/file1 -not -newer /tmp/file2 -printf "%s\n" | awk '{sum += $0} END{print sum}'
$ rm /tmp/file1 /tmp/file2

Can you please give me little explanation like whats is the procedure and what the script is performing.

Appreciate your help


All times are GMT -5. The time now is 04:51 AM.