File Size Shell Script

BlackLabel · 10-27-2005, 01:13 PM

Hello all,

It has been a while since college and my brain is foggy on shell scripting. I need to create a script that will add up the file size of each file created previous day.

More specifically:
Each day between 2 and 36 files are created. The day after we have been manually entering in the file size of each file into a spreadsheet. I want to be able to run a script and have it add up the file sizes and redirect it to a file with the sum.

I have:
ls -al | awk '{sum = sum + $5} END {print sum}'

but I need it to only calculate files that are one day behind the system date. Any ideas?

homey · 10-27-2005, 01:20 PM

Use find with ctime, mtime or atime to find files of a particular age.

BlackLabel · 10-27-2005, 01:29 PM

So something like:

find / (DIR) $ -name "CORE*" -or -name "c-" $ -mtime
+1 -exec ls -al | awk '{sum = sum + $5} END {print sum}' {}\;

where (DIR) equals the desired directory?

destuxor · 10-27-2005, 04:15 PM

Instead of your "ls -al | awk '{sum = sum + $5} END {print sum}'" statement, can't you just use the du command? So:

Code:

#!/bin/bash
SUM = 0
find -mtune +1 -exec "$SUM = $SUM + `du {}`" \;

That probably doesn't work, but it might help.

BlackLabel · 10-27-2005, 06:04 PM

Thanks!

I was able to work out the specifics to:

Code:

find . \( -name "CORE_*" -o -name "c-*" \) -mtime -1 -exec ls -l {} \; |awk '/^-/ {total += $5} END {printf "%15.2f\n",total}'

works great, all I need to do is put it in a .sh file and should be good to go. Feel free to play golf with it, this was as short as I could get it

Akilan Perumal · 11-27-2007, 10:03 AM

---------------------------------------------------------------------------------------------------

#!/bin/sh

for i in `find . $ -name "aki1*" -o -name "c-*" $ -exec ls {} \;`; do

echo $i

echo "Moving"

mv $i new.`date +%Y%m%d`

touch aki1

done

--------------------------------------------------------------------------------------------------

PAix · 11-27-2007, 06:57 PM

Hi BlackLabel,
It strikes me that your question wasn't fully answered. The following code may not be the shortest solution, but does give you a list of filenames and filesizes along with the sum of the file sizes. I haven't put a count of the files in, largely because you didn't specify that.

It also occurred to me that the 24 hours before requirement hasn't been interpreted/implemented accurately either and a run that was an hour early could conceivably cause problems. My interpretation of 24 hours before is all the files written/modified on the day before 00:00 to 23:59. Currently I assume that you don't normally write files with embeded spaces and don't normally write hidden files.

My approach at the end of the first, throwaway, run is to write the current date suitably formatted to match that of the date in the ls command. So next day the previous day's date is available for use. Of course if the operation only produces files for five working days a week then you could actually run the script with cron at the same time Monday to Friday and on the Following Monday, it will use the Friday date, probably what you actually want. I use .hidden files in the current directory to store the date and various interim products. These could of course all be changed to a separate directory to save poluting your data directory and be a bit more portable if you change operations into a new directory.

I feel that it's the basis of what you want, but would be interested to know how closely if it meets what you actually intended. On the basis that the position of columns may vary in the ls command output on different distros, be aware that I cut on character positons. Also if you use longer filenames than I have anticipated then that might need slight adjustments in the columns in the cut statements..

Code:

#!/bin/sh

if [ -f .myyesterday ]
then
    mydate=$(< .myyesterday)
else
    echo
    echo "File .myyesterday with yesterday's date"
    echo "in form YYYY-MM-DD doesn't exist"
    echo "Recommend manualy edit and re-run"
    echo
    exit 1
fi


# Clear file we will append to
> .mytmp2

# This next line is here for debugging
# echo $mydate

printf "%s\n" $(ls -l |
 cut -c 30-40,47-95 |
 grep "$mydate" | cut -c 12-40) > .mytmp

for file in $(cat .mytmp)
do
#  echo  $(wc -c $file | cut -d' ' -f 1) |
echo $(wc -c $file) >> .mytmp2
done

awk -v mydate=$mydate 'BEGIN { printf "%30s\n\n", ("File Totals for " mydate) }
     { total=total+$1;
       printf   "%-20s %9d\n", $2, $1 };
 END { printf "\n%-20s %9d\n", (mydate " Total"), total }' < .mytmp2

# Store todays date for use tomorrow (comment it out for debugging)
echo $(date +%Y-%m-%d) > .myyesterday

exit 0

Here are a couple of output examples for illustration.

Code:

   File Totals for 2007-11-18

ani.gif                  23969
animated.gif             23969
Arrow1.png               47133
Arrow2.png                1347
mylist.txt                 600
mymagic                   1108
mymagic1                   448
rose.gif                  4169
xArrow1blue0.gif           404
. . .
xArrow2yellow270.gif      1132
xArrow2yellow90.gif       1137

2007-11-18 Total        127163

Code:

    File Totals for 2007-11-27

arr1                       134
check                     6701
check.c                    129

2007-11-27 Total          6964

Of course you might just want to have the script email you and the appropriate OP with the output.

ghostdog74 · 11-27-2007, 07:48 PM

using GNU find

Code:

find /path -mtime +1 -printf "%s\n"|awk '{sum+=$0}END{print sum}'