Get a list of most accessed, "hot", files
 

Hello!

I have used search and google but i can't see to find an answer to this (if it's here please link and spank me for search-failing :)

I admin a server that serves small (static) images as part of a bigger site. There is about 1 TB of images, but most of the are seldom accessed.

The problem: Large cheap disks are slow and i would like to move the most accessed images to something faster (be it a server with faster disk in better raid or a machine with ram-cache or SSD or whatev). But how do i find the "top accessed files in period X"?

Anyone have an idea? It's probably an obvious cat-grep-sed-mangle of something in /proc but i just can't solve it on my own...


Thank you in advance!

BTW: We run Debian stable...

The simplest way I could think of is using the inbuilt find command and its options

The command goes something like this

find / -name *.jpg -atime 1
The above command displays all the files which have been accessed a day ago.

See man find for more exact reference.

Cheers!!!

man find
Read more
and you will find more...

Thanks! Thats obvious, why didn't i think of that myself :)

Given a short value for atime i would get a list i myself could store and process (as in "this file was also returnd as a when you asked the last time"). That could be a working (although maybe not the most elegant) solution.

The other solution, i just realized, is... of course... turn on access-logging and parse through those files. Dunno where my head's at today.


I have gotten enough answers to solve my problem, thanks! If anyone has a cooler way to do it - plase share if you feel you have the time to do so...

filetop
 

#!/usr/bin/perl
use Cwd;
use File::Touch;
use File::Temp qw/tempfile/;
use Time::HiRes qw/sleep time alarm/;
use Term::ReadKey;
my ($wchar, $hchar, $wpixels, $hpixels) = GetTerminalSize();
if($hchar < 10) {print "please increase window size"; exit; }
my $mydir = getcwd;
my ($fh, $tmpfile) = tempfile(UNLINK => 1);

while(1)
{
my $starttime = time;
eval {
local $SIG{ALRM} = sub { die "alarm\n" };
alarm 0.4;
$query = `find -newermm $tmpfile 2>&1`; #change to mm for writes only
touch($tmpfile);
@files = split(/\n/,$query);
alarm 0;
};
system('clear');
foreach $file(@files) { $filecount{$file}++; }
@sorted = sort {$filecount{$b} <=> $filecount{$a}} (keys %filecount);
for ($x = 0;$x < $hchar-2; $x++) {print $filecount{$sorted[$x]}."\t".$sorted[$x]."\n";}
my $endtime = time;
my $length = ($endtime-$starttime);
if ($length > 0.3) {print "program not designed for large trees, please use smaller tree.\n"; exit;}
print "\n$length"."s\n"
}

Interesting first post.

How about inotify ?.

Edit: - didn't realise this was an old post. @jongab, resurrecting such old posts is generally frowned upon.
But welcome to LQ anyway ... ;)