LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Blogs > linux-related notes
User Name
Password

Notices

Just annotations of little "how to's", so I know I can find how to do something I've already done when I need to do it again, in case I don't remember anymore, which is not unlikely. Hopefully they can be useful to others, but I can't guarantee that it will work, or that it won't even make things worse.
Rate this Entry

More than two-and-a-half-fold decrease in ~/.thumbnails disk usage with pngnq

Posted 05-07-2014 at 02:32 AM by the dsc
Updated 05-07-2014 at 02:43 AM by the dsc (dependency)
Tags thumbnails

The shared thumbnail folder can be quite sizeable on linux (or whatever OSs follow the freedesktop.org specifications), you won't have much trouble finding forum threads of people asking if it's OK to delete it, because it's occupying too much space.

One of the reasons it occupies so much space is because the thumbnails are set to be PNG files, which are comparatively large by nature. Thumbnails for a folder full of JPEGs of moderately medium-large resolution (about 800-1200x800-1200) can add up to about half the size of the actual full-sized images! (For an actual example, 4.7 MB of PNG thumbnails for a total of 8.4 MB for 64 jpegs -- now consider such proportion on a larger scale).

PNGs can't get all the blame though, even though I think it's perhaps arguable that JPGs are good enough for this role (and others). Despite being naturally somewhat larger, PNGs can have a rather small file size and still be decent thumbnails, but it seems that part of the standards is that "every program should use the best possible quality when creating the thumbnail", even though "the exact meaning of this is left to the specific program", and with that, developers just followed the advice without questioning much, more focused on the core features of the applications, considering thumbnails just a minor detail, and not much effort was put into making the thumbnails as light as they could be.


Hopefully a specialized application like tumbler daemon can fix the situation to some degree, creating or re-creating the shared thumbnails with more consideration to file size, through a centralized configuration.

I don't know if "tumbler"/"tumblerd" already does that, but if it doesn't, I don't think it's so hard to come up with a script to do a job like that. I've succeeded in shrinking the PNG thumbnails with the necessary exif metadata to a 2.6 times smaller size, with pngnq (1.8 MB for what was previously 4.7 MB -- again, that's for just miserable 64 jpegs).

It was a bit frustrating because Geeqie was overwriting them all the time, so I thought there was something missing, that I wasn't being able to meet the standards with the exiftool trick. But it seems it's just some bug with Geeqie, because the great Konqueror file manager doesn't do the same, and even Geeqie can be made to shut up and just use the smaller thumbnails, if they're chmod-ed to not be writable.


Here's the "pre-pre-alpha" script, that will only shrink the thumbnails on the user's thumbnail folder, and only the newer ones, if executed once again.

Code:
#!/bin/bash
#
# This "software" is completely UNLICENSED!!!!1111
#
# ************************* WARNING **************************
#
# Newbie/layman/ignorant/h4xx0 code can cause permanent loss of data and time. 
# Don't run without thoroughly examination and safety measures such as backups and all that. 
# NO WARRANTIES WHATSOEVER, you're on your own! 
# There's even a "unsafe" flag in one of the commands! _You_have_been_warned!_
#
#

tempdir=/dev/shm/pngnqtn$$

if [ ! -d $tempdir ]  ;
 
 then mkdir $tempdir
 
 fi


if [ ! -f /home/$USER/.thumbnails/pngnqtn-lastrun ] ; then

date --date "last year" "+%Y-%m-%d %H:%M:%S" > /home/$USER/.thumbnails/pngnqtn-lastrun

fi

find /home/$USER/.thumbnails -newermt "$(cat /home/$USER/.thumbnails/pngnqtn-lastrun)" -iname "*.png"  | while read png ; do

pngfn=$(basename $png)

mtime=$(stat -c %y $png) # may be useless

pngnq -n 128 $png -d $tempdir -e .png

# one may want to add a sleep time here to not get too hard on the CPU

 exiftool -all= -tagsfromfile $png -all:all --directory -unsafe $tempdir/$pngfn

mv $tempdir/$pngfn $png

touch -d "$mtime" $png # may be useless

done

date "+%Y-%m-%d %H:%M:%S" > /home/$USER/.thumbnails/pngnqtn-lastrun
I guess it doesn't even clean the temporary folder yet.

(I'm not sure the "touch" part is needed, I guess it's not, it was from my first impression of the freedesktop specifications).

The daemon-ness can perhaps be achieved by having it as a cron job, and perhaps one could do some more trickery into making it run only when the CPU is relatively idle. The program "loadwatch" supposedly does that, but I have to check it yet. Also, in order to make the loop lighter on CPU one could add some sleep time or something more "sophisticated".

It can be tweaked to have different parameters for "normal" and "large" thumbnails, so the normal/smaller could be more aggressively "optimized" while the larger ones are given some slack.


I think it may require this configuration file (~/.ExifTool_config) for exiftool:

Code:
%Image::ExifTool::UserDefined = (
    'Image::ExifTool::PNG::TextualData' => {
        'Thumb::URI' => { Name => 'ThumbURI' },
        'Thumb::MTime' => { Name => 'ThumbMTime' },
    },
);
1; #end
Inferred from: http://u88.n24.queensu.ca/exiftool/f...p?topic=5551.0
Posted in Uncategorized
Views 120 Comments 0
« Prev     Main     Next »
Total Comments 0

Comments

 

  



All times are GMT -5. The time now is 02:49 AM.

Main Menu
Advertisement

My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration