Transforminator

Edit 31/12/2023: A new version 1.1.7 of Cremefraiche is available on rubygems.org. It comprises some bug fixes, the license is now wtfpl-2 and more HTML-garbage may be handled to render the PDF readable.

(...) Crème Fraiche has been developed in response to an inquiry in the Framasoft forum `Framagora'. When I noticed, that there are only commercial tools but no free software available to convert e-mail to PDF, I felt challenged to give it a try.

At first, I planned to convert the EML-files to XML, as this format can then easily be processed into many different kinds of document. The Apache-FOP utility would then be used to create PDF-files from the intermediate XML. And because XML lets you freely handle the contained data in any way you like, just as with `Crème Fraiche' you compose a splendid meal from otherwise frugal ingredients, I chose the name Crème Fraiche for my new utility.

(...)

You can convert email to PDF in many different ways. I found the easiest solution in printing a message to file.

(...)

However, the format of the resulting file may not be to everyone's gusto. Furthermore, when you want to archive more
than one message in the PDF-format, opening and printing each one `to file' is cumbersome. At last, mail-attachments
need to be handled separately and keeping them together with the PDF-file means an additional effort.

(...)

Crème Fraiche is programmed in `Ruby'. To run it, you need a working
Ruby-interpreter on your computer. It should be easy to find Ruby in the
package-management system that comes with your Linux-distribution. If not, or
if you want to get a newer Ruby version, visit the Ruby homepage and get the
latest version of the interpreter directly from there. Crème Fraiche was
originally developed with Ruby 1.9.3 and you should probably avoid running the program with older versions.

The easiest way to download and install the program is by use of the gem-utility. Just execute on a command-line

Code:

user@machine:$ gem install cremefraiche

(...)

With the help of the gem-utility, download and install the gems which Crème Fraiche uses:

• `mail', to parse the components of email
• `prawn', to generate PDF
• `prawn-table', to generate tables in PDF
• `nokogiri', to handle HTML-mail
• `escape', to handle shell-commands where external tools are called.
• `gtk3', to use the optionan graphical user-interface.

Since version 0.3 Crème Fraiche comes with a graphical user-interface that you
can use as an alternative to executing Crème Fraiche from the command-prompt.
Since Crème Fraiche 0.6.1, the Gtk3 library is needed for the graphical
user-interface to run.

Execute Crème Fraiche once without arguments. If you installed the gem, just call `cremefraiche.rb' right away,
otherwise use the following command.

Code:

user@machine:~$ ruby bin/cremefraiche.rb

(...)

If the usage-information is displayed, Crème Fraiche is functional.

If for some reason the required gem-files for prawn, mail, nokogiri, escape or
gtk3 (for the GUI) are not found by the ruby-interpreter, Crème Fraiche will
try to load and install them first. This may once again become necessary after
an update of the ruby-interpreter. The fact is indicated for each of the
missing gems:

Code:

user@machine:~$ cremefraiche.rb
   =>   PSE wait, installing required gem "mail"...

The installation may take some time, as all of the named gems depend themselves on other gems. If your gem-files
are installed for system-wide usage, this operation may require administrator-rights. If a gem cannot be installed, and
an error-message like the following appears, execute cremefraiche.rb once as super-user.

Code:

user@machine:~$ cremefraiche.rb
   =>   PSE wait, installing required gem "mail"...
ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the /usr/local/lib/ruby/gems/2.0.0 directory.

Repeating the command-line as super-user (root or administrator), you will see that the gems and packages, that they
depend on, are installed on your system:

Code:

root@machine:~# cremefraiche.rb
   =>   PSE wait, installing required gem "mail"...
Fetching: mime-types-1.21.gem (100%)
Successfully installed mime-types-1.21
Fetching: polyglot-0.3.3.gem (100%)
(...)

When all required gems are installed the output shown above will be created

At the time of this writing, Crème Fraiche will create output either in English (default), French or German, depending
on the value of the environment-variable `LANG'. In case that this value is not set or cannot be interpreted, the
language is read from the file `LANG' in the program directory. It contains nothing but the two-letter designator of a
locale, ergo one of en, fr or de in lower or upper case. Choose your preferred language-setting or leave the file
empty, which means the default English.

Further below in this document I will explain how you can add a translation to Crème Fraiche, if you want to make the
program support additional languages.

(...)

It is not mandatory for the normal operation of Crème Fraiche to have the PDFTK-utility installed. However this nice
little tool empowers you to manipulate existing PDF-files in many ways. Crème Fraiche can be configured to call pdftk
to attach files to a PDF. See below in the section about mail-attachments how this will be useful.

(...)

I will first explain, which kind of PDF-result you can expect from Crème Fraiche, based on a simple example message,
then show you, how you can influence the results with the help of some configuration-options. At last, I explain the
translation mechanism, which can be extended to support more than the current three languages English, French and
German for the feedback, which Crème Fraiche produces on the standard-output or in a protocol-file (see below in
section Logging).

EML is a rather simple file-format for the textual representation of email. Many email-clients are capable to store
individual emails in EML-files or to export complete email-folders in eml-format. In the latter case, one EML-file will
contain all the messages, which were originally found in that email-folder. Crème Fraiche will handle single mails or
multiple mails in one or many EML-files alike and create 1 PDF-file for each individual message.

How you create these EML-files depends on the email-client, that you use. Mozilla-Thunderbird for example, uses to
store messages by default in eml-format and not just when you choose the pertinent menu-command `save as...', but also
routinely each incoming message or those that you move to different mail-folders. Thus, these mail-folders are in
reality EML-files, each containing one or many emails.

A test message which I have sent to my own email address, looks like this when exported to EML:

Code:

Message-ID: <4F0AD54A.9010908@uplawski.eu>
Date: Mon, 09 Jan 2012 12:53:46 +0100
From: Michael Uplawski <michael.uplawski@uplawski.eu>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:9.0) Gecko/20111222 Thunderbird/9.0.1
MIME-Version: 1.0
To: uplawski@web.de
Subject: TEST priority
X-Priority: 1 (Highest)
X-Enigmail-Version: 1.3.4
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Body text of the test message.
--
GnuPG/OpenPGP: 2048D/74A227D5 2010-04-09 [verfällt: 2012-04-08]
sub: 2048g/4E580A13 2010-04-09 [verfällt: 2012-04-08]
User: Michael Uplawski (privat) <michael.uplawski@uplawski.eu>
Fingerprint: 1E6E 87CF B2F5 EBBC ACB6 7120 0A44 0972 74A2 27D5

Apart from the body of the email message, all the original mail-headers are present in the EML-file.

To produce a PDF-file from the above simple email, you must know the file-name and path of the EML-file. Call Crème
Fraiche with the file as the only argument, like in this example:

Code:

user@machine:~$ cremefraiche.rb /tmp/test_mail.eml

Note: For the time being, Crème Fraiche does not verify that the handled files do really contain text in EML-format. An
attempt to transform other data will usually fail, but the exact behavior of Crème Fraiche depends on the type of the
actual data. The responsibility to choose only valid EML-files is yours alone.

The result of the transformation is 1 PDF-file for each message in the EML-file. In the above example, a new
PDF-document would be stored in the same directory (`/tmp') where the input had been located: `/tmp/test_mail.eml.pdf'.
The name of the new file will always follow the example of the original eml and only the file-extension `.pdf' is
added.

(...)

You will notice, that not all the mail-headers have survived the transformation. Message-ID, User-Agent, MIME-Version, X-Priority, X-Enigmail-Version and Content-Transfer-Encoding disappeared. The Content-type appears to have moved towards the body of the message, where it got highlighted in blue.

When many emails are stored in only one EML-file (e.g. when the file represents a mail-folder), the procedure is not
different in any way. Just call Crème Fraiche with the file as its argument.

(...)

You can process multiple EML-files at one go. Just name them one by one, separated by an empty space, as arguments to
Crème Fraiche.

Code:

user@machine:~$ cremefraiche /tmp/test_mail.eml
      /Mail/Folders/family/Fridolin /Archives/my_mail/2011.eml

It is of no importance, how many mails are stored in each folder. Keep in mind however, that the PDF output is always
created in the same directory, that contains the original EML-file. It is probably best, to first copy all the files,
that you are interested in, to one dedicated folder. This will be even more interesting, when you also need to keep
mail-attachments ready for later reference (see below).

The content of an email can be piped in to the program via the command-line. One dash (-) signals that the
argument-list shall be ignored and the actual content to be transformed shall be read from the standard input.
Example:

Code:

user@machine:~$ cat mailfile.eml | cremefraiche -

See further below for an example usage of this functionality.

You can influence the behaviour of Crème Fraiche. The file `config' in the gem- or program directory contains adaptable
options in `YAML'-format. Like in the exemplary sections shown below, each option is commented and it should be easy for you to save your preferred values.

By default, the configuration-file resides in the sub-directory `lib' of the downloaded source-package or the installed
gem. To modify this file directly, you might need administrator rights.

Each user of Crème Fraiche can have her/his own version of the configuration file and set her/his own preferences
there. These will then override the original settings. To create your first initial copy of the configuration, just
execute cremefraiche with the program argument `user-config':

Code:

user@machine:~$ cremefraiche.rb user-config

This way, Crème Fraiche creates a sub-folder `.cremefraiche' in your home-directory where it places the new file
`config'. You can later execute the identical command to be notified about new or obsolete elements in future versions
of the program and to have your own copy of the configuration updated accordingly.

Note, that an existing user-configuration will be renamed to `config_bak' and the new copy be stored alongside this
file.

Code:

user@machine:~$ cremefraiche.rb user-conf

The following changes can be effectuated on your user-configuration
        + New options:
                additional
        - Obsolete Options:
                obsolete option
Do you want to replace your current user-configuration?
If you answer 'yes' a back-up copy of your old file will still be available.
But your old settings will be replaced by the defaults.
Your answer (Y/n):

If you push `y', unfortunately and for the time being, you will be obliged to once again adapt all the options to your
needs. Use the backup copy for reference. On the other hand, you can also say `n' and just add those options that are
listed as new and remove the old ones from your file. This way, you only miss the comment-section which explains each
option. You can always look these up in the file that you installed with the downloaded program-package. In the future
I might provide some way to access this information in a different way; please be patient.

Instead of listing and explaining each and every option once again on this page, I keep a copy of my own current
configuration in the sub-folder `example' under `doc'.

(...)

Crème Fraiche can handle mail-attachments in two different ways or ignore them completely.
See in the configuration-file the last two options at the end of the file, `save attachments' and `attachment link color':

Code:

# Where attachments shall be stored during the transformation.
#       Default:        false
#       Value:          One of the following
#                               'pdf'            - store files in the PDF (needs pdftk)
#                                                  The Adobe Acrobat-reader can open
#                                                  attachments to PDF-files.
#
#                               a directory path - store files in the given directory.
#                                                  A link to the file shall be provided
#                                                  in the PDF. The directory MUST EXIST.
#
#                               'false'          - do not store the attachments
# save attachments: /tmp/mail_attachments
save attachments: pdf

# The color of the links to saved attachments, if "save attachments"
# above is set to a file path, rather than 'pdf'
#       Default:        008000
#       Value:          a hexadecimal rgb-color value
#                       in single quotes
attachment link color: '008000'

Apart from the value `false', the option `save attachments' can be set to either a file-path or the value `pdf'. In
latter case, and provided, that you have the PDFTK-tool installed, the mail-attachments are again attached to the
PDF-file in creation. This has the immense advantage, that you can move the PDF-file around in the file-system or
transfer them anywhere but never forget that originally, the email-message came with more files attached. The Adobe
Acrobat-reader™ can show the list of attached files and lets you open each one of them with a double-click.

(...)

If you do not want to resort to PDFTK for some reason, see the next paragraph for an alternative way to handle mail-attachments.

When storing attachments in the PDF-file itself is not possible, you can still have a link to each attachment in the
content of the PDF-file. Choose an existing directory where Crème Fraiche shall store all mail-attachments. At the end
of each transformed message, when it contained attachments, a link to the files will be generated. As long as you do
not move them away from the dedicated directory, you can open each attachment by clicking on its link.

(...)

The color of the links is set with the configuration-option `attachment link color'

Crème Fraiche will report back during execution. You can determine where to put these protocol-messages and how much
information you will get from the program by setting the configuration-options `log level' and `log file'. Here is the
relevant section from the configuration-file:

Code:

# The log-level can be fatal, error, warn, info or debug
#       Default: UNKNOWN
#       Value: one of unknown, fatal, error, warn, info or debug
log level: debug

# Path to the protocol-file, where the logger-output will
# be directed, if configured. When left empty, stdout will
# be used.
#       Default:        STDOUT
#       Value:          a file-path
log file: /tmp/CremeFraiche.log

Setting `log file' to an empty value or to STDOUT will make Crème Fraiche log to the screen.

This too, concerns the log-function. Crème Fraiche can create log-messages in different languages; at the time of this
writing those are English (default), French and German. This applies though only to messages on the log-levels info,
error and warn, as I consider debug-messages of minor interest.

As indicated above, under Installation, to change the language, you open the file `LANG' in the program-directory and
just type on the first and only line the two-letter code for the language you want: en, fr or de.

If you want to add a language, you need to translate yourself all the messages, contained in the file `translations' in
the same gem- or program-directory. The structure of that file is simple. Example:

Code:

storing temporary files in %s:
  de: speichere temporäre Dateien in %s
  fr: des fichiers temporaires seront ecrit dans %s

replacing existing file %s:
  de: ersetze vorhandene Datei %s
  fr: fichier %s est remplacé

method %s is undefined for objects of class %s:
  de: Die Methode %s ist für Objekte der Klasse %s undefiniert
  fr: La méthode %s n'est pas défini pour les objets de la class %s

In the example above, each block of three lines stands for one protocol-message. In the first line, you see the
original English message, followed by a colon (. The following lines contain the translation, preceded by a
two-letter code and a colon. The use of the colon in this YAML-syntax unfortunately prevents me from using colons
within the messages. Keep that in mind, when you provide your own translation: No colon other than those at the end of
the English line and after the two-letter code!

Another specialty is the code `%s' which will be replaced by a variable value during the execution of the program.
Keep these two characters in your translation. You are though not obliged to put them at a certain position of the
message, as you see in the last block in the example, where in the German translation, they moved one word to the left.

But that is it. No more background is needed to add a language. Do not forget to change the two-letter code in `LANG'
to the code of the new language.

You can add one translated message at a time and run Crème Fraiche without fear, as messages for which no translation
exists will not create an error but instead are displayed in English.

The GUI-Howto contains a complete scenario but it relies completely on the graphical user-interface to Crème Fraiche.

Mutt is one of the most versatile Mail-User-Agents that you can get for free, and maybe just the best Mail-User-Agent
ever developed. You can provide a keyboard-shortcut in Mutt that will, while you check your mail, call Crème Fraiche on
an arbitrary mail in your inbox or other mail-folder and create a PDF-file from its content.

Since version 0.6.6 files can be `piped-in' to Crème Fraiche and all you need to do is define a Mutt-macro in the
configuration-file (usually .muttrc in your home-directory) to make use of this possibility. Use only one dash '-' as
program-argument to Crème Fraiche.

Code:

macro index X "|cremefraiche.rb -\n" "make PDF"
macro pager X "|cremefraiche.rb -\n" "make PDF"

Now, you can start Mutt with this configuration and in the message-list select an email and push Shift+X or use the
same command from the pager. Crème Fraiche will be called and a new PDF-file is created in your home-directory. This
file is always called 001_mail_message.pdf for all messages which are piped-in to Crème Fraiche. So, subsequent calls
of the same command will always overwrite the file as long as it has not been renamed or moved out of the way.