LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 03-09-2012, 07:34 PM   #1
garyrickert
LQ Newbie
 
Registered: Jan 2009
Location: Plano (Dallas) Texas
Distribution: Centos Win7 W2KPro
Posts: 15

Rep: Reputation: 0
procmailrc decode data from base64 encoded emails - trying to decode with uudeview


Friends:

We do many things with procmail, and it has been invaluable, but I have a need that I have not been able to break I was hoping someone can help with.

The use case is reformatting messages to be sent to our application in a fixed format. While we receive information from many suppliers in many different formats, we extract the data we need, format it so to the users it looks the same, and simplify reporting, filtering, etc.

We have been very successful with this until last week when we took on another supplier that sends their message
base64. What I need to do is decode the base64 (1-text, 1-html) into text and return that data to the message. Whenever I run the uudeview against them, the message is gone. I can run it:

:0 B
* ^Content-Type:.*base64
| uudeview -p $HOME -

and take the decoded pieces to 2 files, but I am just not able to figure out how to get them into the procmailrc processing. Following is the end of the procmail log in case there is something helpful:

procmail: Match on "^Content-Type:.*base64"
procmail: Executing "/usr/bin/uudeview,-p,/home/gooooert,-"
procmail: Assigning "LASTFOLDER=/usr/bin/uudeview -p /home/gooooert -"
From 84f.18.PROD-DirectInquiry3.leads=paagent.com@mxsp2.email-od.com Wed Mar 7 11:22:07 2012
Subject:NewTenantContact
Folder: /usr/bin/uudeview -p /home/gooooert - 7140
Loaded from /tmp/uu8xSAoq: 'NewTenantContact' (NewTenantContact): part 1 Base64
Loaded from /tmp/uu8xSAoq: 'NewTenantContact' (NewTenantContact): part 1 Base64

Any help would be strongly appreciated.
Gary
 
Old 03-10-2012, 08:35 PM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608
This recipe should reformat the message prior to processing, try loading it instead of yours:
Code:
:0
* ^Content-Type: *text/plain
{
        :0 fbw
        * ^Content-Transfer-Encoding: *base64
        | mimencode -u -b

        :0 Afhw
        | formail -I "Content-Transfer-Encoding: 8bit"
}
 
Old 03-11-2012, 07:15 AM   #3
garyrickert
LQ Newbie
 
Registered: Jan 2009
Location: Plano (Dallas) Texas
Distribution: Centos Win7 W2KPro
Posts: 15

Original Poster
Rep: Reputation: 0
mimencode is no longer available

I have found this solution many places, but mimencode is no longer available. If anyone has any idea where it can be obtained, I would love to find it.
 
Old 03-11-2012, 08:16 AM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608Reputation: 3608
Search for the 'metamail' package?
 
Old 09-29-2015, 10:24 AM   #5
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
I know it has been a while since this post was active. I've just started working on my own procmail recipe but did want to mention I've already found that openssl can be used for encoding and decoding for base64 stuff rather than mimencode or uudeview.

openssl enc -d -base64 -in ./<input file with base64 encoding> -out <decoded file>.

That syntax has been successfully used to convert one base64 encoded attachment into the XML file that was attached by the sender. It has also been used to successfully used to convert another base64 encoded attachment into the PDF that was attached by the sender of another email.

To manually do this I found the section in the email (as seen in /var/spool/mail/<user> or as copied wholly into a directory with procmail which starts similar to:
Code:
Content-Transfer-Encoding: base64

PD94bWwgdmVyc2lvbiA9ICcxLjAnIGVuY29kaW5nID0gJ1VURi04Jz8+DQo8IURPQ1RZUEUgY1hN
and ends similar to the following:
Code:
DQogICAgICA8L09yZGVyUmVxdWVzdD4NCiAgIDwvUmVxdWVzdD4NCjwvY1hNTD4NCg==

--_002_252900221442858624275JavaMailiapebs3oauohsebsn01oraclec_--
There will be many lines between the first and last encrypted lines). Those lines are all the encrypted attachment. Just copy the first encrypted line after the "Content-Transfer-Encoding: base64" and after the blank line down to the line that ends in "==" into a separate file then use that as your input file to the openssl syntax shown.

I suspect a modified form of the recipe shown earlier in this post that pipes to openssl rather than mimeencode will be the solution and am getting ready to test same but figured I'd share the openssl syntax as it is at least part of the puzzle.

Last edited by MensaWater; 09-29-2015 at 12:17 PM.
 
Old 09-30-2015, 09:33 AM   #6
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
Just as a follow up I had found a thread at:
http://www.unix.com/shell-programmin...lbox-file.html

That included a link for a Perl script to save email attachments:
http://www.tek-tips.com/faqs.cfm?fid=4138


Quote:
From unknown source, bless his perl skills. Change the TMPDIR path to where you want to save the attachments. I called it unmime, you are free to call it anything you want.

Things to watch out for:

Path to perl RTE.
Executable rights for script.


Define an alias in sendmail's /etc/aliases to call the script, e.g:
detach: |"/your/path/to/script/unmime"

Mailing to the alias will detach the file to your preferred directory. Remember to run "newaliases" after adding the alias.

Script below:

Code:
#!/usr/bin/perl
#
# Un-MIME regular message from stdin.
# Non-text version saved in directory ~/mail/MIME and proper indication is
# left in the dumped message. Text is otherwise dumped and deleted from there.
#
# Intended to be used with mailagent thanks to the following incantation rule:
#
# Mime-Version: /^\d/    { SAVE +mime; FEED ~/mail/unmime; RESYNC; REJECT };
#
# Options:
#   -e: pass the quoted-printable decoder over the message and that's it.
#    -x: translate chars not understood by some iso8859-1 fonts.
#    -X: translate all accents to non-accentuated letters (plain ASCII).

($me = $0) =~ s|.*/(.*)|$1|;

require "getopts.pl";
Getopts('exX');

$opt_x++ if $opt_X;        # -X implies -x

$TMPDIR = "/var/spool/mail/MIME";

use MIME::Parser;

#------------------------------------------------------------
# dump_entity - idempotent routine for dumping an entity
#------------------------------------------------------------

sub dump_entity {
  my ($entity) = @_;
  my $IO;
  my $not_first_part = 0;
  
  # Print the header, converting accents if any
  my $head = $entity->head->original_text;
  $head =~ s/^(Subject:.*)/no_iso_markup($1)/me
    if $head =~ /^Subject:.*=\?iso-8859-1\?Q\?/mi;
  print $head, "\n";
  
  # Output the body:
  my @parts = $entity->parts;
  if (@parts) {            # multipart...
    my $i;
    foreach $i (0 .. $#parts) { # dump each part...
      dump_entity($parts[$i]);
    }
  } else {            # single part...
    # Get MIME type, and display accordingly...
    my ($type, $subtype) = split('/', $entity->head->mime_type);
    #print STDERR "type - $type\n";
    
    my $body = $entity->bodyhandle;
    my $path = $body->path;
    if ($type =~ /^(text|message)$/ || -T $path) {     # text: display it...
      if ($IO = $body->open("r")) {
    print "\n" if $not_first_part++;
    print to_ascii($_) while (defined($_ = $IO->getline));
    $IO->close;
    
    # If message is text/message, chances that we did the right
    # thing are extremely high. So unlink the message if lying on
    # the disk... -- RAM, 19/11/96

    #unlink($path) or warn "$me: can't unlink $path: $!\n"
    #  if defined $path && -f $path;
    
      } else {            # d'oh!
    die "$me: couldn't find/open '$file': $!";
      }
    } else {            # binary: just summarize it...
      my $size = ($path ? (-s $path) : '???');
      print ">>> This is a non-text message, $size bytes long.\n";
      print ">>> It is stored in ", ($path ? "'$path'" : 'core'),".\n\n";
    }
  }
  print "\n";
  
  1;
}

#------------------------------------------------------------
# smart_pack
#------------------------------------------------------------
sub smart_pack {
  my ($hexa) = @_;
  my $val = hex($hexa);
  return "=$hexa" if $val >= 128; # We're smart right there!
  return pack('C', $val);
}

#------------------------------------------------------------
# no_accent
#------------------------------------------------------------
sub no_accent {
  local ($_) = @_;
  tr/\xab\xbb\xe0\xe2\xe7\xe8\xe9\xea\xee\xef\xf4\xf9\xfb/""aaceeeiiouu/;
  return $_;
}

#------------------------------------------------------------
# to_ascii
#------------------------------------------------------------
sub to_ascii {
  my ($l) = @_;
  return $l unless $opt_x;    # Don't loose info unless -x or -X
  $l =~ tr/\x92/'/ if $opt_x;    # ';
  $l = no_accent($l) if $opt_X;
  return $l;
}

#------------------------------------------------------------
# to_txt -- combines =xx packing with no_accent()
#------------------------------------------------------------
sub to_txt {
  my ($l) = @_;
  $l =~ s/=([\da-fA-F]{2})/pack('C', hex($1))/ge;
  return no_accent($l);
}

#------------------------------------------------------------
# no_iso_markup -- removes ugly ?iso-8859-1?Q escapes
#------------------------------------------------------------
sub no_iso_markup {
  local ($_) = @_;
  s/^(.*?)=\?iso-8859-1\?Q\?(.*)\?=/$1 . to_txt($2)/ie;
  s/_/ /g;
  return $_;
}

#------------------------------------------------------------
# unquote_stdin
#------------------------------------------------------------
sub unquote_stdin {
  local $_;
  my $encoded = 0;
  my $in_header = 1;
  while (<STDIN>) {
    $in_header = 0 if /^\s*$/;
    
    # All Subject: line with accents to be "un-mimed" as well.
    s/^(Subject:.*)/no_iso_markup($1)/e
      if $in_header && /^Subject:.*=\?iso-8859-1\?Q\?/i;
    
    # Avoid decoding inlined uuencoded/btoa stuff... since they might
    # accidentally bear valid =xx escapes... The leading \w character
    # is there in case the thing is shar'ed...
    # Likewise, all the lines longer than 60 chars and with no space
    # in them are treated as being encoded iff they begin with M.

    $encoded = 1 if /^\w?begin\s+\d+\s+\S+\s*$/ || /^\w?xbtoa Begin\s*$/;
    $encoded = 0 if /^\w?end\s*$/ || /^\w?xbtoa End/;
    
    if ($encoded || (length > 60 && !/ / && /^M/)) {
      print $_;
    } else {
      # Can't use decode_qp from MIME::QuotedPrint because we might not
      # face a real quoted-printable message...
      # Inline an alternate  version.
      
      s/\s+(\r?\n)/$1/g;    # Trailing white spaces
      s/^=\r?\n//;        # Soft line breaks
      s/([^=])=\r?\n/$1/;    # Soft line breaks, but not for trailing ==
      s/=([\da-fA-F]{2})/smart_pack($1)/ge;    # Hehe
      print to_ascii($_);
    }
  }
  return 1;    # OK
}

#------------------------------------------------------------
# main
#------------------------------------------------------------

sub main {
  return &unquote_stdin if $opt_e;
  
  # Create a new MIME parser:
  my $parser = new MIME::Parser;
  
  # Create and set the output directory:
  $parser->output_dir($TMPDIR);
  
  # Read the MIME message:
  $entity = $parser->read(\*STDIN) or
    die "$me: couldn't parse MIME stream";
  
  # Dump it out:
  dump_entity($entity);
  unlink<$TMPDIR/msg-*.txt> or warn "can't unlink: $!\n";
}

exit(&main ? 0 : -1);
#------------------------------------------------------------
1;
#
# This bit below saves the message body to file, uncomment if wanted
#
#unlink</var/spool/mail/MIME/msg-*.txt> or warn "can't unlink: $!\n";
I saved the above code to a script named unmime.pl and modified the $TMPDIR within it to point to a local user directory rather than /var/spool/mail/MIME (if you leave it on that directory you have to create the directory AND make it writable by the user you're doing the procmail as).

Note the above Perl code requires you to have MIME/Parser.pm Perl module installed as it is in the @INC. On RHEL6 I was able to install that module with "yum install perl-MIME-tools-5.427-4.el6.noarch". If not in an official repository for your distro you can probably get it from CPAN.

My procmail recipe (in testuser's .procmailrc) using the above was simple:
Code:
:0:
* ^From: Oracle SN Operations*
* ^Subject: : Transaction Delivery:*
| /home/testuser/unmime.pl
I was using fetchmail to force getting the email by that testuser from Office365 (I'll do a blog post on that later). After running the fetchmail to download email that came from above source it put two files into the target $TMPDIR I'd defined. One was the body of the email (which was an HTML body so file was created as HTML) and the other was the attachment which was an XML file.
Doing that the resulting XML file created was exactly the same as the one I'd manually created before with the previously mentioned openssl syntax.

By the way it appears there are man pages for enc and base64 as individual commands on RHEL6 so it might not actually be necessary to use the full openssl syntax.
 
Old 10-06-2015, 03:41 PM   #7
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
The blog entry with the final solution I did for extracting email attachments:
http://www.linuxquestions.org/questi...chments-36724/
 
  


Reply

Tags
centos52, procmail


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Decoding Base64 encoded PHP scripts on Linux Server charu Linux - Newbie 2 01-18-2011 11:57 AM
How to decode gpg? ashkael Linux - Newbie 5 02-22-2007 11:18 AM
Base64 decode to image in python genderbender Programming 0 09-06-2006 04:20 PM
suggestions for using xslt to view base64-encoded floating point data in an xml file? zero79 Programming 0 01-10-2006 06:52 PM
nzbperl & uudeview...cant decode? JapanFred Linux - Newbie 1 07-14-2005 04:49 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:40 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration