Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum. |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
03-09-2012, 07:34 PM
|
#1
|
LQ Newbie
Registered: Jan 2009
Location: Plano (Dallas) Texas
Distribution: Centos Win7 W2KPro
Posts: 15
Rep:
|
procmailrc decode data from base64 encoded emails - trying to decode with uudeview
Friends:
We do many things with procmail, and it has been invaluable, but I have a need that I have not been able to break I was hoping someone can help with.
The use case is reformatting messages to be sent to our application in a fixed format. While we receive information from many suppliers in many different formats, we extract the data we need, format it so to the users it looks the same, and simplify reporting, filtering, etc.
We have been very successful with this until last week when we took on another supplier that sends their message
base64. What I need to do is decode the base64 (1-text, 1-html) into text and return that data to the message. Whenever I run the uudeview against them, the message is gone. I can run it:
:0 B
* ^Content-Type:.*base64
| uudeview -p $HOME -
and take the decoded pieces to 2 files, but I am just not able to figure out how to get them into the procmailrc processing. Following is the end of the procmail log in case there is something helpful:
procmail: Match on "^Content-Type:.*base64"
procmail: Executing "/usr/bin/uudeview,-p,/home/gooooert,-"
procmail: Assigning "LASTFOLDER=/usr/bin/uudeview -p /home/gooooert -"
From 84f.18.PROD-DirectInquiry3.leads=paagent.com@mxsp2.email-od.com Wed Mar 7 11:22:07 2012
Subject:NewTenantContact
Folder: /usr/bin/uudeview -p /home/gooooert - 7140
Loaded from /tmp/uu8xSAoq: 'NewTenantContact' (NewTenantContact): part 1 Base64
Loaded from /tmp/uu8xSAoq: 'NewTenantContact' (NewTenantContact): part 1 Base64
Any help would be strongly appreciated.
Gary
|
|
|
03-10-2012, 08:35 PM
|
#2
|
Moderator
Registered: May 2001
Posts: 29,415
|
This recipe should reformat the message prior to processing, try loading it instead of yours:
Code:
:0
* ^Content-Type: *text/plain
{
:0 fbw
* ^Content-Transfer-Encoding: *base64
| mimencode -u -b
:0 Afhw
| formail -I "Content-Transfer-Encoding: 8bit"
}
|
|
|
03-11-2012, 07:15 AM
|
#3
|
LQ Newbie
Registered: Jan 2009
Location: Plano (Dallas) Texas
Distribution: Centos Win7 W2KPro
Posts: 15
Original Poster
Rep:
|
mimencode is no longer available
I have found this solution many places, but mimencode is no longer available. If anyone has any idea where it can be obtained, I would love to find it.
|
|
|
03-11-2012, 08:16 AM
|
#4
|
Moderator
Registered: May 2001
Posts: 29,415
|
Search for the 'metamail' package?
|
|
|
09-29-2015, 10:24 AM
|
#5
|
LQ Guru
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
|
I know it has been a while since this post was active. I've just started working on my own procmail recipe but did want to mention I've already found that openssl can be used for encoding and decoding for base64 stuff rather than mimencode or uudeview.
openssl enc -d -base64 -in ./<input file with base64 encoding> -out <decoded file>.
That syntax has been successfully used to convert one base64 encoded attachment into the XML file that was attached by the sender. It has also been used to successfully used to convert another base64 encoded attachment into the PDF that was attached by the sender of another email.
To manually do this I found the section in the email (as seen in /var/spool/mail/<user> or as copied wholly into a directory with procmail which starts similar to:
Code:
Content-Transfer-Encoding: base64
PD94bWwgdmVyc2lvbiA9ICcxLjAnIGVuY29kaW5nID0gJ1VURi04Jz8+DQo8IURPQ1RZUEUgY1hN
and ends similar to the following:
Code:
DQogICAgICA8L09yZGVyUmVxdWVzdD4NCiAgIDwvUmVxdWVzdD4NCjwvY1hNTD4NCg==
--_002_252900221442858624275JavaMailiapebs3oauohsebsn01oraclec_--
There will be many lines between the first and last encrypted lines). Those lines are all the encrypted attachment. Just copy the first encrypted line after the "Content-Transfer-Encoding: base64" and after the blank line down to the line that ends in "==" into a separate file then use that as your input file to the openssl syntax shown.
I suspect a modified form of the recipe shown earlier in this post that pipes to openssl rather than mimeencode will be the solution and am getting ready to test same but figured I'd share the openssl syntax as it is at least part of the puzzle.
Last edited by MensaWater; 09-29-2015 at 12:17 PM.
|
|
|
09-30-2015, 09:33 AM
|
#6
|
LQ Guru
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
|
Just as a follow up I had found a thread at:
http://www.unix.com/shell-programmin...lbox-file.html
That included a link for a Perl script to save email attachments:
http://www.tek-tips.com/faqs.cfm?fid=4138
Quote:
From unknown source, bless his perl skills. Change the TMPDIR path to where you want to save the attachments. I called it unmime, you are free to call it anything you want.
Things to watch out for:
Path to perl RTE.
Executable rights for script.
Define an alias in sendmail's /etc/aliases to call the script, e.g:
detach: |"/your/path/to/script/unmime"
Mailing to the alias will detach the file to your preferred directory. Remember to run "newaliases" after adding the alias.
Script below:
Code:
#!/usr/bin/perl
#
# Un-MIME regular message from stdin.
# Non-text version saved in directory ~/mail/MIME and proper indication is
# left in the dumped message. Text is otherwise dumped and deleted from there.
#
# Intended to be used with mailagent thanks to the following incantation rule:
#
# Mime-Version: /^\d/ { SAVE +mime; FEED ~/mail/unmime; RESYNC; REJECT };
#
# Options:
# -e: pass the quoted-printable decoder over the message and that's it.
# -x: translate chars not understood by some iso8859-1 fonts.
# -X: translate all accents to non-accentuated letters (plain ASCII).
($me = $0) =~ s|.*/(.*)|$1|;
require "getopts.pl";
Getopts('exX');
$opt_x++ if $opt_X; # -X implies -x
$TMPDIR = "/var/spool/mail/MIME";
use MIME::Parser;
#------------------------------------------------------------
# dump_entity - idempotent routine for dumping an entity
#------------------------------------------------------------
sub dump_entity {
my ($entity) = @_;
my $IO;
my $not_first_part = 0;
# Print the header, converting accents if any
my $head = $entity->head->original_text;
$head =~ s/^(Subject:.*)/no_iso_markup($1)/me
if $head =~ /^Subject:.*=\?iso-8859-1\?Q\?/mi;
print $head, "\n";
# Output the body:
my @parts = $entity->parts;
if (@parts) { # multipart...
my $i;
foreach $i (0 .. $#parts) { # dump each part...
dump_entity($parts[$i]);
}
} else { # single part...
# Get MIME type, and display accordingly...
my ($type, $subtype) = split('/', $entity->head->mime_type);
#print STDERR "type - $type\n";
my $body = $entity->bodyhandle;
my $path = $body->path;
if ($type =~ /^(text|message)$/ || -T $path) { # text: display it...
if ($IO = $body->open("r")) {
print "\n" if $not_first_part++;
print to_ascii($_) while (defined($_ = $IO->getline));
$IO->close;
# If message is text/message, chances that we did the right
# thing are extremely high. So unlink the message if lying on
# the disk... -- RAM, 19/11/96
#unlink($path) or warn "$me: can't unlink $path: $!\n"
# if defined $path && -f $path;
} else { # d'oh!
die "$me: couldn't find/open '$file': $!";
}
} else { # binary: just summarize it...
my $size = ($path ? (-s $path) : '???');
print ">>> This is a non-text message, $size bytes long.\n";
print ">>> It is stored in ", ($path ? "'$path'" : 'core'),".\n\n";
}
}
print "\n";
1;
}
#------------------------------------------------------------
# smart_pack
#------------------------------------------------------------
sub smart_pack {
my ($hexa) = @_;
my $val = hex($hexa);
return "=$hexa" if $val >= 128; # We're smart right there!
return pack('C', $val);
}
#------------------------------------------------------------
# no_accent
#------------------------------------------------------------
sub no_accent {
local ($_) = @_;
tr/\xab\xbb\xe0\xe2\xe7\xe8\xe9\xea\xee\xef\xf4\xf9\xfb/""aaceeeiiouu/;
return $_;
}
#------------------------------------------------------------
# to_ascii
#------------------------------------------------------------
sub to_ascii {
my ($l) = @_;
return $l unless $opt_x; # Don't loose info unless -x or -X
$l =~ tr/\x92/'/ if $opt_x; # ';
$l = no_accent($l) if $opt_X;
return $l;
}
#------------------------------------------------------------
# to_txt -- combines =xx packing with no_accent()
#------------------------------------------------------------
sub to_txt {
my ($l) = @_;
$l =~ s/=([\da-fA-F]{2})/pack('C', hex($1))/ge;
return no_accent($l);
}
#------------------------------------------------------------
# no_iso_markup -- removes ugly ?iso-8859-1?Q escapes
#------------------------------------------------------------
sub no_iso_markup {
local ($_) = @_;
s/^(.*?)=\?iso-8859-1\?Q\?(.*)\?=/$1 . to_txt($2)/ie;
s/_/ /g;
return $_;
}
#------------------------------------------------------------
# unquote_stdin
#------------------------------------------------------------
sub unquote_stdin {
local $_;
my $encoded = 0;
my $in_header = 1;
while (<STDIN>) {
$in_header = 0 if /^\s*$/;
# All Subject: line with accents to be "un-mimed" as well.
s/^(Subject:.*)/no_iso_markup($1)/e
if $in_header && /^Subject:.*=\?iso-8859-1\?Q\?/i;
# Avoid decoding inlined uuencoded/btoa stuff... since they might
# accidentally bear valid =xx escapes... The leading \w character
# is there in case the thing is shar'ed...
# Likewise, all the lines longer than 60 chars and with no space
# in them are treated as being encoded iff they begin with M.
$encoded = 1 if /^\w?begin\s+\d+\s+\S+\s*$/ || /^\w?xbtoa Begin\s*$/;
$encoded = 0 if /^\w?end\s*$/ || /^\w?xbtoa End/;
if ($encoded || (length > 60 && !/ / && /^M/)) {
print $_;
} else {
# Can't use decode_qp from MIME::QuotedPrint because we might not
# face a real quoted-printable message...
# Inline an alternate version.
s/\s+(\r?\n)/$1/g; # Trailing white spaces
s/^=\r?\n//; # Soft line breaks
s/([^=])=\r?\n/$1/; # Soft line breaks, but not for trailing ==
s/=([\da-fA-F]{2})/smart_pack($1)/ge; # Hehe
print to_ascii($_);
}
}
return 1; # OK
}
#------------------------------------------------------------
# main
#------------------------------------------------------------
sub main {
return &unquote_stdin if $opt_e;
# Create a new MIME parser:
my $parser = new MIME::Parser;
# Create and set the output directory:
$parser->output_dir($TMPDIR);
# Read the MIME message:
$entity = $parser->read(\*STDIN) or
die "$me: couldn't parse MIME stream";
# Dump it out:
dump_entity($entity);
unlink<$TMPDIR/msg-*.txt> or warn "can't unlink: $!\n";
}
exit(&main ? 0 : -1);
#------------------------------------------------------------
1;
#
# This bit below saves the message body to file, uncomment if wanted
#
#unlink</var/spool/mail/MIME/msg-*.txt> or warn "can't unlink: $!\n";
|
I saved the above code to a script named unmime.pl and modified the $TMPDIR within it to point to a local user directory rather than /var/spool/mail/MIME (if you leave it on that directory you have to create the directory AND make it writable by the user you're doing the procmail as).
Note the above Perl code requires you to have MIME/Parser.pm Perl module installed as it is in the @INC. On RHEL6 I was able to install that module with "yum install perl-MIME-tools-5.427-4.el6.noarch". If not in an official repository for your distro you can probably get it from CPAN.
My procmail recipe (in testuser's .procmailrc) using the above was simple:
Code:
:0:
* ^From: Oracle SN Operations*
* ^Subject: : Transaction Delivery:*
| /home/testuser/unmime.pl
I was using fetchmail to force getting the email by that testuser from Office365 (I'll do a blog post on that later). After running the fetchmail to download email that came from above source it put two files into the target $TMPDIR I'd defined. One was the body of the email (which was an HTML body so file was created as HTML) and the other was the attachment which was an XML file.
Doing that the resulting XML file created was exactly the same as the one I'd manually created before with the previously mentioned openssl syntax.
By the way it appears there are man pages for enc and base64 as individual commands on RHEL6 so it might not actually be necessary to use the full openssl syntax.
|
|
|
10-06-2015, 03:41 PM
|
#7
|
LQ Guru
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
|
The blog entry with the final solution I did for extracting email attachments:
http://www.linuxquestions.org/questi...chments-36724/
|
|
|
All times are GMT -5. The time now is 11:40 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|