LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 08-12-2019, 04:20 PM   #1
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 1,902

Rep: Reputation: 158Reputation: 158
base64 encoding


Not sure if this topic belongs (If not I am sure someone will tell me

While diagnosing an email issue with my ISP (don't ask) I needed to send a rather large message as a test. I copied and pasted the contents of a 500 kB text file into the body of an email and sent it. On the receiving email server (the ISP's) the message showed a size of 4 MB. Examining the message source showed it to be base64 encoded.

The saved message on the senting email server (Protonmail) showed a size of 122 kB. A copy mailed to another Protonmail address also showed the same size. A copy mailed to a lavabit.com address showed 3.6 MB

I found a base64 encoding page and put the sample text through the process. It produced a 700 kB file. I understand that base64 encoding is used to facilitate email transport. All this makes me wonder...

Why would someone save a text message on their email server as a greatly bloated base64 format and more importantly

If base64 encoding increases the size of the text by 50% or 800% depending on the situation... why is it used when sending an email? Email started when bandwidth was limited and expensive.

I am scratching my head

TIA,

Ken
 
Old 08-12-2019, 04:36 PM   #2
scasey
Senior Member
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.6
Posts: 3,486

Rep: Reputation: 1170Reputation: 1170Reputation: 1170Reputation: 1170Reputation: 1170Reputation: 1170Reputation: 1170Reputation: 1170Reputation: 1170
base64 encoding obfuscates the content of the email body. I believe this is most often done to get around spam filters, although there one can have a rule that treats an encoded body as spam, by definition. I recently corresponded with a mail administrator who's company's cgi script was base64 encoding emails. He did not understand what it was...thought it was just a "bunch of garbage."

I've not seen the bloating you describe for a text or html message. Does the sample you're working with include images?
If so, then there's also MIME encoding happening, maybe.

EDIT: Just looked at an email with an image. The image was also base64 encoded, so it's not just for obfuscating text, although I have seen it used for that purpose.
Still did not see the bloat described. Thunderbird says it's 100KB, and the file on disk on the server is 101215 (or 99k with the -h option)

Last edited by scasey; 08-12-2019 at 05:09 PM.
 
Old 08-12-2019, 05:53 PM   #3
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 1,902

Original Poster
Rep: Reputation: 158Reputation: 158
Thanks scasey,

Interesting you mention spam. The issue I am fighting about with this ISP is that messages from a mailing list to which I subscribe are being refused as spam WHEN I FORWARD THEM TO THE ISP FROM AN EXTERNAL EMAIL PROVIDER. I can receive the messages directly to the ISP's email from the list. This morning I tried to forward one of the emails from the ISP's email to another address at the same domain. I received this message
Quote:
An error occurred while sending mail. The mail server responded: 5.7.1 [P4] Message blocked due to spam content in the message.. Please check the message and try again.
I have seen this when I am connected with a VPN and use Thunderbird on my computer. The originating address is blacklisted by the ISP (although I cannot get them to understand/admit that). So I disconnected the VPN first and connected from and ISP IP address on my computer. Their span filter obviously did not flag the message as it arrived on my account at the ISP from which I was trying to forward it.

In the middle of all this crap I saw a message on the IPS's webmail page about the offending message being too large (140 kB) and thus email addresses, URLs and other action objects were displayed as text. That prompted me to send a bigger email. There were no images in my large email and no "action objects."

I need to just bag my ISP's email and stick with Protonmail. Actually I have pretty much done that. However, over the weekend I needed to forward an offending email from Protonmail to someone with an address at the ISP. That started the whole fiasco.

Thanks again for your input. It is time to go do something else

Ken
 
Old 08-13-2019, 02:07 PM   #4
ehartman
Member
 
Registered: Jul 2007
Location: Delft, The Netherlands
Distribution: Slackware
Posts: 836

Rep: Reputation: 416Reputation: 416Reputation: 416Reputation: 416Reputation: 416
Quote:
Originally Posted by taylorkh View Post
why is it used when sending an email? Email started when bandwidth was limited and expensive.
E-mail was started too when "text" meant "pure Ascii" (bytes with value 0-127), no 8-bit chars (like iso-8859, utf-8 etc). So base64 (and uuencode cq xxencode before that) was invented to send anything that was NOT pure Ascii, because the E-mail might go through systems that didn't use ASCII at all (like IBM mainframes in the BITnet part of internet, which used EBCDIC) as the correct transmission of any byte > 127 wasn't ensured.
And yes, it makes "what you send" larger as the encoding only uses 64 different characters (out of the 256 possible byte values for a binary file), so it could be as much as 4 times the size. Your MUA should use base64 only, though, when it detects any byte > 127 (cq <= 0 when handled as a signed value) in your input.
 
Old 08-13-2019, 02:40 PM   #5
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 1,902

Original Poster
Rep: Reputation: 158Reputation: 158
Thanks ehartman,

I am familiar with uuencode and yenc from back in Usenet days. I had not run across base64 until this latest exercise.

Ken
 
Old 08-13-2019, 03:47 PM   #6
ehartman
Member
 
Registered: Jul 2007
Location: Delft, The Netherlands
Distribution: Slackware
Posts: 836

Rep: Reputation: 416Reputation: 416Reputation: 416Reputation: 416Reputation: 416
Quote:
Originally Posted by taylorkh View Post
I am familiar with uuencode and yenc from back in Usenet days. I had not run across base64 until this latest exercise.
base64 came with the first implementations of MIME, which then was an abbreviation of
Multipurpose Internet Mail Extensions,
to extend E-mail to handle data that was NOT "pure ASCII" (or pure EBCDIC text on computers, using that characterset).
From wikipedia
Quote:
In computer science, Base64 is a group of binary-to-text encoding schemes that represent binary data in an ASCII string format by translating it into a radix-64 representation. The term Base64 originates from a specific MIME content transfer encoding. Each Base64 digit represents exactly 6 bits of data. Three 8-bit bytes (i.e., a total of 24 bits) can therefore be represented by four 6-bit Base64 digits.
 
Old 08-14-2019, 05:21 AM   #7
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 1,902

Original Poster
Rep: Reputation: 158Reputation: 158
Thanks ehartman
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Firefox adult/drug/gambling/alcohol site filter: md5 encryption/deciphering & base64 encoding/decoding l0f4r0 General 17 03-10-2019 04:48 PM
[SOLVED] base64 encoding in program vs piping to base64 command-line utility 0x53h Linux - General 3 08-05-2016 10:39 AM
Chinese encoding not encoding in kate linuxmandrake Linux - Software 1 12-12-2010 08:50 AM
Recommended XML / Base64 encoding libraries? overbored Programming 2 10-14-2004 11:21 AM
problem in installing MIME-Base64 Joe_Zhu Linux - Software 2 01-01-2004 07:15 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 09:36 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration