problems with getmail4 and procmail... From "unknown"

goumba · 02-08-2014, 09:42 AM

Hey Guys,

I'm trying to set up getmail and procmail to retrieve mail from a POP3 server. I tried following the instructions at http://www.brain-dump.org/blog/entry/30 and the getmail FAQ but I am running into some trouble.

My getmailrc:

Code:

[options]
delete = False
verbose = 1
message_log = /home/anthony/Mail/getmaillog
message_log_verbose = 1

[retriever]
type = SimplePOP3SSLRetriever
server = pop.mail.yahoo.com
username = user@host.com
password = mypasswd

[destination]
type = MDA_external
path = /usr/bin/procmail
# I have tried with both arguments = ("-f", "%(sender)") and unixfrom
unixfrom = True

If I tail the procmail log (procmail has a single recipe to dump everything for now, as that's what it was doing anyway because of my problem), every entry says something like

Code:

From unknown  Sat Feb  8 10:29:31 2014
 Subject: Very Interesting
  Folder: Inbox/new/1391873371.9789_1.shuttle1				  17405

(It appears that the) the associated getmail log entry is

Code:

2014-02-08 10:29:31 msg   13/1828 (17849 bytes) msgid AMsPw0MAAPD-TlmsBgGVLXzBMEc from <unknown> delivered to MDA_external command procmail ()

It seems as though the From line is not being prepended to each message as it should be. In fact if I'm reading the getmaill log right, it looks like getmail cannot even figure out who it's from. Anyone have any suggestions to troubleshoot this? I have logs at maximum verbosy (so I think) and the output is as above. I googled, all instructions point to the fact I'm doing it right, and nothing specific to Yahoo's POP3 servers.

yooden · 02-10-2014, 05:53 AM

Is your mail delieverd to mbox format? If yes, what does the From_ header look like?
Do you have a From: header (not the colon)?

goumba · 02-10-2014, 08:31 AM

The From at the beginning is that which is added by getmail, apparently

Code:

From unknown Sun Feb  9 10:06:15 2014
Return-Path: <unknown>
X-Apparently-To:
...

Halfway through the headers, there's a From: (with colon) following Received:

yooden · 02-10-2014, 08:36 AM

Quote:

Originally Posted by goumba

The From at the beginning is that which is added by getmail, apparently

Necessarily, yes. Procmail wouldn't be able to recreate that information.

Next thing I would do is to get out trusty ol' Wireshark and have a look at an incoming mail. Please do that if possible and tell us what you see.

Quote:

Originally Posted by goumba

Halfway through the headers, there's a From: (with colon) following Received:

-v

goumba · 02-10-2014, 10:40 AM

Quote:

Originally Posted by yooden

Necessarily, yes. Procmail wouldn't be able to recreate that information.

Next thing I would do is to get out trusty ol' Wireshark and have a look at an incoming mail. Please do that if possible and tell us what you see.

Yep, having a bit of trouble with it and SSL at the moment... I'll post back as soon as I have the info.

Quote:

-v

?

yooden · 02-10-2014, 10:46 AM

Quote:

Originally Posted by goumba

Quote:

-v

?

You are too quiet, be more verbose. What does the header look like?

goumba · 02-10-2014, 11:09 AM

Quote:

Originally Posted by yooden

You are too quiet, be more verbose. What does the header look like?

Ha, being I gave you the From you didn't ask for as well, I thought the opposite

Here's the whole header (hope it isn't too much, grabbed the first I saw):

Code:

From unknown Mon Feb 10 11:01:19 2014
Return-Path: <unknown>
X-Apparently-To: xxx@yahoo.com via 67.195.15.185; Sun, 04 Sep 2011 12:45:15 -0700
Received-SPF: pass (domain of bounces.amazon.com designates 72.21.212.36 as permitted sender)
X-YMailISG: oY1Le9kWLDuuDRqo35EeBx8ofxENwyv216Vf9QkTBWq23IfW
 eNeNUQAf6u435elm1KwaJ3nWanbR6H.FnzLUXe8V36l28G6AJlTyrG_NLI2J
 EG1rDj4tT4egK2zSJz2oGXvV9FoSarr84oJkf2yffAY4OuAi6_0OG3lmA8NU
 14H0O6AEeO0a1x5n3x5O5_r_jEy9XCRpQWH.z684vzHtXE8mv1GFIHqjJNo_
 ICL4hOhUNtUkbEYQC5ImzQA8VBd7yPwhis0ZBrogsGhc1ivJCELuEXhfiZeJ
 7WqJ5ERWs9jit7u5xIVgxd9.eysxS.wNV6ZDwsV6c1YNcnaP97Dqbg7Txp..
 9M0.kDuKONOc5hlmPl9FKY2aHwAVzc9mVR2e2pbrXCbzmv8Rbyw8oOS8zOAv
 0u2nPpQtWfbvghANcrzgj5cchbcRIVoNRjWay6WIwlpLAuax7Y1hpBG.U20g
 CBA04xlwI_IL_.B4q4ZHHqYy__edSUTWlCTAKLMy5M2q7vV5ycR2_ZzilRS2
 8YsVRehOy8OT22qaPq.E_n0duHwwFB53QYBITE3TuaqWiuOzL805hTNsi2Ar
 y1aDoCPaE8Wl8Ah0lhy_K0.pQVxNIWg1PyJXbp2UwWHa2.YiHPrxcGYpmAtC
 KPTG9dktNKtMpNY7PCH_kAv89nBzD5Or7QiFauKNAImTeUHt.XmGFXO7cPEO
 tT2zJn3aLGU4qjFHSd5vwwnW6_5ZgiW3h6Tu.qRv0f4dSv3ohCuS3ZHJuChS
 sd_DxOeLM6VD9XmLSH9nrjPAm6Cm.Ne80jj84llmvCknDIwpR5h2vu3FOZWu
 IL9LjjHzTEHG1vCtmC3yL1GLZn1A8oULMrYQlFiThWCzwZTDnNSZm6JdO6uo
 NkBNnOzKvcMFqQZIh8_KgdAEcUe866UIRF33ICOYIj.62y6TXCmul17u9zFo
 LcNBEJc8wB1vErxlo19P1ZfySLx.Yxtaap6Aiz7D_qJ6poHpvRxci7KC_eQO
 oOEG9gBVMzk.hSZm9XsmupqFH2soBocg1tSfRd54I0fOd5UoX.i9O4VU9ZSv
 dpi_jcPdmUbxDh6_GUPDLQu8pe78..hqJiRUyjOhrGwQT7q.Hqya6NV7vty5
 fdfBbf.TcUGgkTtY47cqiJatFjQ_aDyYcw_JXfj4eV5cA__AdT3EHvveWLB4
 aVcPOkT6EwdFsR6nVKUVQdVrKpunaNRx.bZZmxFI6A--
X-Originating-IP: [72.21.212.36]
Authentication-Results: mta1017.mail.sk1.yahoo.com  from=comcast.net; domainkeys=neutral (no sig);  from=comcast.net; dkim=neutral (no sig)
Received: from 127.0.0.1  (EHLO smtp-out-212-36.amazon.com) (72.21.212.36)
  by mta1017.mail.sk1.yahoo.com with SMTP; Sun, 04 Sep 2011 12:45:15 -0700
Date: Sun, 4 Sep 2011 19:45:14 +0000 (UTC)
From: Victoria <xxxx@comcast.net>
To: "xxx@yahoo.com" <xxx@yahoo.com>
Message-ID: <urn.correios.msg.201109041945147338c8b6588048ccbe23a15af58bfb7d@1315165514400.rte-svc-na-6007.iad6.amazon.com>
Subject: Victoria has a Wish List to Share
MIME-Version: 1.0
Content-Type: multipart/mixed; 
	boundary="----=_Part_3560811_2644046.1315165514398"
Bounces-to: 201109041945147338c8b6588048ccbe23a15af58bfb7d@bounces.amazon.com

yooden · 02-10-2014, 11:23 AM

Quote:

Originally Posted by goumba

Ha, being I gave you the From you didn't ask for as well, I thought the opposite

I asked for both, I called one "From_", the other "From:". It's a bit unclear, but that was the common way to address these back when I frequented the Mutt lists.

Quote:

Here's the whole header (hope it isn't too much, grabbed the first I saw):
[...]

Looks harmless, except for the "unknown".

The only things mildly suprising is the single Received: line and Amazon. Could you ask Victoria whether Amazon provides MTA services for Comcast? With a DNS check maybe?

Would be difficult to switch to Fetchmail? (I know, I know, but it would only be for a short time...)

Other then that, let's see what Wireshark has to say about it.

(BTW: Switching away from Procmail was one of the best things I ever did, this thing is like Sendmail on acid. Have a look at Maildrop.)

yooden · 02-10-2014, 11:28 AM

Quote:

Originally Posted by yooden

Necessarily, yes. Procmail wouldn't be able to recreate that information.

Wait, Procmail could drop the info, then recreate the field empty.

What does the procmail.cf look like? Any non-obvious conf's included, system-wide perhaps?

goumba · 02-10-2014, 12:08 PM

Quote:

Originally Posted by yooden

I asked for both, I called one "From_", the other "From:". It's a bit unclear, but that was the common way to address these back when I frequented the Mutt lists.

Looks harmless, except for the "unknown".

The only things mildly suprising is the single Received: line and Amazon. Could you ask Victoria whether Amazon provides MTA services for Comcast? With a DNS check maybe?

Would be difficult to switch to Fetchmail? (I know, I know, but it would only be for a short time...)

Other then that, let's see what Wireshark has to say about it.

(BTW: Switching away from Procmail was one of the best things I ever did, this thing is like Sendmail on acid. Have a look at Maildrop.)

Aha, that email was sent by her via Amazon, so that's why the odd line.

I'll give fetchmail a try. Maildrop produces the same result (the "unknown" From). Of course if I remove unixfrom = True, or the arguments line, then the first From line is missing altogether.

goumba · 02-21-2014, 11:59 AM

Sorry for the long delay, got busy otherwise.

Well, I couldn't get Wireshark to play nice, so could not get the packets decrypted (ok, so I really didn't know what I was doing as far as SSL goes).

Tried the same exact emails with fetchmail and procmail (using the same procmail conf), and everything works as expected, so I'm going to have to look into an issue with getmail4 itself. Thanks for your help.

I'm going to use fetchmail for now until I can fix this, and set up with maildrop.

yooden · 02-21-2014, 03:12 PM

Fetchmail is usually considered a step down, but if it works, it works.

goumba · 02-21-2014, 04:41 PM

Yeah I know. I remember years ago using it, and when I went about setting it up again, every page said use getmail instead because of various issues with fetchmail. However, as you said, it works. If I do come across a solution, I'll post it here for all.