[MAILSERVER] Mail reported as spam by GMail, Outlook, Yahoo

Tonio- · 06-05-2017, 05:23 PM

Hello,

I'm trying to configure a mailserver for 2 weeks, but I still have a problem: the mails I (or my users) send to Gmail, Outlook or Yahoo as flagged as spam.

Config details

I use Postfix + Dovecot for the mail management.
Everything is hosted on a container, visualized in a dedicated server.
This container have a dedicated IP address (let's call it 1.2.3.4), different than the dedicated server's one (let's call it 111.222.333.444).

My DNS logic is

*.mydomain.tld -> IP of the dedicated server
mydomain.tld -> IP of the dedicated server
mail.mydomail.tld -> IP of the mail container
Reverse DNS from this IP to mail.mydomain.tld
MX mydomain.tld -> mail.mydomain.tld

Here my DNS zone file:

Code:

$TTL 3600
@	IN SOA dns200.anycast.me. tech.ovh.net. (2017053100 86400 3600 3600000 300)
                        IN NS     ns200.anycast.me.
                        IN NS     dns200.anycast.me.
                        IN MX 1   mail.mydomain.tld.
                        IN A      111.222.333.444
                        IN TXT    "v=spf1 a mx a:mail.mydomain.tld a:mydomain.tld mx:mail.mydomain.tld mx:mydomain.tld ip4:1.2.3.4 ip4:111.222.333.444 ~all"
                    600 IN TXT    "1|www.mydomain.tld"
*                       IN A      111.222.333.444
_adsp._domainkey        IN TXT    "dkim=all"
_dmarc                  IN TXT    "v=DMARC1; p=reject; sp=reject; rua=mailto:admin@mydomain.tld!8m; ruf=mailto:admin@mydomain.tld!10m; rf=afrf; pct=100; ri=86400"
mail                    IN A      1.2.3.4
mail._domainkey         IN TXT    "v=DKIM1; k=rsa; p=MYDOMAINKEY"
www                     IN TXT    "3|welcome"
www                     IN TXT    "l|fr"

Postconf -n:

Code:

alias_database = mysql:/etc/postfix/mysql-virtual-alias-maps.cf
alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf
append_dot_mydomain = no
biff = no
config_directory = /etc/postfix
inet_interfaces = 1.2.3.4, mail1.mydomain.lan, mail1,mail.mydomain.tld, localhost
maximal_backoff_time = 18m
maximal_queue_lifetime = 1d
milter_default_action = accept
milter_protocol = 2
minimal_backoff_time = 6m
mydestination = localhost, $myhostname
myhostname = mail.mydomain.tld
mynetworks = 127.0.0.0/8 192.168.100.0/24
myorigin = /etc/mailname
non_smtpd_milters = inet:localhost:12301
queue_run_delay = 6m
recipient_delimiter = +
relayhost =
smtp_bind_address = 1.2.3.4
smtpd_banner = $myhostname ESMTP $mail_name
smtpd_client_restrictions = permit_mynetworks, permit
smtpd_helo_restrictions = reject_invalid_hostname reject_non_fqdn_hostname
smtpd_milters = inet:localhost:12301
smtpd_recipient_restrictions = permit_sasl_authenticated, reject_invalid_hostname, reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, permit_auth_destination, reject_unauth_destination, check_policy_service unix:private/policyd-spf, reject
smtpd_sasl_auth_enable = yes
smtpd_sasl_path = private/auth
smtpd_sasl_type = dovecot
smtpd_sender_login_maps = ldap:/etc/postfix/ldap-virtual-mailbox-maps.cf mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
smtpd_sender_restrictions = reject_unknown_sender_domain, reject_unlisted_sender, reject_authenticated_sender_login_mismatch, permit
smtpd_tls_auth_only = yes
smtpd_tls_cert_file = /etc/ssl/certs/dovecot.pem
smtpd_tls_key_file = /etc/ssl/private/dovecot.pem
smtpd_use_tls = yes
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = ldap:/etc/postfix/ldap-virtual-mailbox-maps.cf mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
virtual_transport = lmtp:unix:private/dovecot-lmtp

What I already did

Triple check my conf
Set the SPF
Set DKIM
Set DMARC
Check with https://www.unlocktheinbox.com/
Check with http://mxtoolbox.com/
Checked in many blacklist: everything is ok

Misc:
In the headers of a Yahoo email, I have: X-YahooFilteredBulk: 1.2.3.4, I'm not sure of what does that mean.

Yahoo Message:

Code:

X-Apparently-To: email@yahoo.com; Mon, 05 Jun 2017 21:19:18 +0000
Return-Path: <antoine@mydomain.tld>
X-YahooFilteredBulk: 1.2.3.4
Received-SPF: pass (domain of mydomain.tld designates 1.2.3.4 as permitted sender)
X-YMailISG: NmnYGtUWLDuAfw7vmxO9guyd29n_SJrHE0Lb6B5IldCsUmZb
 c.JVM.MOkpsOyW.QRJul22QWdV.4BuUu2SXtKES.nMDR94465rdxyOxm5Wzy
 nFwAM6lx1Vkywp2ljFhe426EpqXJvaXjfh7qIk8nfMKha7O4zDXmDGRJf5Ph
 bokiiqTxc_uXvpeBHLAh6w_YrGWcEwPzdllWS3MQgkeu3d1YI2LbpFTG9IdH
 nkmKCcrOtjumjanqO.AfWLrY6I3HSLpYlpq7Y_JQ6FJsn4vaKcHHbhMi8qIf
 hK4stG6Cj8ADjYBhQav3mK.hGR_UuwRk.jyC8TNanamDZVQ1gXFbRPKjrvhq
 rHDx4ax2IPY3PPnOZPrB2YsRn5Dza4TAo8XwSbr_iiMGPa.5BwHIeckg8J7_
 v2.EbXHd.okInPqUz2KSV0NLJDNDstTj62edrHiLe0J4d_uu9ktyqBAGNaE0
 1ax0M__YWlNmo4sjCpdCZq61vxdAdubGC9nrakSZljRODetS7bLlgaBEbn8j
 7FpLBdEPMr7japaoU7YaYo7f1t6G0O5dzh.JKjBuyqe9bEzdYjyIu6z1NqYk
 0_AmiVmkuRtVlJ4zwETfQShkXCkk0SR7pv2pqT738TACPQnTZVdVZ7TUenoW
 3RR9ukCPJrBIdZCi_pCS.tdjDQe9NkxDBuo8zt3CbrrD_pgpLrmWb_OctknR
 B0hV.YmqqVV6qnkuLkWjokSgP1.gXabxK4MyC18xPq0bbYMcI9GfjU.1dp1w
 0LCz5PgEeP3wHc_y0ycHHAMrZSw.0fjIt8L.QieyiSbTTbNnmyA5DdkWtRqQ
 NyqNiWn9i7QL9uA8jU4OczvhhWy2CdGobCvhjCclWdadWJqNdKAqht1nu.44
 JDcc3yz7wv42Jqz7NsdAk9VM3yvgqyHkqRFRn9u5Glg6nkxOCAYqdz5SZ3uZ
 Ikm3C7_RNtYMgemcprpdr_xqKqz6uhlMxnlFMsTLmHY6KtKiw50ON3MZkjo2
 Xa3iY9MwUule1wUNTTUGy4CZjCdUQAvIl1tyYS.7SJzEvBhIN_LX6UnNSL4s
 Y2VsyJUbCAAXYg2QTXV5yaDJr9WmSE0H3mfsbnn3Z_dXM2YgHMjfMZvNKa5u
 QWAobol_8oNYXVs6gL9t4yuL4RLfjavdVSpenCkBXXT7dlEIUgP_vGYikY7v
 HPsbu3pZMUxIEbEe1DKhR9wN4Np7bJKvDc232kbthOAfIJnYLAPK6Fmiq6JH
 4uzMHdvDvJ6NqGnh30C.fGPcBRR3CCllFk6miYhNZpMSJIw0z5DKnpYCXcjC
 UfFAbcD.VOiSdgWOrGVVTcDbprXA_KLolBeR9JC4odjbOHxRSKDT5G0PFIkj
 W9B3tazltXZW3WHezbClfl9m37UmGHmkK87FKciQqpjDctreTiC9yMfeOk0G
 gMdPvH6eHau9oikfuMvnrgcfdno9myi1xNMIrc9LvR3_NqFX66dE4deLKU_H
 2EqmE8BgGqJjivdpAWLfqZQ-
X-Originating-IP: [1.2.3.4]
Authentication-Results: mta1117.mail.gq1.yahoo.com  from=mydomain.tld; domainkeys=neutral (no sig);  from=mydomain.tld; dkim=pass (ok)
Received: from 127.0.0.1  (EHLO mail.mydomain.tld) (1.2.3.4)
  by mta1117.mail.gq1.yahoo.com with SMTP; Mon, 05 Jun 2017 21:19:17 +0000
Received: from [192.168.1.220] (c-98-210-109-176.hsd1.ca.comcast.net [98.210.109.176])
	by mail.mydomain.tld (Postfix) with ESMTPSA id 1C1EA14C0EBA
	for <email@yahoo.com>; Mon,  5 Jun 2017 21:19:15 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mydomain.tld; s=mail;
	t=1496697556; bh=R8eR4kmLWkKynnHnZxw8ip8txd1LkVgyl2JAdEHbtio=;
	h=To:From:Subject:Date;
	b=ABAe3zHUe8kZnSTZ9f+1ET2yt+ux06PeHT2L4GWPZNRfy3E/uQTOsbAodJhAS0cJJ
	 Qiqw2GdOpWqbtsC9cl4/3bBMSr6FVNU8WN+XM1RJC8uAmcwqSb0c6VBANCNW7AbMwf
	 tmaQYoVm2nhHt3+1YDkgOjfYvK8PVEMJ0Mcu9yXY=
To: email@yahoo.com
From: ME <myemail@mydomain.tld>
Subject: Hi
Message-ID: <ae5afb63-506b-ffa5-03ba-88b58aad6ff0@mydomain.tld>
Date: Mon, 5 Jun 2017 14:19:14 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.1.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
Content-Length: 9

EMAIL CONTENT

Gmail Message:

Code:

Delivered-To: myemail@gmail.com
Received: by 10.31.182.13 with SMTP id g13csp194520vkf;
        Mon, 5 Jun 2017 14:16:50 -0700 (PDT)
X-Received: by 10.223.176.25 with SMTP id f25mr14587144wra.74.1496697410840;
        Mon, 05 Jun 2017 14:16:50 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1496697410; cv=none;
        d=google.com; s=arc-20160816;
        b=vB4Tmmnr4A8aVlu21bxFrc3i46XMJei+0RH7dPzcZP6zHkTDVV2Ymj0zQQ5BeTu7KJ
         OEYGre9lgiTfrEjrB//bby7VJnhu0zayGVerDVGddZ/7hQ1of78GxwWtjMm4wKi8wfE9
         44iYui2qdB3444hd3oj8KUTuR/ucE90xzUxmgi3a2PEefuezqLiSZw74r1myLkdHCXvd
         Lj71cbflz4YwNELhkThSoXqD+8CYT72ayrdw0+xI1a9kqhELFCY9TKBBdKvs5pWCXy02
         M9bNch+oX7gPO1RaZoC7CitUBSCZScWOAw7CNvhdAIy4z3CmNkz36E3atSpbqx6clssH
         x89A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=content-transfer-encoding:content-language:mime-version:user-agent
         :date:message-id:to:subject:from:dkim-signature
         :arc-authentication-results;
        bh=OLYzleSV97zJUpddJUkH+RtmG5agjgSFbH+7reBP2oE=;
        b=DKIMSIGNATURE==
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=pass header.i=@mydomain.tld;
       spf=pass (google.com: domain of myemail@mydomain.tld designates 1.2.3.4 as permitted sender) smtp.mailfrom=myemail@mydomain.tld;
       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=mydomain.tld
Return-Path: <myemail@mydomain.tld>
Received: from mail.mydomain.tld (mail.mydomain.tld. [1.2.3.4])
        by mx.google.com with ESMTP id w22si36747796wrb.325.2017.06.05.14.16.50
        for <myemail@gmail.com>;
        Mon, 05 Jun 2017 14:16:50 -0700 (PDT)
Received-SPF: pass (google.com: domain of myemail@mydomain.tld designates 1.2.3.4 as permitted sender) client-ip=1.2.3.4;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@mydomain.tld;
       spf=pass (google.com: domain of myemail@mydomain.tld designates 1.2.3.4 as permitted sender) smtp.mailfrom=myemail@mydomain.tld;
       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=mydomain.tld
Received: from [192.168.1.220] (c-98-210-109-176.hsd1.ca.comcast.net [98.210.109.176]) by mail.mydomain.tld (Postfix) with ESMTPSA id 0728414C0594 for <myemail@gmail.com>; Mon,
  5 Jun 2017 21:16:49 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mydomain.tld; s=mail; t=1496697410; bh=OLYzleSV97zJUpddJUkH+RtmG5agjgSFbH+7reBP2oE=; h=From:Subject:To:Date; b=uaT0emH0uk6oB/nmx5wgroVkqp3vgy5uv5GIyFIRtgfCgnSTvF0YWQgS1lAV6Rwbv
	 9o1ME8WCtZ1fcYPR5Ec1kEMePdwcrJDjMyf1ek5LBS2IFlIfIPjsHfHYjxNTvmkIr2
	 kZCEwjC3Dy5lrZEGxJLeW+/oPQKejeBUrZVquCbo=
From: Antoine <myemail@mydomain.tld>
Subject: Subject of the emai
To: myemail@gmail.com
Message-ID: <0adb2271-9c9b-6316-f59e-7d949522fc59@mydomain.tld>
Date: Mon, 5 Jun 2017 14:16:48 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit

EMAIL CONTENT

I really don't know what to do now. If anyone have an idea ?

Thanks a lot,
Tonio

wpeckham · 06-05-2017, 05:30 PM

Is your mail server properly registered as your domain MX (mail) server in the upstream DNS?

Habitual · 06-05-2017, 06:27 PM

and you identified the mail host as a "mail container" suggesting docker, or other?

Just sayin'

Tonio- · 06-05-2017, 07:25 PM

Hello,

Thanks for your answers!

For the DNS, I think this is the line "IN MX 1 mail.mydomain.tld.". Is that right configured ? Should I put the IP address instead?

For the container, I'm using ProxmoxVE on the dedicated server, and I've created some Containers for my different usages.

af7567 · 06-06-2017, 04:49 AM

Do you send out a lot of emails? If a few users on those services mark your mail as spam then it can end up being marked as spam for everyone using the service. I had that problem with AOL even though my DNS/SPF/DKIM/DMARC was all fine. It just took one person marking mails as spam (who was an elderly friend who thought she was pressing the delete button) and then I had to contact AOL to unblock my mail server.

edit: It could also just be that you have been sending a large amount of mail and you are using a new IP address which isn't trusted as mentioned on the outlook.com postmaster site https://mail.live.com/mail/troubleshooting.aspx

bathory · 06-06-2017, 05:25 AM

Quote:

For the DNS, I think this is the line "IN MX 1 mail.mydomain.tld.". Is that right configured ? Should I put the IP address instead?

Nope. MX always takes a FQDN as argument.

Quote:

For the container, I'm using ProxmoxVE on the dedicated server, and I've created some Containers for my different usages.

Apparently 98.210.109.176 is listed in some RBL lists.

TenTenths · 06-06-2017, 05:53 AM

98.210.109.176 also looks like it's an ISP dynamic assigned IP and doesn't "reverse" back to the FQDN of the mail server.

This alone can add several "spam points" to mail scoring.

Edit: In addition there are many mail servers that now won't accept mail if the reverse doesn't match.

wpeckham · 06-06-2017, 06:57 AM

The MX record in YOUR dns may not matter. Are you registered as a proper mail domain with your ISP so they forward the valid record. Newer major DNS services check things like this before forwarding mail. (Due to the amount of spam arriving from temporary invalid mail servers.)

af7567 · 06-06-2017, 07:38 AM

Quote:

Originally Posted by TenTenths

98.210.109.176 also looks like it's an ISP dynamic assigned IP and doesn't "reverse" back to the FQDN of the mail server.

This seems to be the IP address of the client PC which submitted the mail to the mail server. The mail server IP address has been replaced by 1.2.3.4. If we did know the IP address and domain name of the mail server it might make finding the problem easier

TenTenths · 06-06-2017, 08:00 AM

Quote:

Originally Posted by af7567

This seems to be the IP address of the client PC which submitted the mail to the mail server. The mail server IP address has been replaced by 1.2.3.4. If we did know the IP address and domain name of the mail server it might make finding the problem easier

Well spotted.

Tonio- · 06-06-2017, 11:34 AM

Quote:

Originally Posted by af7567

Do you send out a lot of emails? If a few users on those services mark your mail as spam then it can end up being marked as spam for everyone using the service. I had that problem with AOL even though my DNS/SPF/DKIM/DMARC was all fine. It just took one person marking mails as spam (who was an elderly friend who thought she was pressing the delete button) and then I had to contact AOL to unblock my mail server.

edit: It could also just be that you have been sending a large amount of mail and you are using a new IP address which isn't trusted as mentioned on the outlook.com postmaster site https://mail.live.com/mail/troubleshooting.aspx

It's a new service that I'm launching, still in beta. So if we sent 20 mails to these providers, it's a max.
At the beginning of the service (3 weeks ago), I used another IP address (the one of the dedicated server: 111.222.333.444), maybe we sent a few mails with this IP, and now the new one.

Quote:

Originally Posted by wpeckham

The MX record in YOUR dns may not matter. Are you registered as a proper mail domain with your ISP so they forward the valid record. Newer major DNS services check things like this before forwarding mail. (Due to the amount of spam arriving from temporary invalid mail servers.)

I'm not sure to understand this part. Actually I have my domain and it's DNS zone managed on my host company system.

Quote:

Originally Posted by af7567

This seems to be the IP address of the client PC which submitted the mail to the mail server. The mail server IP address has been replaced by 1.2.3.4. If we did know the IP address and domain name of the mail server it might make finding the problem easier

Yep exactly.
The domain is: mail.nebulae.co and the IP 217.182.239.228, if that could help.

scasey · 06-06-2017, 11:51 AM

Quote:

The domain is: mail.nebulae.co and the IP 217.182.239.228, if that could help.

A dedicated IP hosted by OVH in France. I get quite a bit of spam from OVH IP addresses, but I find them to be very responsive to abuse reports and seldom need to block them.

The domain TLD is Columbian. South American TLDs tend to be abusive as well...but as you say, neither the IP nor the domain seem to be block-listed in several RBLs I checked.

Also, I don't see anything in the headers you posted indicating that the emails were spammy.

The email is not being *rejected* by Yahoo! or gmail, right? Only routed to the spam folder of the receiving user? (which is you, I presume). That is usually managed by settings over which the recipient has control. Have you looked at the spam settings on the receiving accounts?
For example, adding the sending email address to the allowed senders might [should] prevent the email from being flagged as spam.

Generally, everything else looks good to me.

Tonio- · 06-06-2017, 11:55 AM

Quote:

Originally Posted by scasey

A dedicated IP hosted by OVH in France. I get quite a bit of spam from OVH IP addresses, but I find them to be very responsive to abuse reports and seldom need to block them.

The domain TLD is Columbian. South American TLDs tend to be abusive as well...but as you say, neither the IP nor the domain seem to be block-listed in several RBLs I checked.

Also, I don't see anything in the headers you posted indicating that the emails were spammy.

The email is not being *rejected* by Yahoo! or gmail, right? Only routed to the spam folder of the receiving user? (which is you, I presume). That is usually managed by settings over which the recipient has control. Have you looked at the spam settings on the receiving accounts?
For example, adding the sending email address to the allowed senders might [should] prevent the email from being flagged as spam.

Generally, everything else looks good to me.

Thanks for this details!

Right, the emails are not being *rejected*, just moved to spam folder (so in my server log, I have a 250 code, saying everything is okay).
The receiving user is me and some of my betatesters.
My concern is that the mails are tagged as spam with the default configuration. So all my users will have to tell all their contact to unspam them, it doesn't seem to be a viable solution.

scasey · 06-06-2017, 12:03 PM

Quote:

Originally Posted by Tonio-

Thanks for this details!

Right, the emails are not being *rejected*, just moved to spam folder (so in my server log, I have a 250 code, saying everything is okay).
The receiving user is me and some of my betatesters.
My concern is that the mails are tagged as spam with the default configuration. So all my users will have to tell all their contact to unspam them, it doesn't seem to be a viable solution.

All your user's contacts are on Yahoo or Gmail? Do you have somewhere else to send a test email? Does it get marked there?

My point is that if the mail is being delivered successfully, then there is nothing you can do to "fix" anything. IMO, it's not *your* problem. Your server is delivering the mail and it's being accepted.

Tonio- · 06-06-2017, 12:07 PM

Quote:

Originally Posted by scasey

All your user's contacts are on Yahoo or Gmail? Do you have somewhere else to send a test email? Does it get marked there?

Yep, almost all other providers are working well!

Quote:

Originally Posted by scasey

My point is that if the mail is being delivered successfully, then there is nothing you can do to "fix" anything. IMO, it's not *your* problem. Your server is delivering the mail and it's being accepted.

Damn, that sucks. Okay then. But I don't understand how other people achieve to send emails to this providers without been tagged as spam (with similar configurations). That's really strange..