[SOLVED] php mail() not reaching some destinations

dogpatch · 05-27-2017, 03:07 PM

Quote:

Originally Posted by Ser Olmy

You don't create SPF records on your server, you do that in the DNS domain of the sender e-mail address.

For instance, if you're sending e-mails from phpmailer@mydomain.com, there should be an SPF record in mydomain.com indicating that your server is a valid source of mails from that domain.

SPF records are nothing but DNS TXT records containing text in a very specific rule format. . .

I do indeed see a TXT record for my domain name, with the following content:

Code:

v=spf1 +a +mx +ip4:31.22.4.4 +ip4:82.163.176.17 ~all

May i assume that these two ip addresses are what i ought to validate via the above mentioned spam block databases?

Ser Olmy · 05-27-2017, 03:28 PM

Quote:

Originally Posted by dogpatch

I do indeed see a TXT record for my domain name, with the following content:

Code:

v=spf1 +a +mx +ip4:31.22.4.4 +ip4:82.163.176.17 ~all

May i assume that these two ip addresses are what i ought to validate via the above mentioned spam block databases?

Yes, perhaps.

The SPF record means:

SPF version 1 (v=spf1)
Accept mail from any host with a hostname in the domain (+a)
Accept mail from servers listed in the MX records (+mx)
Accept mail from IPv4 address 31.22.4.4 (+ip4:31.22.4.4)
Accept mail from IPv4 address 82.163.176.17 (+ip4:82.163.176.17)
Mail from other hosts should be treated as "SOFTFAIL", meaning the sending server is not validated, and mails could/should be subjected to further checks and spam filtering (the tilde in ~all)

The question is whether your server is one of the servers specified above.

There's another potential issue here. Even if your server is either 31.22.4.4 or 82.163.176.17, and the IPs are not blacklisted anywhere, delivery could still fail if your server also has an IPv6 address, and is able to use that when communicating with external mail servers.

Linux has full dual-stack capability (IPv4 and IPv6 simultaneously), and many applications support both protocols as well. If your MTA supports dual-stack and finds that the receiving domain has an MX record pointing to a hostname with an IPv6 address (AAAA record), if will probably prefer IPv6. And unless that IPv6 address has a reverse pointer to a hostname in your domain (and believe me, it hasn't), the SPF rule won't match your server anymore, and you're back in SOFTFAIL territory.

The first order of business should be to figure out which (routable) IP addresses have been assigned to your server, either directly or via a NAT router.

dogpatch · 05-27-2017, 04:08 PM

I think ip 31.22.4.4 is assigned to my server. This ip passes (is not listed) in either the spamhaus or spamcop databases, but is listed in sorbs. Sorbs shows the listing highlighted in green, which seems to indicate an older non-active listing, but am not sure. The sorbs site was a little difficult for me to navigate, and permitted only 2 queries, so i guess i have to wait before trying again to see why it's listed, and how to get it unlisted from sorbs.

Quote:

Originally Posted by Ser Olmy

Linux has full dual-stack capability (IPv4 and IPv6 simultaneously), and many applications support both protocols as well. If your MTA supports dual-stack and finds that the receiving domain has an MX record pointing to a hostname with an IPv6 address (AAAA record), if will probably prefer IPv6. And unless that IPv6 address has a reverse pointer to a hostname in your domain (and believe me, it hasn't), the SPF rule won't match your server anymore, and you're back in SOFTFAIL territory.

Getting way over my head now. How do i find this information for my domain, and what, if anything, can i do about it?

Ser Olmy · 05-27-2017, 04:29 PM

Quote:

Originally Posted by dogpatch

Getting way over my head now. How do i find this information for my domain, and what, if anything, can i do about it?

First, browse to http://whatismyipaddress.com from your server. If you don't have a GUI, no matter; running links http://www.whatismyipaddress.com or lynx http://www.whatismyipaddress.com (one of them ought to be there, and if not, they should be a simple package install away) in an SSH or console session will do just fine.

Right below the line "Your IP address is:" you'll see which IP address you're using when contacting other servers and the Internet in general. This address needs to be in the SPF record.

If you need to change the SPF record for some reason (missing/wrong IPv4 address, IPv6 on the server), you'll probably have to do that via your domain registrar. Most have a web interface for managing DNS zones, but if they don't you could just contact the support department.

dogpatch · 05-27-2017, 05:09 PM

Quote:

Originally Posted by Ser Olmy

First, browse to http://whatismyipaddress.com from your server

But that gives me the temporary ip assigned to me by my isp in Nicaragua. My hosting server is located somewhere in North America or possibly Europe, can't browse from there.

Quote:

Originally Posted by Ser Olmy

If you need to change the SPF record for some reason (missing/wrong IPv4 address, IPv6 on the server), you'll probably have to do that via your domain registrar. Most have a web interface for managing DNS zones, but if they don't you could just contact the support department.

My hosting service does give me cPanel, one whose functions is a Zone Editor, where i can view and modify the various 'A', 'TXT' and other records associated with my domains and subdomains. The fact that i see ip 31.22.4.4 in many of these automatically generated records makes me think that must be my server's ip address. Does that sound like a reasonable assumption? If there were an IPv6 address associated with my server, how would i know that? I see nothing that looks like IPv6 in any of these records.

dogpatch · 05-27-2017, 05:38 PM

Quote:

Originally Posted by scasey

We could tell how your mail is being sent from the headers, as Ser Olmy suggested...the challenge being that sanitizing them for a post would likely obfuscate what we need to see.

Here's an actual email header, pretty simple, only slightly sanitized. I have no control over the two 'return-path' lines, they are automatically generated and any return-path values i specify are overwritten.

Code:

Return-Path: <tigerber#sv16.byethost16.org>
Delivered-To: clare#tigerberry.com
Received: from sv16.byethost16.org
        by sv16.byethost16.org (Dovecot) with LMTP id oezuKW38KVniHQUAysSrog
        for <clare#tigerberry.com>; Sat, 27 May 2017 18:23:41 -0400
Return-path: <tigerber#sv16.byethost16.org>
Envelope-to: clare#tigerberry.com,
 clare#tigerberry.com
Delivery-date: Sat, 27 May 2017 18:23:41 -0400
Received: from tigerber by sv16.byethost16.org with local (Exim 4.89)
        (envelope-from <tigerber#sv16.byethost16.org>)
        id 1dEk7Z-001Q97-FH; Sat, 27 May 2017 18:23:41 -0400
To: Clarence <clare#tigerberry.com>
Subject: test email
Reply-To: <jim#tigerberry.com>
return-receipt-to: <jim#tigerberry.com>
From: Jimmy <jim#tigerberry.com>
To: Clarence <clare#tigerberry.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Message-Id: <E1dEk7Z-001Q97-FH#sv16.byethost16.org>
Date: Sat, 27 May 2017 18:23:41 -0400

Everything from the 'To:' header to the 'Content-Transfer-Encoding:' header is supplied by me. Everything above that, and the last two lines, are generated automatically.

(Just now noticed there are two 'To:' headers, the first one generated by mail(), the second identical one supplied in my script. Is that a problem?)

Quote:

Originally Posted by scasey

What are the values of SMTP and sendmail_path in your php.ini?

I have no clue where to find the php.ini, probably inaccessible to me.

scasey · 05-27-2017, 06:08 PM

Quote:

Originally Posted by dogpatch

My hosting service does give me cPanel, one whose functions is a Zone Editor, where i can view and modify the various 'A', 'TXT' and other records associated with my domains and subdomains. The fact that i see ip 31.22.4.4 in many of these automatically generated records makes me think that must be my server's ip address. Does that sound like a reasonable assumption? If there were an IPv6 address associated with my server, how would i know that? I see nothing that looks like IPv6 in any of these records.

31.22.4.4 is an IP hosted in the UK...and yes, the SORBs listing for that IP shows only 3 "spam" entries from 2010, so you're not being blocked there.
82.163.176.17 is also hosted in the UK (the same company I think). It is not in SORBs at all, although *I* have blocked email from that netblock since 12/2006 because they reject(ed) abuse reports. I've removed that block today.
Use:

Code:

whois IPaddress

to see where the IP is hosted. That's how my scripts find reporting addresses for UCE.
also,

Code:

dig yourdomainname.com any

will show you all the NS records for your domain. You can then use whois for the IP address of the A record to identify your hosting company.

Again, mail blocked by SORBs (or any RBL -- or by my personal filters) will generate a BOUNCE message. Since you're not seeing those, you need to figure out where they're going.

When I send out script-generated emails, the default From: and Return-Path: headers get set to anonymous@ If yours does the same and you don't have anything set up to accept mail to anonymous@ , you won't receive BOUNCE messages [been there]. It's possible to configure the From: and Return-Path to be sure it's a legitimate email address, but I don't know how to do that with php mail.

I don't see the answer to the question asked by Ser Olmy: What is the From: header on your email?

I think the discussion about SPF has been interesting, but given that your SPF record is "permissive," I doubt that is the issue in your case. And again, missing or mis-coded SPF will (Ok, should) cause the receiving server to generate a BOUNCE. Once you figure out why you're not seeing the BOUNCE messages and correct that, THEN you'll be able to see why your email from php mail is being rejected...IMO

[edit]
Your post made while I was working on this one...will review your headers and respond momentarily

scasey · 05-27-2017, 06:28 PM

sv16.byethost16.org is 82.163.176.17, so that's the server that's delivering your mail...and is one of those listed in the SPF record.

This example appears to be a "local" delivery...from and to the same domain, so there's not much info about routing to a foreign address, or what that foreign server might see.

Can you post the headers of an email from this server to your gmail account? That may reveal something more.

Also, suggest you munge email addresses...maybe replace @ with #...to prevent harvesting.

dogpatch · 05-27-2017, 11:07 PM

Quote:

Originally Posted by scasey

sv16.byethost16.org is 82.163.176.17, so that's the server that's delivering your mail...and is one of those listed in the SPF record.

This example appears to be a "local" delivery...from and to the same domain, so there's not much info about routing to a foreign address, or what that foreign server might see.

Can you post the headers of an email from this server to your gmail account? That may reveal something more.

Also, suggest you munge email addresses...maybe replace @ with #...to prevent harvesting.

Can't get on gmail right now, and don't know how to read their email headers, anyway. Here's the best i can do: I've sent an email to my gmail account which forwards it back to a different account on my server. Here's what i see in the header that comes back to me, sanitized a bit better this time:

Code:

Return-Path: <mygmailaddr+caf_=jim=tigerberry.com#gmail.com>
Delivered-To: jim#tigerberry.com
Received: from sv16.byethost16.org
        by sv16.byethost16.org (Dovecot) with LMTP id wT2NGhFEKlnzwQoAysSrog
        for <jim#tigerberry.com>; Sat, 27 May 2017 23:29:21 -0400
Return-path: <mygmailaddr+caf_=jim=tigerberry.com#gmail.com>
Envelope-to: jim#tigerberry.com
Delivery-date: Sat, 27 May 2017 23:29:21 -0400
Received: from mail-lf0-f47.google.com ([209.85.215.47]:36191)
        by sv16.byethost16.org with esmtps (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128)
        (Exim 4.89)
        (envelope-from <mygmailaddr+caf_=jim=tigerberry.com#gmail.com>)
        id 1dEotN-002xjf-AA
        for jim#tigerberry.com; Sat, 27 May 2017 23:29:21 -0400
Received: by mail-lf0-f47.google.com with SMTP id h4so21012131lfj.3
        for <jim#tigerberry.com>; Sat, 27 May 2017 20:29:19 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-original-authentication-results:x-gm-message-state:delivered-to
         :to:subject:reply-to:return-receipt-to:from:to:mime-version
         :content-transfer-encoding:message-id:date;
        bh=RGQO7vTIC7+Jl5bw1i1+4ouPRDzPjuH0Ia3pKVLCRl8=;
        b=dEclqLHGlppPrOG8DxoRQsm4zaiJmQXn6Oi2dmpVIDPQwWypBOPP6/cADHloeF+hPg
         sMJgX5uEJFfIs4L4VrOqBrxycCBFoJWWsqpefmhqiVHmLcAw7rSu67pv+An+cnkX3H8i
         JPDbRF1ygYUv/O4okiPHlLo5sP5Fa+VVF9tzKBmB8lpb4aUae8CQSRGEklm49/3Op4Z8
         Sun9XXx6FYz3Ee9lEglH3bf8jb+zy6uGL8CHdS1Vr3wDOmh3mSAQwkw0pOgLjm8qdUie
         3Hm0b5BmWzkZK1uGXExxAguUiVV5dt/eBogewSg/DuN6DkzrEuJ2gFg8IpQqLdEvTpeD
         8PBQ==
X-Original-Authentication-Results: mx.google.com;       spf=pass (google.com: domain of tigerber#sv16.byethost16.org designates 82.163.176.17 as permitted sender) smtp.mailfrom=tigerber#sv16.byethost16.org;       dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com
X-Gm-Message-State: AODbwcBJDUDL+abGUM5ClOzZIsplUGXsuTjLb1WDIupSUMaDhtcIdvqJ
        m3lMj3qITnwcOhnQREPjrlruzHAbfOgh1os7FsXh6Q==
X-Received: by 10.46.87.77 with SMTP id r13mr2618640ljd.69.1495942158500;
        Sat, 27 May 2017 20:29:18 -0700 (PDT)
X-Forwarded-To: jim#tigerberry.com
X-Forwarded-For: mygmailaddr#gmail.com jim#tigerberry.com
Delivered-To: mygmailaddr#gmail.com
Received: by 10.25.234.140 with SMTP id y12csp899291lfi;
        Sat, 27 May 2017 20:29:17 -0700 (PDT)
X-Received: by 10.28.145.12 with SMTP id t12mr8010976wmd.46.1495942157574;
        Sat, 27 May 2017 20:29:17 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1495942157; cv=none;
        d=google.com; s=arc-20160816;
        b=IxXzxd7kf3TtJsjpDh1B5vOafFs6PlAccHnVfddeCFDGlfmuQAzvtrvfY6T95Rf41R
         3W0QXEKCz+PiPeGfcgWn5GtGhLjE7ALadKdmz1z55RkW5f8QnAETP6roOoT1Zo2kFSln
         yebF7GG1kc0n2Dpr8IkyI+Y0r6Je4q6QuymkAb4Jw5QxbgeF2ebhL2fnyC5Rt1Eruwxx
         aMNlZOiZQCbM/ZYpLiCkROHbvFUQEMHHgmUmyrSxqGrY2wcpLCtf66pMhp7/AdmWMlbH
         YP2+5R0CJ/fiF4OlnIgaROi8E/OkEqQ20U0riiO919LVcJ1qLJcqtfZjB9/Y1MuH6/lg
         cY6g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=date:message-id:content-transfer-encoding:mime-version:to:from
         :return-receipt-to:reply-to:subject:to:arc-authentication-results;
        bh=RGQO7vTIC7+Jl5bw1i1+4ouPRDzPjuH0Ia3pKVLCRl8=;
        b=G/4GDujKtNYntlk/eFuwQ8dBa2AehtXnqknwA8ei6VOdxGKcVo2UKmty6DBJX0Kdu2
         0zPHPeqf+vfzefAB5rky/KXWlmyC8vXk5qWgwj0EJedOsTr1VxmQvI2WVpObfX5csGg7
         XfN5PskILlbk81v2W60ewlWDv0/9iX/ibmrANsyjFurtJEDqqyWkRs/xYW1YZ65tdrM5
         soVqB9BIGqeq5Ma1vZ4cOVj6mZ7L/2a7l0BVCspj5JpDYXzOjrOdo+mrjtQlH8EMbrQn
         SRCMadQ6p8hE7aUq7XBdbRx8d3fq68XP7og+z2ZTBDCLZSGlw4X2y3Wcp/3X/tvezIEt
         0wGg==
ARC-Authentication-Results: i=1; mx.google.com;
       spf=pass (google.com: domain of tigerber#sv16.byethost16.org designates 82.163.176.17 as permitted sender) smtp.mailfrom=tigerber#sv16.byethost16.org;
       dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com
Received: from sv16.byethost16.org (sv16.byethost16.org. [82.163.176.17])
        by mx.google.com with ESMTPS id g4si18907860wmf.17.2017.05.27.20.29.17
        for <mygmailaddr#gmail.com>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Sat, 27 May 2017 20:29:17 -0700 (PDT)
Received-SPF: pass (google.com: domain of tigerber#sv16.byethost16.org designates 82.163.176.17 as permitted sender) client-ip=82.163.176.17;
Authentication-Results: mx.google.com;
       spf=pass (google.com: domain of tigerber#sv16.byethost16.org designates 82.163.176.17 as permitted sender) smtp.mailfrom=tigerber#sv16.byethost16.org;
       dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com
Received: from tigerber by sv16.byethost16.org with local (Exim 4.89)
        (envelope-from <tigerber#sv16.byethost16.org>)
        id 1dEotK-002xj7-At; Sat, 27 May 2017 23:29:18 -0400
To: Jimmy <mygmailaddr#gmail.com>
Subject: Re:test for spam filter
Reply-To: <clare#tigerberry.com>
return-receipt-to: <clare#tigerberry.com>
From: Clarence <clare#tigerberry.com>
To: Jimmy <mygmailaddr#gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Message-Id: <E1dEotK-002xj7-At#sv16.byethost16.org>
Date: Sat, 27 May 2017 23:29:18 -0400
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - sv16.byethost16.org
X-AntiAbuse: Original Domain - gmail.com
X-AntiAbuse: Originator/Caller UID/GID - [826 501] / [47 12]
X-AntiAbuse: Sender Address Domain - sv16.byethost16.org
X-Get-Message-Sender-Via: sv16.byethost16.org: authenticated_id: tigerber/only user confirmed/virtual account not confirmed
X-Authenticated-Sender: sv16.byethost16.org: tigerber
X-Source: 
X-Source-Args: /usr/sbin/proxyexec -q -d -s /var/lib/proxyexec/cagefs.sock/socket /bin/cagefs.server 
X-Source-Dir: tigerberry.com:/public_html/mvoss/l

I see that you (scasey) nailed it when you said my server's address is 82.163.176.17. Other than that, i don't understand much of this. It may not reveal much even to you, since gmail now accepts mail from my server since i went into their spam folder and said, 'No, email from this address is not spam'.

Besides, mayn't gmail be dropping or altering some of the headers it originally received from me, before forwarding the email?

Won't be able to log into gmail until Monday. When i do, how might i look at the raw headers in an email in my gmail inbox or spam folder?

scasey · 05-28-2017, 04:18 AM

Quote:

Originally Posted by dogpatch

I see that you (scasey) nailed it when you said my server's address is 82.163.176.17. Other than that, i don't understand much of this. It may not reveal much even to you, since gmail now accepts mail from my server since i went into their spam folder and said, 'No, email from this address is not spam'.

That doesn't matter to us...telling gmail that your mail is not spam wouldn't change the headers.

Quote:

Originally Posted by dogpatch

Besides, mayn't gmail be dropping or altering some of the headers it originally received from me, before forwarding the email?

Probably not. We can see their recipt of the mail and the forwarding in the headers you posted.

Quote:

Originally Posted by dogpatch

Won't be able to log into gmail until Monday. When i do, how might i look at the raw headers in an email in my gmail inbox or spam folder?

There are help/support pages on gmail to tell you how to do that...something about view source maybe?

Frankly, I'm stumped. I'm at the point where I question the report from your recipient that they didn't receive the email. If that recipient is on a legitimate email server, you would get a BOUNCE if your mail wasn't delivered. Note that it is possible for email perceived as spam to be received and silently deleted, if the recipient configures to do that...
Sorry...

dogpatch · 05-29-2017, 08:25 AM

Quote:

Originally Posted by scasey

Frankly, I'm stumped. I'm at the point where I question the report from your recipient that they didn't receive the email. If that recipient is on a legitimate email server, you would get a BOUNCE if your mail wasn't delivered. Note that it is possible for email perceived as spam to be received and silently deleted, if the recipient configures to do that...
Sorry...

Or - - perthaps you have been successful in giving me good advice above. The gmail header posted was from an email sent after implementing some of the suggestions received. Sorry, should have noted that above.

Will live with this for awhile, and, if i see further problems, will report back with specifics. If not, i will at some point mark the thread as solved.

Thank you, all!

One further question: One of the changes i made was to eliminate the 'X-mailer' header which advertised my mail as coming from php. Do you foresee any problem in eliminating this header?

scasey · 05-30-2017, 03:55 PM

Quote:

Originally Posted by dogpatch

One further question: One of the changes i made was to eliminate the 'X-mailer' header which advertised my mail as coming from php. Do you foresee any problem in eliminating this header?

You only run the risk of someone rejecting mail because it doesn't have that header. I don't know how likely that is. It's certainly not required by the RFCs, AFAIK.

dogpatch · 06-03-2017, 02:40 PM

Have received several replies to recent emails, including one from the aforementioned associate who was previously not receiving my emails, and an automatic reply. Thread marked as 'solved'