Hello!

I manage a small mail server that hosts a few domain's e-mail. In the last couple of weeks, I have been getting replies from various sources that shows our domain name and IP as the source of spam.

While looking into this, I found the /var/spool/mqueue file and several files in there. Some research took me to the qfs* files and their format. I see details that show my public IP address as the source, but also a $s[192.168.0.28] line as well. (This is NOT our internal IP address space]. As this is not a publicly routable IP address, I am confused as to how these emails have entered into my system.

1. Is my server compromised and a trojan generating the e-mail?
2. Is someone outside of my facility somehow relaying through my server via a spoofing technique?

I am running Fedora Core 15, sendmail 8.14.5.
I am using local-host-names for the various domains we support, access file to note any machines that need to RELAY (currently all turned off for testing) and I require authorization for all outside connections to RELAY (for cell phone e-mail connections).

Any ideas?

TIA

Mark

Fedora Core 15???? Start there...the latest is 20, so you're FIVE major versions behind, which include a LOT of security fixes and updates to sendmail, now at 8.14.9. If you are managing a server, use a server-class distro with a long life cycle, like CentOS.

Also, never put down to maliciousness what you can attribute to stupidity. Have you asked any of your users on the domains you host, if they have done anything fishy? It could be as simple as someones Windows PC with a trojan/infection that's shoveling out emails, and your server is just along for the ride, since they have access to it. Check your domains first...if you don't have many, disable them one at a time for a period, and watch the mail logs...if the spams stop going out, you've found your culprit.

Checking your system for a rootkit would be a good idea, and also check the integrity of your firewall and DMZ servers. Without more information, we can't tell you what else to check, since we don't know your environment.

Thanks for the thoughts! I did take an inventory of every device that connects from inside the firewall. I have an assortment of android phones, tablets, a sony DVD that includes internet access, a Wii game, two laptops (Windows Vista and Windows 8), two desktops PC's (Windows 7 and Windows 8) as well as a Fedora 18 client machine. I have attempted to isolate each machine to see if the source of the spam is coming from any of them. It does not appear to be so.

What was interesting to me was the content of the mail queue files that I was able to see. Here is a part (edited)

::::::::::::::
qfs7509RGu027444
::::::::::::::
V8
T1407197371
K1407247771
N41
P3840488
I253/1/9935
MDeferred: Connection timed out with 110mail.net.
Fwbs
$_Dynamic-IP-1815508495.cable.net.co [181.55.84.95] (may be forged)
$rESMTP
$s[192.168.1.6]
${daemon_flags}
${if_addr}www.xxx.yyy.zzz <-This was my public IP address
S<me@mydomain.com> <-This was my personal email address
MDeferred: Connection timed out with 110mail.net.
rRFC822; esgf74@110mail.net
RPFD:<esgf74@110mail.net>
MDeferred: Connection timed out with mp3-world.us.
rRFC822; spokefawovasp@mp3-world.us
RPFD:<spokefawovasp@mp3-world.us>
H??Received: from [192.168.1.6] (Dynamic-IP-1815508495.cable.net.co [181.55.84.9
5] (may be forged))
(authenticated bits=0)
by mydomain.com (8.14.5/8.14.5) with ESMTP id s7509RGu027444
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
Mon, 4 Aug 2014 20:09:31 -0400
H??Full-Name: My Name
H??Subject:
H??From: nmsVIAGRA <me@mydomain.com>
H??Content-Type: text/plain;
charset=charset="iso-8859-2"
H??X-Mailer: iPhone Mail (10A523)
H??Message-Id: <5558BADB-1067-9945-5AFC-3EE804CC0818@alpha2.com>
H??Date: Tue, 5 Aug 2014 02:56:11 -0700
H??To: "ebonylance@uswest.net" <ebonylance@uswest.net>
H??Content-Transfer-Encoding: quoted-printable


Notice that this mail came in from a dynamic IP address (first clue),
the sending IP source is listed as 192.168.1.6 (second clue as this is a non-routable address and is NOT my internal net)

My sendmail is configured to NOT allow unresolvable domains, and I have turned off all internal sources of mail (localhost, etc.) I use tcpdump to watch for outgoing packets with a destination port of 25 and a source host of my public IP. I see that there is outgoing traffic, but I cannot locate it's source.

What I am looking for is a tool that might be better than tcpdump, or maybe another way to use tcpdump. I was hoping to find the originating process id for the packets...

I understand that I need to upgrade the server software. It has been running since FC 15 was current without any issues until now. If I was sure that cleaning the machine and reloading the software would solve the problem, I would do that over a week-end. I was just concerned that maybe the spammer was getting in some other way and the upgrade would not solve that problem.

Thoughts and comments are welcome.

Well, that's one step further in troubleshooting, at least.
Did you check for a rootkit on your server yet? And if this is your home system, and you've got a small number of devices, I'd strongly suggest using the access.db feature in sendmail. That way, the ONLY way for mail to get relayed through your sendmail server, is if you manually add the address/host into the access database. This site has an explanation of it, with example:
http://www.faqs.org/docs/linux_network/x15291.html
Unless you scan for a rootkit or some other malware, you won't know. FC15 is VERY old, and doing a fresh install (that is, format drives, reload only DATA from backups) wouldn't be a bad thing. It will apply a bunch of security updates and patches all at once, and if there ARE any nasty things, chances are it'll get rid of them.

You also should be able to get into your router, and block/filter traffic there.

Thanks for your post. I do use the access.db file to control relaying.

Since I am getting pretty close to eliminating all other options, I am thinking that the service has been compromised. It is the only explanation that fits the facts. The spam appears to be originating on the server (as long as I have cut off relaying from all other sources via access.db this appears to be logical). My firewall prevents any internal PC from going out on ports 25,465 and 587 (I block packets being forwarded through the server with these destination ports). I am also using SELinux features on this server.

What I can't get my head around is how someone could get into my system. Passwords are fairly complex, I have denyhosts running and I have never in 15 years of running linux been compromised.

Unless I can think of another scenario I will need to rebuild the server with the latest FC over the weekend.

Thanks for your thoughts.