LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 05-18-2004, 01:45 AM   #1
J_Szucs
Senior Member
 
Registered: Nov 2001
Location: Budapest, Hungary
Distribution: SuSE 6.4-11.3, Dsl linux, FreeBSD 4.3-6.2, Mandrake 8.2, Redhat, UHU, Debian Etch
Posts: 1,126

Rep: Reputation: 58
My victory over spams


In the past few days I had a great victory over spams: out of the 200 spams my users used to get daily, not more than 2 could get through our mail server.

The victory is due to spammers' stupidity: they think they have a greater chance to push their spams through our mail filter if they masquerade them (in the 'Received:' header field) as if they were sent by the IP address of our mail server.
It was a great fault from them: our mail server is so configured that it would never insert our IP address in the header of any legal mails.

So, all I had to do is to insert a procmail rule that trashes mails with our IP address in the 'Received:' header field, and some 90% of spams is catched!

Spamassassin with the bayesian filter catches another 9%, so there is only some 1 % or less of spams getting through, meanwhile I have virtually no false positives.

This is the first time that the great majority of the mails in the mailboxes of my users is legal mail, while there are hardly any spams there.

As long as spammers are this stupid...

Last edited by J_Szucs; 05-18-2004 at 02:31 AM.
 
Old 05-18-2004, 06:53 AM   #2
fancypiper
Guru
 
Registered: Feb 2003
Location: Sparta, NC USA
Distribution: Ubuntu 10.04
Posts: 5,141

Rep: Reputation: 58
But they still steal valuable bandwidth that could be much better used downloading Linux/BSD ISOs..

If people were just smart enough not to connect a Windows computer to any kind of network, bayesian filters, etc wouldn't be needed.....
 
Old 05-18-2004, 08:15 AM   #3
J_Szucs
Senior Member
 
Registered: Nov 2001
Location: Budapest, Hungary
Distribution: SuSE 6.4-11.3, Dsl linux, FreeBSD 4.3-6.2, Mandrake 8.2, Redhat, UHU, Debian Etch
Posts: 1,126

Original Poster
Rep: Reputation: 58
Quote:
If people were just smart enough not to connect a Windows computer to any kind of network, bayesian filters, etc wouldn't be needed.....
You are right, and I do what I can to change this around me.

Since I am regarded as The Windows Guru among my colleagues, I am often asked for help with their Windows machines at work or home.
Nowadays, I always refuse any support for Windows or MSOffice, but offer them to install a fresh Linux and OpenOffice with my support.

Unfortunately, I could only turn two of my colleagues to Linux, suprisingly the most computer illiterate ones.

Those who think they have some computer knowledge stick with Windows and would not even consider to convert or even to have a dual-boot system.
However, I think many of them would change their mind if Microsoft were just a bit more eager to fight against stolen Microsoft softwares...

Last edited by J_Szucs; 05-18-2004 at 08:24 AM.
 
Old 09-21-2004, 01:56 AM   #4
J_Szucs
Senior Member
 
Registered: Nov 2001
Location: Budapest, Hungary
Distribution: SuSE 6.4-11.3, Dsl linux, FreeBSD 4.3-6.2, Mandrake 8.2, Redhat, UHU, Debian Etch
Posts: 1,126

Original Poster
Rep: Reputation: 58
My victory over spams and viruses is total now!

Last week's statistics of the mailbox of the user who has a highly outstanding mail traffic on my mail server:

Mails total: some 300
Spams total: some 270
(False virus notifications and viral mails are also regarded as spams!)

Out of the 270 spams:
- blocked on the mail server: 268
- reached the user but marked as spam: 2
- reached the user without being marked as spam: 0

False positives: 0

Summary: 100% of the spams was recognized, less than 1% of them could reach the user's mailbox, meanwhile there were no false positives at all!

What the greatest is that this result is reached with a mailserver that is so configured that it shall never ever make a fault by blocking a false positive. And actually, this has not occured for years, now.

Procmail + spamassassin with bayesdb rocks!

Last edited by J_Szucs; 09-21-2004 at 02:07 AM.
 
Old 09-21-2004, 02:23 AM   #5
theYinYeti
Senior Member
 
Registered: Jul 2004
Location: France
Distribution: Arch Linux
Posts: 1,897

Rep: Reputation: 61
Could you please post the relevant parts of your procmail and postfix configuration, that makes you sure your server won't use your IP by itself, and leads to being able to reject 90% of spam, and being sure there's no false positive among them?
I also run postfix, with a very simple popfile filter ("categorizer" would be more appropriate), and I'd be interested in reproducing this behaviour.

Yves.
 
Old 09-21-2004, 07:05 AM   #6
J_Szucs
Senior Member
 
Registered: Nov 2001
Location: Budapest, Hungary
Distribution: SuSE 6.4-11.3, Dsl linux, FreeBSD 4.3-6.2, Mandrake 8.2, Redhat, UHU, Debian Etch
Posts: 1,126

Original Poster
Rep: Reputation: 58
There is nothing special, it is a default sendmail setup on FreeBSD.

I just happened to analize the structure of local mails originating from our mail server and also the mails relayed by it, and suddenly realised that the VERY FIRST Received header field never contains the public IP of our mailserver.

If it is a mail originating from localhost, then the very first "Received" header looks like this:
Received: (from someuser@localhost)

If it is a relayed mail, it looks like this:
Received: from someothermachine (someothermachine [xxx.xxx.xxx.xxx])

But, if it is a spoofed spam, then it looks like this:
Received: from our.own.ip.add ([som.oth.ip.add])

The structure of the Received header is like this:
Received: from A (B [C])

where
- A is the identifier by which the mail client on the sender machine identifies itself for the first mail server that gets the mail for relaying
- B is the full hostname of the sender machine, as determined by dnslookup performed by the first relaying mailserver (which is the only server that knows the real IP address of the sender!!!)
- C is the IP address of the sender machine, known by the first mailserver relaying the mail

Spammers usually put our IP address or domain name in field A, but fields B and C are contradictory, as those contain name and IP addresses of a machine for which our mailserver would never act as a mail relay. This reveals that the mail is spoofed, and probably spam.

A brand new tendency that spammers falsify not only the first (from) line of the Received header, but also the second (by) line. This is accidentally also catched by my procmail rules.

Last edited by J_Szucs; 09-21-2004 at 07:23 AM.
 
Old 09-21-2004, 07:12 AM   #7
theYinYeti
Senior Member
 
Registered: Jul 2004
Location: France
Distribution: Arch Linux
Posts: 1,897

Rep: Reputation: 61
Thank you!
 
Old 09-21-2004, 07:29 AM   #8
J_Szucs
Senior Member
 
Registered: Nov 2001
Location: Budapest, Hungary
Distribution: SuSE 6.4-11.3, Dsl linux, FreeBSD 4.3-6.2, Mandrake 8.2, Redhat, UHU, Debian Etch
Posts: 1,126

Original Poster
Rep: Reputation: 58
Just one note: under "the very first Received header" I mean the Received header appended by the first mailserver, and that happens to be the innermost Received header in the mail.
 
Old 09-21-2004, 09:59 AM   #9
scowles
Member
 
Registered: Sep 2004
Location: Texas, USA
Distribution: Fedora
Posts: 620

Rep: Reputation: 31
First, I have never been able to find a document that explains all the different header fields. I mean, if you take a look at sendmail.cf for the section that describes the header format:

HReceived: $?sfrom $s $.$?_($?s$|from $.$_)
$.$?{auth_type}(authenticated$?{auth_ssf} bits=${auth_ssf}$.)
$.by $j ($v/$Z)$?r with $r$. id $i$?{tls_version}
(version=${tls_version} cipher=${cipher} bits=${cipher_bits} verify=${ve
rify})$.$?u
for $u; $|;
$.$b

What the #$%*... Some of the above is self explanatory, but it would be nice if the fields were documented in a format that could be cross-referenced to an RFC of some sort.

Anyway, I "think" the OP might be testing the HELO announcement made by the remote MTA. At least thats what my analysis reveals when looking at postfix headers. The formats look similar for the first line.

If your using postfix, you can test if the remote MTA is sending the same name or IP address as your MTA during the HELO handshake by implementing smtpd_helo_restrictions tests in main.cf. If anyone is interested, I can post how I have this feature implemented at my end. FWIW: My nightly postfix report (pflogsumm) shows that the HELO test (on average) REJECTS about 20 sites a day. Not much, but they all add up. YMMV
 
Old 09-22-2004, 09:09 AM   #10
J_Szucs
Senior Member
 
Registered: Nov 2001
Location: Budapest, Hungary
Distribution: SuSE 6.4-11.3, Dsl linux, FreeBSD 4.3-6.2, Mandrake 8.2, Redhat, UHU, Debian Etch
Posts: 1,126

Original Poster
Rep: Reputation: 58
Well, your post gives me the idea of putting some uniq pattern into the "Received" header, the presence of which verifies that the mail was actually sent/relayed by our mailserver.

This might simplify my procmail rules!

As for testing HELO statement and rejecting mails based on it not that too risky? If you make a fault in the configuration, you loose mails, since they could not even arrive.
 
Old 09-22-2004, 02:57 PM   #11
dukeinlondon
Member
 
Registered: May 2003
Location: London
Distribution: kubuntu 8.10
Posts: 593
Blog Entries: 1

Rep: Reputation: 30
Quote:
Originally posted by J_Szucs


Those who think they have some computer knowledge stick with Windows and would not even consider to convert or even to have a dual-boot system.
.
Too true. They are quite proud of their knowledge of how to circumvent windows problems (however serious). But They also can't believe they've been spending all that time resolving problems they shoudn't have had in the first place.

Now, I've got a friend who just installed mandrake just to have a look and there's a few things that sounds obvious to the long time linux user that are hard to understand for non-ideological newbies : why does the dvd player not work (no codecs) ? When installing packages, why isn't there a choice of putting a desktop shortcut and how do you find the application in the menus ? And what do you do when you DON't find it in the menu ?
 
Old 09-22-2004, 03:22 PM   #12
scowles
Member
 
Registered: Sep 2004
Location: Texas, USA
Distribution: Fedora
Posts: 620

Rep: Reputation: 31
Well, your post gives me the idea of putting some uniq pattern into the "Received" header, the presence of which verifies that the mail was actually sent/relayed by our mailserver.

What you are proposing is actually quite common.

As for testing HELO statement and rejecting mails based on it not that too risky?

HELO test being risky? IMO, No MTA on this planet should ever announce its HELO name with your MTA's hostname or IP address. If it is, its SPAM or the e-mail admin for the remote MTA has no clue.

Long pause...

OK, I just enabled sendmail on my test system and verified that Field A (as you state in your post) is the HELO name that the remote MTA announced itself as. The data in the parens that follow the HELO name (fields b and c) can be decoded as (rDNS name of the remote MTA [IP of remote MTA])

So, based on my findings, I think you are actually testing the HELO name by testing the e-mail header through a procmail rule. With postfix, I'm testing the HELO as its received in the e-mail envelope, not the data portion.
 
Old 09-22-2004, 06:25 PM   #13
J_Szucs
Senior Member
 
Registered: Nov 2001
Location: Budapest, Hungary
Distribution: SuSE 6.4-11.3, Dsl linux, FreeBSD 4.3-6.2, Mandrake 8.2, Redhat, UHU, Debian Etch
Posts: 1,126

Original Poster
Rep: Reputation: 58
Quote:
What you are proposing is actually quite common.
For a lame like me its quite new.
Quote:
No MTA on this planet should ever announce its HELO name with your MTA's hostname or IP address
You are right, but spammers will not do that, either. They will mostly announce themselves under your DOMAIN name (or IP address, or a valid mailbox name).
Unfortunately, this is just what my Mozilla mail client on windows does. I.e. it announces itself as "foo.bar" in field A, where foo.bar is our domain name.

Now, if mozilla is on a notebook of a roaming user, and that user sends a mail e.g. from home via our mailserver, then our domain name will be in field A in the Received header of his mail (as Mozilla will still announce itself under our domain name).
So, I must test fields B and C, too, to find out if it is a legitimate mail or not. (Though it is true: Mozilla never announces itself under our IP address, so filtering by IP address is easy)
Consequently, if a mail has our domain name in field A, I must also check if field C contains a valid IP address. The valid IP addresses are those inside our LAN, and also those inserted by my popbeforesmtp script into /etc/mail/access for roaming users.

Often spammers do not insert our domain name, but a valid user (mailbox) name in field A. These are also catched via checking field C.

Sometimes spammers do not spoof the Received header, but spoof the From header (as if the mail was sent by one of my users). I catch these the same way: I check if the mail was sent from a legitimate IP address or not.

A new trick from spammers: they completely falsify the very first Received field, like this:
Received from somehost ( somehost [som.ip.add.res])
by our.mail.server
for ourvalidmailbox@ourdom.ain
I wonder how they do it? Maybe they relay the mail through a second mailserver of theirs, and that latter mailserver is temporarily setup to identify itself under the hostname of our mailserver?
Anyway, these mails are catched by IP address, too, as long as they use a spoofed sender in the From header field.

Last edited by J_Szucs; 09-22-2004 at 06:59 PM.
 
Old 09-22-2004, 07:44 PM   #14
scowles
Member
 
Registered: Sep 2004
Location: Texas, USA
Distribution: Fedora
Posts: 620

Rep: Reputation: 31
Sounds like you and I are testing/rejecting/accepting e-mails based on similar criteria. Just in a different way.
 
Old 09-22-2004, 08:14 PM   #15
scowles
Member
 
Registered: Sep 2004
Location: Texas, USA
Distribution: Fedora
Posts: 620

Rep: Reputation: 31
I wonder how they do it? Maybe they relay the mail through a second mailserver of theirs, and that latter mailserver is temporarily setup to identify itself under the hostname of our mailserver? Anyway, these mails are catched by IP address, too, as long as they use a spoofed sender in the From header field.

FWIW: If your interested in learning some of the tricks spammers use now-a-days (like hijacking an open proxy server), checkout: http://www.dnsbl.us.sorbs.net/faq/proxy.shtml

Reading the above link sure woke me up!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Major Victory for Freedom Lleb_KCir General 16 05-10-2005 11:35 AM
chop chop, dlink dwl 650 rev M problems? victory! rhoyerboat Linux - Wireless Networking 0 02-08-2005 06:04 AM
Starting Mozilla spams /var/log/messages Tinkster Slackware 7 08-22-2004 03:39 AM
Victory hexes in LGeneral for Linux woodywellhung Linux - Games 0 05-27-2004 05:02 AM
A small victory for PHP and open-source. Pcghost General 2 01-15-2004 07:40 AM


All times are GMT -5. The time now is 01:58 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration