LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 09-13-2006, 11:53 AM   #1
mastmind0
LQ Newbie
 
Registered: May 2003
Posts: 4

Rep: Reputation: 0
Postfix RCPT TO and TO headers


Does anyone know how to do something like this:
http://support.sssolutions.net/kb/in...v2&id=123&c=10
using Postfix?

Basically I want to be able to compare the RCPT TO addresses to the actual TO/CC addresses in the header. If an address in the RCPT TO is not in the TO/CC headers, then I want to discard the message. Obviously this will cause problems for anyone who does a BCC, but I can live with killing that feature. As I'm sure all of us are, I'm tired of getting emails where the TO in the header says something like "getmore@RxMeds.com", and obviously the RCPT TO line contained my email address. If I could compare and make sure that all RCPT TO addresses are listed in the TO/CC headers, that should eliminate that problem

Any help would be greatly appreciated. Also, if anyone sees any drawbacks to the above, please point them out.

Thanks!
 
Old 09-13-2006, 01:45 PM   #2
ramram29
Member
 
Registered: Jul 2003
Location: Miami, Florida, USA
Distribution: Debian
Posts: 848
Blog Entries: 1

Rep: Reputation: 47
Run spamassassin - it can take care of this for you.

In postfix there is a way to do this using header_checks also but it will slow it down - I'd rather use spamassassin.
 
Old 09-13-2006, 07:32 PM   #3
generic_user
Member
 
Registered: Sep 2006
Location: San Francisco, Ca.
Distribution: Redhat/Fedora/CentOS
Posts: 39

Rep: Reputation: 15
Quote:
Originally Posted by ramram29
Run spamassassin - it can take care of this for you.

In postfix there is a way to do this using header_checks also but it will slow it down - I'd rather use spamassassin.
I can say with certainty that spamassassin will be a much slower solution to this than using postfix! Much better to stop the email before it has to be processed by other pipes like spamassasin (which is a resource hog).
 
Old 09-13-2006, 07:36 PM   #4
generic_user
Member
 
Registered: Sep 2006
Location: San Francisco, Ca.
Distribution: Redhat/Fedora/CentOS
Posts: 39

Rep: Reputation: 15
Quote:
Originally Posted by mastmind0
Does anyone know how to do something like this:

Basically I want to be able to compare the RCPT TO addresses to the actual TO/CC addresses in the header. If an address in the RCPT TO is not in the TO/CC headers, then I want to discard the message. Obviously this will cause problems for anyone who does a BCC, but I can live with killing that feature. As I'm sure all of us are, I'm tired of getting emails where the TO in the header says something like "getmore@RxMeds.com", and obviously the RCPT TO line contained my email address. If I could compare and make sure that all RCPT TO addresses are listed in the TO/CC headers, that should eliminate that problem

Any help would be greatly appreciated. Also, if anyone sees any drawbacks to the above, please point them out.

Thanks!
It sounds like you want to stop backscatter. This url will answer your questions:
damn.. I can't post urls..

Search google for "backscatter postfix howto"

Note that the ultimate check would be to compare RCPT TO or MAIL FROM with the contents of the actual email... for this you need a SENDERID plugin.
 
Old 09-14-2006, 09:20 AM   #5
ramram29
Member
 
Registered: Jul 2003
Location: Miami, Florida, USA
Distribution: Debian
Posts: 848
Blog Entries: 1

Rep: Reputation: 47
Quote:
Originally Posted by generic_user
I can say with certainty that spamassassin will be a much slower solution to this than using postfix! Much better to stop the email before it has to be processed by other pipes like spamassasin (which is a resource hog).
Like I originally said above, you can use postfix header_checks or you can use spamassassin. However, I think you are actually wrong. Your SMTP server should be as fast as possibe and if you have mega SPAM and use it to do header_checks then you will bring it down to it's knees very soon. My recommendation is to let the SMTP server (postfix) receive as much mail as possible as fast as possible and hand it down to another program for filtering SPAM (spamassassin).

I respect your opinion but I follow what other experts say: This how SPAM best works. Postfix can easily recieve 100 regular messages per second unchecked an put them in it's queue. Even if it receives 1000 messages per second it will put them all in its queue and continue. Spamassassin will then pick up the message and filter it and resend it to postfix but this time reinserting the message in the queue will be much faster because it is local.

If you do header checks with postfix you and you get 1000 messages with SPAM then postfix will have to check the message and reinsert it into itself. That's too much work and it will slow your CPU and blow up your memory. In the UNIX/Linux processes are designed with many minute programs that all do one thing and pass it on to the next program. Hence, making things more efficient, faster and stable.

Furthermore, you get a very rich SPAM filter by using spamassassin.
 
Old 09-14-2006, 10:04 AM   #6
mastmind0
LQ Newbie
 
Registered: May 2003
Posts: 4

Original Poster
Rep: Reputation: 0
Unfortunately, even if I wanted to use Postfix to perform the header_checks it doesn't look like I'd be able to since postfix doesn't seem to have the ability to combine multiple checks like spamassassin does via macros. So it looks like spamassassin may be my best bet, however I've written many custom rules for spamassassin and I'm having a problem seeing how spamassassin can do this. Normally I check headers for predefined values, like "header MYRULE X-Original-To =~ /username\@domain\.com/". Not sure how I can do something like "header MYRULE X-Original-To =~ To|CC" (obviously this won't work, but that's pretty much what I need it to do). Anyone have any ideas?

Thanks for the responses I've gotten so far, I really do appreciate it.
 
Old 09-14-2006, 03:16 PM   #7
generic_user
Member
 
Registered: Sep 2006
Location: San Francisco, Ca.
Distribution: Redhat/Fedora/CentOS
Posts: 39

Rep: Reputation: 15
Quote:
Originally Posted by ramram29
Like I originally said above, you can use postfix header_checks or you can use spamassassin. However, I think you are actually wrong. Your SMTP server should be as fast as possibe and if you have mega SPAM and use it to do header_checks then you will bring it down to it's knees very soon. My recommendation is to let the SMTP server (postfix) receive as much mail as possible as fast as possible and hand it down to another program for filtering SPAM (spamassassin).
What RFC states that "SMTP server should be as fast as possible"? Gerylisting, and some other tricks to prevent spam rely on slowing the server down. There's one trick where you purposefully add a long pause before sending the HELO query in an attempt to get the spammer to send its data before being asked for it.

Quote:
I respect your opinion but I follow what other experts say: This how SPAM best works. Postfix can easily recieve 100 regular messages per second unchecked an put them in it's queue. Even if it receives 1000 messages per second it will put them all in its queue and continue. Spamassassin will then pick up the message and filter it and resend it to postfix but this time reinserting the message in the queue will be much faster because it is local.
So, by your own admission, postfix now has to process the email twice... before spamassasin, and afterwards... Thus cutting its ability to process new emails in half.

Quote:
If you do header checks with postfix you and you get 1000 messages with SPAM then postfix will have to check the message and reinsert it into itself.
wrong, it checks the header (read: before accepting the data of the email). If the rules don't match, it drops the sender... No bandwidth lost accepting the spam itself, and no point in having spamassassin (or any other list of piped processes) parse the data... It's a big fat waste.

Quote:
That's too much work and it will slow your CPU and blow up your memory. In the UNIX/Linux processes are designed with many minute programs that all do one thing and pass it on to the next program. Hence, making things more efficient, faster and stable.

Furthermore, you get a very rich SPAM filter by using spamassassin.
Heh.. performing a regex on a header will absolutely not "blow up your memory" nor is it likely to "slow your CPU". We're talking 3 lines of text or less. I'd be amazed to see a system today that would be bogged down cpu or memory wise before running into trouble with bandwidth to their isp first.
 
Old 09-14-2006, 03:21 PM   #8
generic_user
Member
 
Registered: Sep 2006
Location: San Francisco, Ca.
Distribution: Redhat/Fedora/CentOS
Posts: 39

Rep: Reputation: 15
Quote:
Originally Posted by mastmind0
Unfortunately, even if I wanted to use Postfix to perform the header_checks it doesn't look like I'd be able to since postfix doesn't seem to have the ability to combine multiple checks like spamassassin does via macros. So it looks like spamassassin may be my best bet, however I've written many custom rules for spamassassin and I'm having a problem seeing how spamassassin can do this. Normally I check headers for predefined values, like "header MYRULE X-Original-To =~ /username\@domain\.com/". Not sure how I can do something like "header MYRULE X-Original-To =~ To|CC" (obviously this won't work, but that's pretty much what I need it to do). Anyone have any ideas?

Thanks for the responses I've gotten so far, I really do appreciate it.
There's nothing stopping you from writing your own program or script to process emails through postfix... The same way that spamassassin does it, or any other "plugin".

Quote:
Basically I want to be able to compare the RCPT TO addresses to the actual TO/CC addresses in the header. If an address in the RCPT TO is not in the TO/CC headers, then I want to discard the message.
Can you give me an example of when this would happen? There might be an easier solution to your problem.
 
Old 09-16-2006, 12:35 AM   #9
gani
Member
 
Registered: Jun 2004
Location: Metro Manila, Philippines
Distribution: Linuxmint, Slackware
Posts: 356

Rep: Reputation: 34
If you really want to lessen your postfix MTA's CPU cycles and at the same time using SpamAssassin and ClamAV through Amavisd-new, insert Postgrey and postgrey will outright reject spam and virusmails to initially reduce your spams and virusmails to not less than 80% (as per my assumption) and spamassassin and clamav will just scan very few of them.

Postgrey is a postfix policy service that provides temporary RCPT timeout (greylisting) which is by default set to 500 secs. Then the sending MTA which mostly are RFC compliant will keep hold of the mail until some few minutes and will then retry sending it in which at this time greylisting would surely be over and the mail will be accepted and verified by spamassassin and clamav for any bad content.

Greylisting through postgrey is proven very effective since most of today's spams and virusmails are being sent through 'zombies' of Botnets which are simply scattered PCs on the Net and not really MTA's and thus retrying after a failure and RFC blah, blah are out of their minds. Greylisting therefore will greatly reduce unsolicited mails flooding your MTA and which is useless and a waste of CPU resource welcoming them, scanning and then rejecting or putting in your quarantine. Here, they are outright rejected!

If you want to test it, send me an email <gani@gigalink.com.ph> while you are doing a 'tail -f /var/log/maillog' and observe your log reports and at the same time watching your clock to find out when will your MTA do the retry. My own observation revealed retries interval of 10 - 11 mins only.

Then after the first successful delivery after the greylisting, send me emails again and try to observe and compare.

Learn about Posgrey here (mine is just a plain intro):

http://isg.ee.ethz.ch/tools/postgrey/
 
Old 09-17-2006, 10:31 PM   #10
ramram29
Member
 
Registered: Jul 2003
Location: Miami, Florida, USA
Distribution: Debian
Posts: 848
Blog Entries: 1

Rep: Reputation: 47
Quote:
Originally Posted by generic_user
So, by your own admission, postfix now has to process the email twice... before spamassasin, and afterwards... Thus cutting its ability to process new emails in half.


wrong, it checks the header (read: before accepting the data of the email). If the rules don't match, it drops the sender... No bandwidth lost accepting the spam itself, and no point in having spamassassin (or any other list of piped processes) parse the data... It's a big fat waste.


Heh.. performing a regex on a header will absolutely not "blow up your memory" nor is it likely to "slow your CPU". We're talking 3 lines of text or less. I'd be amazed to see a system today that would be bogged down cpu or memory wise before running into trouble with bandwidth to their isp first.
Yes, for your knowledge, postfix is a collection of many tiny programs that all have queues, one program passess the message to the other; that's what makes postfix so secure also. If you add parameters to the header checks and you get mega messages then postfix will slow down.

You are the one that is wrong. I read this in the several books that I've read on postfix and in several fine tuning pages. For further proof of the need to correct you open the following page:

http://www.postfix.org/QSHAPE_README...maildrop_queue

Read the section:

Congestion in this queue is indicative of an excessive local message submission rate or perhaps excessive CPU consumption in the cleanup(8) service due to excessive body_checks.


Not only am I telling you this but also the authors and the developers that made postfix. If you disagree with them as noted in your statements above then you are trully hard-headed.
 
Old 09-18-2006, 01:07 AM   #11
generic_user
Member
 
Registered: Sep 2006
Location: San Francisco, Ca.
Distribution: Redhat/Fedora/CentOS
Posts: 39

Rep: Reputation: 15
Quote:
Originally Posted by ramram29
Congestion in this queue is indicative of an excessive local message submission rate or perhaps excessive CPU consumption in the cleanup(8) service due to excessive body_checks.


Not only am I telling you this but also the authors and the developers that made postfix. If you disagree with them as noted in your statements above then you are trully hard-headed.
I'm sorry you aren't able to differentiate between a header_check and a body_check. We have been talking header_checks haven't we? To quote you from your original post:
Quote:
In postfix there is a way to do this using header_checks
and even if we were talking body_checks, I guarantee you'll see more cpu cycles and memory used if you attempt to have spamassassin do the check instead of postfix. Btw, you're aware of the message size limits in spamassassin?
http://www.mail-archive.com/dev@spam.../msg14752.html

from the developers own mouth:
Quote:
Also, I upped the limit to 256MB, which I hope is well beyond the max -- SA will
definitely cause insane memory-usage on that, anyway -- documented it, and
added a test for it in the args-parsing code too.
Simply put, if the message is too large (read: spam with image attachments), spamsassin will choak and die. This is why they had to program in such a small default size limit of a mear 250k.

I'm not saying that spamassasin is useless... I use it! It's just not as efficient at doing some of the things that postfix can do, and will simply fail at other things that postfix can do.

From the man page of header_checks:
Quote:
Postfix header or body_checks are designed to stop a flood
of mail from worms or viruses;

Last edited by generic_user; 09-18-2006 at 01:11 AM.
 
Old 09-18-2006, 07:20 AM   #12
mastmind0
LQ Newbie
 
Registered: May 2003
Posts: 4

Original Poster
Rep: Reputation: 0
Quote:
Can you give me an example of when this would happen? There might be an easier solution to your problem.
Okay, so pretty much this is a general (obviously very simplified) example of an incoming message to an email server:

Code:
< 220 email.server.com ESMTP Postfix
> ehlo mydomain.com
< 250-email.server.com
< 250-PIPELINING
< 250-SIZE 104857600
< 250-VRFY
< 250-ETRN
< 250-STARTTLS
< 250-ENHANCEDSTATUSCODES
< 250 8BITMIME
> mail from:<myusername@mydomain.com>
< 250 2.1.0 Ok
> rcpt to:<someuser@email.server.com>
< 250 2.1.5 Ok
> rcpt to:<someuser2@email.server.com>
< 250 2.1.5 Ok
> data
< 354 End data with <CR><LF>.<CR><LF>
> Subject: This is the subject
> To: bogususer@thisisspam.com
> 
> This is the body
> .
< 250 2.0.0 Ok: queued as ADA255D4071
> quit
< 221 2.0.0 Bye
So... I would consider the above email to be spam, simply because the RCPT TO recipients do not in anyway match the TO: header field. Obviously, if there was a CC line, I would want to check that against the RCPT TO also. In my opinion there shouldn't be any reason why the TO/CC headers shouldn't contain the email addresses received from RCPT TO.

I've read all of the Postfix documents on backscatter, and email address verification and such, but nothing can tell me how to verify that the TO/CC header recipients are the TRUE recipients (as determined by the RCPT TO lines).

That's pretty much what I'm looking for... A way to compare TO/CC headers with RCPT TO entries and if the TO/CC headers do not match the RCPT TO lines, then discard the email (and of course log that it was discarded).

Last edited by mastmind0; 09-18-2006 at 07:25 AM.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Postfix send mail problem(In RH9, kernal 2.4.20, postfix 2.1.5) minor Linux - General 6 09-23-2019 10:09 PM
Logging mail subject headers in Postfix timjames Linux - Software 1 03-20-2009 02:07 PM
Exim: temporarily rejected RCPT bulliver Linux - General 3 05-02-2004 05:08 AM
Blank emails with no headers??!!!!! (postfix/spamassassin/procmail) birkinshawc Linux - Software 0 03-10-2004 08:59 AM
Mail problems 503 5.0.0 Need RCPT (recipient) Hylke Hogewind Linux - Software 3 04-03-2003 07:50 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 05:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration