Linux - ServerThis forum is for the discussion of Linux Software used in a server related context.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Basically I want to be able to compare the RCPT TO addresses to the actual TO/CC addresses in the header. If an address in the RCPT TO is not in the TO/CC headers, then I want to discard the message. Obviously this will cause problems for anyone who does a BCC, but I can live with killing that feature. As I'm sure all of us are, I'm tired of getting emails where the TO in the header says something like "getmore@RxMeds.com", and obviously the RCPT TO line contained my email address. If I could compare and make sure that all RCPT TO addresses are listed in the TO/CC headers, that should eliminate that problem
Any help would be greatly appreciated. Also, if anyone sees any drawbacks to the above, please point them out.
Run spamassassin - it can take care of this for you.
In postfix there is a way to do this using header_checks also but it will slow it down - I'd rather use spamassassin.
I can say with certainty that spamassassin will be a much slower solution to this than using postfix! Much better to stop the email before it has to be processed by other pipes like spamassasin (which is a resource hog).
Basically I want to be able to compare the RCPT TO addresses to the actual TO/CC addresses in the header. If an address in the RCPT TO is not in the TO/CC headers, then I want to discard the message. Obviously this will cause problems for anyone who does a BCC, but I can live with killing that feature. As I'm sure all of us are, I'm tired of getting emails where the TO in the header says something like "getmore@RxMeds.com", and obviously the RCPT TO line contained my email address. If I could compare and make sure that all RCPT TO addresses are listed in the TO/CC headers, that should eliminate that problem
Any help would be greatly appreciated. Also, if anyone sees any drawbacks to the above, please point them out.
Thanks!
It sounds like you want to stop backscatter. This url will answer your questions:
damn.. I can't post urls..
Search google for "backscatter postfix howto"
Note that the ultimate check would be to compare RCPT TO or MAIL FROM with the contents of the actual email... for this you need a SENDERID plugin.
I can say with certainty that spamassassin will be a much slower solution to this than using postfix! Much better to stop the email before it has to be processed by other pipes like spamassasin (which is a resource hog).
Like I originally said above, you can use postfix header_checks or you can use spamassassin. However, I think you are actually wrong. Your SMTP server should be as fast as possibe and if you have mega SPAM and use it to do header_checks then you will bring it down to it's knees very soon. My recommendation is to let the SMTP server (postfix) receive as much mail as possible as fast as possible and hand it down to another program for filtering SPAM (spamassassin).
I respect your opinion but I follow what other experts say: This how SPAM best works. Postfix can easily recieve 100 regular messages per second unchecked an put them in it's queue. Even if it receives 1000 messages per second it will put them all in its queue and continue. Spamassassin will then pick up the message and filter it and resend it to postfix but this time reinserting the message in the queue will be much faster because it is local.
If you do header checks with postfix you and you get 1000 messages with SPAM then postfix will have to check the message and reinsert it into itself. That's too much work and it will slow your CPU and blow up your memory. In the UNIX/Linux processes are designed with many minute programs that all do one thing and pass it on to the next program. Hence, making things more efficient, faster and stable.
Furthermore, you get a very rich SPAM filter by using spamassassin.
Unfortunately, even if I wanted to use Postfix to perform the header_checks it doesn't look like I'd be able to since postfix doesn't seem to have the ability to combine multiple checks like spamassassin does via macros. So it looks like spamassassin may be my best bet, however I've written many custom rules for spamassassin and I'm having a problem seeing how spamassassin can do this. Normally I check headers for predefined values, like "header MYRULE X-Original-To =~ /username\@domain\.com/". Not sure how I can do something like "header MYRULE X-Original-To =~ To|CC" (obviously this won't work, but that's pretty much what I need it to do). Anyone have any ideas?
Thanks for the responses I've gotten so far, I really do appreciate it.
Like I originally said above, you can use postfix header_checks or you can use spamassassin. However, I think you are actually wrong. Your SMTP server should be as fast as possibe and if you have mega SPAM and use it to do header_checks then you will bring it down to it's knees very soon. My recommendation is to let the SMTP server (postfix) receive as much mail as possible as fast as possible and hand it down to another program for filtering SPAM (spamassassin).
What RFC states that "SMTP server should be as fast as possible"? Gerylisting, and some other tricks to prevent spam rely on slowing the server down. There's one trick where you purposefully add a long pause before sending the HELO query in an attempt to get the spammer to send its data before being asked for it.
Quote:
I respect your opinion but I follow what other experts say: This how SPAM best works. Postfix can easily recieve 100 regular messages per second unchecked an put them in it's queue. Even if it receives 1000 messages per second it will put them all in its queue and continue. Spamassassin will then pick up the message and filter it and resend it to postfix but this time reinserting the message in the queue will be much faster because it is local.
So, by your own admission, postfix now has to process the email twice... before spamassasin, and afterwards... Thus cutting its ability to process new emails in half.
Quote:
If you do header checks with postfix you and you get 1000 messages with SPAM then postfix will have to check the message and reinsert it into itself.
wrong, it checks the header (read: before accepting the data of the email). If the rules don't match, it drops the sender... No bandwidth lost accepting the spam itself, and no point in having spamassassin (or any other list of piped processes) parse the data... It's a big fat waste.
Quote:
That's too much work and it will slow your CPU and blow up your memory. In the UNIX/Linux processes are designed with many minute programs that all do one thing and pass it on to the next program. Hence, making things more efficient, faster and stable.
Furthermore, you get a very rich SPAM filter by using spamassassin.
Heh.. performing a regex on a header will absolutely not "blow up your memory" nor is it likely to "slow your CPU". We're talking 3 lines of text or less. I'd be amazed to see a system today that would be bogged down cpu or memory wise before running into trouble with bandwidth to their isp first.
Unfortunately, even if I wanted to use Postfix to perform the header_checks it doesn't look like I'd be able to since postfix doesn't seem to have the ability to combine multiple checks like spamassassin does via macros. So it looks like spamassassin may be my best bet, however I've written many custom rules for spamassassin and I'm having a problem seeing how spamassassin can do this. Normally I check headers for predefined values, like "header MYRULE X-Original-To =~ /username\@domain\.com/". Not sure how I can do something like "header MYRULE X-Original-To =~ To|CC" (obviously this won't work, but that's pretty much what I need it to do). Anyone have any ideas?
Thanks for the responses I've gotten so far, I really do appreciate it.
There's nothing stopping you from writing your own program or script to process emails through postfix... The same way that spamassassin does it, or any other "plugin".
Quote:
Basically I want to be able to compare the RCPT TO addresses to the actual TO/CC addresses in the header. If an address in the RCPT TO is not in the TO/CC headers, then I want to discard the message.
Can you give me an example of when this would happen? There might be an easier solution to your problem.
If you really want to lessen your postfix MTA's CPU cycles and at the same time using SpamAssassin and ClamAV through Amavisd-new, insert Postgrey and postgrey will outright reject spam and virusmails to initially reduce your spams and virusmails to not less than 80% (as per my assumption) and spamassassin and clamav will just scan very few of them.
Postgrey is a postfix policy service that provides temporary RCPT timeout (greylisting) which is by default set to 500 secs. Then the sending MTA which mostly are RFC compliant will keep hold of the mail until some few minutes and will then retry sending it in which at this time greylisting would surely be over and the mail will be accepted and verified by spamassassin and clamav for any bad content.
Greylisting through postgrey is proven very effective since most of today's spams and virusmails are being sent through 'zombies' of Botnets which are simply scattered PCs on the Net and not really MTA's and thus retrying after a failure and RFC blah, blah are out of their minds. Greylisting therefore will greatly reduce unsolicited mails flooding your MTA and which is useless and a waste of CPU resource welcoming them, scanning and then rejecting or putting in your quarantine. Here, they are outright rejected!
If you want to test it, send me an email <gani@gigalink.com.ph> while you are doing a 'tail -f /var/log/maillog' and observe your log reports and at the same time watching your clock to find out when will your MTA do the retry. My own observation revealed retries interval of 10 - 11 mins only.
Then after the first successful delivery after the greylisting, send me emails again and try to observe and compare.
Learn about Posgrey here (mine is just a plain intro):
So, by your own admission, postfix now has to process the email twice... before spamassasin, and afterwards... Thus cutting its ability to process new emails in half.
wrong, it checks the header (read: before accepting the data of the email). If the rules don't match, it drops the sender... No bandwidth lost accepting the spam itself, and no point in having spamassassin (or any other list of piped processes) parse the data... It's a big fat waste.
Heh.. performing a regex on a header will absolutely not "blow up your memory" nor is it likely to "slow your CPU". We're talking 3 lines of text or less. I'd be amazed to see a system today that would be bogged down cpu or memory wise before running into trouble with bandwidth to their isp first.
Yes, for your knowledge, postfix is a collection of many tiny programs that all have queues, one program passess the message to the other; that's what makes postfix so secure also. If you add parameters to the header checks and you get mega messages then postfix will slow down.
You are the one that is wrong. I read this in the several books that I've read on postfix and in several fine tuning pages. For further proof of the need to correct you open the following page:
Congestion in this queue is indicative of an excessive local message submission rate or perhaps excessive CPU consumption in the cleanup(8) service due to excessive body_checks.
Not only am I telling you this but also the authors and the developers that made postfix. If you disagree with them as noted in your statements above then you are trully hard-headed.
Congestion in this queue is indicative of an excessive local message submission rate or perhaps excessive CPU consumption in the cleanup(8) service due to excessive body_checks.
Not only am I telling you this but also the authors and the developers that made postfix. If you disagree with them as noted in your statements above then you are trully hard-headed.
I'm sorry you aren't able to differentiate between a header_check and a body_check. We have been talking header_checks haven't we? To quote you from your original post:
Quote:
In postfix there is a way to do this using header_checks
and even if we were talking body_checks, I guarantee you'll see more cpu cycles and memory used if you attempt to have spamassassin do the check instead of postfix. Btw, you're aware of the message size limits in spamassassin? http://www.mail-archive.com/dev@spam.../msg14752.html
from the developers own mouth:
Quote:
Also, I upped the limit to 256MB, which I hope is well beyond the max -- SA will
definitely cause insane memory-usage on that, anyway -- documented it, and
added a test for it in the args-parsing code too.
Simply put, if the message is too large (read: spam with image attachments), spamsassin will choak and die. This is why they had to program in such a small default size limit of a mear 250k.
I'm not saying that spamassasin is useless... I use it! It's just not as efficient at doing some of the things that postfix can do, and will simply fail at other things that postfix can do.
From the man page of header_checks:
Quote:
Postfix header or body_checks are designed to stop a flood
of mail from worms or viruses;
Last edited by generic_user; 09-18-2006 at 01:11 AM.
Can you give me an example of when this would happen? There might be an easier solution to your problem.
Okay, so pretty much this is a general (obviously very simplified) example of an incoming message to an email server:
Code:
< 220 email.server.com ESMTP Postfix
> ehlo mydomain.com
< 250-email.server.com
< 250-PIPELINING
< 250-SIZE 104857600
< 250-VRFY
< 250-ETRN
< 250-STARTTLS
< 250-ENHANCEDSTATUSCODES
< 250 8BITMIME
> mail from:<myusername@mydomain.com>
< 250 2.1.0 Ok
> rcpt to:<someuser@email.server.com>
< 250 2.1.5 Ok
> rcpt to:<someuser2@email.server.com>
< 250 2.1.5 Ok
> data
< 354 End data with <CR><LF>.<CR><LF>
> Subject: This is the subject
> To: bogususer@thisisspam.com
>
> This is the body
> .
< 250 2.0.0 Ok: queued as ADA255D4071
> quit
< 221 2.0.0 Bye
So... I would consider the above email to be spam, simply because the RCPT TO recipients do not in anyway match the TO: header field. Obviously, if there was a CC line, I would want to check that against the RCPT TO also. In my opinion there shouldn't be any reason why the TO/CC headers shouldn't contain the email addresses received from RCPT TO.
I've read all of the Postfix documents on backscatter, and email address verification and such, but nothing can tell me how to verify that the TO/CC header recipients are the TRUE recipients (as determined by the RCPT TO lines).
That's pretty much what I'm looking for... A way to compare TO/CC headers with RCPT TO entries and if the TO/CC headers do not match the RCPT TO lines, then discard the email (and of course log that it was discarded).
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.