Server Setup - Multiple Questions

Ruler2112 · 05-12-2008, 05:33 PM

I know that authentication and encryption are two different animals. Most of the tutorials that I've found seem to implement both simultaneously though. Because of this, I've kind of linked them in my brain, even though they really probably shouldn't be.

I think I'll take your advice and try again to get them both working. I'm just getting really frustrated trying to set everything up using tutorials/HOW-TOs that are mainly geared toward software current as of between 3 and 7 years ago, not understanding most of what I'm reading/doing, needing to have a functional machine relatively soon, and only progressing by seemingly random chance. (One of the HOW-TOs actually got chucked across my office into a wall Friday late afternoon, and I'm normally not a very excitable guy.) I haven't even started on the webmail, virus scanning, spam filtering, web server, firewall, or masquerading yet and already dropped LDAP functionality because I just don't understand how the security on it works.

</vent mode off>

I really do appreciate all your time, advice, and patience dude.

I'll post back when I have more information on the authentication and/or encryption troubles.

Ruler2112 · 05-12-2008, 06:40 PM

OK, looks like I've got SASL going. Again, not 100% on what I changed to make it work... Kept having "fatal: SASL per-process initialization failed" pop up every 30 seconds or so after I tried to send mail. Only way to get it to stop was to reboot. (No saslauthd process that I was able to see; is it possible for a process to hide itself from ps -ef on FreeBSD?)

One thing I did notice is that sending mail takes a long time (10-15 seconds or more) after enabling AUTH. I didn't have to generate certificates like so many tutorials wanted; could this possibly have something to do with it?

***edit - Nevermind the delay. Just the first time one goes to send does it take that long; subsequent messages fly right through.***

I'm going on to try to get TLS working now... I hope the stars are aligned just right!

Also, please don't take my vent in the last post as my venting at you; you've been great, trying to help me work through this. If anything, I'm frustrated with myself for being too dense to understand. (I come from a coding background; this is the first public server I've set up. I think writing and make work a 20,000 line program is far easier than tweaking all the settings for this kind of thing!)

Ruler2112 · 05-12-2008, 06:53 PM

One more question before I shut up for a while.

This machine has several domains on it - domain1.com, domain2.com, domain3.com, and so forth. There are 6 total domains, each goes with a different department of the same company, but all the users are the same. For example, user_name@domain1.com and user_name@domain2.com both go to the same mailbox. (I originally thought this was going to be difficult to set up, but it turns out that it was less work than getting them segregated.) The certificate generation process needs a particular domain name to be used for the certificate. Do I generate one certificate for all the domains or do I generate a separate one for each domain and somehow tell Postfix to use the appropriate one when a client connects to a given domain?

Ruler2112 · 05-13-2008, 02:34 PM

This thing is trying to drive me insane - I swear it! Didn't touch a thing on the server since last night and tried sending a message before starting on TLS/SSL and am now getting messages in /var/log/maillog and displayed to the console that "no sasl authentication mechanisms" when trying to send.

I'm telling you - this machine is out to get me...

Ruler2112 · 05-13-2008, 02:49 PM

OK, I must've forgotten to either postmap the config file or reload postfix after a change last night, because I figured it out. If I have noplaintext under smtpd_sasl_security_options, sending mail fails. With this commented out, it works.

With this commented out, I get the following lines in /var/log/maillog when sending mail:

Code:

warning: SASL authentication failure: no secret in database
SASL-MD5 authentication failed: authentication failure

I then see it use the sasl_method=PLAIN and the message is sent. I'm going to start googling for the first error, as I have a hunch that is what's causing the second, but wanted to update this thread with the information that I had a brain-fart. (Not re-generating the config database/reloading postfix after config change last night...)

Ruler2112 · 05-13-2008, 03:05 PM

Wait a minute... after reading about a dozen pages found via google, I got to thinking. If I get TLS/SSL working on this box, it doesn't matter if the authentication uses PLAIN as the authentication type as it'll be encrypted anyway - correct?

If the above thinking is right, I'm now back to generating certificates and setting up TLS.

billymayday · 05-13-2008, 04:02 PM

I think you'll find that PLAIn just refers to the type of password expected by the backend, so you assertion above is correct.

I'll try and find a good certificate link for you today

Edit - on the multiple domains, look at the postfix docs for virtual hosts - it's pretty simply. I haven't done TLS for virtuals, but I'd expect you to need 1 certificate per relay - so if you set your clients to all send via smtp.example.com, you'll only need one. If they send via smtp.domain1.com, smtp.comain2.com, etc., you'll need multiple.

billymayday · 05-13-2008, 06:20 PM

Here are a couple of links on TLS/certificates. The latter is certificates in some detail

http://postfix.state-of-mind.de/patr...s_support.html
http://www.eclectica.ca/howto/ssl-cert-howto.php

Ruler2112 · 05-19-2008, 04:21 PM

Thank you so much for the tips and pointers. With what you provided me, I've been able to set up TLS, SSL, and SASL. I also found a document explaining how to enable TLS for communications between mail servers (for those servers that support it). This is done by the last 4 lines - very straight-forward once you see it. I do have another question though. A chunk of my main.cf is below for reference.

The behavior I'd like to see happen is this. If somebody checks their e-mail via POP, they're allowed in without needing authentication. (This way, BlackBerry, iPhone, etc devices are certain to work as all the security is on the server. Further, I won't need to beat my head into a wall figuring out the specific setup on each and every hand-held client.) I know that postfix processes the conditions sequentially and stops when it finds a match. I put the check for pop-before-smtp before the authentication lines for this reason, expecting the behavior I just described to be evident. However, when I test this, authentication is required and as long as the authentication succeeds, one is allowed to send whether they're in the pop-before-smtp database or not. Pop-before-smtp was working and tested before TLS and SASL was put in and was working as expected.

Does this sound like a reasonable design? If so, does anybody have advice as to what I may change in order to make postfix behave like this?

Code:

unknown_hostname_reject_code = 554
unknown_client_reject_code = 450
message_size_limit = 20000000
smtpd_delay_reject = yes
smtpd_helo_required = yes
strict_rfc821_envelopes = yes

smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = no

smtpd_use_tls = yes
smtpd_tls_security_level = encrypt
smtpd_tls_auth_only = yes
smtpd_tls_key_file = /etc/postfix/key.pem
smtpd_tls_cert_file = /etc/postfix/cert.pem
smtpd_tls_CAfile = /etc/postfix/cacert.pem
smtpd_tls_loglevel = 0
smtpd_tls_received_header = no
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom

smtpd_helo_restrictions =
  permit_mynetworks,
  check_helo_access hash:/etc/postfix/helo_access,
  reject_non_fqdn_hostname,
  reject_invalid_hostname,
  permit

smtpd_sender_restrictions =
  permit_sasl_authenticated,
  permit_mynetworks,
  reject_non_fqdn_sender,
  reject_unknown_sender_domain,
  permit

smtpd_client_restrictions =
  reject_rbl_client zen.spamhaus.org

smtpd_recipient_restrictions =
  reject_unauth_pipelining,
  reject_non_fqdn_recipient,
  reject_unknown_recipient_domain,
  reject_invalid_hostname,
  reject_non_fqdn_hostname,
  reject_non_fqdn_sender,
  reject_unknown_sender_domain,
  permit_mynetworks,
  check_client_access hash:/etc/postfix/pop-before-smtp,
  permit_sasl_authenticated,
  reject_unauth_destination,
  reject_rbl_client zen.spamhaus.org,
  check_policy_service inet:127.0.0.1:10023
  check_policy_service unix:private/spfpolicy
  permit

maps_rbl_domains = zen.spamhaus.org

# Haven't gotten to these yet...
#header_checks = regexp:/etc/postfix/header_checks
#body_checks = regexp:/etc/postfix/body_checks

smtp_use_tls = yes
smtp_tls_note_starttls_offer = yes
smtp_tls_key_file = /etc/postfix/key.pem
smtp_tls_cert_file = /etc/postfix/cert.pem

Last, if anybody sees anything bad in the above, please speak up! I gleaned the configuration settings from many different sources across the internet and while I read up on most of them, am not 100% of what I was reading on many of them.

Ruler2112 · 05-29-2008, 02:37 PM

OK, I've decided to back-burner the problem with pop-before-smtp for now. Maybe I'll re-visit it again when everything else is done, but this depends on if I have any trouble with setting up the various hand-held devices to use SASL. It also depends on if anybody has ideas because quite frankly, I've tried everything I can think of in this regard; PostFix doesn't seem to honor the pop-before-smtp line being before the authentication lines of main.cf.

I was able to get the pf firewall working and based on port scans I've done while connected to each of the different interfaces, it's working flawlessly. This was surprisingly easy to do, much easier than I thought it'd be.

There's one major problem that I am aware of left to overcome. (Aside from negotiating the PPPoE connection, which I cannot do until the line becomes active; should be next week sometime for this.) I decided not to use SpamAssassin for now. Some of the documentation I've found says that with a good filter set, 99+% of the spam will be blocked anyways. I figure if the amount of spam is too much, I can install it in the future. However, I definitely want virus scanning of incoming messages. I set up amavisd-new and clamav to do a virus scan on all incoming messages. For some reason, this completely breaks sending. Here is what postqueue -p says after trying to send a message; this same message is contained in a bounce sent back to the sender.

Code:

host 127.0.0.1[127.0.0.1] said: 451-4.5.0 Error in processing,
id=20310-02, virus_scan FAILED: virus_scan: ALL VIRUS SCANNERS FAILED:
ClamAV-clamd av-scanner FAILED: CODE(0x8d283bc) unexpected , output="/var/amavis/tmp/amavis-20080526T184139-20310/parts:
lstat() failed.
ERROR 451 4.5.0 " at (eval 113) line 523. (in reply to end of DATA command)

I added a line to main.cf right under the body check comments in my previous post.

Code:

content_filter = smtp-amavis:[127.0.0.1]:10024

I also added 2 lines to master.cf as instructed in the HOWTO I followed to get this working. I've verified with sockstat -4 that amavisd is listening on port 10024. I also verified that both clamd and amavisd are running. I uncommented the section in amavisd.conf that deals with clamav, as well as changing the few lines in the same file that deal with my specific server. I honestly don't know where to go next; none of the log files seem to give any further indication of where the problem lie - the virus scanner isn't scanning the message is the most specific I've been able to find. The only clue that I've noticed, which I'm not sure is even a valid clue is in the above error message - "unexpected" followed by a space, then a comma. Unprintable character in one of the config files???

Any ideas?

billymayday · 05-29-2008, 03:14 PM

Can you post the clamav section of amavisd.conf and the corresponding section of clamd.conf. Also, what user are you running clamd as?

Ruler2112 · 05-29-2008, 04:48 PM

I didn't see where to disable SpamAssassin in amavis, but to enable clam I simply uncommented the 4 lines in amavisd.conf:

Code:

# ### http://www.clamav.net/
 ['ClamAV-clamd',
   \&ask_daemon, ["CONTSCAN {}\n", "/var/run/clamav/clamd"],
   qr/\bOK$/, qr/\bFOUND$/,
   qr/^.*?: (?!Infected Archive)(.*) FOUND$/ ],
# # NOTE: run clamd under the same user as amavisd, or run it under its own
# #   uid such as clamav, add user clamav to the amavis group, and then add
# #   AllowSupplementaryGroups to clamd.conf;
# # NOTE: match socket name (LocalSocket) in clamav.conf to the socket name in
# #   this entry; when running chrooted one may prefer socket "$MYHOME/clamd".

Here's the entire uncommented contents of clamd.conf:

Code:

LogFile /var/log/clamav/clamd.log
LogTime yes
LogSyslog yes
LogFacility LOG_MAIL
PidFile /var/run/clamav/clamd.pid
TemporaryDirectory /tmp
DatabaseDirectory /var/db/clamav
LocalSocket /var/run/clamav/clamd
FixStaleSocket yes
TCPAddr 127.0.0.1
StreamMaxLength 25M
User clamav
AllowSupplementaryGroups yes
DetectBrokenExecutables yes
ScanPDF yes
ScanMail yes

I do not know how to tell who clamd is running under. I start it via clamav_clamd_enable="YES" in /etc/rc.conf. According to the config file, it should use clamav as the user. While I can confirm that this user exists, ps -ax doesn't show what user owns a given process.

Something that I also left out of the last message (actually forgot about it until I was gathering config files for this post) is that greylisting also does not work as expected, though I have limited testing facilities until the box goes live. I send a message through it to a user on the same machine and it arrives immediately with no waiting period for the greylisting process. It's possible that I'm missing something though - there's only so much I can do with a machine sitting on my desk and having 1 PC to communicate with it. I've included chunks of config files related to this here as well, along with the pieces of those same config files that might relate to the amavis/clam problem.

Here's the bulk of the customized part of main.cf:

Code:

unknown_hostname_reject_code = 554
unknown_client_reject_code = 450
message_size_limit = 20000000
smtpd_delay_reject = yes
smtpd_helo_required = yes
strict_rfc821_envelopes = yes

smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = no

smtpd_use_tls = yes
smtpd_tls_security_level = encrypt
smtpd_tls_auth_only = yes
smtpd_tls_key_file = /etc/postfix/key.pem
smtpd_tls_cert_file = /etc/postfix/cert.pem
smtpd_tls_CAfile = /etc/postfix/cacert.pem
smtpd_tls_loglevel = 0
smtpd_tls_received_header = no
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom

smtpd_helo_restrictions =
  permit_mynetworks,
  check_helo_access hash:/etc/postfix/helo_access,
  reject_non_fqdn_hostname,
  reject_invalid_hostname,
  permit

smtpd_sender_restrictions =
  permit_sasl_authenticated,
  permit_mynetworks,
  reject_non_fqdn_sender,
  reject_unknown_sender_domain,
  permit

smtpd_client_restrictions =
  reject_rbl_client zen.spamhaus.org

smtpd_recipient_restrictions =
  reject_unauth_pipelining,
  reject_non_fqdn_recipient,
  reject_unknown_recipient_domain,
  reject_invalid_hostname,
  reject_non_fqdn_hostname,
  reject_non_fqdn_sender,
  reject_unknown_sender_domain,
  permit_mynetworks,
#  check_client_access hash:/etc/postfix/pop-before-smtp,
  permit_sasl_authenticated,
  reject_unauth_destination,
  reject_rbl_client zen.spamhaus.org,
  check_policy_service inet:127.0.0.1:10023
  check_policy_service unix:private/spfpolicy
  permit

maps_rbl_domains = zen.spamhaus.org

#header_checks = regexp:/etc/postfix/header_checks
#body_checks = regexp:/etc/postfix/body_checks

content_filter = smtp-amavis:[127.0.0.1]:10024

smtp_use_tls = yes
smtp_tls_note_starttls_offer = yes
smtp_tls_key_file = /etc/postfix/key.pem
smtp_tls_cert_file = /etc/postfix/cert.pem

And finally, some lines from master.cf:

Code:

policy    unix  -       n       n       -       -       spawn
  user=nobody
  argv=/usr/local/sbin/postfix-policyd-spf
smtp-amavis unix -      -       y       -       2       smtp
  -o smtp_data_done_timeout=1200
  -o disable_dns_lookups=yes
127.0.0.1:10025 inet  n -       y       -       -       smtpd
  -o content_filter=
  -o local_recipient_maps=
  -o local_recipient_maps=
  -o relay_recipient_maps=
  -o smtpd_restriction_classes=
  -o smtpd_client_restrictions=
  -o smtpd_helo_restrictions=
  -o smtpd_sender_restrictions=
  -o smtpd_recipient_restrictions=permit_mynetworks,reject
  -o mynetworks=127.0.0.0/8
  -o strict_rfc821_envelopes=yes

Thanks again for your time and help.

billymayday · 05-29-2008, 05:34 PM

the clamav user should be added to the amavis group. Did you do that do you know?

billymayday · 05-29-2008, 06:44 PM

Tell me, if you stop and restart amavisd, what does tha maillog say - you should see it starting/checking various decoders (gzip and the like) and there should be something about clam. I suspect it isn't even finding it. You do have the clamd daemon running I assume.

Edit - on the greylisting, you can only really test once, since the IP will be put in the greylist database. You can delete the database to wipe it.

Ruler2112 · 05-30-2008, 11:44 AM

Arrrrgggghhhh!!!! How'd I miss that!?!? It's right below the config lines for clam in the amavisd.conf file! Sure enough, adding the clam user to the amavisd group and restarting the daemons brought everything up and I'm now able to send. However, this uncovered another problem. While I think I know what's going on, I haven't the foggiest on how to fix it without disabling TLS entirely.

After sending a message, I get a bounce with the following. My theory is that amavisd is forwarding the message to a local port for sending after virus scanning and isn't using TLS. Does this thinking make sense? If so, how would I go about solving it? (I'd assume to somehow tell postfix to not require a STARTTLS command when connecting from 127.0.0.1.)

Code:

<email@address.com>: host 127.0.0.1[127.0.0.1] said: 530 5.7.0 Must issue a
    STARTTLS command first (in reply to end of DATA command)

That's a good point about testing the greylist. It was so easy to set up compared to everything else that I just glossed over it quickly and didn't even think that I couldn't test more than once.