User Tools

Site Tools


mailserver_configuration_with_postfix_courier-imap_procmail_spamassassin_clamav

Configuring a mail server on FreeBSD

A step-by-step practical guide to installing & configuring your mailserver on FreeBSD 6.2, by Sebastiaan Giebels [sgie bels_freebsdATpc probleemloos.nl].

admitted, I created a logo myself for procmail, as there is no official one All logos are copyrighted by their respective copyright holders (except for procmail, where I got creative myself).

This section of My FreeBSD Installation and Configuration Guide will show you how you can handle e-mail on your FreeBSD server. It includes the Postfix mail transfer agent, ClamAV as a e-mail virusscanner, SpamAssassin to detect SPAM, procmail to do advanced custom mail processing/sorting, the courier IMAP / IMAPS server, the Squirrelmail webmail application, and a basic console e-mail reader (mutt). The Postfix/courier-imap/clamsmtp/clamav/spamassassin/procmail setup is pretty common.

This will probably always be 'Work in progress', but I think it's pretty usable in its current state. I would welcome any comments or corrections. By continuing reading you agree to the disclaimer.

Commercial FreeBSD support NEW!
We can help you with many of your configuration and installation problems, by phone, chat, or e-mail from our office in Eindhoven, the Netherlands/Nederland. Contact [freebsd_live_supportATpc probleemloos.nl] to find out how we can help you with your BSD issues & ask for our hourly rates. Languages spoken: dutch, english, german.

How things fit together

In a mail system, there are various programs for all the tasks:

  1. accepting mail from the internet,
  2. scanning e-mail for virusses,
  3. determining if a message is spam,
  4. deciding each e-mail goes where,
  5. storing mail on harddisk,
  6. making mail readable for e-mail clients,
  7. and finally the e-mail client/reader itself.

For many of the tasks above there is a different program needed. The path the e-mail messages on my FreeBSD system take is as follows:

Internet → Postfix → ClamSMTP → ClamAV → Postfix → Procmail → Spamassassin → (returns to) Procmail → [Maildir Storage] → Courier-IMAP → Any e-mail client (mutt, Mozilla Thunderbird, pine or even webmail applications like squirrelmail.

Postfix

Postfix is, just like Sendmail, a MTA. It will listen for incoming e-mail and forward or store it. Official URL: http://www.postfix.org/

Virtual Domain Hosting: http://www.postfix.org/VIRTUAL_README.html

Install package: (version 2.2.9,1)

pkg_add -r postfix

You need user “postfix” added to group “mail”.

Would you like me to add it [y]? y

Would you like to activate Postfix in /etc/mail/mailer.conf [n]? n

nano /etc/rc.conf
postfix_enable="YES"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

I like to make a backup of the original configuration before I make changes:

cp /usr/local/etc/postfix/main.cf /usr/local/etc/postfix/main.cf.bak

Now, we're going to do some configuration:

nano /usr/local/etc/postfix/main.cf
soft_bounce = yes  //(for the time being)//
mydestination = $myhostname, $mydomain, localhost.$mydomain, localhost, /usr/local/etc/postfix/mydestinations
unknown_local_recipient_reject_code = 450  //(for the time being)//
home_mailbox = Maildir/
recipient_delimiter = .
mynetworks_style = host
virtual_alias_maps = hash:/usr/local/etc/postfix/virtual

Optional: (only needed when we want to send e-mail out)

relayhost = <your_ISPs_outgoing_mailserver>

You can hide the hostname postfix shows when someone connects on port 25, change the smtpd_banner setting in main.cf, e.g.:

smtpd_banner = localhost ESMTP Sendmail

There are more than one location for the aliases file. The original is /etc/mail/aliases, and there's a symlinked version in /etc/aliases.

nano /etc/aliases

Add this after “# Pretty much everything else in this file points to “root”, so # you would do well in either reading root's mailbox or forwarding # root's email from here.”:

    #Person who should get root's mail
    root:<your_freebsd_username>
    *:<your_freebsd_username>

I have removed all the other lines in the file.

newaliases
/usr/local/sbin/postalias /etc/aliases

Virtual mailboxes, see <http://www.cluesoft.be/howto/postfix.htm>:

cd /usr/local/etc/postfix/
nano virtual
@<my_hostname:_freebsd61.example.org>   <my_freebsd_username>
@<localhost.example.org>                <my_freebsd_username>
@<my_first_domain_name:_example.org>    <my_freebsd_username>
@<optionally_my_2nd_domain_name>        <my_freebsd_username>

Instead of <your_freebsd_username> you can also put a e-mail addres at a different host (e.g. your_hotmail_account@hotmail.com). This will require configuration of “relayhost = …” in the section above.

This will create /usr/local/etc/postfix/virtual.db (a hash-file) from the contents of /usr/local/etc/postfix/virtual:

postmap /usr/local/etc/postfix/virtual

Remember to 'postmap' your files (only aliases & virtual) after editing and before reloading postfix to make postfix able to read it.

Let's tell Postfix for which domains to accept mail for:

nano /usr/local/etc/postfix/mydestinations
localhost
<hostname>
<first_domain_name>
<second_domain_name>
<etcetera>

Check /etc/hosts too …

Let's configure the Maildir-folders for the mail to be stored in. As username (non-root):

cd ~
maildirmake Maildir

Finally, reload Postfix to activate all the previous changes:

postfix reload   (or 'postfix start')

Use the following command to troubleshoot postfix:

tail /var/log/maillog

reload postfix:

postfix reload

Testing it

Test it:

telnet localhost 25

Will tell you: 220 <freebsd61.example.org> ESMTP Postfix. Enter:

  mail from: my_real@email-addres.com

And after '250 Ok', enter:

  rcpt to: <freebsd_username>@<some_of_your_configured_domains.com>

And after '250 Ok', enter:

  data

And after '354 End data with <CR><LF>.<CR><LF>', enter some lines of text, finish with an . on an empty line, it will give: '250 Ok: queued as …'

  quit

to end this smtp-session

cd to the maildir where you expect the file to be delivered, and check its presence. it wil probably be in the '/Maildir/new' folder of the users home dir.

Testing against 'open smtp relay'

I will now check my Postfix SMTP server to make sure no-one from the internet can send e-mail trough my server, when the e-mail is not for me.
More info: http://whatis.techtarget.com/definition/0,,sid9_gci782509,00.html
For security reasons, I urge you to check your Postfix installation. From another pc (not the FreeBSD pc you are testing), open a telnet session to your FreeBSD machine on port 25 (in Windows: Start → Run → “telnet <freebsd_ip_address> 25” (without the quotes)
If you are using linux or freebsd on another pc, just use:

telnet <freebsd_ip_address> 25

No matter if you are connecting from Windows, FreeBSD, Linux, you will get something like this:

Trying <freebsd_ip_address>...
Connected to <freebsd_full_qualified_address>.
Escape character is '^]'.
220 <freebsd62.example.org> ESMTP Postfix
</hmtl>
  mail from: hotmail@hotmail.com
<code conf>
250 2.1.0 Ok
rcpt to: some_address@hotmail.com

If you get the following message, everything is ok, and your Postfix installation is not acting like a 'open smtp relay':

454 4.7.1 <some_address@hotmail.com>: Relay access denied

If however the server replies:

250 2.1.5 Ok

And if you are sure you are not connecting from the FreeBSD machine itself, your Postfix installation is not ok. Use the command

postfix stop

to stop the postfix SMTP server, then recheck your configuration!

Procmail

Install the package: (version procmail-3.22_6)

pkg_add -r procmail

Now, to have mail sorted, you will need to create a '.procmailrc' file in the home directory for every user you wish to configure procmail for (you can also use a global procmail configuration file, but I won't explain how to do that here) I'll show you how to save a backup / spare copy of every e-mail message going trough procmail, as bad procmail configuration files can result in a lot of lost mail messages.

Switch to the user you wish to configure procmail for:

su <my_freebsd_username>
cd ~<my_freebsd_username>
mkdir .procmail
mkdir .procmail/backup
chmod 644 .procmailrc
chmod -R 644 .procmail
nano .procmailrc
# This is your procmail configuration file.
############################################################################################
# TEST YOUR .PROCMAILRC CONFIGURATION AFTER EVERY CHANGE TO AVOID PERMAMENTLY LOST E-MAIL! #
#################################################################################################### 
# I cannot stress this enough:                                                                     #
# If there are errors in the procmail configuration file, your mail might not                      #
# get delivered to the place you expect it to go.                                                  #
# Also, don't change too many recipes at once, change, test, and change again.                     #
# Leave the backup recipe in place,                                                                #
#  ### AND BE SURE TO TEST THAT BACKUP MAIL GETS STORED IN THE DESIGNATED FOLDER! ###              #
# If the backup folder does not exist, or if it has the wrong permissions, messages WILL get lost! #
####################################################################################################
 
# Set the path, important when you're accessing external applications from procmail:
PATH=/bin:/usr/bin:/usr/local/bin
 
#Set to on when debugging, leave off to prevent logfile from getting too big
VERBOSE=yes
 
# Procmail base directory (for additional configuration files etc.)
PMDIR=$HOME/.procmail
 
# Where to store the procmail logfile:
LOGFILE=$PMDIR/procmaillog
 
# Default directory to use for storing e-mails:
MAILDIR=$HOME/Maildir
DEFAULT=$MAILDIR
 
# Directory to use for storing backups:
BACKUPMAILDIR=$PMDIR/backup
 
# Make a copy of every passing mail to a specific folder
# (I'd like to use $HOME/.procmail/backup)
# Make sure that folder exists and is writable!
# If it would store every mail passing trough procmail this directory
# would grow very large. So we're removing files more than 14 days old.
BACKUPMAILDIR=$PMDIR/backup/
:0 c
$BACKUPMAILDIR
:0 ic
| cd $BACKUPMAILDIR && find $BACKUPMAILDIR -name "msg*" -ctime +14 -exec rm {} \;
 
 
# Insert your own recipes below this line
#==============================================================================
 
# put your own procmail recipes here, such as:
#  - guaranteed delivery for mail to postmaster@.., abuse@.., .. 
# Do not scan e-mail sent to the following address for spam:
#:0:
#* ^TO.*\<IMP>abuse@freebsd62.example.org</IMP>
#$MAILDIR/
 
#  - spam filtering
#  - message tagging
#  - sorting messages in specific maildirs based on recipient- or sender address
#  - marking messages as 'read' or 'seen' using procmail and 'mv' (see below)
 
#==============================================================================
# Insert your own recipes above this line
 
# If none of the above recipes matched, make sure the mail does get delivered (to INBOX)
:0
$MAILDIR/

Configure Postfix to use Procmail for message delivery:

nano /usr/local/etc/postfix/main.cf
mailbox_command = /usr/local/bin/procmail -a "$EXTENSION"

To make postfix aware of the changes to its configuration file, reload postfix (as root user):

postfix reload

This should give the message:

postfix/postfix-script: refreshing the Postfix mail system

If you have problems configuring procmail, check the file ~/.procmail/procmaillog for errors (make sure you've kept the 'VERBOSE=yes' like in .procmailrc) In .procmailrc, you can set the variable 'LOG', and anything assigned to this variable will be appended to $LOGFILE. Use this for adding your own debugging messages. Be sure you've read 'man procmailrc' for the complete syntax.

Use this recipe to move messages that were marked as spam by SpamAssassin to the 'SPAM' maildir:

# If the message contains the header 'X-Spam-Status: Yes',
# move it to the spam-maildir. (you might want to combine this wih the 'mark-as-seen' recipe below).
:0h:
* ^X-Spam-Status: Yes
$MAILDIR/.SPAM/

Generally, you can mark e-mail messages as 'seen' (so that they won't generate a 'new mail' notification in your mail client). I'll mark the message as 'seen' just by adding ':2,S' at the end of the filename. The correct way would additionally involve moving the file from the 'new' folder to the 'cur' folder too, but as this would take more programming, and any undetected error in this file will probably have you lose new mail, I won't take this step.

You can only use this with the 'maildir' format. Do not use this when using mailbox format. Change your existing recipe that should look a lot like this:

:0 <any flags>
* <any conditions in procmail format>
<any destination>

To:

:0 <any flags>
* <any conditions in procmail format>
{
  :0c:
  <any destination maildir>
 
  # This will mark the message as read. 
  # Make sure the previous 'delivery' begins with :0c:
  TRAP="mv \"$LASTFOLDER\" \"$LASTFOLDER:2,S\""
  :0
  /dev/null
}
Logging and debugging

Insert extra logging functions troughout the procmailrc file, to help diagnose problems quickly. I like to have VERBOSE=on always, but I'll have to prevent the log-file from getting too big. >50MB can get troublesome.

NOTICE="
 
<insert here the text you want to see in the procmail logfile>
"

If you set 'VERBOSE=off', you'll have to replace 'NOTICE=' wit 'LOG=' to still see your debug messages in the logfile. ('LOG' is a special procmailrc keyword, 'NOTICE' isn't).

If you're going to do some procmail configuring yourself, I urge you to read http://www.iki.fi/era/procmail/mini-faq.html

And remember: - Make sure that the 'backup' recipe is working correct. (There's also a smaller recipe for this example, see 'man procmailex', search for 'backup') - Make sure the backups of the backup-recipe will not eat up all diskspace over time (better: make it preserve mails for a week or so, depending on how much mail you're receiving daily) - Test thouroughly. Perhaps your recipe does match manager@somecompany.com but not “Manager manager@somecompany.com”. You'll only find out by testing. - Try to completely understand the recipes that you use, add extensive comments to the recipes which will be of great help when you'll be reading them later. - Use file locking (with the ':' at the end of the first line of a rule) whenever there's a rule that will be writing to files. - Don't put any comments on a condition line (that is, a line starting with '*')

Also read: http://userpages.umbc.edu/~ian/procmail.html And: 'man procmailrc'

Finally, make sure the mail backups and procmail.log file don't fill up your disk.

Procmail logrotate: http://www.cv.nrao.edu/~pmurphy/Talks/procmail2003/allparts.shtml

Courier-imap/imaps

What is IMAP, and why should I use it?

Quoted from LifeHacker.com: “IMAP (Internet Message Access Protocol) keeps everything – even your sent mail – on a server. In short, IMAP gets you e-mail portability without having to use a Web-based e-mail interface.”, and: “Being a multi-computer user myself, I'm an IMAP user and fan - it's the best way to keep your email sync'ed on every computer with access”

As all e-mail (received, sent, and even drafts/concepts) are saved on a central server, I never have to worry about which computer I use to read or write my mail. Having just one central place where all my e-mail messages are stored makes making daily backups very easy.

First, install the CA-Cert root certificate (Official URL: http://www.cacert.org): (version 1.2)

pkg_add -r ca-roots

As I want to use imaps (the secure/encrypted imap protocol), I'm making sure openssl is installed (it might be already installed on your system):

cd /usr/ports/security/openssl
make
...
=> openssl-0.9.8e.tar.gz doesn't seem ...
make install

Build and install the port:

/usr/port/courier-imap
make config

As I had some problems with courier-imap (connection closes after '. select inbox'), I deselected all options, and enabled 'openssl' and 'auth_userdb'.

Alternatively, you can install the pre-built package:

pkg_add -r courier-imap

For user authentication, we'll be using userdb. Install the required packages: (version 0.58_2)

pkg_add -r courier-authlib-userdb courier-authlib

(This will also install courier-authlib-base)

Configuring package:

cd /usr/local/etc/courier-imap
cp imapd-ssl.dist imapd-ssl
cp imapd.dist imapd

The Thunderbird e-mail client uses 10 connections by default, for 10 users this can require up to 100 connections. We double that to prevent problems in the future:

nano /usr/local/etc/courier-imap/imapd
    MAXDAEMONS=200
    MAXPERIP=20

Edit /usr/local/etc/authlib/authdaemonrc, remove the authentication types that aren't used (leaving only 'authuserdb' as the authentication method to use):

nano -w /usr/local/etc/authlib/authdaemonrc
    authmodulelist="authuserdb"

Make a userdatabase for IMAP users (so users can have access to the IMAP server without the need for them to have a full shell account on the server) (URL: http://www.courier-mta.org/?makeuserdb.html): (URL: http://www.de.daemonnews.org/200308/courier-imap.html) http://www.courier-mta.org/?makeuserdb.html

/usr/local/sbin/pw2userdb | grep uid | grep <my_freebsd_username> > /usr/local/etc/userdb
chmod go-rwx /usr/local/etc/userdb

If you are adding additional users, use '»' (2x '>') instead of '>' to make sure the userdb doesn't get overwritten.:

/usr/local/sbin/pw2userdb | grep uid | grep <next_freebsd_username> >> /usr/local/etc/userdb

We're going to create a password to let this user have access to the IMAP server.

userdbpw
    Password: <my_mothers_maiden_name>
    Reenter password: <my_mothers_maiden_name>
    <your_encrypted_password_appears_here>

Copy the encrypted password string.

nano -w /usr/local/etc/userdb

Change the text between 'systempw=' and the next '|' character to the encrypted password string you just copied, and save the userdb file (CTRL-O, CTRL-X). The '-w' parameter to nano makes sure nano won't wrap long lines (which would make the userdb unreadable to makeuserdb).

To convert 'userdb' to 'userdb.db' (the file which is read by courier-authdeamond):

makeuserdb

Check if the userdb.dat file is created by entering:

ls -al /usr/local/etc/userdb*

Now, we will convert the userdb to a file which courier can read. The command 'makeuserdb' creates the file '/usr/local/etc/userdb.dat' from the contents of '/usr/local/etc/userdb':

/usr/local/sbin/makeuserdb

We're going to make sure these files can only be read by root:

chmod 0600 /usr/local/etc/userdb
chmod 0600 /usr/local/etc/userdb.dat

Now we're going to start the authentication daemon which will check the passwords for the IMAP service:

mkdir /usr/local/var
mkdir /var/run/authdaemond
/usr/local/sbin/authdaemond start

If, in the future, you have made additional changes to the userdb.dat file and you want to reload them into the authentication engine, you can use this command:

/usr/local/sbin/authdaemond restart

If you didn't follow the postfix installation, or if you are adding new users to your IMAP server, you should go there now and read the instructions on how to create a 'Maildir' folder. This is where the messages of the user will be stored.

In the folder /usr/share/skel/ you can setup the template for the home directories for future users you might add. To automatically have a Maildir folder created when you're adding a new user, use this command. As root:

maildirmake /usr/share/skel/Maildir

Start the courier IMAP daemon:

/usr/local/etc/rc.d/courier-imap-imapd.sh start

Configure Courier-IMAP and IMAPS start up upon boot:

nano /etc/rc.conf
    courier_imap_imapd_enable="YES"
    #courier_imap_imapd_ssl_enable="YES"
    courier_authdaemond_enable="YES"

To see which ports are used by courier:

sockstat -l | grep courier
root     couriertcp 3224  3  tcp4   *:143                 *:*

Test your IMAP server with telnet:

telnet localhost 143

Enter the following data:

. login <imap_username> <imap_password>

If you get '. OK LOGIN Ok.' as an answer, you are logged in (which would mean that authdaemond is running ok.

Now check if you can open the inbox:

. SELECT INBOX

If you're getting a reply like this, everything is ok:

* FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent)
* OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] 
Limited
* 0 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1187180023] Ok
* OK [MYRIGHTS "acdilrsw"] ACL
. OK [READ-WRITE] Ok

If you're getting errors like:

### NO Error in IMAP command received by server.
pid ##### (imapd), uid #####: exited on signal 11 (core dumped)

or:

. select Inbox
Connection closed by foreign host.

Check your installation, configuration, file permissions, and make sure you've done the 'mkdir' stuff. I've re-installed courier-imap from the ports, and deselected all options except 'auth_userdb' and 'openssl'. I suspect FAM to create problems, but I'm not sure on this. You could use 'ktrace' for debugging, although this requires more unix knowledge:

cd ~<my_freebsd_username>
ktrace </path/to/imapd> Maildir

Telnet to port 143, and try again. After it crashes,

kdump | less

When something doesn't go as expected, you might find the error message in '/var/log/maillog'. A good login looks like this:

Aug 15 13:52:43 <your_freebsd_hostname> imapd: LOGIN, user=<your_freebsd_username>, ip=[::ffff:127.0.0.1], protocol=IMAP

Secure IMAP (IMAPS, IMAP-SSL) with Courier is just a few steps away from here. Normal IMAP-traffic (that is: any e-mail message you read) is sent unencrypted over the network. Secure IMAP is encrypted, which means that only your pc can open the message. Luckily, this feature is supported by almost all e-mail clients. I'll show you how to enable it.

/* <FIXME>

– under construction below –

/usr/local/etc/rc.d/courier-imap-imapd-ssl.sh start

pkg_add -r courierpassd

/etc/inetd.conf if not already there: pop3pw stream tcp nowait root /usr/local/sbin/courierpassd courierpassd -s imap “imap” can be replaced with the service you wish to authenticate (and change) passwords against, eg: “pop3”. If you leave out the -s switch altogether, its value defaults to “login”. After modifying /etc/inetd.conf, you must (as root) run:

      killall -HUP inetd

; courier-authlib-base <http://freebsd.qmailrocks.org/imap.htm> scp …/imap.example.org.pem cd /usr/local/share/courier-imap ln -s imap.example.org.pem imapd.pem

/usr/local/etc/rc.d/courier-imap-imapd-ssl.sh restart */

Courier-IMAP

cp imap.example.org.pem /usr/local/share/courier-imap/

cp imapd.cnf /usr/local/etc/courier-imap cp imapd-ssl /usr/local/etc/courier-imap check that in imapd-ssl the following lines exist:

TLS_CERTFILE=/path/to/my/pemfile.pem
IMAPDSTARTTLS=YES
IMAPDSSLSTART=YES

-copying old maildir (with netcat, tar and gzip): on source machine (old):

tar -czf - maildirfolder/ | nc -w 3 ip.new.machine port

on target machine (new):

nc -l port | tar xvzf -

Converting mailbox to maildir

URL: http://batleth.sapienti-sat.org/projects/mb2md/ You'll probably need to 'install Date::Format' to install the required perl module, you can easily do this using the CPAN shell:

perl -MCPAN -e shell

Enter:

install Date::Format
exit

Installation of the mb2md perl script:

cd /usr/src
wget http://batleth.sapienti-sat.org/projects/mb2md/mb2md-3.20.pl.gz
gunzip mb2md*
cp mb2md-3.20.pl /usr/local/bin/

Run it:

perl /usr/local/bin/mb2md-3.20.pl -s /var/mail/<mailboxfile> -d 

/home/<username_for_mail>/Maildir/

Check if all mails are converted successfully, then remove the mailbox file.

Spamassassin

Official URL: http://spamassassin.apache.org/
Other URL: http://alex.kruijff.org/FreeBSD/Retrieve_e-mail.html URL: http://spamassassin.org/dist/INSTALL URL: http://freebsd.peon.net/tutorials/23/

Spamassassin is software that can pretty accurately detect if an e-mail message is spam (unsollicited e-mail). It has a few techniques, which are explained on its website, but I want to explain just a few checks:

The e-mail message is assigned a score, every time one of the rules matches, this score is increased with a few points (depending on the 'severity' of the rule). When the score is more than a set level, the message is marked as being SPAM.

  • Checks on the message content:
  • if the title o the body text of the message contain references to debt relief, mortages, rolex watches, viagra, stock buying hints, etc.
  • if the e-mail contains an image with text to fool the spamchecker. (HTML_IMAGE_ONLY)
  • if the e-mail contains attempts to obfuscate words in spam (e.g. V1A6RA instead of Viagra)
  • if one of the links in the e-mail message is in a database. (Contains an URL listed in the SBL or SURBL blocklist)
  • Checks on the message header:
  • if the IP from which the message is sent is in a database containing open SMTP relays/proxies

Databases containing those IP adresses are for example NJABL and SORBS

  • if the date of the message is far in the future (DATE_IN_FUTURE)

Marking all unsollicited e-mail as being spam is only part 1 of the story, putting the messages marked as spam in a seperate folder in the Maildir instead of delivering it to the 'INBOX'. Because SpamAssassin might consider some messages as spam when they actually aren't, I've chosen not to delete them directly, but to place them in a folder named 'SPAM', so the user can choose what to do with it. I'd suggest you do the same (I'll show you how to do this).

SpamAssassin Installation:

cd /usr/ports/mail/p5-Mail-SpamAssassin
make 

Options:

         Options for p5-Mail-SpamAssassin 3.1.8              
                     │ │   [X] AS_ROOT        Run spamd as root (recommended)           │ │
                     │ │   [ ] DOMAINKEYS     DomainKeys support                        │ │
                     │ │   [ ] DKIM           DomainKeys Identified Mail                │ │
                     │ │   [X] SSL            Build with SSL support for spamd/spamc    │ │
                     │ │   [X] GNUPG          Install GnuPG (for sa-update)             │ │
                     │ │   [ ] MYSQL          Add MySQL support                         │ │
                     │ │   [ ] PGSQL          Add PostreSQL support                     │ │
                     │ │   [X] RAZOR          Add Vipul's Razor support                 │ │
                     │ │   [ ] SPF_QUERY      Add SPF query support                     │ │
                     │ │   [ ] RELAY_COUNTRY  Relay country support                     │ │
                     │ │   [ ] TOOLS          Install SpamAssassin tools                │ │
I also tagged DOMAINKEYS, DKIM, SPF_QUERY and RELAY_COUNTRY, just for testing out these new functions, but it should work fine with just the default options tagged.
make install

Configure spamd to start on the next boot:

echo 'spamd_enable="YES"' >> /etc/rc.conf

Start it manually:

/usr/local/etc/rc.d/sa-spamd start

You should get a message indicating that spamd has been started.

Test it:

spamc < some_saved_spam_msg

<FIXME>Download an example spam message here

Would print (inserted into the original message) a result like this one:

...
X-Spam-Flag: YES
X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on
        example.org
X-Spam-Level: *******************
X-Spam-Status: Yes, score=19.7 required=5.0 tests=DK_POLICY_SIGNSOME,
        DNS_FROM_RFC_ABUSE,DNS_FROM_RFC_DSN,DNS_FROM_RFC_POST,EXTRA_MPART_TYPE,
        FORGED_OUTLOOK_TAGS,HTML_MESSAGE,HTML_SHOUTING5,MIME_HTML_MOSTLY,
        MPART_ALT_DIFF,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E4_51_100,RAZOR2_CHECK,
        RCVD_IN_DSBL,RCVD_IN_NJABL_DUL,RCVD_IN_NJABL_PROXY,RCVD_IN_XBL,TRACKER_ID
        autolearn=spam version=3.1.8
... original message continued ...

To fine-tune SpamAssassins behaviour, you can write configuration files for it, although without configuration files it will use default values, which work fine for me. You can use this online spamassassin-config-creator to make config files for you: URL: http://www.yrex.com/spam/spamconfig.php I haven't yet checked where I should put this configuration file. (~freebsd/.spamassassin/user_prefs?)

Now it's time to tell procmail (by editing it's configuration file .procmailrc) to run each incoming e-mail trough SpamAssassin (replace <freebsd_username> with the username of the user you wish to configure SpamAssassin for).:

cd ~<freebsd_username>
nano .procmailrc

Look for the following procmail 'recipe':

:0
*< 100000000
$MAILDIR/

Then insert the following text just before that 'recipe':

# The next rule sends all incoming e-mail with a size <256000 
# bytes though spamc (=the SpamAssassin client).
# SpamAssassin will change the headers of the e-mail to 
# include it's opinion on the message being spam or not.
:0fw: spamassassin.lock    
* < 2560000
| /usr/local/bin/spamc
 
# You can optionally add extra rules, like this one, which 
# would prevent e-mail with a 'from' addres containing '.nl' 
# undergoing spamfiltering (effectively, this says that all 
# mail from e-mail senders on Dutch domains (= '.nl') is good 
# and should never be filtered):
#  * !^From:.*\.nl([^a-z]|$)
#place this line before the '* < 256000' line.
 
# This recipe will deliver all e-mail messages which SpamAssassin
# has judged to be spam in the 'SPAM' folder, by looking for the 
# line 'X-Spam-Status' that SpamAssassin added to the e-mail message:
:0h:
* ^X-Spam-Status: Yes
$MAILDIR/.SPAM/

Now, whenever a new mail message arrives, procmail will run it trough SpamAssassin, and SpamAssassin will add a header to the e-mail message that looks like this one:

X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on
	<freebsd62.example.org>
X-Spam-Level: ***
X-Spam-Status: No, score=3.9 required=5.0 tests=DNS_FROM_RFC_ABUSE,
	DNS_FROM_RFC_POST,FORGED_HOTMAIL_RCVD2,SPF_SOFTFAIL autolearn=no version=3.1.8

You can check it by viewing the source of the e-mail message in your mailreader.

If you have problems getting procmail to send the mail to spamc, change 'VERBOSE=no' to 'VERBOSE=yes' in .procmailrc, and check the file ~<freebsd_username>/.procmail/log for errors when new mail arrives. Change the VERBOSE-option back to 'no' when you've solved the problem. See the procmail item for more information

FIXME: problem with truncated mails when spamassassin considers it spam

Extra reading material: http://en.wikipedia.org/wiki/DomainKeys
http://wiki.apache.org/spamassassin/RelayCountryPlugin (plugin to expose the countries that a mail was relayed from)

Spamassassin has 'rules' to determine if a e-mail is spam or not. These rules are updated regularly, so you might want to download new updates to improve spam filtering on your server. read: http://wiki.apache.org/spamassassin/RuleUpdates

Work in progress: fgrep -a -i -o -r -E -m 1 –color=auto “[A-Za-z\.]+@mydomain.com” * filter files for e-mail adresses

ClamAV

Clam Antivirus On-Demand Virusscanner.
Available for free for *BSD and Linux (there's a Windows version too, called ClamWin). Can scan e-mail attachments (with the help of ClamSMTP) or do on-demand scans with the 'clamscan' command.

Official URL: http://www.clamav.net

Install port:

cd /usr/ports/security/clamav
make
clamav options:
[ ] MILTER        Compile the milter interface              . .
[ ] LDAP          libmilter was built with LDAP             
[ ] CURL          Support URL downloading                   
[ ] STDERR        Print logs to stderr instead of stdout    
[ ] EXPERIMENTAL  Build experimental code  

? [X] LIBUNRAR Support for external Unrar library

make install

Start the ClamAV daemon:

/usr/local/etc/rc.d/clamav-clamd start

run the Clam antivirus updater (FreshClam)

freshclam

Test it on a file (you can download a non-harmful file for testing at the eicar.com website, which upon scanning with clamscan, should be detect as being a 'Anti Virus Test File')

clamscan <file_to_scan>

Clamscan can handle zip-files too.

nano rc.conf

Add the following lines to have ClamAV (clamd to be exact) started upon boot:

clamav_clamd_enable="YES"
clamav_freshclam_enable="YES"

ClamSMTP

Postfix filter to virus scan incoming e-mail with ClamAV Official URL: http://memberwebs.com/nielsen/software/clamsmtp clamsmtp-1.7 An ClamAV anti-virus SMTP Filter

I've just installed the Postfix mailserver, and mail is coming in nicely. Not only normal e-mails, but also SPAM (which I'm probably going to find a solution for next week, if I can find some spare time), but also an e-mail with a virus in its attachment. Getting rid of these virusses will probaly be much easier than getting rid of those unsollicited e-mail messages..

A search on Google for 'postfix' and 'clamav' sends me to this site: http://www.mail-archive.com/clamav-users@lists.clamav.net/msg23390.html “Popular choices for integrating clamav with postfix include clamsmtp and amavisd-new.”

From the search results of my searches on 'amavisd-new postfix' and 'clamsmtp postfix', I think I'll go for clamsmtp (less text in the installation instructions). Read the original thing here: http://memberwebs.com/nielsen/software/clamsmtp/postfix.html

Port installation: (version 1.7)

cd /usr/ports/security/clamsmtp
make
clamsmtp options:
[X] LOCAL_CLAMAV        RUN_DEPEND on security/clamav
[ ] LOCAL_CLAMAV_DEVEL  RUN_DEPEND on security/clamav-devel
make install  
cp /usr/local/etc/clamsmtpd.conf-sample /usr/local/etc/clamsmtpd.conf
nano /usr/local/etc/clamsmtpd.conf
# Address to listen on (defaults to all local addresses on port 10025)
Listen: 127.0.0.1:10025
# User to switch to
User: clamav

Test if all clamav applications run without errors: Start Clamd, the virusscanner daemon:

clamd

Start the ClamSMTPd in a new screen in debug mode (so you can see any errors):

screen
clamsmtpd -d 4 -f /usr/local/etc/clamsmtpd.conf

We first must configure postfix to allow clamsmtpd to connect on port 10026 before we can test this.

Make sure clamsmtpd gets started upon boot:

nano /etc/rc.conf
clamsmtpd_enable="YES"

Now let's integrate it with the Postfix MTA:

cd /usr/local/etc/postfix
nano main.cf    

add:

content_filter = scan:[127.0.0.1]:10025
nano master.cf    

add:

# AV scan filter (used by content_filter)
scan      unix  -       -       n       -       16      smtp
        -o smtp_send_xforward_command=yes
        -o smtp_enforce_tls=no
# For injecting mail back into postfix from the filter
127.0.0.1:10026 inet  n -       n       -       16      smtpd
        -o content_filter=
        -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
        -o smtpd_helo_restrictions=
        -o smtpd_client_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks_style=host
        -o smtpd_authorized_xforward_hosts=127.0.0.0/8

Reload the Postfix configuration:

postfix reload

Next, reboot. ?

Test it from any computer:

telnet localhost 10025
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
telnet: Unable to connect to remote host
clamsmtpd: 100000: CLAMAV > SCAN /tmp//clamsmtpd.bnSrem
clamsmtpd: 100000: CLAMAV < /tmp//clamsmtpd.bnSrem: Access denied. ERROR
clamsmtpd: 100000: clamav error: /tmp//clamsmtpd.bnSrem: Access denied. ERROR
clamsmtpd: 100000: CLAMAV connection closed
clamsmtpd: 100000: CLIENT > 451 Local Error
clamsmtpd: 100000: from=<my_real_e-mail_address>, to=<my_freebsd_username>@<freebsd62.example.org>, status=CLAMAV-ERROR

You have not configured the user to switch to in clamsmtpd.conf. The correct handling of an e-mail message looks like this:

  ...
  clamsmtpd: 100000: from=<my_real_e-mail_address>, to=<my_freebsd_username>@<freebsd62.example.org>, status=CLEAN

However, if things don't work as expected, use clamsmtpd in debug mode:

clamsmtpd -d 4 -f /usr/local/etc/clamsmtpd.conf
clamsmtpd: 100000: CLAMAV: couldn't connect to: /var/run/clamav/clamd: Connection refused
clamsmtpd: 100000: CLIENT > 451 Local Error

/var/log/clamav/clamd.log:

ERROR: Socket file /var/run/clamav/clamd exists. Either remove it, or configure a different one.

So I did a:

rm /var/run/clamav/clamd

and a clamd restart did the trick. I'll have to make something that checks if all important services are running ok.. I Still don't know what created this problem, and I expect to see it occur again.

Test it from any computer:

telnet freebsd62.example.org 25

It will tell you: “220 <freebsd62.example.org> ESMTP Postfix.”. Now enter:

mail from: <my_real_e-mail_address>

And after '250 Ok', enter (replacing the text with the < and > characters):

rcpt to: <freebsd_username>@<freebsd62.example.org>

(You can replace '<freebsd62.example.org>' with 'localhost' instead of your fully qualified domain name, this should work too) And after '250 Ok', enter:

data

And after '354 End data with <CR><LF>.<CR><LF>', enter some lines of text, finish with an single dot (.) on an empty line:

This is a test message.
.

It will give back something like: '250 Ok: queued as …'. Finish this SMTP-session by entering:

quit

Check your maildir for arrival of this new message. Check the file for presence of the text

X-Virus-Scanned: ClamAV using ClamSMTP

Which will confirm that this e-mail message has passed the virusscanner before delivery.

Test by e-mailing yourself the Standard Anti-Virus Test File (this not a real virus, but something for proofing virusscanners), visit http://www.eicar.org/anti_virus_test_file.htm do download it and mailing it yourself, or let it be e-mailed to you at http://www.aleph-tec.com/eicar/index.php. Things are fine when the messages won't arrive in your mailbox (..and all the others do).

Check the contents of the folder /var/spool/postfix/defer to see if any messages are undelivered. If something went wrong, you can get postfix to 'flush' your mailqueue (this means: perform a deliviry retry on all waiting/undelivered e-mail) with the command:

postqueue -f  

Re-check /var/spool/postfix/defer/… after the flush, to see if it solved the issue.

Sep 9 02:50:13 freebsd61 clamsmtpd: 100001: from=<my_real_email_address@example.org>, to=<freebsd_username>@freebsd61.example.org, status=CLAMAV-ERROR

One time on one server, e-mails weren't delivered (but were instead kept in the /var/spool/postfix/deferred folder, waiting for delivery). Trying telnet to localhost port 25 on the server, and simulating an e-mail going through the system, I got an error message:

451 Local Error

After I killed clamsmtpd, and restarted it in debug/log-to-console mode, I noticed this error message:

clamsmtpd: 100000: CLAMAV: couldn't connect to: /var/run/clamav/clamd: Connection refused

In the LogFile /var/log/clamav/clamd.log (I enabled logging in /usr/local/etc/clamd.conf) :

ERROR: Socket file /var/run/clamav/clamd exists.
 Either remove it, or configure a different one.

Ah.. I don't know what went wrong, but I tried the following, and it worked:

rm /var/run/clamav/*
killall clamd
clamd

Fetchmail

To retreive mail and forward it to postfix or procmail for further handling and filtering. Install package (as root):

pkg_add -r fetchmail
pkg_add -r getmail

The configuration file for regular users is ~/.fetchmailrc, but these are not read by the fetchmail daemon, when running fetchmail from rc.conf) So I'll make a global configuration, using the file /usr/local/etc/fetchmailrc. I am using procmail in my example, but you can also redirect messages trough postfix and the installed virusscanner.

As root, enter:

nano /usr/local/etc/fetchmailrc

Insert the following text, replace <pop3_server_of_my_isp>, <pop3_username>, <pop3_password> and <local_freebsd_username> with the correct values:

# Your fetchmail configuration goes here.
set syslog
set logfile "/var/log/fetchmail.log"
set daemon 60
 
poll <pop3_server_of_my_isp>
 with proto POP3
 interval 100
 auth password
 user "<pop3_username>" there
 with password "<pop3_password>"
 is <local_freebsd_username> here
 mda "/usr/bin/procmail -d %s"

Prevent other users from viewing this file (as it contains your pop3 username and password!)

chmod 0600 /usr/local/etc/fetchmailrc
chown fetchmail:fetchmail /usr/local/etc/fetchmailrc

Edit /etc/rc.conf and add: fetchmail_enable=“YES”

If you are configuring fetchmail as a local user, create the $HOME/.fetchmail folder, edit ~/.fetchmailrc (remember to chown/chmod) and change the logfile directive to “$HOME/.fetchmail/fetchmail.log”, and disable syslog.

What is NTLM/MSN authentication?

Mutt

Install package: (version 1.4.2.1_2)

pkg_add -r mutt

(will also install urlview-0.9_2, mime-support-3.35.1, ispell-3.2.06_14)

nano ~/.muttrc
    set spoolfile={<freebsd_username>@imap.example.org}
    set folder=imaps://<freebsd_username>@imap.example.org/
    set ssl_starttls=no
    set imap_user="<freebsd_username>"
    set imap_pass="<my_mothers_maiden_name>"
    set mask="\..*"
    set ssl_use_sslv3

The 'imaps:…' (instead of 'imap:…') will make mutt connect over the secure IMAP-SSL protocol, which I would certainly recommend if you are connecting to it over the internet. Replace 'imap.example.org' with the address of the IMAP-machine you want to connect to (and don't forget to change <freebsd_username> and <my_mothers_maiden_name> too.

To increase the readability, I added another line (this is optional):

    source ~/.muttcolor
nano ~/.muttcolor

The color scheme I use:

color hdrdefault 	red 		black
color quoted 		brightblue 	black
color signature 	red 		black
color indicator 	brightyellow 	red
color error 		brightred 	black
color status 		yellow 		blue
color tree 		magenta 	black # the thread tree in the index menu
color tilde 		magenta 	black
color message 		brightcyan 	black
color markers 		brightcyan 	black
color attachment 	brightmagenta 	black
color search 		black 		green	# how to hilite search patterns in the pager
 
color header 		brightred 	black 	^(From|Subject):
color body 		magenta 	black 	"(ftp|http|https)://[^ ]+" 	# point out URLs
color body 		magenta 	black 	[-a-z_0-9.]+@[-a-z_0-9.]+  	# e-mail addresses
color body      	yellow 		default "[;:]-[)/(|]"  			# colorise smileys
color body      	yellow 		default "[;:][)/(|]"")]]"")]]"
color underline 	brightgreen 	black

As we have stored the password in the .muttrc file, it's wise to disable others reading the file:

chmod go-r .mutt*

Test it:

mutt

Now mutt comes in color, so I can find unread messages quickly. Use 'c' to change folder, enter-key to go down a folder, and the spacebar to open it.

Combating return or bounce spam

Advanced bounce spam e-mail filtering on catch-all domains Requirements:

  1. procmail
  2. spamassassin
  3. fuzzy e-mail matching perl script (and perl, ofcourse)
  4. download link

work in progress

Squirrelmail webmail

Webmail application for IMAP servers

IS NOT EQUAL TO: http://www.courier-mta.org/sqwebmail/
?? pkg_add -r mhash php5-mhash
!NOT pkg_add -r squirrelmail
URL: http://www.squirrelmail.org

cd /usr/ports/mail/squirrelmail
make
make install
cd /usr/local/www/squirrelmail/config
./conf.pl
  D
  courier
  S
  Q
  2
  1
  freebsd61.example.org

http://freebsd61.example.org/squirrelmail/src/configtest.php

http://freebsd61.example.org/squirrelmail/index.php

Mozilla Thunderbird

Add-on, so I can quickly change the 'from' e-mail address when composing a message to someone: (sometimes I apply with a myname_thatservice@mydomain.com e-mail address. When I want to opt-out/unsubscribe I have to use that same e-mail address) URL: https://addons.mozilla.org/en-US/thunderbird/addon/594

mailserver_configuration_with_postfix_courier-imap_procmail_spamassassin_clamav.txt · Last modified: 2017-01-18 18:41 by 127.0.0.1