gene02
August 7th, 2007, 07:22 PM
HOWTO dspam with postfix
dspam (http://dspam.nuclearelephant.com/) is a spam filter for mail servers. I have found it to be less of a resource hog than spamassassin and easier to retrain. It took me a while to get a robust dspam installation going with email-based retraining support. The best tutorial I found was http://dspamwiki.expass.de/Installation/Postfix/NealesSetup, but I had to make some small modifications to get it working the way I wanted. This is my tutorial for how to get mysql-backed dspam running on a postfix-based system:
You should already have a running postfix and mysql installation.
Install the following packages:
dspam
dspam-doc
libdspam7
libdspam7-drv-mysql
I found that the mysql database, user, and tables were not automatically created, so I manually created a dspamdb mysql database and dspam user for that database. Then I created the tables:
mysql -u dspam -p -D dspamdb < /usr/share/doc/libdspam7-drv-mysql/mysql_objects-XXX.sql
mysql -u dspam -p -D dspamdb < /usr/share/doc/libdspam7-drv-mysql/virtual_users.sql
Note that mysql_objects-XXX.sql is one of the /usr/share/doc/libdspam7-drv-mysql/mysql_objects-*.sql files. Choose whichever looks the most appropriate.
If you don't see /etc/dspam/dspam.d/mysql.conf, copy it from /usr/share/doc/libdspam7-drv-mysql/ and edit the user, password, and database name configuration near the top of the file.
Now, edit the dspam configuration at /etc/dspam/dspam.conf. We will be configuring dspam to deliver all mail, tagged as spam or innocent, rather than quaranteening it. Users can then forward mistagged emails to dspam for retraining. dspam will be called from postfix and re-inject the tagged email back into the postfix pipeline.
First make sure it is set up to use mysql, by checking that the StorageDriver line says:
StorageDriver /usr/lib/dspam/libmysql_drv.so
We will use postfix's fake sendmail as the trusted delivery agent:
TrustedDeliveryAgent "/usr/sbin/sendmail"
And comment out the untrusted delivery agent:
#UntrustedDeliveryAgent "/usr/bin/procmail -d %u"
Comment out all the delivery configuration as the emails will be injected into postfix's own delivery pipeline:
#DeliveryHost 127.0.0.1
#DeliveryPort 10026
#DeliveryIdent localhost
#DeliveryProto SMTP
Comment out all the trusted users except for these:
Trust root
Trust dspam
Trust mail
Trust postfix
Change the Preference section to deliver all emails, whether spam or not, and put the dpsam signature in the headers. This will mean that you need to include the headers when forwarding emails for retraining:
Preference "spamAction=deliver"
Preference "signatureLocation=headers" # 'message' or 'headers'
Preference "showFactors=off"
Preference "spamSubject=SPAM"
#Preference "spamAction=tag"
#Preference "signatureLocation=message" # 'message' or 'headers'
#Preference "showFactors=on"
#Preference "spamAction=tag"
#Preference "spamSubject=SPAM"
Change ServerParameters to deliver all email:
ServerParameters "--deliver=innocent, spam"
Woops, I forgot to mention this in the original edit:
Edit /etc/default/dspam and change "START=no" to "START=yes"
Next, we have to modify the postfix configuration to work with dspam. Make backup copies of any file that you modify here so that you can quickly roll it back if there are any problems.
Edit /etc/postfix/master.cf and add these lines:
dspam unix - n n - 10 pipe
flags=Ru user=dspam argv=/usr/bin/dspam --deliver=innocent,spam --user $user -i -f $sender -- $recipient
dspam-retrain unix - n n - 10 pipe
flags=Ru user=dspam argv=/usr/local/bin/dspam-retrain $nexthop $sender $recipient
That will set up dspam as a transport which gets invoked for each incoming email. We'll get back to dspam-retrain later.
Edit /etc/postfix/main.cf.
This cludge will setup postfix so that only incoming mail gets filtered by spamd. Include, as the last restriction, the following smtpd_recipient_restrictions line:
check_client_access pcre:/etc/postfix/dspam_filter_access
This is my full smptd_recipient_restrictions as illustration. Don't copy the whole thing unless you know what all the lines mean, this can break your postfix setup!
smtpd_recipient_restrictions =
reject_unauth_pipelining
reject_non_fqdn_recipient
reject_non_fqdn_hostname
reject_non_fqdn_sender
permit_mynetworks
permit_sasl_authenticated
reject_unauth_destination
reject_invalid_hostname
reject_unknown_recipient_domain
check_policy_service inet:127.0.0.1:10031
reject_unlisted_recipient
check_recipient_access hash:/etc/postfix/access
check_policy_service inet:127.0.0.1:12525
check_client_access pcre:/etc/postfix/dspam_filter_access
permit
create the file /etc/postfix/dspam_filter_access with the following contents:
/./ FILTER dspam:dspam
Add these dspam config lines at the end of main.cf:
dspam_destination_recipient_limit = 1
dspam-add_destination_recipient_limit = 1
dspam-fp_destination_recipient_limit = 1
In order to do the retraining, we need to set up some special dspam email addresses. Create the file /etc/postfix/transport with the following contents:
spam@your.server dspam-retrain:spam
ham@your.server dspam-retrain:innocent
your.server should be your mail server domain.
Edit the transport_maps section of main.cf to include:
transport_maps = hash:/etc/postfix/transport
Apparently, dpsam gets run before aliases are read, so move all aliases in /etc/aliases into /etc/postfix/virtual.
Update the local_recipient_maps line to include virtual and transport:
local_recipient_maps = proxy:unix:passwd.byname $alias_maps hash:/etc/postfix/virtual $transport_maps
Finally, make sure to uncomment the recipient_delimiter line. We'll be using this for retraining:
recipient_delimiter = -
The last missing piece is the retrain script. This will parse the misclassified emails forwarded to dspam for retraining. I modified the version at http://dspamwiki.expass.de/Installation/Postfix/NealesSetup (http://dspamwiki.expass.de/DspamRetrainScript) to make it a little more robust and to report through syslog. You can find my version at http://www.smalltime.com/gene/dspam-retrain.pl.gz. My version requires the installation of the Logger::Syslog perl module.
wget http://www.smalltime.com/gene/dspam-retrain.pl.gz
gunzip dspam-retrain.pl.gz
sudo mv dspam-retrain.pl /usr/local/bin/dspam-retrain
sudo chmod 744 /usr/local/bin/dspam-retrain
sudo cpan -e shell
<... answer all the questions if you've never used CPAN before ...>
install Logger::Syslog
exit
At this point we should be just about ready to go. Now we just need to get everything reloaded:
sudo /etc/init.d/dspam restart
sudo postalias /etc/aliases
sudo postmap /etc/postfix/dspam_filter_access
sudo postmap /etc/postfix/transport
sudo postmap /etc/postfix/virtual
sudo postfix reload
Now try sending yourself an email and see if it gets through. It should have an X-Dspam-Result header line. Keep an eye on /var/log/syslog and if anything looks amiss, revert to your backup postfix files while you figure out what the problem was.
For retraining, forward missed spams to spam-username@your.server and falsely tagged innocent emails to ham-username@your.server, where username is your user name and your.server is your server domain name. Make sure that the X-Dspam-Signature header line is included in any forwarded retrain emails. You can take a look at the dspam stats after a few emails have gone through with this command:
sudo dspam_stats -H
You can also use the dspam_train command to get the dspam database up and running if you already have a collection of known good and spammy emails.
Please let me know if you try to follow this and have any problems or corrections.
dspam (http://dspam.nuclearelephant.com/) is a spam filter for mail servers. I have found it to be less of a resource hog than spamassassin and easier to retrain. It took me a while to get a robust dspam installation going with email-based retraining support. The best tutorial I found was http://dspamwiki.expass.de/Installation/Postfix/NealesSetup, but I had to make some small modifications to get it working the way I wanted. This is my tutorial for how to get mysql-backed dspam running on a postfix-based system:
You should already have a running postfix and mysql installation.
Install the following packages:
dspam
dspam-doc
libdspam7
libdspam7-drv-mysql
I found that the mysql database, user, and tables were not automatically created, so I manually created a dspamdb mysql database and dspam user for that database. Then I created the tables:
mysql -u dspam -p -D dspamdb < /usr/share/doc/libdspam7-drv-mysql/mysql_objects-XXX.sql
mysql -u dspam -p -D dspamdb < /usr/share/doc/libdspam7-drv-mysql/virtual_users.sql
Note that mysql_objects-XXX.sql is one of the /usr/share/doc/libdspam7-drv-mysql/mysql_objects-*.sql files. Choose whichever looks the most appropriate.
If you don't see /etc/dspam/dspam.d/mysql.conf, copy it from /usr/share/doc/libdspam7-drv-mysql/ and edit the user, password, and database name configuration near the top of the file.
Now, edit the dspam configuration at /etc/dspam/dspam.conf. We will be configuring dspam to deliver all mail, tagged as spam or innocent, rather than quaranteening it. Users can then forward mistagged emails to dspam for retraining. dspam will be called from postfix and re-inject the tagged email back into the postfix pipeline.
First make sure it is set up to use mysql, by checking that the StorageDriver line says:
StorageDriver /usr/lib/dspam/libmysql_drv.so
We will use postfix's fake sendmail as the trusted delivery agent:
TrustedDeliveryAgent "/usr/sbin/sendmail"
And comment out the untrusted delivery agent:
#UntrustedDeliveryAgent "/usr/bin/procmail -d %u"
Comment out all the delivery configuration as the emails will be injected into postfix's own delivery pipeline:
#DeliveryHost 127.0.0.1
#DeliveryPort 10026
#DeliveryIdent localhost
#DeliveryProto SMTP
Comment out all the trusted users except for these:
Trust root
Trust dspam
Trust mail
Trust postfix
Change the Preference section to deliver all emails, whether spam or not, and put the dpsam signature in the headers. This will mean that you need to include the headers when forwarding emails for retraining:
Preference "spamAction=deliver"
Preference "signatureLocation=headers" # 'message' or 'headers'
Preference "showFactors=off"
Preference "spamSubject=SPAM"
#Preference "spamAction=tag"
#Preference "signatureLocation=message" # 'message' or 'headers'
#Preference "showFactors=on"
#Preference "spamAction=tag"
#Preference "spamSubject=SPAM"
Change ServerParameters to deliver all email:
ServerParameters "--deliver=innocent, spam"
Woops, I forgot to mention this in the original edit:
Edit /etc/default/dspam and change "START=no" to "START=yes"
Next, we have to modify the postfix configuration to work with dspam. Make backup copies of any file that you modify here so that you can quickly roll it back if there are any problems.
Edit /etc/postfix/master.cf and add these lines:
dspam unix - n n - 10 pipe
flags=Ru user=dspam argv=/usr/bin/dspam --deliver=innocent,spam --user $user -i -f $sender -- $recipient
dspam-retrain unix - n n - 10 pipe
flags=Ru user=dspam argv=/usr/local/bin/dspam-retrain $nexthop $sender $recipient
That will set up dspam as a transport which gets invoked for each incoming email. We'll get back to dspam-retrain later.
Edit /etc/postfix/main.cf.
This cludge will setup postfix so that only incoming mail gets filtered by spamd. Include, as the last restriction, the following smtpd_recipient_restrictions line:
check_client_access pcre:/etc/postfix/dspam_filter_access
This is my full smptd_recipient_restrictions as illustration. Don't copy the whole thing unless you know what all the lines mean, this can break your postfix setup!
smtpd_recipient_restrictions =
reject_unauth_pipelining
reject_non_fqdn_recipient
reject_non_fqdn_hostname
reject_non_fqdn_sender
permit_mynetworks
permit_sasl_authenticated
reject_unauth_destination
reject_invalid_hostname
reject_unknown_recipient_domain
check_policy_service inet:127.0.0.1:10031
reject_unlisted_recipient
check_recipient_access hash:/etc/postfix/access
check_policy_service inet:127.0.0.1:12525
check_client_access pcre:/etc/postfix/dspam_filter_access
permit
create the file /etc/postfix/dspam_filter_access with the following contents:
/./ FILTER dspam:dspam
Add these dspam config lines at the end of main.cf:
dspam_destination_recipient_limit = 1
dspam-add_destination_recipient_limit = 1
dspam-fp_destination_recipient_limit = 1
In order to do the retraining, we need to set up some special dspam email addresses. Create the file /etc/postfix/transport with the following contents:
spam@your.server dspam-retrain:spam
ham@your.server dspam-retrain:innocent
your.server should be your mail server domain.
Edit the transport_maps section of main.cf to include:
transport_maps = hash:/etc/postfix/transport
Apparently, dpsam gets run before aliases are read, so move all aliases in /etc/aliases into /etc/postfix/virtual.
Update the local_recipient_maps line to include virtual and transport:
local_recipient_maps = proxy:unix:passwd.byname $alias_maps hash:/etc/postfix/virtual $transport_maps
Finally, make sure to uncomment the recipient_delimiter line. We'll be using this for retraining:
recipient_delimiter = -
The last missing piece is the retrain script. This will parse the misclassified emails forwarded to dspam for retraining. I modified the version at http://dspamwiki.expass.de/Installation/Postfix/NealesSetup (http://dspamwiki.expass.de/DspamRetrainScript) to make it a little more robust and to report through syslog. You can find my version at http://www.smalltime.com/gene/dspam-retrain.pl.gz. My version requires the installation of the Logger::Syslog perl module.
wget http://www.smalltime.com/gene/dspam-retrain.pl.gz
gunzip dspam-retrain.pl.gz
sudo mv dspam-retrain.pl /usr/local/bin/dspam-retrain
sudo chmod 744 /usr/local/bin/dspam-retrain
sudo cpan -e shell
<... answer all the questions if you've never used CPAN before ...>
install Logger::Syslog
exit
At this point we should be just about ready to go. Now we just need to get everything reloaded:
sudo /etc/init.d/dspam restart
sudo postalias /etc/aliases
sudo postmap /etc/postfix/dspam_filter_access
sudo postmap /etc/postfix/transport
sudo postmap /etc/postfix/virtual
sudo postfix reload
Now try sending yourself an email and see if it gets through. It should have an X-Dspam-Result header line. Keep an eye on /var/log/syslog and if anything looks amiss, revert to your backup postfix files while you figure out what the problem was.
For retraining, forward missed spams to spam-username@your.server and falsely tagged innocent emails to ham-username@your.server, where username is your user name and your.server is your server domain name. Make sure that the X-Dspam-Signature header line is included in any forwarded retrain emails. You can take a look at the dspam stats after a few emails have gone through with this command:
sudo dspam_stats -H
You can also use the dspam_train command to get the dspam database up and running if you already have a collection of known good and spammy emails.
Please let me know if you try to follow this and have any problems or corrections.