Re: greylisting multiple mail servers, greylisting with SPF, challenge response

["Jeremy C. Reed" <reed%reedmedia.net@localhost>]

On Thu, 27 Aug 2015, Matthias Scheler wrote:

> > I workaround these by adding individual IPs or blocks to my pf rules to 
> > bypass the spamd (so goes direct to mail server).
> 
> It sounds like you need a better greylisting software. I would recommend
> "milter-greylist" which works with Sendmail and Postfix.

Thanks. I installed it from pkgsrc and it appears to be working fine. 

> It makes whitelisting e.g. Microsoft's "outlook.com" very easy:
> 
> 	#	Outlook.com
> 	racl whitelist domain .outbound.protection.outlook.com

I am hoping I don't have to do that (in this case using spf).

> > I can automate updating the pf whitelist table from DNS SPF records, but 
> > that doesn't help with unknown senders.
> 
> Not sure what you mean by that. But "milter-greylist" has builtin
> SPF support.

I built the package with
 PKG_OPTIONS.milter-greylist="dnsrbl p0f postfix-milter spamassassin spf"
(the default I changed from "sendmail-milter" to "postfix-milter")

Thanks for pointing me to milter-greylist (and thank you manu@).

I have a few comments about it (maybe later I will discuss at their 
list):

1) user "smmsp" was the default and it appeared to work. I changed to 
user "postfix" though.

2) lots of logging with milter-greylist: (unknown id). I looked at 
source code and it appears that maybe this is because postfix doesn't 
have queueid. I am not sure if this matters, but maybe it could have a 
friendlier log output (maybe generic "postfix-queue")?

3) changed default dumpfreq from 1 to 60. Warnings in docs about 
dumping to frequent, so seems like one second default is too frequent. I 
don't know.

4)  changed global setting to greylist for 15 minutes instead 
of 30: greylist 15m

It has been long time since I researched, but some common servers used 
to retry to me like: 
1 minute, then 3 minutes, then 9 minutes, then 27 minutes, then 60 
minutes. So if default retry greylisting is 30 minutes, I may need to 
wait 60 minutes. (I haven't analyzed the timing recently.)

5) changed global setting to keep whitelisted for 10 days 
instead of 1: autowhite 10d

I used to use 36 days. 10d has not meaning to me, but the default 1 day 
seems much too short.

6) changed how long greylist tuples are retained instead of 
default 5d: timeout 6d

I cannot remember why, but I think I saw some mail servers not retry 
until after 5 days. Something broken maybe but allow another day to try.

7) I used a few DNSRBLs and then greylist them with delay of 6h (instead 
of 15 minutes as custom defined above). If they are in a DNSRBL, I don't 
block here. I think this means that later they can get through. The 
postfix also used the reject_rbl_client for same. Maybe by delaying some 
will make it that other DNSRBL lists also contain the IP. As an example:

dnsrbl "SORBS DUN" dnsbl.sorbs.net 127.0.0.10
racl greylist dnsrbl "SORBS DUN" delay 6h

8) I had a bunch of spamtraps that before were used to tarpit smtp 
connections with very slow conversations. So if the email is sent to, 
that sending IP was tarpitted. Now I have:

racl blacklist rcpt paytonbarlenequark%bsdnewsletter.com@localhost flushaddr
(and several others)

I confirmed that when this RCPT TO: is sent, it gets denied "Go away!" 
and then the previously open IP is not back in "Greylisting in action".
This may be a better solution than I had before as I found that some 
legitimate sending servers also mailed to my spamtraps. This new 
solution will allow them back in via greylisting (as long as they don't 
keep sending to my spamtraps).

9) Later I saw logs about postfix/smtp rejecting some emails to my 
spamtraps due to DNSRBL and I was confused why weren't logging about 
blacklisted. I assume the postfix order was to do the 
smtpd_client_restrictions before my milter was used.  So this means that 
DNSRBL blocked spamtraps won't flush my milter-greylist whitelist 
entries. I guess this is fine -- if later the DNSRBL delists the IP then 
I can feel fine with greylist delay going from my configured 6 hours 
down to my 15 minutes.

10) The tarpit feature is not documented in man pages. The README 
actually confused me and I still don't understand. But anyways, it 
didn't work for me: 
 	libmilter >= 8.14 is required for tarpit
For now my spamtrap solution above seems fine.  But I would like to 
consider the idea to slow down conversations to waste their time and 
potentially help others.

11) My previous setup added IPs to my spamtrap tarpit if they first 
communicated with a different MX that wasn't the first. I sometimes 
wonder if that is dangerous as maybe some network problem caused the 
first connection to highest priority MX was lost so it fell back 
correctly to another MX.

12) When I first enabled (prior to any DNSRBL) I got a bunch of spam 
allowed (SPF-compliant, bypassing greylist). I saw that the spammers had 
correct SPF DNS TXT records. I think maybe it would be useful to 
greylist the sender at least one time even if SPF matched.  So maybe the 
tuple wouldn't just record the IP (since that may change) but record 
that it should be checked against SPF second time (so ignore the IP as 
part of the tuple in second connection if using SPF).  Maybe this would 
mean that the spammer would be delayed -- and maybe due to spammer's 
resources they won't try again -- or maybe they will get listed in a 
DNSRBL within the next few minutes before they try again.

13) Later on I will automate some statistics on how many were 
greylisted, blocked due to spamtraps, passed due to SPF, etc.

By the way, I also upgraded from a very old postfix. The install message 
suggested:
 	/usr/pkg/sbin/postfix upgrade-configuration
but that didn't work since couldn't know where my configurations were. 
For example, I was upgrading from /etc/postfix/ to 
/usr/pkg/etc/postfix/.  So I coped files into place (after doing some 
backups). Then got a series of other errors like:
postfix: fatal: /usr/libexec/postfix/postfix-script: No such file or 
directory
(and then post-install and then another one ...)

I am still interested in using a challenge response system for some of 
my email addresses. Any one have suggestions for that?