Subject: Re: Merging sendmail queues.. (plus one more question at the bottom)
To: Netbsd-help@NetBSD.org, Chuck Yerkes <chuck+nbsd@2004.snew.com>
From: Marc Tooley <netbsdMLpost@quake.ca>
List: netbsd-help
Date: 03/02/2004 13:38:01
On Tuesday 02 March 2004 12:12, Chuck Yerkes wrote:
> Quoting Marc Tooley (netbsdMLpost@quake.ca):
> > I haven't found much in the way of merging sendmail queues around,
> > and asking in the usual places didn't garner much, so I thought I'd
> > ask here. I know I should be moving to postfix or something else,
> > but one thing at a time! :)
>
> Oh pish!  postfix is for those not smart enough to figure out
> sendmail and it's power ;)   (kidding! people).

=]

> qtool.pl is in the contrib area of sendmail.  This should be
> used (or is the basis for tools for you).

Absolutely PERFECT. Thank you, that's precisely what I was looking for. 
Your note was much appreciated, and tremendously helpful and detailed. 
Thanks again!

> For chroot, I use the SafeEnvironment option in the mc when
> I need it.  Other neat options on machines that have NO local
> delivery (no programs, no users) is to redirect port 25 to,
> say, port 20025 and have sendmail running there not as root.

In my case all I'm doing (or trying to do, with varying degrees of 
success) is providing a way for chroot'd users to send mail to the 
primary mail mechanism outside the chroot and via a command-line 
program, and in a recordable, byte-countable way.

> In reality, on DMZ machines, I runas another user, in another group.
> It must be able to create/write in the /var/spool/mqueue/ hierarchy
> (I use queue groups and have, in total, around 50 directories there.
> 10k outbound daemons messages end up being around 400 messages/dir
> in the DAEMON queuegroup - fine for FFS1)

Wow. That's big.

> Back to moving:
> you never want to move a message that's in process (eg. sendmail is
> receiving or trying to send it).  mv is bad for that.

No problem, I think. I can schedule the queue run and make sure it's 
finished before moving the remaining messages out into the outer 
mqueue.

> 1) file/queuenames will be unique on a machine.  That was a fix
> for problems we'd run into moving queue files around on a stopped
> machine.

Excellent. I was hoping that at the very least the names were randomized 
to the point where collision was *very* unlikely, but this is even 
better.

> PERHAPS THE EASIEST ANSWER:
> You can also just have the "outer" sendmail INCLUDE the dirs
> used by the "inner" chrooted sendmail as a queuegroup.

This would work well, but I think the qtool.pl allows me more 
flexibility in accounting for the incoming/outgoing messages on a 
per-user basis. Glomping both queues together I'll perhaps leave for 
when things get too busy to be worrying about silly byte-counts. :)

> In practice, at large enough sites, putting a fallback MX cascade
> does that job - first attempt will try to send for 20 seconds total,
> if it fails, queue it, next attempt will try for more time, if
> fail, run it off to fallback1.  Repeat if necessary for fallback2.
> At several hundred thousand messages/hr, you can justify more
> machines. In more sane environments, I really just want older
> messages bumped over to a "lessLikelyToEverWork" queue.

Policies like this are always interesting to hear about.

> I'd note that _FFR_QUEUERETURN_DSN is a GREAT thing to compile with
> (an FFR because a new feature mandates a version bump (8.13) and
> typically they want more things to bump it).  It allows:
> Timeout.Queuereturn.dsn=1d - dump messages from MAILER-DAEMON after
> 1d, let the rest use the default 5d.  Very useful for handling
> bouncing spam.

Indeed!

> Also net-snmp-5.1, if compiled with MIBII/mta_sendmail, will let
> snmp monitor queuegroups, queue sizes, and so forth.

Got it, thanks.

By the by, is there any pattern matching for virtusertable entries being 
planned or looked at? A capability like:

   someone-*@somedomain.com myUserName
   @somedomain.com error:nouser Go away, nobody here.

... would be interesting, as it would allow for nearly 
automatically-generated one-offs to be used for things like tracking 
spammer harvesters. I just got my first spam on a new email address I 
put up specifically to let me know the address came from a websites, 
but then I realized after looking at the logs that the (duh) the 
harvester looked just like a normal browser and I'd out-smarted myself. 
With patterns, a someone-10.4.5.9@somedomain.com (or even just a 
nondescript hash or something) could weed out the harvesting spammers 
and I could make a useful complaint to an ISP.

-Marc