Subject: Re: Merging sendmail queues..
To: None <netbsd-help@NetBSD.org>
From: Chuck Yerkes <chuck+nbsd@2004.snew.com>
List: netbsd-help
Date: 03/02/2004 15:12:56
Quoting Marc Tooley (netbsdMLpost@quake.ca):
> I haven't found much in the way of merging sendmail queues around, and 
> asking in the usual places didn't garner much, so I thought I'd ask 
> here. I know I should be moving to postfix or something else, but one 
> thing at a time! :)

Oh pish!  postfix is for those not smart enough to figure out
sendmail and it's power ;)   (kidding! people).


> Is it possible to just "mv" the queue files into the system's primary 
qtool.pl is in the contrib area of sendmail.  This should be
used (or is the basis for tools for you).

> I have a chroot environment that I'm using sendmail from within, and 
> then subsequently merge the chroot mqueue with the outer. Currently 
> what I'm doing is delivering solely within the chroot directly to the 
> queue, and then at regular intervals taking a snapshot of those 
> messages waiting delivery (for bytecount purposes) and actually running 
> the queue. I am using a 1h MinQueueAge for that part.

For chroot, I use the SafeEnvironment option in the mc when
I need it.  Other neat options on machines that have NO local
delivery (no programs, no users) is to redirect port 25 to,
say, port 20025 and have sendmail running there not as root.

Sendmail is pretty quick about bind to port 25, become another user.
It's no longer setuid in 8.12.

In reality, on DMZ machines, I runas another user, in another group.
It must be able to create/write in the /var/spool/mqueue/ hierarchy
(I use queue groups and have, in total, around 50 directories there.
10k outbound daemons messages end up being around 400 messages/dir
in the DAEMON queuegroup - fine for FFS1)

> Is it possible to just "mv" the queue files into the system's primary 
> mail queue after the first-run attempt and not worry too much about 
> collision if I check the existence of the destination file beforehand? 
> Or, does sendmail have a "better" way of doing this that I'm just not 
> aware of?

Back to moving:
you never want to move a message that's in process (eg. sendmail is
receiving or trying to send it).  mv is bad for that.

1) file/queuenames will be unique on a machine.  That was a fix
for problems we'd run into moving queue files around on a stopped
machine.

PERHAPS THE EASIEST ANSWER:
You can also just have the "outer" sendmail INCLUDE the dirs
used by the "inner" chrooted sendmail as a queuegroup.

Sendmail respects sendmail's locking - we expect there to
be multiple sendmail processes.


As far as moving to cascade old mail out of the way:
It would have been nice, and was desired by people involved, but
not involved enough, to allow sendmail to migrate messages from
one queuegroup to another - eg. after 1day move it to the "slowmail"
queuegroup.

In practice, at large enough sites, putting a fallback MX cascade
does that job - first attempt will try to send for 20 seconds total,
if it fails, queue it, next attempt will try for more time, if
fail, run it off to fallback1.  Repeat if necessary for fallback2.
At several hundred thousand messages/hr, you can justify more machines.
In more sane environments, I really just want older messages bumped
over to a "lessLikelyToEverWork" queue.


8.12 was tested and developed under very very high email volumes.
qtool.pl is the necessary tool to move things around.


I'd note that _FFR_QUEUERETURN_DSN is a GREAT thing to compile with
(an FFR because a new feature mandates a version bump (8.13) and
typically they want more things to bump it).  It allows:
Timeout.Queuereturn.dsn=1d - dump messages from MAILER-DAEMON after
1d, let the rest use the default 5d.  Very useful for handling
bouncing spam.

> What I'd like to do, though, to improve the accuracy of the bytecount 
> run attempt *merge* the inner chroot mqueue with the outer so it can be 
> dealt with normally by the primary sendmail daemon, after the first 
> attempt done from within the chroot environment.

Also net-snmp-5.1, if compiled with MIBII/mta_sendmail, will let
snmp monitor queuegroups, queue sizes, and so forth.