Subject: PR/31127 CVS commit: pkgsrc/security/amavisd-new
To: None <pkg-manager@netbsd.org, gnats-admin@netbsd.org,>
From: Marc Recht <recht@netbsd.org>
List: pkgsrc-bugs
Date: 09/04/2005 20:24:02
The following reply was made to PR pkg/31127; it has been noted by GNATS.
From: Marc Recht <recht@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/31127 CVS commit: pkgsrc/security/amavisd-new
Date: Sun, 4 Sep 2005 20:23:14 +0000 (UTC)
Module Name: pkgsrc
Committed By: recht
Date: Sun Sep 4 20:23:14 UTC 2005
Modified Files:
pkgsrc/security/amavisd-new: Makefile distinfo
Log Message:
update to amavisd-new 2.3.3
patch provided by eggert at macvaerk dot dtu dot dk
in PR 31127
changes:
Version 2.3.3 is a maintenance release over 2.3.2. Besides fixing known
problems and providing some optimizations, no new features were added.
If using SpamAssassin older than 3.1, an upgrade of either SA to 3.1,
or an upgrade of amavisd-new to 2.3.3 is recommended.
- privacy: add a safety fuse / workaround around calls to SA to detect
SA's failure (in SA versions before 3.1) to catch a failed exec() in a
forked process, which could produce runaway process clones. See SA bug
report #4370. An incident of a mail copy being delivered to unrelated
recipient reported by Joel Nimety;
- privacy: turn warning into a fatal error when a quarantine ID of a message
requested for a quarantine release does not match the requested mail_id;
- security: require minimal version 1.35 of Compress::Zlib to avoid
vulnerability in the zlib compression library;
- the dsn_cutoff_level should have been ignored if undefined according to
documentation, but was not, causing DSN to be suppressed regardless of
spam level; discovered by Gary V;
- ensure the banned check is not performed if all recipients agree
it is not needed, even in presence of $banned_namepath_re;
undesired behaviour (not strictly incorrect) reported by Joel Nimety;
- missing import of lookup_ip_acl in module Amavis::In::AMCL caused
failure in sendmail milter setup when using the new AM.PDP protocol;
reported by Mic And;
- document and explicitly define handling of syntactically invalid IP address
in lookup_ip_acl: it matches a zero-length-mask net, a constant lookup table,
or a hash entry with an undef key, but no other entries in IP lookup tables;
syntactically invalid IP addresses are now logged;
- fix parsing if IPv6 address in $notify_method and $forward_method in case
of dynamic destination override (the use of '*' in method fields);
- check during startup that $myhostname is a fully qualified domain name
(or 'localhost', if you must), and abort if it isn't, otherwise a non-FQDN
can end up in places where RFC 2822 does not allow it; if uname(3) does not
provide a FQDN, then an assignment to $myhostname must be done explicitly
in amavisd.conf;
- when quarantining to a single file in mbox format the 'From ...' line
needs an English date, regardless of current locale; fixed by globally
setting locale LC_TIME to "C";
- pass on the parameter BODY=8BITMIME on MAIL FROM when submitting to MTA
when original message reception indicated it is needed (RFC 1652).
Note that mail forwarding may now fail if the feeding MTA requests
BODY=8BITMIME SMTP service extension (or just passes data with msb set),
but the MTA on the output side does not allow the use of the BODY parameter
in SMTP. In case of Postfix this may only happen when receiving service
on port 10025 is misconfigured and does not announce ESMTP capability
and support for the SMTP service extension 8BITMIME;
- RFC 2554 requires auth_param to be xtext-encoded addr-spec (no angle
brackets) or "<>", not the xtext-encoded addr-spec enclosed in angle
brackets (when specifying submitter during authentication); fixed;
- apply some sanity limit on collected bad-header samples to ensure that
a grossly broken mail does not unnecessarily fill up memory;
- when sending recipient warnings for viruses, banned files, or bad headers,
recipient address must not be rfc2822-quoted twice; fixed;
- fix interpretation of $defang_all to really imply all; previously it only
affected clean messages;
- in quarantined mail the reported spam score in X-Spam-Status header field
now includes maximum of all by-recipient score boosts (less surprising
when soft-whitelisting through @score_sender_maps is in use); suggested
by Mike Cappella and Gary V;
- when a policy delegation protocol attribute "request" is not "AM.PDP"
(perhaps it is a Postfix policy delegation request) don't attempt to find
and open a mail file;
- do_ascii and do_unarj: set environment variable TMPDIR or a command line
temporary directory option to "$tempdir/parts" instead of $TEMPBASE
to minimize possible pollution of top level directory;
- don't abort even if amavisd.conf returns undef as a final value,
as long as there are no errors reading or interpreting it;
- if during 'amavisd stop' or 'amavisd reload' the old running daemon does
not go away for one minute after sending it a SIGTERM, use a bigger
hammer and send it a SIGKILL; suggested by Sven Riedel;
- extend LDAP lookups to allow multiple search attributes (multiple
occurrences of %m in a query); a patch by Michael Hall (and a similar
one by Matthias Bandemer);
- LDAP lookup on an empty envelope address (e.g. a null return path)
adds another lookup key "<>", as it is difficult if not impossible
to have LDAP attributes with empty string as a value; by Michael Hall;
- LDAP.schema: drop "MUST ( mail )" from objectclass 'amavisAccount';
suggested by Michael Hall;
- updated comments and documentation, most notably the README.chroot;
- contributed file Macintosh.tar.gz updated by Dale Walsh;
COMPATIBILITY
- replaced 'hits=' with 'score=' in inserted X-Spam-Status header field
(and in some internal log entries) for compatibility with a changed
default in SpamAssassin 3.1;
- insert X-Spam-Score header field for compatibility with SA (previously
insertion of this header field was commented-out because the information
is redundant, as the score already appears in X-Spam-Status);
OPTIMIZATION
- speed up sending a mail header or full defanged (rewritten) mail over SMTP
back to MTA by a factor of 4 by buffering header fields into large chunks
to avoid bottleneck in Net::Cmd::datasend, which has lots of overhead for
line-by-line writes. Previously slow writes mostly affected mail messages
with extreme header lengths (such as results of a broken mail loop), or
when delivering defanged messages, particularly at sites with large MTA
mail size limits, sometimes to a point of exceeding timeout limits;
reported by Dominik Weber and Ralf Hildebrandt;
- move subroutine lookup_ip_acl() and associated ip_to_vec() into its own
dedicated new package Amavis::Lookup::IP; provide a constructor to pre-parse
IP lookup tables to speed up IP lookups in lookup_ip_acl; prepare pre-parsed
commonly used IP lookup tables (@mynetworks_maps, @publicnetworks_maps,
@inet_acl);
- optimized reading loop in SMTP DATA state, receiving data is now about
35% faster when mail size limit is not enforced (which is a default);
no speedup when mail size limit _is_ enforced;
- cache results of evaluated macros during a single call to expand(),
as macro calls often come in pairs, like: [?%e||\[%e\] ]
or [? %#T ||, Tests: [%T|,]]; together with the above optimization in
pre-parsed IP lookups it shaves off 25% of time in preparing main log entry;
- set locale LC_TIME to "C" globally, avoid changing and restoring locale
for every log write and when generating RFC2822 timestamps;
- added an optimization note in README.sql about indexes and about
SELECT count(*) in MySQL with InnoDB; investigation by Paolo Cravero;
---------------------------------------------------------------------------
June 29, 2005
amavisd-new-2.3.2 release notes
INCOMPATIBILITY with 2.3.1 and earlier versions:
If running amavisd daemon in chroot please note:
Each child process now opens its own syslog connection or a file descriptor
to a log file, and no longer inherits a connection from its parent.
When running in chroot jail and logging to syslog, the syslog client
routines need syslogd socket to be present in the chroot subtree to be
able to establish a connection with syslogd, otherwise logging output
may be lost. Additional syslogd sockets (to be made available in the
jail) may be requested from the syslogd daemon, see its documentation.
This requirement is equivalent to the requirement of chrooted Postfix
services (see Postfix documentation file BASIC_CONFIGURATION_README).
BUG FIXES since 2.3.1:
- do not enforce $MAXFILES limit during top-level MIME decoding to avoid
tempfailing mail; MIME parts are still counted, so a limit exceeded may
still be reported during subsequent decoding, but this is handled more
gracefully and does not cause preserved temporary directories to be left
behind; reported by Marcin Lemanski; suggested by Stephane Lentz and
Robert LeBlanc (noted in the 2.0 release notes);
- use recv() instead of read() to get results from daemonized virus scanners
in an attempt to avoid a bogus Perl I/O status on some Linux installations
(reported by Sander Steffann); we now get a meaningful status codes like
ECONNRESET instead of a bogus EBADF (Bad file descriptor);
- ignore status ECONNRESET when reading results of a daemonized virus scanner
from a socket, specific to some Linux versions; thanks to Sander Steffann
for the initial report and extensive help in debugging the Perl problem;
- run_av and other similar code sections: replace line-by-line reads by
block-by-block reads wherever possible to avoid inappropriate status report
EBADF (Bad file descriptor) caused by Perl I/O bug when last line is not
terminated by a newline. The problem was affecting reading response from
some command line virus checkers; reported by Sander Steffann;
- ignore status EAGAIN when reading results on a pipe from a forked process;
the status EAGAIN seems to be an artifact of Perl I/O on some installations;
reported by several people to cause problems on FreeBSD with Perl 5.8.7
(but Perl 5.8.6 is fine); thanks to Bart Matterne for testing and feedback;
- allow one level of indirection when collecting %needed_protocols;
global setting $protocol='COURIER' did not work, a workaround was needed
with previous version, e.g.: $policy_bank{'QMQPqq'}={protocol=>'QMQPqq'};
reported by Nicklas Bondesson and Martin Orr;
- fix a bug (introduced with 2.3.0) in Courier and QMQPqq setups, where global
information about processed message wasn't always reset and could leak
into processing of a subsequent message; reported by Nicklas Bondesson;
- SQL: fix arguments in calls to last_insert_id(), failing under PostgreSQL
(MySQL didn't mind); pointed out by Henrik Krohns;
- if module SAVI is loaded, insist it is version 0.30 or later;
incompatibility with earlier versions reported by Andrzej Kukula;
- make use of the new Net::Server 0.88 hook run_n_children_hook() to
reload SAVI database; removes a need to apply SAVI patch to Net::Server;
the Net::Server hook was suggested by Paul B. Henson and others,
and incorporated into Net::Server 0.88 by Paul Seamons;
- reopen log file or syslog connection in each child process to make it use
its own file descriptor; also minimizes transients when syslogd is restarted
and its socket re-created, as reported by Les Ault. When running in chroot
please make sure a syslogd socket is also available in the chroot jail,
see README.chroot for syslogd options (and BASIC_CONFIGURATION_README
in Postfix documentation for the Postfix equivalent);
- close log file or syslog in forked process before exec, just to play nicely;
- do_lha: fix extracting archive member filename in case of broken archive
or empty name (avoid interpreting creation date as a file name);
do not increment OpsDecByLha counter for empty archives, which are
most likely not lha archives at all;
- obey $final_bad_header_destiny D_DISCARD or D_REJECT even for messages
with bad headers from mailing lists or with a null envelope sender (DSN);
previously such messages were passed; undesired behaviour reported
by Cami Sardinha.
Such messages are still let through with $final_bad_header_destiny set to
D_BOUNCE, as otherwise they will be lost because a bounce is suppressed
for null sender messages and for mail from mailing list. This behaviour
is retained for backwards compatibility, but may need to be reconsidered.
- fix regexp for extracting am_id from amavis-milter helper program requests;
- if fork/exec fails, try to commit suicide in forked process with
POSIX::_exit(1) first, before trying kill('KILL',$$) as a last resort;
- updated $log_templ example in amavisd.conf-sample to match the default;
pointed out by Gary V;
- further reduce a couple of more frequent Perl warnings about the use of
uninitialized values in expressions;
- pre-load additional Perl modules required by SA 3.1 plugins;
- require minimal versions of modules: Time::HiRes 1.49, Archive::Zip 1.14;
- replaced nonexistent variable @sa_spam_modifies_subj_maps by
@spam_modifies_subj_maps in commented-out example in amavisd.conf-sample;
noticed by Joachim Schoenberg;
LDAP CHANGES by Michael Hall:
All the LDAP changes are transparent to the user.
- rewritten some of the code similar to the restructuring of the SQL code
in version amavisd-new-2.3.0. A new package Amavisd::LDAP::Connection was
added which is a LDAP connection object, and the old connection-related code
in Amavis::Lookup::LDAP has been moved to the new package. Amavisd-new will
now try to reconnect (once) while processing a message, similar to SQL;
- added the ability to specify a '%d' (domain) token in the LDAP base DN;
based on idea from Alexander Wittig;
- updated default LDAP port based on whether SSL/TLS is being used or not;
based on idea from Timo Veith;
- updated the search code to query for multiple records and return the results
sorted in 'make_query_keys' order versus doing a query for each key.
As a result performance is enhanced, and the tweaks 'ldap_get_all', and
'use_query_keys' (recently added) are no longer applicable or needed
and have been removed;
- improved LDAP error reporting and misc changes to multivalued attributes;
- documentation changes (amavisd.conf-default, README.lookups);
MINOR IMPROVEMENTS:
- macro %c (commonly used in a log template) reports spam score no longer
as a single number, but as an explicit sum of a SA score and a by-sender
boost score (from @score_sender_maps) when boost score is nonzero;
suggested by Ed Walker;
- enhancement to amavisd-release: if its only command line argument is '-',
then read arguments from stdin, one release request per line, ignoring empty
lines; input lines have the same format as command line arguments, i.e.:
mail_file
mail_file secret_id
mail_file secret_id alt_recip1 alt_recip2 ...
- better handle cases where a persistent temporary file email.txt
as prepared by the SMTP server module gets replaced as a result
of some user program modification (e.g. when invoking altermime);
problems reported by Dinesh Shah and Leonardo Rodrigues;
To generate a diff of this commit:
cvs rdiff -r1.13 -r1.14 pkgsrc/security/amavisd-new/Makefile
cvs rdiff -r1.5 -r1.6 pkgsrc/security/amavisd-new/distinfo
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.