Module Name:	pkgsrc
Committed By:	recht
Date:		Sun Sep  4 20:23:14 UTC 2005

Modified Files:
	pkgsrc/security/amavisd-new: Makefile distinfo

Log Message:
update to amavisd-new 2.3.3
patch provided by eggert at macvaerk dot dtu dot dk
in PR 31127

changes:

Version 2.3.3 is a maintenance release over 2.3.2. Besides fixing known
problems and providing some optimizations, no new features were added.
If using SpamAssassin older than 3.1, an upgrade of either SA to 3.1,
or an upgrade of amavisd-new to 2.3.3 is recommended.

- privacy: add a safety fuse / workaround around calls to SA to detect
  SA's failure (in SA versions before 3.1) to catch a failed exec() in a
  forked process, which could produce runaway process clones. See SA bug
  report #4370. An incident of a mail copy being delivered to unrelated
  recipient reported by Joel Nimety;

- privacy: turn warning into a fatal error when a quarantine ID of a message
  requested for a quarantine release does not match the requested mail_id;

- security: require minimal version 1.35 of Compress::Zlib to avoid
  vulnerability in the zlib compression library;

- the dsn_cutoff_level should have been ignored if undefined according to
  documentation, but was not, causing DSN to be suppressed regardless of
  spam level; discovered by Gary V;

- ensure the banned check is not performed if all recipients agree
  it is not needed, even in presence of $banned_namepath_re;
  undesired behaviour (not strictly incorrect) reported by Joel Nimety;

- missing import of lookup_ip_acl in module Amavis::In::AMCL caused
  failure in sendmail milter setup when using the new AM.PDP protocol;
  reported by Mic And;

- document and explicitly define handling of syntactically invalid IP address
  in lookup_ip_acl: it matches a zero-length-mask net, a constant lookup table,
  or a hash entry with an undef key, but no other entries in IP lookup tables;
  syntactically invalid IP addresses are now logged;

- fix parsing if IPv6 address in $notify_method and $forward_method in case
  of dynamic destination override (the use of '*' in method fields);

- check during startup that $myhostname is a fully qualified domain name
  (or 'localhost', if you must), and abort if it isn't, otherwise a non-FQDN
  can end up in places where RFC 2822 does not allow it; if uname(3) does not
  provide a FQDN, then an assignment to $myhostname must be done explicitly
  in amavisd.conf;

- when quarantining to a single file in mbox format the 'From ...' line
  needs an English date, regardless of current locale; fixed by globally
  setting locale LC_TIME to "C";

- pass on the parameter BODY=8BITMIME on MAIL FROM when submitting to MTA
  when original message reception indicated it is needed (RFC 1652).
  Note that mail forwarding may now fail if the feeding MTA requests
  BODY=8BITMIME SMTP service extension (or just passes data with msb set),
  but the MTA on the output side does not allow the use of the BODY parameter
  in SMTP. In case of Postfix this may only happen when receiving service
  on port 10025 is misconfigured and does not announce ESMTP capability
  and support for the SMTP service extension 8BITMIME;

- RFC 2554 requires auth_param to be xtext-encoded addr-spec (no angle
  brackets) or "<>", not the xtext-encoded addr-spec enclosed in angle
  brackets (when specifying submitter during authentication); fixed;

- apply some sanity limit on collected bad-header samples to ensure that
  a grossly broken mail does not unnecessarily fill up memory;

- when sending recipient warnings for viruses, banned files, or bad headers,
  recipient address must not be rfc2822-quoted twice; fixed;

- fix interpretation of $defang_all to really imply all; previously it only
  affected clean messages;

- in quarantined mail the reported spam score in X-Spam-Status header field
  now includes maximum of all by-recipient score boosts (less surprising
  when soft-whitelisting through @score_sender_maps is in use); suggested
  by Mike Cappella and Gary V;

- when a policy delegation protocol attribute "request" is not "AM.PDP"
  (perhaps it is a Postfix policy delegation request) don't attempt to find
  and open a mail file;

- do_ascii and do_unarj: set environment variable TMPDIR or a command line
  temporary directory option to "$tempdir/parts" instead of $TEMPBASE
  to minimize possible pollution of top level directory;

- don't abort even if amavisd.conf returns undef as a final value,
  as long as there are no errors reading or interpreting it;

- if during 'amavisd stop' or 'amavisd reload' the old running daemon does
  not go away for one minute after sending it a SIGTERM, use a bigger
  hammer and send it a SIGKILL; suggested by Sven Riedel;

- extend LDAP lookups to allow multiple search attributes (multiple
  occurrences of %m in a query); a patch by Michael Hall (and a similar
  one by Matthias Bandemer);

- LDAP lookup on an empty envelope address (e.g. a null return path)
  adds another lookup key "<>", as it is difficult if not impossible
  to have LDAP attributes with empty string as a value; by Michael Hall;

- LDAP.schema: drop "MUST ( mail )" from objectclass 'amavisAccount';
  suggested by Michael Hall;

- updated comments and documentation, most notably the README.chroot;

- contributed file Macintosh.tar.gz updated by Dale Walsh;

COMPATIBILITY

- replaced 'hits=' with 'score=' in inserted X-Spam-Status header field
  (and in some internal log entries) for compatibility with a changed
  default in SpamAssassin 3.1;

- insert X-Spam-Score header field for compatibility with SA (previously
  insertion of this header field was commented-out because the information
  is redundant, as the score already appears in X-Spam-Status);

OPTIMIZATION

- speed up sending a mail header or full defanged (rewritten) mail over SMTP
  back to MTA by a factor of 4 by buffering header fields into large chunks
  to avoid bottleneck in Net::Cmd::datasend, which has lots of overhead for
  line-by-line writes. Previously slow writes mostly affected mail messages
  with extreme header lengths (such as results of a broken mail loop), or
  when delivering defanged messages, particularly at sites with large MTA
  mail size limits, sometimes to a point of exceeding timeout limits;
  reported by Dominik Weber and Ralf Hildebrandt;

- move subroutine lookup_ip_acl() and associated ip_to_vec() into its own
  dedicated new package Amavis::Lookup::IP; provide a constructor to pre-parse
  IP lookup tables to speed up IP lookups in lookup_ip_acl; prepare pre-parsed
  commonly used IP lookup tables (@mynetworks_maps, @publicnetworks_maps,
  @inet_acl);

- optimized reading loop in SMTP DATA state, receiving data is now about
  35% faster when mail size limit is not enforced (which is a default);
  no speedup when mail size limit _is_ enforced;

- cache results of evaluated macros during a single call to expand(),
  as macro calls often come in pairs, like:  [?%e||\[%e\] ]
  or [? %#T ||, Tests: [%T|,]];  together with the above optimization in
  pre-parsed IP lookups it shaves off 25% of time in preparing main log entry;

- set locale LC_TIME to "C" globally, avoid changing and restoring locale
  for every log write and when generating RFC2822 timestamps;

- added an optimization note in README.sql about indexes and about
  SELECT count(*) in MySQL with InnoDB; investigation by Paolo Cravero;

---------------------------------------------------------------------------
                                                              June 29, 2005
amavisd-new-2.3.2 release notes

INCOMPATIBILITY with 2.3.1 and earlier versions:

If running amavisd daemon in chroot please note:

  Each child process now opens its own syslog connection or a file descriptor
  to a log file, and no longer inherits a connection from its parent.
  When running in chroot jail and logging to syslog, the syslog client
  routines need syslogd socket to be present in the chroot subtree to be
  able to establish a connection with syslogd, otherwise logging output
  may be lost. Additional syslogd sockets (to be made available in the
  jail) may be requested from the syslogd daemon, see its documentation.
  This requirement is equivalent to the requirement of chrooted Postfix
  services (see Postfix documentation file BASIC_CONFIGURATION_README).

BUG FIXES since 2.3.1:

- do not enforce $MAXFILES limit during top-level MIME decoding to avoid
  tempfailing mail;  MIME parts are still counted, so a limit exceeded may
  still be reported during subsequent decoding, but this is handled more
  gracefully and does not cause preserved temporary directories to be left
  behind; reported by Marcin Lemanski; suggested by Stephane Lentz and
  Robert LeBlanc (noted in the 2.0 release notes);

- use recv() instead of read() to get results from daemonized virus scanners
  in an attempt to avoid a bogus Perl I/O status on some Linux installations
  (reported by Sander Steffann); we now get a meaningful status codes like
  ECONNRESET instead of a bogus EBADF (Bad file descriptor);

- ignore status ECONNRESET when reading results of a daemonized virus scanner
  from a socket, specific to some Linux versions; thanks to Sander Steffann
  for the initial report and extensive help in debugging the Perl problem;

- run_av and other similar code sections: replace line-by-line reads by
  block-by-block reads wherever possible to avoid inappropriate status report
  EBADF (Bad file descriptor) caused by Perl I/O bug when last line is not
  terminated by a newline. The problem was affecting reading response from
  some command line virus checkers; reported by Sander Steffann;

- ignore status EAGAIN when reading results on a pipe from a forked process;
  the status EAGAIN seems to be an artifact of Perl I/O on some installations;
  reported by several people to cause problems on FreeBSD with Perl 5.8.7
  (but Perl 5.8.6 is fine); thanks to Bart Matterne for testing and feedback;

- allow one level of indirection when collecting %needed_protocols;
  global setting $protocol='COURIER' did not work, a workaround was needed
  with previous version, e.g.: $policy_bank{'QMQPqq'}={protocol=>'QMQPqq'};
  reported by Nicklas Bondesson and Martin Orr;

- fix a bug (introduced with 2.3.0) in Courier and QMQPqq setups, where global
  information about processed message wasn't always reset and could leak
  into processing of a subsequent message; reported by Nicklas Bondesson;

- SQL: fix arguments in calls to last_insert_id(), failing under PostgreSQL
  (MySQL didn't mind); pointed out by Henrik Krohns;

- if module SAVI is loaded, insist it is version 0.30 or later;
  incompatibility with earlier versions reported by Andrzej Kukula;

- make use of the new Net::Server 0.88 hook run_n_children_hook() to
  reload SAVI database; removes a need to apply SAVI patch to Net::Server;
  the Net::Server hook was suggested by Paul B. Henson and others,
  and incorporated into Net::Server 0.88 by Paul Seamons;

- reopen log file or syslog connection in each child process to make it use
  its own file descriptor; also minimizes transients when syslogd is restarted
  and its socket re-created, as reported by Les Ault. When running in chroot
  please make sure a syslogd socket is also available in the chroot jail,
  see README.chroot for syslogd options (and BASIC_CONFIGURATION_README
  in Postfix documentation for the Postfix equivalent);

- close log file or syslog in forked process before exec, just to play nicely;

- do_lha: fix extracting archive member filename in case of broken archive
  or empty name (avoid interpreting creation date as a file name);
  do not increment OpsDecByLha counter for empty archives, which are
  most likely not lha archives at all;

- obey $final_bad_header_destiny D_DISCARD or D_REJECT even for messages
  with bad headers from mailing lists or with a null envelope sender (DSN);
  previously such messages were passed; undesired behaviour reported
  by Cami Sardinha.

  Such messages are still let through with $final_bad_header_destiny set to
  D_BOUNCE, as otherwise they will be lost because a bounce is suppressed
  for null sender messages and for mail from mailing list. This behaviour
  is retained for backwards compatibility, but may need to be reconsidered.

- fix regexp for extracting am_id from amavis-milter helper program requests;

- if fork/exec fails, try to commit suicide in forked process with
  POSIX::_exit(1) first, before trying kill('KILL',$$) as a last resort;

- updated $log_templ example in amavisd.conf-sample to match the default;
  pointed out by Gary V;

- further reduce a couple of more frequent Perl warnings about the use of
  uninitialized values in expressions;

- pre-load additional Perl modules required by SA 3.1 plugins;

- require minimal versions of modules: Time::HiRes 1.49, Archive::Zip 1.14;

- replaced nonexistent variable @sa_spam_modifies_subj_maps by
  @spam_modifies_subj_maps in commented-out example in amavisd.conf-sample;
  noticed by Joachim Schoenberg;

LDAP CHANGES by Michael Hall:

All the LDAP changes are transparent to the user.

- rewritten some of the code similar to the restructuring of the SQL code
  in version amavisd-new-2.3.0. A new package Amavisd::LDAP::Connection was
  added which is a LDAP connection object, and the old connection-related code
  in Amavis::Lookup::LDAP has been moved to the new package. Amavisd-new will
  now try to reconnect (once) while processing a message, similar to SQL;

- added the ability to specify a '%d' (domain) token in the LDAP base DN;
  based on idea from Alexander Wittig;

- updated default LDAP port based on whether SSL/TLS is being used or not;
  based on idea from Timo Veith;

- updated the search code to query for multiple records and return the results
  sorted in 'make_query_keys' order versus doing a query for each key.
  As a result performance is enhanced, and the tweaks 'ldap_get_all', and
  'use_query_keys' (recently added) are no longer applicable or needed
  and have been removed;

- improved LDAP error reporting and misc changes to multivalued attributes;

- documentation changes (amavisd.conf-default, README.lookups);

MINOR IMPROVEMENTS:

- macro %c (commonly used in a log template) reports spam score no longer
  as a single number, but as an explicit sum of a SA score and a by-sender
  boost score (from @score_sender_maps) when boost score is nonzero;
  suggested by Ed Walker;

- enhancement to amavisd-release: if its only command line argument is '-',
  then read arguments from stdin, one release request per line, ignoring empty
  lines; input lines have the same format as command line arguments, i.e.:
     mail_file
     mail_file secret_id
     mail_file secret_id alt_recip1 alt_recip2 ...

- better handle cases where a persistent temporary file email.txt
  as prepared by the SMTP server module gets replaced as a result
  of some user program modification (e.g. when invoking altermime);
  problems reported by Dinesh Shah and Leonardo Rodrigues;


To generate a diff of this commit:
cvs rdiff -r1.13 -r1.14 pkgsrc/security/amavisd-new/Makefile
cvs rdiff -r1.5 -r1.6 pkgsrc/security/amavisd-new/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.