PR/31127 CVS commit: pkgsrc/security/amavisd-new

[Marc Recht <recht%netbsd.org@localhost>]

The following reply was made to PR pkg/31127; it has been noted by GNATS.

From: Marc Recht <recht%netbsd.org@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: PR/31127 CVS commit: pkgsrc/security/amavisd-new
Date: Sun,  4 Sep 2005 20:23:14 +0000 (UTC)

 Module Name:   pkgsrc
 Committed By:  recht
 Date:          Sun Sep  4 20:23:14 UTC 2005
 
 Modified Files:
        pkgsrc/security/amavisd-new: Makefile distinfo
 
 Log Message:
 update to amavisd-new 2.3.3
 patch provided by eggert at macvaerk dot dtu dot dk
 in PR 31127
 
 changes:
 
 Version 2.3.3 is a maintenance release over 2.3.2. Besides fixing known
 problems and providing some optimizations, no new features were added.
 If using SpamAssassin older than 3.1, an upgrade of either SA to 3.1,
 or an upgrade of amavisd-new to 2.3.3 is recommended.
 
 - privacy: add a safety fuse / workaround around calls to SA to detect
   SA's failure (in SA versions before 3.1) to catch a failed exec() in a
   forked process, which could produce runaway process clones. See SA bug
   report #4370. An incident of a mail copy being delivered to unrelated
   recipient reported by Joel Nimety;
 
 - privacy: turn warning into a fatal error when a quarantine ID of a message
   requested for a quarantine release does not match the requested mail_id;
 
 - security: require minimal version 1.35 of Compress::Zlib to avoid
   vulnerability in the zlib compression library;
 
 - the dsn_cutoff_level should have been ignored if undefined according to
   documentation, but was not, causing DSN to be suppressed regardless of
   spam level; discovered by Gary V;
 
 - ensure the banned check is not performed if all recipients agree
   it is not needed, even in presence of $banned_namepath_re;
   undesired behaviour (not strictly incorrect) reported by Joel Nimety;
 
 - missing import of lookup_ip_acl in module Amavis::In::AMCL caused
   failure in sendmail milter setup when using the new AM.PDP protocol;
   reported by Mic And;
 
 - document and explicitly define handling of syntactically invalid IP address
   in lookup_ip_acl: it matches a zero-length-mask net, a constant lookup table,
   or a hash entry with an undef key, but no other entries in IP lookup tables;
   syntactically invalid IP addresses are now logged;
 
 - fix parsing if IPv6 address in $notify_method and $forward_method in case
   of dynamic destination override (the use of '*' in method fields);
 
 - check during startup that $myhostname is a fully qualified domain name
   (or 'localhost', if you must), and abort if it isn't, otherwise a non-FQDN
   can end up in places where RFC 2822 does not allow it; if uname(3) does not
   provide a FQDN, then an assignment to $myhostname must be done explicitly
   in amavisd.conf;
 
 - when quarantining to a single file in mbox format the 'From ...' line
   needs an English date, regardless of current locale; fixed by globally
   setting locale LC_TIME to "C";
 
 - pass on the parameter BODY=8BITMIME on MAIL FROM when submitting to MTA
   when original message reception indicated it is needed (RFC 1652).
   Note that mail forwarding may now fail if the feeding MTA requests
   BODY=8BITMIME SMTP service extension (or just passes data with msb set),
   but the MTA on the output side does not allow the use of the BODY parameter
   in SMTP. In case of Postfix this may only happen when receiving service
   on port 10025 is misconfigured and does not announce ESMTP capability
   and support for the SMTP service extension 8BITMIME;
 
 - RFC 2554 requires auth_param to be xtext-encoded addr-spec (no angle
   brackets) or "<>", not the xtext-encoded addr-spec enclosed in angle
   brackets (when specifying submitter during authentication); fixed;
 
 - apply some sanity limit on collected bad-header samples to ensure that
   a grossly broken mail does not unnecessarily fill up memory;
 
 - when sending recipient warnings for viruses, banned files, or bad headers,
   recipient address must not be rfc2822-quoted twice; fixed;
 
 - fix interpretation of $defang_all to really imply all; previously it only
   affected clean messages;
 
 - in quarantined mail the reported spam score in X-Spam-Status header field
   now includes maximum of all by-recipient score boosts (less surprising
   when soft-whitelisting through @score_sender_maps is in use); suggested
   by Mike Cappella and Gary V;
 
 - when a policy delegation protocol attribute "request" is not "AM.PDP"
   (perhaps it is a Postfix policy delegation request) don't attempt to find
   and open a mail file;
 
 - do_ascii and do_unarj: set environment variable TMPDIR or a command line
   temporary directory option to "$tempdir/parts" instead of $TEMPBASE
   to minimize possible pollution of top level directory;
 
 - don't abort even if amavisd.conf returns undef as a final value,
   as long as there are no errors reading or interpreting it;
 
 - if during 'amavisd stop' or 'amavisd reload' the old running daemon does
   not go away for one minute after sending it a SIGTERM, use a bigger
   hammer and send it a SIGKILL; suggested by Sven Riedel;
 
 - extend LDAP lookups to allow multiple search attributes (multiple
   occurrences of %m in a query); a patch by Michael Hall (and a similar
   one by Matthias Bandemer);
 
 - LDAP lookup on an empty envelope address (e.g. a null return path)
   adds another lookup key "<>", as it is difficult if not impossible
   to have LDAP attributes with empty string as a value; by Michael Hall;
 
 - LDAP.schema: drop "MUST ( mail )" from objectclass 'amavisAccount';
   suggested by Michael Hall;
 
 - updated comments and documentation, most notably the README.chroot;
 
 - contributed file Macintosh.tar.gz updated by Dale Walsh;
 
 COMPATIBILITY
 
 - replaced 'hits=' with 'score=' in inserted X-Spam-Status header field
   (and in some internal log entries) for compatibility with a changed
   default in SpamAssassin 3.1;
 
 - insert X-Spam-Score header field for compatibility with SA (previously
   insertion of this header field was commented-out because the information
   is redundant, as the score already appears in X-Spam-Status);
 
 OPTIMIZATION
 
 - speed up sending a mail header or full defanged (rewritten) mail over SMTP
   back to MTA by a factor of 4 by buffering header fields into large chunks
   to avoid bottleneck in Net::Cmd::datasend, which has lots of overhead for
   line-by-line writes. Previously slow writes mostly affected mail messages
   with extreme header lengths (such as results of a broken mail loop), or
   when delivering defanged messages, particularly at sites with large MTA
   mail size limits, sometimes to a point of exceeding timeout limits;
   reported by Dominik Weber and Ralf Hildebrandt;
 
 - move subroutine lookup_ip_acl() and associated ip_to_vec() into its own
   dedicated new package Amavis::Lookup::IP; provide a constructor to pre-parse
   IP lookup tables to speed up IP lookups in lookup_ip_acl; prepare pre-parsed
   commonly used IP lookup tables (@mynetworks_maps, @publicnetworks_maps,
   @inet_acl);
 
 - optimized reading loop in SMTP DATA state, receiving data is now about
   35% faster when mail size limit is not enforced (which is a default);
   no speedup when mail size limit _is_ enforced;
 
 - cache results of evaluated macros during a single call to expand(),
   as macro calls often come in pairs, like:  [?%e||\[%e\] ]
   or [? %#T ||, Tests: [%T|,]];  together with the above optimization in
   pre-parsed IP lookups it shaves off 25% of time in preparing main log entry;
 
 - set locale LC_TIME to "C" globally, avoid changing and restoring locale
   for every log write and when generating RFC2822 timestamps;
 
 - added an optimization note in README.sql about indexes and about
   SELECT count(*) in MySQL with InnoDB; investigation by Paolo Cravero;
 
 ---------------------------------------------------------------------------
                                                               June 29, 2005
 amavisd-new-2.3.2 release notes
 
 INCOMPATIBILITY with 2.3.1 and earlier versions:
 
 If running amavisd daemon in chroot please note:
 
   Each child process now opens its own syslog connection or a file descriptor
   to a log file, and no longer inherits a connection from its parent.
   When running in chroot jail and logging to syslog, the syslog client
   routines need syslogd socket to be present in the chroot subtree to be
   able to establish a connection with syslogd, otherwise logging output
   may be lost. Additional syslogd sockets (to be made available in the
   jail) may be requested from the syslogd daemon, see its documentation.
   This requirement is equivalent to the requirement of chrooted Postfix
   services (see Postfix documentation file BASIC_CONFIGURATION_README).
 
 BUG FIXES since 2.3.1:
 
 - do not enforce $MAXFILES limit during top-level MIME decoding to avoid
   tempfailing mail;  MIME parts are still counted, so a limit exceeded may
   still be reported during subsequent decoding, but this is handled more
   gracefully and does not cause preserved temporary directories to be left
   behind; reported by Marcin Lemanski; suggested by Stephane Lentz and
   Robert LeBlanc (noted in the 2.0 release notes);
 
 - use recv() instead of read() to get results from daemonized virus scanners
   in an attempt to avoid a bogus Perl I/O status on some Linux installations
   (reported by Sander Steffann); we now get a meaningful status codes like
   ECONNRESET instead of a bogus EBADF (Bad file descriptor);
 
 - ignore status ECONNRESET when reading results of a daemonized virus scanner
   from a socket, specific to some Linux versions; thanks to Sander Steffann
   for the initial report and extensive help in debugging the Perl problem;
 
 - run_av and other similar code sections: replace line-by-line reads by
   block-by-block reads wherever possible to avoid inappropriate status report
   EBADF (Bad file descriptor) caused by Perl I/O bug when last line is not
   terminated by a newline. The problem was affecting reading response from
   some command line virus checkers; reported by Sander Steffann;
 
 - ignore status EAGAIN when reading results on a pipe from a forked process;
   the status EAGAIN seems to be an artifact of Perl I/O on some installations;
   reported by several people to cause problems on FreeBSD with Perl 5.8.7
   (but Perl 5.8.6 is fine); thanks to Bart Matterne for testing and feedback;
 
 - allow one level of indirection when collecting %needed_protocols;
   global setting $protocol='COURIER' did not work, a workaround was needed
   with previous version, e.g.: $policy_bank{'QMQPqq'}={protocol=>'QMQPqq'};
   reported by Nicklas Bondesson and Martin Orr;
 
 - fix a bug (introduced with 2.3.0) in Courier and QMQPqq setups, where global
   information about processed message wasn't always reset and could leak
   into processing of a subsequent message; reported by Nicklas Bondesson;
 
 - SQL: fix arguments in calls to last_insert_id(), failing under PostgreSQL
   (MySQL didn't mind); pointed out by Henrik Krohns;
 
 - if module SAVI is loaded, insist it is version 0.30 or later;
   incompatibility with earlier versions reported by Andrzej Kukula;
 
 - make use of the new Net::Server 0.88 hook run_n_children_hook() to
   reload SAVI database; removes a need to apply SAVI patch to Net::Server;
   the Net::Server hook was suggested by Paul B. Henson and others,
   and incorporated into Net::Server 0.88 by Paul Seamons;
 
 - reopen log file or syslog connection in each child process to make it use
   its own file descriptor; also minimizes transients when syslogd is restarted
   and its socket re-created, as reported by Les Ault. When running in chroot
   please make sure a syslogd socket is also available in the chroot jail,
   see README.chroot for syslogd options (and BASIC_CONFIGURATION_README
   in Postfix documentation for the Postfix equivalent);
 
 - close log file or syslog in forked process before exec, just to play nicely;
 
 - do_lha: fix extracting archive member filename in case of broken archive
   or empty name (avoid interpreting creation date as a file name);
   do not increment OpsDecByLha counter for empty archives, which are
   most likely not lha archives at all;
 
 - obey $final_bad_header_destiny D_DISCARD or D_REJECT even for messages
   with bad headers from mailing lists or with a null envelope sender (DSN);
   previously such messages were passed; undesired behaviour reported
   by Cami Sardinha.
 
   Such messages are still let through with $final_bad_header_destiny set to
   D_BOUNCE, as otherwise they will be lost because a bounce is suppressed
   for null sender messages and for mail from mailing list. This behaviour
   is retained for backwards compatibility, but may need to be reconsidered.
 
 - fix regexp for extracting am_id from amavis-milter helper program requests;
 
 - if fork/exec fails, try to commit suicide in forked process with
   POSIX::_exit(1) first, before trying kill('KILL',$$) as a last resort;
 
 - updated $log_templ example in amavisd.conf-sample to match the default;
   pointed out by Gary V;
 
 - further reduce a couple of more frequent Perl warnings about the use of
   uninitialized values in expressions;
 
 - pre-load additional Perl modules required by SA 3.1 plugins;
 
 - require minimal versions of modules: Time::HiRes 1.49, Archive::Zip 1.14;
 
 - replaced nonexistent variable @sa_spam_modifies_subj_maps by
   @spam_modifies_subj_maps in commented-out example in amavisd.conf-sample;
   noticed by Joachim Schoenberg;
 
 LDAP CHANGES by Michael Hall:
 
 All the LDAP changes are transparent to the user.
 
 - rewritten some of the code similar to the restructuring of the SQL code
   in version amavisd-new-2.3.0. A new package Amavisd::LDAP::Connection was
   added which is a LDAP connection object, and the old connection-related code
   in Amavis::Lookup::LDAP has been moved to the new package. Amavisd-new will
   now try to reconnect (once) while processing a message, similar to SQL;
 
 - added the ability to specify a '%d' (domain) token in the LDAP base DN;
   based on idea from Alexander Wittig;
 
 - updated default LDAP port based on whether SSL/TLS is being used or not;
   based on idea from Timo Veith;
 
 - updated the search code to query for multiple records and return the results
   sorted in 'make_query_keys' order versus doing a query for each key.
   As a result performance is enhanced, and the tweaks 'ldap_get_all', and
   'use_query_keys' (recently added) are no longer applicable or needed
   and have been removed;
 
 - improved LDAP error reporting and misc changes to multivalued attributes;
 
 - documentation changes (amavisd.conf-default, README.lookups);
 
 MINOR IMPROVEMENTS:
 
 - macro %c (commonly used in a log template) reports spam score no longer
   as a single number, but as an explicit sum of a SA score and a by-sender
   boost score (from @score_sender_maps) when boost score is nonzero;
   suggested by Ed Walker;
 
 - enhancement to amavisd-release: if its only command line argument is '-',
   then read arguments from stdin, one release request per line, ignoring empty
   lines; input lines have the same format as command line arguments, i.e.:
      mail_file
      mail_file secret_id
      mail_file secret_id alt_recip1 alt_recip2 ...
 
 - better handle cases where a persistent temporary file email.txt
   as prepared by the SMTP server module gets replaced as a result
   of some user program modification (e.g. when invoking altermime);
   problems reported by Dinesh Shah and Leonardo Rodrigues;
 
 
 To generate a diff of this commit:
 cvs rdiff -r1.13 -r1.14 pkgsrc/security/amavisd-new/Makefile
 cvs rdiff -r1.5 -r1.6 pkgsrc/security/amavisd-new/distinfo
 
 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.