ANNOUNCE: amavisd-new-2.10.0-rc1 release candidate is available

[Mark Martinec]

A release candidade RC1 of the coming version 2.10.0 of amavisd-new
is available at:

   http://www.ijs.si/software/amavisd/amavisd-new-2.10.0-rc1.tar.xz

Release notes are at:

   http://www.ijs.si/software/amavisd/release-notes.txt


amavisd-new-2.10.0 release notes

Contents:
   COMPATIBILITY
   BUG FIXES
   NEW FEATURES


COMPATIBILITY

- new requirement: perl module Net::LibIDN needs to be installed;

- uses a perl module File::LibMagic if installed, instead of spawning
   a file(1) utility;

- Support for international email relies heavily on perl to do the
   right thing in its support of Unicode, so using a reasonably recent
   version of perl is recommended. Amavisd was tested with perl 5.18
   and 5.20.1. Versions of perl older than 5.12 may cause problems
   with handling, encoding, and decoding of Unicode characters.
   It is reasonable to expect that versions 5.14 and 5.16 are fine too,
   but have not been tested.

- Default log templates and notification templates have changed
   in details (like in decoding of international e-mail addresses), so
   if locally customized templates are in use these will benefit from
   updating - otherwise expect some mojibake in log and notifications.

- Logging via syslog expects that syslogd (or equivalent) will not
   clobber UTF-8 characters. It may be necessary to tell syslogd to
   accept C1 control characters unchanged, e.g. by adding a command line
   option "-8" to syslogd. Failing to do so may leave logged entries
   (like sender and recipient address, From, Subject) with UTF-8
   characters in international mail garbled or poorly readable in syslog.
   On FreeBSD one should add:  syslogd_flags="-8"  to rc.conf.

- International domain names (IDN) encoded in ASCII-compatible encoding
   found in e-mail addresses and in Message-ID header field will be 
decoded
   to Unicode for presentation purposes (syslog, JSON structured log,
   notifications). This decoding does not affect a mail message itself.


BUG FIXES

- releasing a message from an SQL quarantine was broken in version 2.9.1
   due to introduction of parent_mail_id(); patches provided by Stef 
Simoens
   and Gionatan Danti;

- if checking of a message was aborted prematurely (like due to a 
timeout
   or some fatal error), JSON log could receive a copy of a previous
   log entry;

- prevent non-ASCII non-UTF-8 bytes from reaching a JSON log/report
   (which can lead to Elasticsearch complaining);

- allow SMTP commands MAIL FROM and RCPT TO to accept options without
   values, as allowed by the RFC 5321 syntax;

- in delivery status notification (DSN) the field Received-From-MTA
   specified 'smtp' as mta-name-type, instead of a 'dns' as prescribed
   in RFC 3464;

- avoid failure of os_fingerprint in certain cases where the
   $os_fingerprint_method uses an asterisk in place of a host IP address
   or port number; the reported error was:
     os_fingerprint FAILED: Insecure dependency in socket
       while running with -T switch
       at /usr/lib/perl/5.18/IO/Socket.pm line 80
   reported by -ben;


NEW FEATURES

- added support for Internationalized Email:
   * RFC 6530 - Overview and Framework for Internationalized Email
   * RFC 6531 - SMTP Extension for Internationalized Email (SMTPUTF8)
   * RFC 6532 - Internationalized Email Headers
   * RFC 6533 - Internationalized Delivery Status Notifications

   This supports UTF-8 (EAI) in SMTP/LMTP sender addresses, recipient
   addresses, and message header section. Feature parity with Postfix
   version 2.12 (support introduced in development snapshot 20140715).

   The SMTPUTF8 extension is supported by Gmail since 2014-08-05:
     
http://googleblog.blogspot.com/2014/08/a-first-step-toward-more-global-email.html


- added support for Internationalized Domain Names (IDN) according
   to IDNA (RFC 5890, RFC 5891; RFC 3490);

   * A-labels in ASCII-compatible encoding of domain names are converted
     to U-labels for presentation/logging purposed;

   * U-labels are converted to A-labels when feeding a mail message
     to an MTA which does not announce support for SMTPUTF8 extension
     (instead of rejecting them as invalid mail address);

   * for lookup purposes an international domain name is converted to
     ASCII-compatible encoding when used as a query key in DNS lookups
     and in lookups into hash, list, SQL and LDAP lookup tables (but not
     in regexp table lookups). These tables are expected to contain 
domain
     names in their ASCII representation (ACE). For convenience of config
     files a subroutine idn_to_ascii() is available, which encodes a
     Unicode domain name to ACE (like ToASCII in RFC 3490);

   * configuration settings $localhost_name and notification sender and
     recipient mail addresses may have their domains in Unicode, these
     will be converted to ACE automatically where necessary;


- use a perl module File::LibMagic when available, instead of spawning
   a file(1) utility for classifying contents of mail parts.
   By using a direct interface to libmagic library the startup cost
   of spawning an external process is avoided. Benchmarking shows that
   using libmagic is significantly faster especially for checking a small
   number of files - takes 4 ms for checking one file with libmagic
   vs. 27 ms with a spawned file(1); based on a patch by Markus Benning;

- new macros: mime_decode, header_field_octets, mail_addr_decode,
   and mail_addr_decode_octets;

- updated do_cabextract (extraction of Microsoft cabinet .cab archives)
   to recognize a slightly changed output of cabextract version 1.2;
   patch by Thomas Jarosch;


OTHER

- RFC 6533: recognize a MIME type 'message/global' as similar
   to 'message/rfc822', and 'message/global-headers' as similar
   to 'text/rfc822-headers' where appropriate (e.g. in bounce killer);

- header validity check now distinguishes 'non-ASCII and invalid UTF-8'
   from 'non-ASCII but valid UTF-8' characters in a mail header section.
   By default valid UTF-8 strings in a mail header section are not 
treated
   as error even if mail is not flagged as international mail (SMTPUTF8),
   as these are quite common in practice. To treat non- MIME-encoded 
UTF-8
   in a header section as error the test can be enabled by:
     $allowed_header_tests{'utf8'} = 1;

- prefer sanitizing/protecting control characters as hex code (like 
\x7F)
   instead of octal (like \177) (e.g. in logging and DSN);




Mark