ANNOUNCE: amavisd-new-2.8.1 has been released

Mark Martinec Mark.Martinec+amavis at
Fri Jun 28 22:06:30 CEST 2013

Version 2.8.1 of amavisd-new has been released.

It is available at:

amavisd-new-2.8.1 release notes


- when 0MQ (a.k.a. ZeroMQ) is used between Amavis components as an
  internal messaging protocol, make sure to replace all 0MQ-enabled
  Amavis components on upgrading amavisd, as the internal protocol
  has changed slightly, taking advantage of 0MQ multi-part messages
  for better performance. Affected programs are: amavis-services,
  amavisd-status, amavisd-snmp-subagent-zmq, and amavisd.

  NOTE: The Crossroads I/O project (libxs) ceased development on
    July 2012, to be replaced by nanomsg eventually by the same author.

    The 0MQ library (libzmq) is currently (2013) the best choice,
    the preferred library version is 3.2.2 or later along with
    the ZMQ::LibZMQ3 Perl interface module and ZMQ::Constants.
    The older version 2 of the library, along with an older perl
    module ZeroMQ, should be fine too, but lacks support for IPv6.

- amavisd is compatible with perl 5.18.0 and with SpamAssassin 3.4.0


- fixed a bug in the SMTP client code, where the final SMTP status did
  not reflect a failure status of a DATA command from a back-end MTA.
  This caused a reception of a mail message to be confirmed but a message
  was then lost, as it could not be passed to a back-end MTA. The bug
  went unnoticed for years, as the commonly used MTAs normally reject
  either at the MAIL FROM, at RCPT TO, or at the data-dot stage, but not
  at the DATA command. Reported by Deniska-rediska;

- fixed calling an external spam scanner DSPAM or Bogofilter, which
  failed with a message:
    auto-learning with spam scanner ... failed: error running program
  Reported by Tonio;

- if a configuration file path as given through a command line option -c
  or as an argument to include_config_files() was not an absolute path,
  and that file contained an error, the do() would search the @INC list
  for alternative files of the same name, and reported an unrelated error
  (typically: No such file or directory) instead of reporting the true
  reason for a failure;

- fixed a regular expression in amavisd.conf for an 'Avast!' AV entry
  to properly extract a virus name; a patch by Ralf Hildebrandt;

- added LDAP errors LOCAL_ERROR and OPERATIONS_ERROR to the set of
  expected error conditions which lets amavisd retry the failed
  operation; a patch by Quanah Gibson-Mount;


- new Redis storage for the "pen pals" feature;

- improved IPv6 support;

- support for p0f v3;

- new macros ip_trace_all and ip_trace_public;

- amavisd-status now shows a bar graph display
  of the number of active processes;

- the timing report log entry can show CPU usage
  at log level 2 if a module Unix::Getrusage is available;


- new Redis storage for the "pen pals" feature: instead of (or in addition
  to) the existing SQL storage for keeping data on past mail messages
  and contributing negative spam score to recent/ongoing correspondence,
  this data can now be kept on a Redis server. Unlike the SQL backend,
  to minimize memory usage the Redis backend keeps only data which are
  required for pen pals operation.

  Redis storage for pen pals can offer a small speedup compared to a
  well-tuned SQL server, offers automatic expiration of data based on a
  configured time-to-live setting, and a simpler setup (no need to manually
  set up an SQL schema). A drawback is that a Redis server keeps all data
  in memory (with optional periodic persistence on disk), which might be
  of concern for busy sites with a long time-to-live setting. Potential
  drawbacks of a Redis server are also its lack of sophisticated access
  controls, and lack of IPv6 support in a current version.

  A redis database may be shared between hosts running amavisd. It can
  be accessed either locally over a Unix socket, or using an INET socket
  (IPv4 only) over a loopback interface (better security) or over a local
  network. Currently (version 2.6.14) a Redis server does not offer
  access over IPv6, which is planned (but not promised) for version 2.8.

  Required dependencies when Redis support is enabled are a perl module
  "Redis" ( ) version 1.954 or later,
  and a redis server ( ) with support for Lua scripting
  (i.e. version 2.6 or later). Most pen pals application-level details on
  queries and storage management is delegated to Lua scripts running on a
  Redis server.

  Expiration time of items stored in a redis database is controlled by
  a setting $storage_redis_ttl, which is a time-to-live time in seconds
  and defaults to 16 days:
    $storage_redis_ttl = 16*24*60*60;

  Redis support in amavisd is enabled by setting a list @storage_redis_dsn
  to a nonempty value (similar to @storage_sql_dsn for an SQL support).
  If @storage_redis_dsn is empty, the redis support code is not loaded
  and does not occupy any storage.

  The configuration setting @storage_redis_dsn is a list of hashrefs
  (a hashref is a { ... } in perl syntax), each of which specifies
  one Redis server that can be used: if there is more than one entry
  in the list, a connection to each server is attempted until one is
  found where connection succeeds. Each entry is an associative array
  of key/value options which are passed on to a new() method of a perl
  Redis module unmodified and unverified. Usual options are: 'server',
  'sock', 'reconnect' - see documentation of a Redis module for details.
  All Redis module options have their default value, so it is alright
  to specify an empty hash, which means to connect to a default server.

  Apart from options which are passed to a Redis module, two additional
  options are interpreted by amavisd itself and are not passed on to a
  Redis perl module. The 'db_id' options is an optional database index,
  used in a SELECT redis command to choose a (sub)database to use.
  By default a database index is 0. The 'ttl' option overrides a global
  time-to-live setting as specified in $storage_redis_ttl, allowing to
  chose different expiration times of stored items for each server.


  # disables Redis (is a default)
  @storage_redis_dsn = ();

  # enables Redis, use a single default local redis server, all
  # defaults are supplied by a Redis perl module, database index 0
  @storage_redis_dsn = ( {} );

  # access a local redis server over a loopback interface on TCP port
  # 6379, select database index 1, try reconnecting for 20 seconds
  # before giving up when a redis server is down (or restarting)
  @storage_redis_dsn =
    ( { server => 'localhost:6379', reconnect => 20, db_id => 1 } );

  $storage_redis_ttl = 16*24*60*60;  # expiration time for data 16 days
  @storage_redis_dsn = (
    { sock => '/tmp/redis.sock',  reconnect => 20, db_id => 1 },
    { server => 'localhost:6379', reconnect => 20, db_id => 1 },
    { server => '', db_id => 1, ttl => 5*24*60*60 },

  Some existing settings also affect Redis pen pals operation:
    $database_sessions_persistent, $penpals_bonus_score,
    $penpals_halflife, $penpals_threshold_low, $penpals_threshold_high.

  Starting with version 3.4.0 the module SpamAssassin can also use Redis
  storage for its global Bayes database. Amavisd and SpamAssassin can use
  the same Redis server for their databases, although it is sensible that
  they use separate (sub)databases by choosing a different database index
  (redis SELECT command) through a 'db_id' configuration option, which
  defaults to 0.

  NOTE: As more experience with Redis is gained, it is possible that
  a redis storage schema may change in future versions, possibly in an
  incompatible way. As its purpose is short-term storage, this should
  not be of great concern.

- improved IPv6 support: can now communicate with amavisd
  processes over an INET6 socket (or over an INET or UNIX socket as before).
  Extended the protocol between amavisd and to allow
  queries on IPv6 addresses;

- rewritten to support a newer p0f v3 output format,
  while still recognizing an older p0f v2 output format;
  suggested by Jernej Porenta;

  NOTE: the p0f v3 does not provide a compact output on stdout like p0f v2
  could by using an option -l, so leave out the option -l with p0f v3;

- improved IPv6 support: program amavisd-snmp-subagent-zmq can now
  attach as an AgentX to a Net-SNMP daemon snmpd over an INET6 socket;
  the AgentX socket is specified by $agentx_sock_specs near the beginning
  of a file amavisd-snmp-subagent-zmq, e.g.:
    $agentx_sock_specs = 'tcp6:localhost:705';  # talk to snmpd over IPv6
    $agentx_sock_specs = 'tcp:localhost:705';   # talk to snmpd over IPv4
    $agentx_sock_specs = '/var/agentx/master';  # talk over a UNIX socket

- improved IPv6 support: program amavisd-submit can now submit a mail
  message to amavisd over an INET6 socket;

- a macro W can now produce a list of all virus scanners invoked,
  along with a list of virus names each scanner detected;
  suggested by Patrick Ben Koetter;

- new macros ip_trace_all and ip_trace_public:

  ip_trace_all  provides a list of IP addresses found in the 'Received from'
     trace of a mail header, one entry for each Received header field,
     including possibly invalid IP addresses and private IP addresses;
     Missing addresses are substituted by with a '?' (e.g. in Received
     header fields for local or other non-IP mail submissions).
     The list order corresponds to the order of 'Received' header fields
     as found in a mail header, top-down, i.e. first entry of the list
     is the topmost (the most recent) 'Received' header field, so
     chronologically in reverse;

  ip_trace_public  provides a list of valid public IP addresses found in the
     'Received from' trace of a mail header.  Missing, invalid or private
     IP addresses are not included in this list, so there may be more
     'Received' header fields in a mail header then entries in this list.
     The list order corresponds to the order of 'Received' header fields
     as found in a mail header, top-down, i.e. first entry of the list
     is the topmost (the most recent) 'Received' header field with a valid
     public IP address, so chronologically in reverse;

  suggested by Tomislav Mihaliček;

- templates for administrator notifications, recipient notifications,
  and sender notifications now use macro 'ip_trace_all' instead of macros
  'e' and 't' in order to report the full 'received' trace, not just the
  first hop;

- macro supplementary_info recognizes new arguments: VERSION, SUBVERSION,
  and RULESVERSION, providing additional information from SpamAssassin
  correspond to equivalent SpamAssassin tags;

- a new command line option -X allows controlling some exotic features,
  useful for example in debugging or automatic testing. The option takes
  one argument which is a comma-separated list of keywords. Currently
  the only recognized option is '-X no_conf_file_writable_check',
  which disables security checks on configuration files, which can be
  useful in automatic testing, but is dangerous to use in production.
  Suggested by Alexander Wirt;

- a configuration setting $sa_debug may now specify a comma-separated
  list of SpamAssassin debug facilities, complementing a similar method
  of specifying these facilities through a command line option -d.
  If $sa_debug looks like a simple boolean (or is undefined), the
  traditional semantics still applies: a false prepends an 'info'
  to the list, while a true prepends 'info' and 'all' to the list
  of SpamAssassin debug facilities.

   $sa_debug = 0;  # same as:  $sa_debug = 'info';
   $sa_debug = 1;  # same as:  $sa_debug = 'info,all';
   $sa_debug = 'info,dns,async,bayes';

- pass the size of an original mail body as a 'supplementary attribute'
  to SpamAssassin for the benefit of a 'check_body_length' eval rule
  (new with SpamAssassin 3.4.0). The original mail body size may
  differ from the message as seen by SpamAssassin in case of truncation
  of large messages to mail_body_size_limit.

- to the output of amavisd-status add a simple bar graph display (with
  an exponential-decay peak indicator) of the number of active processes;

- if a module Unix::Getrusage is available, the timing report log entry
  (at log level 2) is enhanced: in addition to total elapsed time
  (wall clock) spent in processing a message, it also shows a sum of
  CPU user and system times spent by amavisd process and its spawned

  old format example:
    size: 3815, TIMING [total 1901 ms] - ...

  new format example:
    size: 3815, TIMING [total 1901 ms, cpu 657 ms] - ...

  Additionally, a separate RUSAGE log entry is produced at log level 2,
  indicating resource usage spent by the last task. A field maxrss is
  a gauge (an absolute current value), all other fields are counters,
  so a difference between a previous and a current value is shown in
  the log. Each field value is a pair of numbers delimited by a plus:
  the first value corresponds to resource usage by the reporting amavisd
  child process, the second value corresponds to its spawned processes
  (e.g. file(1), gzip(1), etc.).

  Example (wrapped for clarity):

    size: 3815, RUSAGE minflt=10114+5223, majflt=0+0,
      nswap=0+0, inblock=0+0, oublock=9+0,
      msgsnd=819+9, msgrcv=211+3, nsignals=0+0,
      nvcsw=128+19, nivcsw=32+41, maxrss=164304+194012,
      ixrss=520+14016, idrss=66300+128392, isrss=24960+7680,
      utime=0.390+0.079, stime=0.079+0.108

  See getrusage(2) Unix man page for details.


- the 'amavisd genrsa' command will now warn if the requested DKIM
  signing key size is below 1024 bits, as required by RFC 6376;

- on amavisd startup a check on available private DKIM signing keys
  (as declared by dkim_key) will now warn if a key size is below
  1024 bits as required by RFC 6376, and log an information message
  if a key size is below a configured $dkim_minimum_key_bits size
  (defaults to 1024, currently 768 would still be a sensible value);

- for purposes of DKIM-based whitelisting (@author_to_policy_bank_maps)
  and @signer_reputation_maps spam scores, valid signatures with
  public keys shorter than $dkim_minimum_key_bits bits (default 1024,
  equivalent to a lower limit as presently used by Google) are now
  ignored, with an informational message logged at level 1.
  To disable this check, set $dkim_minimum_key_bits to undef or to 0.

- consider Unique local addresses (ULA) fc00::/7 non-public (RFC 4193),
  dropped site-local addresses fec0::/10 (deprecated by RFC 3879),
  adjusting the default setting of @mynetworks accordingly;

- consider the "Shared Address Space" non-public (RFC 6598);

- adjust parsing the <zone_id> syntax of a scoped IPv6 address
  as per RFC 6874;

- updated an AV entry for a Sophos Anti Virus: the scanning program
  used to be named 'sweep', now it is 'savscan'; thanks to mefiX;

- updated a default value of @virus_name_to_spam_score_maps: updated
  entry for Doppelstern and added entries for Bofhland and PORCUPINE;

- increase an arbitrary sanity limit on %smtp_reason_by_ccat strings
  from 100 to 450 characters (RFC 5321 allows 512 character reply lines);

- relax testing file type of a configuration file, now a configuration
  may also be passed to amavisd through a named pipe (fifo), possibly
  facilitating testing or unusual deployments;

- relax a requirement that a $QUARANTINEDIR directory needs to be
  writable: if $*_quarantine_method template settings include a
  subdirectory (e.g.: $spam_quarantine_method='local:W%P/spam/%m.gz'),
  such subdirectories must already exist and should be writable,
  but the top-level $QUARANTINEDIR directory need not be writable;

- convert an IPv4-mapped IPv6 address into a plain IPv4 dot-quad form
  when found in Received header fields, in socket local or peer address,
  in ADDR field of an XFORWARD smtp extension command, or in an AM.PDP
  attribute client_address. See draft-cmetz-v6ops-v4mapped-api-harmful
  and draft-itojun-v6ops-v4mapped-harmful for potential caveats;

- drop a support for direct queries to p0f v2, as it never worked well
  due to bugs in p0f v2. The p0f v3 changed the query protocol, but
  a query does not include port numbers (see RFC 6302), so using the interface is still the only reliable approach;

- use sysread() instead of read() when reading from /dev/urandom
  to avoid leaving entropy data in I/O buffers; also changed interface
  name to sub read_random_bytes(), which now reads directly into a
  scalar buffer, provided by an argument;

- fix uniform random distribution when generating a random PIN for an
  attachment password (when releasing and $release_format is 'attach')
  (not a security issue);

- added keepalive options to a call to Net::LDAP->new, recognized since
  Net::LDAP 0.53; a patch by Quanah Gibson-Mount;

- removed option inet6 from a default LDAP setup ( $ldap_sys_default ),
  as Net::LDAP changed semantics in an incompatible way;
  presumably the Net::LDAP now does the right thing by default;
  suggested by Quanah Gibson-Mount;

- use a low-level 0MQ interface instead of ZeroMQ / ZMQ abstractions;
  (i.e. ZeroMQ raw interface or ZMQ::LibZMQ3 or ZMQ::LibZMQ2);

- taking advantage of 0MQ multi-part messages the number of IP packets
  transmitted is now radically decreased in favour of sending larger
  but fewer packets;

- when generating 'Abuse Reporting Format (ARF) Reports' add a field
  Source-IP and use UTC timestamps in the Arrival-Date field, in accordance
  with RFC 6692;

- drop (opportunistic) loading of a module Devel::SawAmpersand and testing
  the Devel::SawAmpersand::sawampersand(), variables $&, $` and $' are
  no longer slow since Perl 5.17.7, the PL_sawampersand became a constant,
  there is no longer any need of report it;

- documentation update: remove vestiges of a field 'spam_modifies_subj'
  in README.sql, README.sql-mysql, this field was obsoleted in 2.7.0;
  thanks to Patrick Ben Koetter;


More information about the amavis-users mailing list