100% CPU on amavisd childs on with amavis 2.6.0 / amavis-milter 1.5.0

david touzeau david.touzeau at fr.kaspersky.com
Fri Apr 15 17:36:50 CEST 2011


Dear

I'm using amavis 2.6.0 and amavisd-milter 1.5.0 on Postfix 2.8.2 with
libmilter on i386 
each time amavisd child hang to 100% CPU
By googlize have made severals parameters in order to reduce the CPU
hang but now way, child come up to 100% cpu
I have  reduced Spamassassin plugins that trying to check DNS and
internet connexion.
But the problem still here 

where i'm wrong ?


Here it is the Spamassassin local.cf (/etc/spamassassin/local.cf)
=============================================================================================
report_safe 1
lock_method flock
required_score 5.0
use_bayes 1
bayes_auto_learn 1
skip_rbl_checks 1
auto_whitelist_file_mode 0666
auto_whitelist_path /etc/spamassassin/auto_whitelist
header KEYWORD_RULE_1	Subject =~ /MACHROUR/i
score KEYWORD_RULE_1	7
header MAX_RPCPT_1            To =~ /(.*?(@)domain\.ma){50,}/i
header MAX_CC_1            Cc =~ /(.*?(@)domain\.ma){50,}/i
score MAX_CC_1            10
describe MAX_RPCPT_1  Sent to 50+ one.ma recipients
describe MAX_CC_1  Sent to 50+ Cc one.ma recipients
score MAX_RPCPT_1	10
rewrite_header Subject ***** SPAM *****
bayes_ignore_header X-Bogosity
bayes_ignore_header X-Spam-Flag
bayes_ignore_header X-Spam-Status
header DSPAM_SPAM X-DSPAM-Result =~ /^Spam$/
describe DSPAM_SPAM DSPAM claims it is spam
score DSPAM_SPAM 0.5
header DSPAM_HAM X-DSPAM-Result =~ /^Innocent$/
describe DSPAM_HAM DSPAM claims it is ham
score DSPAM_HAM -0.

and the loaded plugins
loadplugin Mail::SpamAssassin::Plugin::Check
loadplugin Mail::SpamAssassin::Plugin::HTTPSMismatch
loadplugin Mail::SpamAssassin::Plugin::URIDetail
loadplugin Mail::SpamAssassin::Plugin::Bayes
loadplugin Mail::SpamAssassin::Plugin::BodyEval
loadplugin Mail::SpamAssassin::Plugin::HTMLEval
loadplugin Mail::SpamAssassin::Plugin::HeaderEval
loadplugin Mail::SpamAssassin::Plugin::MIMEEval
loadplugin Mail::SpamAssassin::Plugin::RelayEval
loadplugin Mail::SpamAssassin::Plugin::URIEval
loadplugin Mail::SpamAssassin::Plugin::WLBLEval
loadplugin Mail::SpamAssassin::Plugin::ImageInfo
loadplugin Mail::SpamAssassin::Plugin::AWL
loadplugin Mail::SpamAssassin::Plugin::AutoLearnThreshold
loadplugin Mail::SpamAssassin::Plugin::WhiteListSubject
loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody
loadplugin Mail::SpamAssassin::Plugin::HTTPSMismatch
loadplugin HitFreqsRuleTiming /etc/spamassassin/HitFreqsRuleTiming.pm

Here it is the amavisd.conf
=============================================================================================
use strict;

# Configuration file with artica builder 2011-04-15 16::30:05
# COMMONLY ADJUSTED SETTINGS:

# PERFORMANCES:
$max_servers = 15;
$child_timeout=8*60;
$max_requests=10;
$pid_file='/var/spool/postfix/var/run/amavisd-new/amavisd-new.pid';
$daemon_user = 'postfix';
$daemon_group = 'postfix';
$TEMPBASE = "$MYHOME/tmp";   # working directory, needs to exist, -T
$ENV{TMPDIR} = $TEMPBASE;    # environment variable TMPDIR, used by SA,
etc.
$QUARANTINEDIR = '/var/virusmails';  # -Q
$log_level = 1;# verbosity 0..5, -d
$log_recip_templ 	= undef;    # disable by-recipient level-0 log entries
$DO_SYSLOG = 1;              # log via syslogd (preferred)
$syslog_facility 	= 'mail';   # Syslog facility as a string
$syslog_priority 	= 'debug';  # Syslog base (minimal) priority as a
string,
#$LOGFILE 		    = "/var/log/amavis/amavis.log";
$MAXLEVELS = 14;
$MAXFILES = 1500;
$MIN_EXPANSION_QUOTA =      100*1024;  # bytes  (default undef, not
enforced)
$MAX_EXPANSION_QUOTA = 300*1024*1024;  # bytes  (default undef, not
enforced)

$enable_db = 0;              # enable use of BerkeleyDB/libdb (SNMP and
nanny)
$enable_global_cache = 0;    # enable use of libdb-based cache if
$enable_db=1
$nanny_details_level = 2;    # nanny verbosity: 1: traditional, 2:
detailed
$enable_dkim_verification = 0;
$enable_dkim_signing = 0;
$enable_ldap  = 0;

@local_domains_maps = ( ['one.ma','one.org.ma'] );  # list of all local
domains
@mynetworks = qw(192.168.3.0/24);

$unix_socketname =
"/var/spool/postfix/var/run/amavisd-new/amavisd-new.sock";  #
amavisd-release or amavis-milter
$inet_socket_port = 10024;   # listen on this local TCP port(s)
$mydomain  = "one.ma";  # domain name
$myhostname = "smtp.one.ma";  # must be a fully-qualified domain name!
#adding net acl 192.168.3.8 127.0.0.1 of all interfaces prevent DENIED
ACCESS from IP 172.16.X.YYY
@inet_acl = qw(192.168.3.8 127.0.0.1);
$forward_method ='pipe:flags=q argv=/usr/sbin/sendmail -Ac -i -f
${sender} -- ${recipient}';
$notify_method = $forward_method;



$policy_bank{'MYNETS'} = {   # mail originating from @mynetworks
	originating => 1,  # is true in MYNETS by default, but let's make it
explicit
	os_fingerprint_method => undef,  # don't query p0f for internal clients
};

$interface_policy{'SOCK'} = 'AM.PDP-SOCK'; # only applies with
$unix_socketname

# Use with amavis-release over a socket or with Petr Rehor's
amavis-milter.c
# (with amavis-milter.c from this package or old amavis.c client use
'AM.CL'):
$policy_bank{'AM.PDP-SOCK'} = {
  protocol => 'AM.PDP',
  notify_method  => 'pipe:flags=q argv=/usr/sbin/sendmail -Ac -i -f
${sender} -- ${recipient}',  auth_required_release => 0,  # do not
require secret_id for amavisd-release
};

# Use in combination with dedicated artica-filters
$policy_bank{'killAll'} = {protocol => 'AM.PDP',final_destiny_by_ccat =>
{ (CC_CATCHALL) => D_DISCARD }};
$sa_tag_level_deflt  = -999;  # add spam info headers if at, or above
that level
$sa_tag2_level_deflt = 6.31;  # add 'spam detected' headers at that
level
$sa_tag3_level_deflt = 10.0;  # add 'spam detected' headers at that
level
$sa_kill_level_deflt = 15;  # triggers spam evasive actions (e.g. blocks
mail)
$sa_dsn_cutoff_level = 9;   # spam level beyond which a DSN is not sent
$sa_crediblefrom_dsn_cutoff_level = 18; # likewise, but for a likely
valid From
$sa_quarantine_cutoff_level = 25; # spam level beyond which quarantine
is off
#$penpals_bonus_score = 8;    # (no effect without a @storage_sql_dsn
database)
#$penpals_threshold_high = $sa_kill_level_deflt;  # don't waste time on
hi spam
$bounce_killer_score = 100;  # spam score points to add for joe-jobbed
bounces

$sa_configpath='/etc/spamassassin';
$sa_siteconfigpath='/etc/spamassassin/local.cf';
$sa_mail_body_size_limit = 400*1024; # don't waste time on SA if mail is
larger
$sa_local_tests_only = 0;    # only tests which do not require internet
access?
#$sa_spam_subject_tag = ***SPAM*** _SCORE_ (_REQD_)
@spam_subject_tag_maps  = undef;
@spam_subject_tag2_maps = ('***SPAM*** _SCORE_ (_REQD_)');
@spam_subject_tag3_maps =('***SPAM*** _SCORE_ (_REQD_)');

$timestamp_fmt_mysql = 1;

$virus_admin               = "undef";  # notifications recip.
$mailfrom_notify_admin     = "root\@localhost.localdomain";  #
notifications sender
$mailfrom_notify_recip     = "root\@localhost.localdomain";  #
notifications sender
$mailfrom_notify_spamadmin = "root\@localhost.localdomain"; #
notifications sender
$mailfrom_to_quarantine = ''; # null return path; uses original sender
if undef

$smtpd_recipient_limit=50;

$enable_dkim_signing = 0;
# WHITELISTING, list of whitelisted senders...
@whitelist_sender_maps = ( new_RE(

	qr'.*@agencedusud\.gov\.ma$'i
,	qr'.*@artemis\.ma$'i
,	qr'.*@attijariwafa\.com$'i
,	qr'.*@aui\.ma$'i
,	qr'.*@barid\.oncf\.ma$'i
,	qr'.*@bmcebank\.co\.ma$'i
,	qr'.*@bnpparibas\.com$'i
,	qr'.*@brams\.net$'i
,	qr'.*@ca-cdm\.ma$'i
,	qr'.*@capitalconsulting\.com$'i
,	qr'.*@capitalconsulting\.fr$'i
,	qr'.*@capitalconsulting\.ma$'i
,	qr'.*@cas\.apd\.ma$'i
,	qr'.*@cfcim\.org$'i
,	qr'.*@chadbourne\.com$'i
,	qr'.*@cnesten\.org\.ma$'i
,	qr'.*@colgp\.it$'i
,	qr'.*@cpm\.co\.ma$'i
,	qr'.*@creditagricole\.ma$'i
,	qr'.*@cspservices\.de$'i
,	qr'.*@dbvib\.com$'i
,	qr'.*@devoteam\.com$'i
,	qr'.*@ec\.europa\.eu$'i
,	qr'.*@edison\.it$'i
,	qr'.*@es\.abb\.com$'i
,	qr'.*@finances\.gov\.ma$'i
,	qr'.*@fr\.rolandbeger\.com$'i
,	qr'.*@garradhassan\.com$'i
,	qr'.*@gide\.com$'i
,	qr'.*@h2energy-maroc\.com$'i
,	qr'.*@hip\.lu$'i
,	qr'.*@hp\.com$'i
,	qr'.*@iam\.ma$'i
,	qr'.*@intelcom\.co\.ma$'i
,	qr'.*@inwi\.ma$'i
,	qr'.*@ipplc\.ae$'i
,	qr'.*@lazrak\.ma$'i
,	qr'.*@ma\.rolandberger\.com$'i
,	qr'.*@mecamidi\.com$'i
,	qr'.*@meditel\.ma$'i
,	qr'.*@munisys\.net\.ma$'i
,	qr'.*@nareva-ona\.com$'i
,	qr'.*@net2s\.ma$'i
,	qr'.*@nexans\.com$'i
,	qr'.*@ocpgroup\.ma$'i
,	qr'.*@onep\.org\.ma$'i
,	qr'.*@pop3\.amadeus\.net$'i
,	qr'.*@ree\.es$'i
,	qr'.*@royalairmaroc\.com$'i
,	qr'.*@saaidihdid\.com$'i
,	qr'.*@sap\.com$'i
,	qr'.*@scania\.co\.ma$'i
,	qr'.*@senelec\.sn$'i
,	qr'.*@siemens\.com$'i
,	qr'.*@simmons-simmons\.com$'i
,	qr'.*@snep\.ma$'i
,	qr'.*@socgen\.com$'i
,	qr'.*@sonelgaz\.dz$'i
,	qr'.*@wana\.ma$'i
,	qr'.*@yahoo\.fr$'i

));

@addr_extension_virus_maps      = ('virus');
@addr_extension_banned_maps     = ('banned');
@addr_extension_spam_maps       = ('spam');
@addr_extension_bad_header_maps = ('badh');
# $recipient_delimiter = '+';  # undef disables address extensions
altogether
# when enabling addr extensions do also Postfix/main.cf:
recipient_delimiter=+

$path =
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/sbin:/usr/bin:/bin:/usr/share/artica-postfix/bin';
$dspam = '';
$defang_virus  = 1;  # MIME-wrap passed infected mail
$defang_banned = 1;  # MIME-wrap passed mail containing banned name
# for defanging bad headers only turn on certain minor contents
categories:
$defang_by_ccat{+CC_BADH.",3"} = 1;  # NUL or CR character in header
$defang_by_ccat{+CC_BADH.",5"} = 1;  # header line longer than 998
characters
$defang_by_ccat{+CC_BADH.",6"} = 1;  # header field syntax error

$final_virus_destiny      = D_DISCARD;
$final_banned_destiny     = D_BOUNCE;
$final_spam_destiny       = D_BOUNCE;
$final_bad_header_destiny = D_PASS;
$mailfrom_to_quarantine = undef;
$spam_quarantine_method   = 'local:spam-%m.gz';
$defang_by_ccat{CC_SPAMMY, 1};
$quarantine_method_by_ccat{+CC_SPAMMY} = $spam_quarantine_method;
$quarantine_to_maps_by_ccat{CC_SPAMMY} = \@spam_quarantine_to_maps;
$final_destiny_by_ccat{+CC_SPAMMY} =$final_spam_destiny;
$defang_by_ccat{+CC_SPAMMY.",1"} = 1;
$quarantine_method_by_ccat{CC_SPAMMY.",1"} = $spam_quarantine_method;
$quarantine_to_maps_by_ccat{CC_SPAMMY.",1"} = \@spam_quarantine_to_maps;
$final_destiny_by_ccat{+CC_SPAMMY.",1"} =$final_spam_destiny;
$bad_header_quarantine_method = undef;

# $os_fingerprint_method = 'p0f:*:2345';  # to query p0f-analyzer.pl

#Sender Notifications
$warnbadhsender = 0;
$warnvirussender = undef;
$warnspamsender = undef;
$warnbannedsender = undef;

#Recipient Notifications
$warnvirusrecip = 0;
$warnbannedrecip = 0;
$warnbadhrecip = 0;

#Admin Notifications
$newvirus_admin = undef;
$virus_admin = undef;
$spam_admin = undef;
$bad_header_admin = undef;

#templates Notifications
$notify_sender_templ      =
read_text("/usr/local/etc/amavis/template-dsn.txt");
$notify_virus_sender_templ=
read_text("/usr/local/etc/amavis/template-virus-sender.txt");
$notify_virus_admin_templ =
read_text("/usr/local/etc/amavis/template-virus-admin.txt");
$notify_virus_recips_templ=
read_text("/usr/local/etc/amavis/template-virus-recipient.txt");
$notify_spam_sender_templ =
read_text("/usr/local/etc/amavis/template-virus-sender.txt");
$notify_spam_admin_templ  =
read_text("/usr/local/etc/amavis/template-spam-admin.txt");


# REMAINING IMPORTANT VARIABLES ARE LISTED HERE BECAUSE OF LONGER
ASSIGNMENTS

@keep_decoded_original_maps = (new_RE(
# qr'^MAIL$',   # retain full original message for virus checking (can
be slow)
  qr'^MAIL-UNDECIPHERABLE$', # recheck full mail if it contains
undecipherables (enable by EnableScanSecurity)
  qr'^(ASCII(?! cpio)|text|uuencoded|xxencoded|binhex)'i,
# qr'^Zip archive data',     # don't trust Archive::Zip
));


$banned_filename_re=undef;

# See http://support.microsoft.com/default.aspx?scid=kb;EN-US;q262631
# and http://www.cknow.com/vtutor/vtextensions.htm


# ENVELOPE SENDER SOFT-WHITELISTING / SOFT-BLACKLISTING

@score_sender_maps = ({

  '.' => [new_RE(
    [qr'^(bulkmail|offers|cheapbenefits|earnmoney|foryou)@'i         =>
5.0],
    [qr'^(greatcasino|investments|lose_weight_today|market\.alert)@'i=>
5.0],
    [qr'^(money2you|MyGreenCard|new\.tld\.registry|opt-out|opt-in)@'i=>
5.0],
    [qr'^(optin|saveonlsmoking2002k|specialoffer|specialoffers)@'i   =>
5.0],
    [qr'^(stockalert|stopsnoring|wantsome|workathome|yesitsfree)@'i  =>
5.0],
    [qr'^(your_friend|greatoffers)@'i                                =>
5.0],
    [qr'^(inkjetplanet|marketopt|MakeMoney)\d*@'i                    =>
5.0] ),

read_hash("/usr/local/etc/sender_scores_sitewide"),
   {
     'nobody at cert.org'                        => -3.0,
     'cert-advisory at us-cert.gov'              => -3.0,
     'owner-alert at iss.net'                    => -3.0,
     'slashdot at slashdot.org'                  => -3.0,
     'securityfocus.com'                      => -3.0,
     'ntbugtraq at listserv.ntbugtraq.com'       => -3.0,
     'security-alerts at linuxsecurity.com'      => -3.0,
     'mailman-announce-admin at python.org'      => -3.0,
     'amavis-user-admin at lists.sourceforge.net'=> -3.0,
     'amavis-user-bounces at lists.sourceforge.net' => -3.0,
     'spamassassin.apache.org'                => -3.0,
     'notification-return at lists.sophos.com'   => -3.0,
     'owner-postfix-users at postfix.org'        => -3.0,
     'owner-postfix-announce at postfix.org'     => -3.0,
     'owner-sendmail-announce at lists.sendmail.org'   => -3.0,
     'sendmail-announce-request at lists.sendmail.org' => -3.0,
     'donotreply at sendmail.org'                => -3.0,
     'ca+envelope at sendmail.org'               => -3.0,
     'noreply at freshmeat.net'                  => -3.0,
     'owner-technews at postel.acm.org'          => -3.0,
     'ietf-123-owner at loki.ietf.org'           => -3.0,
     'cvs-commits-list-admin at gnome.org'       => -3.0,
     'rt-users-admin at lists.fsck.com'          => -3.0,
     'clp-request at comp.nus.edu.sg'            => -3.0,
     'surveys-errors at lists.nua.ie'            => -3.0,
     'emailnews at genomeweb.com'                => -5.0,
     'yahoo-dev-null at yahoo-inc.com'           => -3.0,
     'returns.groups.yahoo.com'               => -3.0,
     'clusternews at linuxnetworx.com'           => -3.0,
     lc('lvs-users-admin at LinuxVirtualServer.org')    => -3.0,
     lc('owner-textbreakingnews at CNNIMAIL12.CNN.COM') => -5.0,
     'sender at example.net'                     =>  3.0,
     '.example.net'                           =>  1.0,

   },
  ],
});


@decoders = (
  ['mail', \&do_mime_decode],
  ['asc',  \&do_ascii],
  ['uue',  \&do_ascii],
  ['hqx',  \&do_ascii],
  ['ync',  \&do_ascii],
  ['F',    \&do_uncompress, ['unfreeze','freeze -d','melt','fcat'] ],
  ['Z',    \&do_uncompress, ['uncompress','gzip -d','zcat'] ],
  ['gz',   \&do_uncompress,  'gzip -d'],
  ['gz',   \&do_gunzip],
  ['bz2',  \&do_uncompress,  'bzip2 -d'],
  ['lzo',  \&do_uncompress,  'lzop -d'],
  ['rpm',  \&do_uncompress, ['rpm2cpio.pl','rpm2cpio'] ],
  ['cpio', \&do_pax_cpio,   ['pax','gcpio','cpio'] ],
  ['tar',  \&do_pax_cpio,   ['pax','gcpio','cpio'] ],
  ['deb',  \&do_ar,          'ar'],
# ['a',    \&do_ar,          'ar'],  # unpacking .a seems an overkill
  ['zip',  \&do_unzip],
  ['7z',   \&do_7zip,       ['7zr','7za','7z'] ],
  ['rar',  \&do_unrar,      ['rar','unrar'] ],
  ['arj',  \&do_unarj,      ['arj','unarj'] ],
  ['arc',  \&do_arc,        ['nomarch','arc'] ],
  ['zoo',  \&do_zoo,        ['zoo','unzoo'] ],
  ['lha',  \&do_lha,         'lha'],
# ['doc',  \&do_ole,         'ripole'],
  ['cab',  \&do_cabextract,  'cabextract'],
  ['tnef', \&do_tnef_ext,    'tnef'],
  ['tnef', \&do_tnef],
# ['sit',  \&do_unstuff,     'unstuff'],  # broken/unsafe decoder
  ['exe',  \&do_executable, ['rar','unrar'], 'lha', ['arj','unarj'] ],
);


@av_scanners = (
['ClamAV-clamd',\&ask_daemon, ["CONTSCAN {}",
"/var/run/clamav/clamav.sock"],qr/\bOK$/, qr/\bFOUND$/,qr/^.*?: (?!
Infected Archive)(.*) FOUND$/m ]
);
@av_scanners_backup = ();
1; 

=============================================================================================
here it is the top command you will see 2 child on 100% CPU and one
using 35%
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
11591 postfix   39  19 83684  68m 3668 R  100  2.2   1:47.59
amavisd                                                                                                                                                                                                            
28300 postfix   39  19 86048  70m 3640 R  100  2.3  12:48.36
amavisd                                                                                                                                                                                                            
11593 postfix   39  19 86736  71m 3788 S   35  2.3   0:07.41
amavisd           


                                                                                                                                                                                                 
20919 mailflt3  20   0 38768 9544 5284 S    3  0.3   0:00.09
ap-mailfilter                                                                                                                                                                                                      
30318 root      39  19 47024  13m 6520 S    2  0.5   2:04.84
php5                                                                                                                                                                                                               
 2255 mailflt3  20   0  217m 117m  588 S    1  3.9 711:59.75
ap-process-serv                                                                                                                                                                                                    
 3803 postfix   39  19  167m 117m 6264 S    1  3.9   1:13.24
clamd                                                                                                                                                                                                              
 1941 postfix   39  19  8360 3260 1892 S    1  0.1   0:00.30
cleanup                                                                                                                                                                                                            
19495 mailflt3  20   0 59536 6908  696 S    1  0.2   0:19.95
kas-milter                                                                                                                                                                                                         
 6470 root      39  19  1868  576  472 S    0  0.0   3:33.81
vnstatd                                                                                                                                                                                                            
20912 root      20   0  2576 1172  812 R    0  0.0   0:00.02
top                                                                                                                                                                                                                
21024 root      39  19 42664  12m 6196 S    0  0.4   0:05.61
php5                                                                                                                                                                                                               
27436 postfix   39  19 51136 1008  720 S    0  0.0   0:01.19
amavisd-milter                                                                                                                                                                                                     
30685 postfix   39  19 12732 4284 2888 S    0  0.1   0:00.20
smtpd                                                                                                                                                                                                              
    1 root      20   0  2032  656  564 S    0  0.0   1:48.80
init             





More information about the amavis-users mailing list