High CPU usage (need help)

[Mark Martinec]

David,

> Dear issue was not solved
> by default the time limit is 299.970 s where can i change the time limit
> in order to reduce it to 60s ??

$child_timeout = 50;

The time limiting has been reworked in 2.7.0.  For shorter limits,
especially in a pre-queue setup, the following features of 2.7.0(-pre*)
are worth considering:

(from release notes)

- a master_deadline option and its API equivalent, available in SpamAssassin
  since version 3.3.0, allows for time limiting on lengthy rules checking,
  while still providing results when a time limit is exceeded; this makes
  it more suitable for time-sensitive setups like a pre-queue filtering setup;

- reworked sub-task time limiting in amavisd, along with its counterpart
  solution in SpamAssassin, makes it better suited to a real-time nature
  of pre-queue filtering setups, where one has no control over how long
  SMTP clients are willing to wait at the data-end stage;


> i have found this
> The amavis process handle 100% CPU and last event was :
[...]
> Apr 25 19:15:05 smtp ch1-19447-01-61)[19447]: (19447-01-61) SA dbg:
> message: decoding base64
> Apr 25 19:15:05 smtp ch1-19447-01-61)[19447]: (19447-01-61) SA dbg:
> check: check_main, time limit in 299.970 s
> Apr 25 19:15:05 smtp ch1-19447-01-61)[19447]: (19447-01-61) SA dbg:
> check: running tests for priority: 0
> Apr 25 19:15:05 smtp ch1-19447-01-61)[19447]: (19447-01-61) SA dbg:
> rules: running head tests; score so far=0
> 
> ==> freeze here

Try to reproduce this from a SpamAssassin command line:

  # su vscan -c 'spamassassin -D -t <0.msg'

using a message left in an amavisd temporary directory (email.txt)
when it gets stuck on scanning a message. Then you can start narrowing
down on the offending rules.

> in this email, tests_pri_0 took 43% time and learn 50% of 1817ms that is
> very long for a little mail.
> 
> Apr 25 18:06:56 smtp ch2-11507-02)[11507]: (11507-02) TIMING-SA total
> 1817 ms - parse: 4 (0.2%), extract_message_metadata: 14 (0.8%),
> get_uri_detail_list: 0.94 (0.1%), tests_pri_0: 796 (43.8%),
> tests_pri_500: 3 (0.2%), learn: 990 (54.5%), get_report: 1.22 (0.1%)

The 796 ms for tests_pri_0 is not unreasonable.

The 990 ms for learn is quite high. A high database contention?
Database needs cleaning/reoptimization? Switching bayes to SQL would
probably solve it.

The SpamAssassin trunk (to be 3.4.0) offers a setting:

  bayes_auto_learn_on_error 1

which greatly reduces the frequency of 'learn' operations:


$ man Mail::SpamAssassin::Plugin::AutoLearnThreshold

       bayes_auto_learn_on_error (0 | 1)        (default: 0)
           With "bayes_auto_learn_on_error" off, autolearning will be
           performed even if bayes classifier already agrees with the new
           classification (i.e.  yielded BAYES_00 for what we are now trying
           to teach it as ham, or yielded BAYES_99 for spam). This is a
           traditional setting, the default was chosen to retain backwards
           compatibility.

           With "bayes_auto_learn_on_error" turned on, autolearning will be
           performed only when a bayes classifier had a different opinion from
           what the autolearner is now trying to teach it (i.e. it made an
           error in judgement). This strategy may or may not produce better
           future classifications, but usually works very well, while also
           preventing unnecessary overlearning and slows down database growth.


Mark