Inbound doesn't catch SaneSecurity signature, Outbound does

[francis picabia]

> On Mon, Sep 24, 2012 at 5:08 AM, Mark Martinec

>> Again, it is not the same message:

OK, now I have a sample case which is simply a mail forward
set up on the user's Exchange account.

Inbound (Redhat) was undetected, and outbound (Debian) did detect.

On Oct 25 I made a new amavisd.conf for the Redhat
system (mx10) which is having the problem not
detecting some phishing signatures.  The new
config file was based on the Debian config files where
the filtering has proven to be superior (smtp).

clamscan run with the quarantined file
on the Redhat system that missed it detects the phishing
signature, and I've not updated SaneSecurity signatures
since this email passed through.

$ clamscan virus-wXQFj8Xeu4G2
virus-wXQFj8Xeu4G2: Doppelstern.Scam4.732.UNOFFICIAL FOUND

----------- SCAN SUMMARY -----------
Known viruses: 2160751
Engine version: 0.97.5
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 0.27 MB
Data read: 0.20 MB (ratio 1.35:1)
Time: 53.426 sec (0 m 53 s)

Here are traces on inbound (not caught) and outbound (caught) 35 seconds later.

Not caught:

Oct 27 13:55:00 mx10 amavis[23335]: (23335-08) LMTP::10024
/var/amavis/tmp/amavis-20121027T134540-23335:
<mslin-homer at hmrc.gov.uk> -> <wlu at exchange.example.com> SIZE=217278
BODY=8BITMIME Received: from mx10.example.com ([127.0.0.1]) by
localhost (mx10.example.com [127.0.0.1]) (amavisd-new, port 10024)
with LMTP for <wlu at exchange.example.com>; Sat, 27 Oct 2012 13:55:00
-0300 (ADT)
Oct 27 13:55:00 mx10 amavis[23335]: (23335-08) Checking: o-t83BXo4jcl
[207.189.223.49] <mslin-homer at hmrc.gov.uk> ->
<wlu at exchange.example.com>
Oct 27 13:55:00 mx10 amavis[23335]: (23335-08) p003 1 Content-Type:
multipart/mixed
Oct 27 13:55:00 mx10 amavis[23335]: (23335-08) p001 1/1 Content-Type:
text/plain, size: 0 B, name:
Oct 27 13:55:00 mx10 amavis[23335]: (23335-08) p002 1/2 Content-Type:
application/msword, size: 157500 B, name: Receipt.rtf
Oct 27 13:55:02 mx10 amavis[23335]: (23335-08) SPAM-TAG,
<mslin-homer at hmrc.gov.uk> -> <wlu at exchange.example.com>, Yes,
score=11.137 tagged_above=0 required=6.2 tests=[MISSING_HEADERS=1.207,
RCVD_IN_BL_SPAMCOP_NET=4, RCVD_IN_PSBL=2.7, RCVD_IN_RP_RNBL=1.284,
REPLYTO_WITHOUT_TO_CC=1.946] autolearn=disabled
Oct 27 13:55:02 mx10 amavis[23335]: (23335-08) FWD via SMTP:
<mslin-homer at hmrc.gov.uk> -> <wlu at exchange.example.com>,BODY=7BIT 250
2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as ABE1319CC89
Oct 27 13:55:02 mx10 amavis[23335]: (23335-08) Passed SPAMMY,
[207.189.223.49] [64.95.245.102] <mslin-homer at hmrc.gov.uk> ->
<wlu at exchange.example.com>, Message-ID:
<2787.64.95.245.102.1351263437.squirrel at email.peakpeak.com>, mail_id:
o-t83BXo4jcl, Hits: 11.137, size: 217278, queued_as: ABE1319CC89, 2461
ms
Oct 27 13:55:02 mx10 amavis[23335]: (23335-08) TIMING-SA total 2114 ms
- parse: 20 (0.9%), extract_message_metadata: 38 (1.8%),
get_uri_detail_list: 0.31 (0.0%), tests_pri_-1000: 10 (0.5%),
tests_pri_-950: 2 (0.1%), tests_pri_-900: 2 (0.1%), tests_pri_-400:
1.77 (0.1%), tests_pri_0: 920 (43.5%), check_dkim_adsp: 118 (5.6%),
check_spf: 421 (19.9%), poll_dns_idle: 1417 (67.0%), check_razor2: 296
(14.0%), tests_pri_500: 1080 (51.1%), get_report: 1.62 (0.1%)
Oct 27 13:55:02 mx10 amavis[23335]: (23335-08) TIMING [total 2468 ms]
- SMTP greeting: 2 (0%)0, SMTP LHLO: 1 (0%)0, SMTP pre-MAIL: 1 (0%)0,
SMTP pre-DATA-flush: 3 (0%)0, SMTP DATA: 55 (2%)3, check_init: 1
(0%)3, digest_hdr: 2 (0%)3, digest_body_dkim: 4 (0%)3, gen_mail_id: 1
(0%)3, mime_decode: 33 (1%)4, get-file-type1: 19 (1%)5, parts_decode:
0 (0%)5, check_header: 1 (0%)5, AV-scan-1: 99 (4%)9, spam-wb-list: 2
(0%)9, SA parse: 21 (1%)10, SA check: 2089 (85%)95, update_cache: 8
(0%)95, decide_mail_destiny: 1 (0%)95, fwd-connect: 7 (0%)95,
fwd-mail-pip: 3 (0%)95, fwd-rcpt-pip: 0 (0%)95, fwd-data-chkpnt: 0
(0%)95, write-header: 2 (0%)95, fwd-data-contents: 12 (0%)96,
fwd-end-chkpnt: 84 (3%)99, prepare-dsn: 1 (0%)99, main_log_entry: 10
(0%)100, update_snmp: 3 (0%)100, SMTP pre-response: 0 (0%)100, SMTP
response: 0 (0%)100, unlink-2-files: 0 (0%)100, rundown: 1 (0%)100


Caught on outbound:

Oct 27 13:55:36 smtp amavis[19984]: (19984-14) loaded policy bank
"MYNETS" over "ORIGINATING"
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) LMTP::10026
/var/lib/amavis/tmp/amavis-20121027T131704-19984:
<mslin-homer at hmrc.gov.uk> -> <john.doe at gmail.com> Received: from
smtp.example.com ([XXX.YYY.201.5]) by localhost (thabit.example.com
[127.0.0.1]) (amavisd-new, port 10026) with LMTP for
<wen.wilsonlu at gmail.com>; Sat, 27 Oct 2012 13:55:36 -0300 (ADT)
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) Checking: wXQFj8Xeu4G2
ORIGINATING/MYNETS [XXX.YYY.200.97] <mslin-homer at hmrc.gov.uk> ->
<john.doe at gmail.com>
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) p003 1 Content-Type:
multipart/mixed
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) p001 1/1 Content-Type:
text/plain, size: 0 B, name:
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) p002 1/2 Content-Type:
application/msword, size: 157500 B, name: Receipt.rtf
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) run_av (ClamAV-clamd):
/var/lib/amavis/tmp/amavis-20121027T131704-19984/parts INFECTED:
Doppelstern.Scam4.732.UNOFFICIAL
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) virus_scan:
(Doppelstern.Scam4.732.UNOFFICIAL), detected by 1 scanners:
ClamAV-clamd
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) Virus
Doppelstern.Scam4.732.UNOFFICIAL matches (?-xism:.*), sender addr
ignored
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) local delivery: <> ->
virus-quarantine, mbx=/var/virusmails/w/virus-wXQFj8Xeu4G2
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) dkim: candidate
originators: 2822.From:<virusalert at example.com>,
2821.mail_from:<virusalert at example.com>
Oct 27 13:55:36 smtp amavis[19984]: (19984-14) dkim: signing (author),
From: <virusalert at example.com>, KEY.key_ind=>0, a=>rsa-sha256,
c=>relaxed/simple, d=>example.com, s=>smtp, ttl=>1814400,
x=>1353171336.72979
Oct 27 13:55:37 smtp amavis[19984]: (19984-14) SEND via SMTP:
<virusalert at example.com> ->
<virusalert at example.com>,ENVID=AM..20121027T165536Z at thabit.example.com
250 2.0.0 Ok, id=19984-14, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok:
queued as 0723A1F4528
Oct 27 13:55:37 smtp amavis[19984]: (19984-14) Blocked INFECTED
(Doppelstern.Scam4.732.UNOFFICIAL), ORIGINATING/MYNETS LOCAL
[XXX.YYY.200.97] [64.95.245.102] <mslin-homer at hmrc.gov.uk> ->
<john.doe at gmail.com>, quarantine: w/virus-wXQFj8Xeu4G2, Message-ID:
<2787.64.95.245.102.1351263437.squirrel at email.peakpeak.com>, mail_id:
wXQFj8Xeu4G2, Hits: -, size: 218433, 338 ms
Oct 27 13:55:37 smtp amavis[19984]: (19984-14) TIMING [total 345 ms] -
SMTP greeting: 4 (1%)1, SMTP LHLO: 3 (1%)2, SMTP pre-MAIL: 3 (1%)3,
SMTP pre-DATA-flush: 2 (1%)4, SMTP DATA: 63 (18%)22, check_init: 1
(0%)22, digest_hdr: 3 (1%)23, digest_body_dkim: 2 (1%)24, gen_mail_id:
1 (0%)24, mime_decode: 27 (8%)32, get-file-type1: 19 (6%)37,
parts_decode: 0 (0%)37, check_header: 2 (1%)38, AV-scan-1: 51 (15%)53,
read_snmp_variables: 1 (0%)53, best_try_originator: 2 (1%)54,
update_cache: 1 (0%)54, decide_mail_destiny: 2 (1%)55, notif-quar: 2
(0%)55, stat-mbx: 3 (1%)56, open-mbx: 0 (0%)56, write-header: 1
(0%)56, save-to-local-mailbox: 2 (1%)57, write-header: 38 (11%)68,
fwd-data-dkim: 19 (5%)73, fwd-connect: 23 (7%)80, fwd-mail-pip: 35
(10%)90, fwd-rcpt-pip: 1 (0%)90, fwd-data-chkpnt: 0 (0%)90,
write-header: 1 (0%)90, fwd-data-contents: 4 (1%)92, fwd-end-chkpnt:
13 (4%)95, prepare-dsn: 1 (0%)96, main_log_entry: 8 (2%)98,
update_snmp: 4 (1%)99, SMTP pre-response: 0 (0%)99, SMTP response: 1
(0%)100, unlink-2...
Oct 27 13:55:37 smtp amavis[19984]: (19984-14) ...-files: 0 (0%)100,
rundown: 1 (0%)100
Oct 27 13:55:37 smtp amavis[19984]: (19984-14) extra modules loaded:
unicore/lib/gc_sc/Digit.pl, unicore/lib/gc_sc/SpacePer.pl
Oct 27 13:57:38 smtp amavis[19984]: (19984-14) loaded policy bank "ORIGINATING"


Another test I did was to reverse the roles of primary and secondary
MX where the Debian
system good at catching these was now primary MX.  In two weeks like this,
there were only 2 emails caught on the outbound with phishing signatures,
and both had arrived on the Redhat system (running as secondary MX
during that time).

The above trace is with the Redhat system mx10 recently back to the
role of primary MX.
With this set up, there is more likelihood of the Debian SMTP
detecting phishing signatures
the inbound Redhat mx10 missed.

I can only conclude that either:

1.  There is a configuration difference between the two amavis
instances which matters
(I've tried to eliminate by building a new config for Redhat out of
Debian /etc/amavis/conf.d files), or
2.  There is a build difference between the two amavis binaries or
their libraries.

The Redhat system has amavisd-new-2.6.6 while Debian is
amavisd-new-2.6.4 (20090625)

The Redhat system does block between 50 to 350 Sanesecurity signatures per day,
so it is generally working OK.

What else can I do to trace the problem and/or improve the chances of
the Redhat system
actually blocking all of the signatures rather than most?