From foreplayboy at gmail.com Wed Feb 1 04:29:54 2012 From: foreplayboy at gmail.com (kshitij mali) Date: Wed, 1 Feb 2012 08:59:54 +0530 Subject: amavis is not able to work please help Message-ID: Hi Mark, I have installed amavisd-new 2.6.4-4 rpm for rhel4 after installing all of is dependancy i have configure the postfix and amavis properly but also i am getting error while postfix send email to amavis for scanning please find the error screen shot below [image: image.png] please let me know what is wrong they same config is working on other machine. Regards, Kshitij -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 185872 bytes Desc: not available URL: From njones at megan.vbhcs.org Wed Feb 1 04:50:24 2012 From: njones at megan.vbhcs.org (Noel Jones) Date: Tue, 31 Jan 2012 21:50:24 -0600 Subject: amavis is not able to work please help In-Reply-To: References: Message-ID: <4F28B680.5010303@megan.vbhcs.org> On 1/31/2012 9:29 PM, kshitij mali wrote: > Hi Mark, > > I have installed amavisd-new 2.6.4-4 rpm for rhel4 after installing > all of is dependancy i have configure the postfix and amavis properly > but also i am getting error while postfix send email to amavis for > scanning please find the error screen shot below > > image.png > please let me know what is wrong they same config is working on > other machine. > > > Regards, > Kshitij > > The postfix/master smtpd: bad command startup message suggests you fat-fingered your postfix master.cf modifications. There are probably other postfix errors logged giving more information. http://www.postfix.org/DEBUG_README.html#logging -- Noel Jones From sdavies at sdc.com.au Wed Feb 1 11:01:11 2012 From: sdavies at sdc.com.au (Stephen Davies) Date: Wed, 1 Feb 2012 20:31:11 +1030 Subject: Amavisd with bogofilter Message-ID: <201202012031.12053.sdavies@sdc.com.au> The attached diff against amavisd-new-2.7.0 replaces Spam Assassin in amavisd with Bogofilter. The resulting amavisd has been running here for over twelve hours and seems to work perfectly. The only other changes were to the configuration file entries for: $sa_tag_level_deflt = 0; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 3; # add 'spam detected' headers at that level $sa_kill_level_deflt = 5; # triggers spam evasive actions (e.g. blocks mail) The first because I like to see spam info headers and the last to force quarantine of all spam as detected by bogofilter. Enjoy, Stephen -------------------------------------------------------- 944c945 < ['SpamAssassin', 'Amavis::SpamControl::SpamAssassin' ], --- > # ['SpamAssassin', 'Amavis::SpamControl::SpamAssassin' ], 957a959,962 > ['Bogofilter', 'Amavis::SpamControl::ExtProg', 'bogofilter', > [ qw(-e -u -v)], > score_factor => 1.0, > ] 1045c1050 < $final_spam_destiny = D_PASS; --- > $final_spam_destiny = D_DISCARD; 1121a1127,1131 > X-Bogosity > ); > $allowed_added_header_fields{lc($_)} = 0 for qw( > X-Spam-Status X-Spam-Level X-Spam-Flag X-Spam-Score > X-Spam-Report X-Spam-Checker-Version X-Spam-Tests 1123,1124d1132 < $allowed_added_header_fields{lc('X-Spam-Report')} = 0; < $allowed_added_header_fields{lc('X-Spam-Checker-Version')} = 0; 24142c24150 < if ($curr_head=~/^((?:X-DSPAM|X-CRM114)[^:]*?)[ \t]*:[ \t]*(.*)$/s) { --- > if ($curr_head=~/^((?:X-DSPAM|X-CRM114|X-Bogosity)[^:]*?)[ \t]*:[ \t]*(.*)$/s) { 24220a24229,24253 > my($bogo_line) = $header_field{lc('X-Bogosity')}; > my($bogo_status,$bogo_score,$bogo_tests); > if (defined $bogo_line) { > ($bogo_status,$bogo_tests,$bogo_score)=split(/,/,$bogo_line); > $bogo_score =~ s/ *spamicity=//; > s/[ \t\r\n]+\z// for ($bogo_status, $bogo_score); > if ($bogo_status eq "Spam"){ > $spam_score = 5; > } else { > if ($bogo_status eq "Ham"){ > $spam_score = 0; > } else { > $spam_score = 2; > } > } > # $spam_tests = sprintf("%s. %s Spamicity=%s", > # $scanner_name, $bogo_status, $bogo_score); > $spam_tests = $bogo_line; > $msginfo->supplementary_info('AUTOLEARN','yes'); > # $msginfo->supplementary_info('BOGOSTATUS', > # sprintf("%s ( %s )", $bogo_status,$bogo_score)); > # $msginfo->supplementary_info('BOGOSCORE', $bogo_score); > do_log(2,"%s result: score=%s (%s), status=%s", $scanner_name, > $spam_score, $bogo_score, $bogo_status); > } -- ============================================================================= Stephen Davies Consulting P/L Voice: 08-8177 1595 Adelaide, South Australia. Fax : 08-8177 0133 Records & Collections Management. Mobile:040 304 0583 From michael.reincke at atlas-elektronik.com Wed Feb 1 11:49:21 2012 From: michael.reincke at atlas-elektronik.com (Michael Reincke) Date: Wed, 01 Feb 2012 11:49:21 +0100 Subject: Warn virus recipients only for non spam mail? Message-ID: <4F2918B1.20109@atlas-elektronik.com> Hello, is it possible just to warn recipients of virus mails only if SA score is below $sa_kill_level_deflt?? $warnvirusrecip =1 send a notify on all virus mails to recipient. Regards Michael Reincke -- Dipl.-Math. Michael Reincke System Services ATLAS ELEKTRONIK GmbH Sebaldsbruecker Heerstrasse 235 28309 BREMEN GERMANY Telefon / Phone +49 (0)421 457-2302 Telefax / Fax +49 (0)421 457-2977 michael.reincke at atlas-elektronik.com www.atlas-elektronik.com Gesch?ftsf?hrung / Management Board: Dieter Rottsieper (Vorsitz / Chairman), Volker Paltzo Vorsitzender des Aufsichtsrats / Chairman Supervisory Board: Dr. Stefan Zoller Sitz der Gesellschaft / Registered Office: Bremen Register / Commercial Register: Amtsgericht Bremen, HRB 21570 -------------- next part -------------- An HTML attachment was scrubbed... URL: From werner at aloah-from-hell.de Wed Feb 1 13:57:45 2012 From: werner at aloah-from-hell.de (Werner Detter) Date: Wed, 01 Feb 2012 13:57:45 +0100 Subject: LDAP-Support in amavisd-new - missing placeholder %d Message-ID: <4F2936C9.5000706@aloah-from-hell.de> Hi everybody, it seems lika as amavisd-new is not capable to use a domain placeholder like %d ? Is there a specific reason for it? How did you manage if you do use your own ldap-scheme. Thanks for your help, Werner From Mark.Martinec+amavis at ijs.si Wed Feb 1 16:23:34 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Wed, 1 Feb 2012 16:23:34 +0100 Subject: amavis is not able to work please help In-Reply-To: <4F28B680.5010303@megan.vbhcs.org> References: <4F28B680.5010303@megan.vbhcs.org> Message-ID: <201202011623.34020.Mark.Martinec+amavis@ijs.si> Kshitij, > I have installed amavisd-new 2.6.4-4 rpm for rhel4 after installing all of > is dependancy i have configure the postfix and amavis properly > but also i am getting error while postfix send email to amavis for scanning > please find the error screen shot below > > [image: image.png] > please let me know what is wrong they same config is working on other > machine. Amavisd could not connect back to postfix on a port 10025, that postfix smtpd service was not available. > Noel wrote: > The postfix/master smtpd: bad command startup message suggests you > fat-fingered your postfix master.cf modifications. > > There are probably other postfix errors logged giving more information. > http://www.postfix.org/DEBUG_README.html#logging Indeed. The SEGV (signal 11) crash on a smtpd service needs to be investigated and resolved. Marl From michael.scheidell at secnap.com Wed Feb 1 17:23:40 2012 From: michael.scheidell at secnap.com (Michael Scheidell) Date: Wed, 1 Feb 2012 11:23:40 -0500 Subject: amavis is not able to work please help In-Reply-To: <201202011623.34020.Mark.Martinec+amavis@ijs.si> References: <4F28B680.5010303@megan.vbhcs.org> <201202011623.34020.Mark.Martinec+amavis@ijs.si> Message-ID: <4F29670C.2000208@secnap.com> On 2/1/12 10:23 AM, Mark Martinec wrote: > Indeed. The SEGV (signal 11) crash on a smtpd service needs > to be investigated and resolved. > good chance OP is using a different db for transport/aliases/, etc and needs to recompile the *.db files -- Michael Scheidell, CTO o: 561-999-5000 d: 561-948-2259 >*| *SECNAP Network Security Corporation * Best Mobile Solutions Product of 2011 * Best Intrusion Prevention Product * Hot Company Finalist 2011 * Best Email Security Product * Certified SNORT Integrator ______________________________________________________________________ This email has been scanned and certified safe by SpammerTrap(r). For Information please see http://www.spammertrap.com/ ______________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mark.Martinec+amavis at ijs.si Wed Feb 1 19:12:48 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Wed, 1 Feb 2012 19:12:48 +0100 Subject: LDAP-Support in amavisd-new - missing placeholder %d In-Reply-To: <4F2936C9.5000706@aloah-from-hell.de> References: <4F2936C9.5000706@aloah-from-hell.de> Message-ID: <201202011912.48517.Mark.Martinec+amavis@ijs.si> Werner, > it seems lika as amavisd-new is not capable to use a domain placeholder > like %d ? Is there a specific reason for it? How did you manage if you > do use your own ldap-scheme. There is a placeholder %m, which gets replaced by a complete chain of a progressively stripped e-mail address, e.g.: user+foo at sub.example.com user at sub.example.com user+foo user @sub.example.com @.sub.example.com @.example.com @.com @. or by setting $ldap_lookups_no_at_means_domain to true, into a: user+foo at sub.example.com user at sub.example.com user+foo@ user@ sub.example.com .sub.example.com .example.com .com . A query filter template like: (&(objectClass=amavisAccount)(mail=%m)) gets expanded into an OR of all mail=%m comparisions, e.g.: (&(objectClass=amavisAccount)(|(mail=v1)(mail=v2)...(mail=vn))) So if you only have full domains listed in LDAP, then only domains get compared, full email or subdomains will never be able to match any domain name in a database. Mark From werner at aloah-from-hell.de Wed Feb 1 19:44:46 2012 From: werner at aloah-from-hell.de (Werner Detter) Date: Wed, 01 Feb 2012 19:44:46 +0100 Subject: LDAP-Support in amavisd-new - missing placeholder %d In-Reply-To: <201202011912.48517.Mark.Martinec+amavis@ijs.si> References: <4F2936C9.5000706@aloah-from-hell.de> <201202011912.48517.Mark.Martinec+amavis@ijs.si> Message-ID: <4F29881E.5050005@aloah-from-hell.de> Hi Mark, > or by setting $ldap_lookups_no_at_means_domain to true, > into a: > > user+foo at sub.example.com > user at sub.example.com > user+foo@ > user@ > sub.example.com > .sub.example.com > .example.com > .com > . > > A query filter template like: > (&(objectClass=amavisAccount)(mail=%m)) > gets expanded into an OR of all mail=%m comparisions, e.g.: > (&(objectClass=amavisAccount)(|(mail=v1)(mail=v2)...(mail=vn))) > > So if you only have full domains listed in LDAP, then only domains > get compared, full email or subdomains will never be able to > match any domain name in a database. thanks for your detailed answer and the hint about "$ldap_lookups_no_at_means_domain". As Domains are not listed with "@" within our LDAP-Directory $ldap_lookups_no_at_means_domain could do the trick. Amazing, I will try this tomorrow. Cheers, Werner From bajodel at gmail.com Thu Feb 2 00:55:59 2012 From: bajodel at gmail.com (Amedeo Rinaldo) Date: Thu, 02 Feb 2012 00:55:59 +0100 Subject: 'X-Envelope-From' missing in 2.7.0 ? In-Reply-To: <201201281345.38553.Mark.Martinec+amavis@ijs.si> References: <4F2385B8.4020201@gmail.com> <201201281345.38553.Mark.Martinec+amavis@ijs.si> Message-ID: <4F29D10F.7040700@gmail.com> Il 28/01/2012 13:45, Mark Martinec ha scritto: > There is a Return-Path header field with exactly the same > contents as X-Envelope-From had. The former is a standard > header field for this purpose, the later was redundant. > > Mark Hi Mark, you are right and I know 'Return-Path' should be exactly the same thing ..but I've noticed some strange behaviours in my quarantined mails. Some of them have in their headers -> [ Return-Path: ] This should be impossible because of my postfix 'reject_non_fqdn_sender'; if I test this sender manually (telnet) I can see: -- -- -- 504 5.5.2 : Sender address rejected: need fully-qualified address -- -- -- It's surely my fault ..but I couldn't understand why some quarantined messages have those Return-Path if that envelope sender are not permitted. In my /var/log/mail.log I cannot find any evidence of that 'weird sender'. Amedeo From foreplayboy at gmail.com Thu Feb 2 05:51:53 2012 From: foreplayboy at gmail.com (kshitij mali) Date: Thu, 2 Feb 2012 10:21:53 +0530 Subject: amavis is not able to work please help In-Reply-To: <4F29670C.2000208@secnap.com> References: <4F28B680.5010303@megan.vbhcs.org> <201202011623.34020.Mark.Martinec+amavis@ijs.si> <4F29670C.2000208@secnap.com> Message-ID: Hi Marc, please find the below error from the postfix + amavis +++++++++++++++++++++++++++++++++++++++++++++++++++++++ Feb 1 10:21:15 D1OKH680RL postfix/master[11324]: daemon started -- version 2.7.2-RC2, configuration /etc/postfix Feb 1 10:21:39 D1OKH680RL postfix/postsuper[11331]: Requeued: 1 message Feb 1 10:21:43 D1OKH680RL postfix/pickup[11326]: C37B84A3BF: uid=502 from= orig_id=EDD8F4A3BC Feb 1 10:21:43 D1OKH680RL postfix/cleanup[11334]: C37B84A3BF: message-id=<> Feb 1 10:21:43 D1OKH680RL postfix/qmgr[11325]: C37B84A3BF: from=, size=550, nrcpt=1 (queue active) Feb 1 10:21:43 D1OKH680RL postfix/master[11324]: warning: process /usr/libexec/postfix/smtpd pid 11339 killed by signal 11 Feb 1 10:21:43 D1OKH680RL postfix/master[11324]: warning: /usr/libexec/postfix/smtpd: bad command startup -- throttling Feb 1 10:21:43 D1OKH680RL amavis[10849]: (10849-01) (!)FWD via SMTP: -> , 451 4.5.0 From MTA([127.0.0.1]:10025) during fwd-connect (Negative greeting: at (eval 72) line 596.): id=10849-01 Feb 1 10:21:44 D1OKH680RL amavis[10849]: (10849-01) Blocked MTA-BLOCKED, MYNETS LOCAL [127.0.0.1] -> , mail_id: 6U1+v+c9a2rh, Hits: -, size: 550, 186 ms Feb 1 10:21:44 D1OKH680RL postfix/smtp[11336]: C37B84A3BF: to=, relay=127.0.0.1[127.0.0.1]:10024, delay=1380, delays=1379/0.01/0.02/0.18, dsn=4.5.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.5.0 From MTA([127.0.0.1]:10025) during fwd-connect (Negative greeting: at (eval 72) line 596.): id=10849-01 (in reply to end of DATA command)) +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ main.cf file +++++++++++++++++++++++++++++++++++++++++++++++++++++ remote_header_rewrite_domain = local_header_rewrite_clients = # the server parameters section # don not forget to run the postmap command when changing relay_domains or transport_maps file # mydomain = apl.com myhostname = D1OKH680RL.$mydomain mynetworks = cidr:/etc/postfix/network_table alias_maps = hash:/etc/postfix/aliases canonical_maps = hash:/etc/postfix/canonical relay_domains = hash:/etc/postfix/relay_domains transport_maps = hash:/etc/postfix/transport message_size_limit = 20971520 append_dot_mydomain = no # Addresses translation recipient_canonical_classes = envelope_recipient, header_recipient recipient_canonical_maps = hash:/etc/postfix/recipient_canonical ################################################################# # Recipient filter # smtpd_sender_restrictions= check_sender_access hash:/etc/postfix/sender_hold smtpd_recipient_restrictions = reject_non_fqdn_recipient, check_recipient_access hash:/etc/postfix/recipient_access, permit_mynetworks, reject_unauth_destination #client filter smtpd_client_restrictions = check_client_access hash:/etc/postfix/access_client, check_client_access cidr:/etc/postfix/network_table, reject ##Queue life time change as per Lingsy 22/2/2011 bounce_queue_lifetime = 3d maximal_queue_lifetime = 3d ################################################################ # Clam Scan for Postfix - done by Augustine on 30th July 2010 ##content_filter = smtp-filter:[127.0.0.1]:10025 content_filter = smtp-amavis:[127.0.0.1]:10024 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ master.cf ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ smtp-amavis unix - - y - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 127.0.0.1:10025 inet n - y - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_address_mappings ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Regards, Kshitij On Wed, Feb 1, 2012 at 9:53 PM, Michael Scheidell < michael.scheidell at secnap.com> wrote: > ** > On 2/1/12 10:23 AM, Mark Martinec wrote: > > Indeed. The SEGV (signal 11) crash on a smtpd service needs > to be investigated and resolved. > > > good chance OP is using a different db for transport/aliases/, etc and > needs to recompile the *.db files > > > > -- > Michael Scheidell, CTO > o: 561-999-5000 > d: 561-948-2259 > > *| *SECNAP Network Security Corporation > > - Best Mobile Solutions Product of 2011 > - Best Intrusion Prevention Product > - Hot Company Finalist 2011 > - Best Email Security Product > - Certified SNORT Integrator > > > ------------------------------ > This email has been scanned and certified safe by SpammerTrap?. > For Information please see http://www.spammertrap.com/ > ------------------------------ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mark.Martinec+amavis at ijs.si Thu Feb 2 16:46:11 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Thu, 2 Feb 2012 16:46:11 +0100 Subject: amavis is not able to work please help In-Reply-To: References: <4F29670C.2000208@secnap.com> Message-ID: <201202021646.11707.Mark.Martinec+amavis@ijs.si> Kshitij, > Feb 1 10:21:43 D1OKH680RL postfix/master[11324]: warning: > /usr/libexec/postfix/smtpd: bad command startup -- throttling The problem is with your postfix installation, please followups to the postfix-users mailing list. The smavisd reported failure to forward is just a consequence of this. > main.cf file > master.cf The problem is not with your configuration files, but with the postfix program itself on your host, or with some library it uses. Mark From Mark.Martinec+amavis at ijs.si Thu Feb 2 17:26:33 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Thu, 2 Feb 2012 17:26:33 +0100 Subject: 'X-Envelope-From' missing in 2.7.0 ? In-Reply-To: <4F29D10F.7040700@gmail.com> References: <4F2385B8.4020201@gmail.com> <201201281345.38553.Mark.Martinec+amavis@ijs.si> <4F29D10F.7040700@gmail.com> Message-ID: <201202021726.33182.Mark.Martinec+amavis@ijs.si> Amedeo, > > There is a Return-Path header field with exactly the same > > contents as X-Envelope-From had. The former is a standard > > header field for this purpose, the later was redundant. > > Hi Mark, you are right and I know 'Return-Path' should be exactly the > same thing ..but I've noticed some strange behaviours in my quarantined > mails. > Some of them have in their headers -> [ Return-Path: ] > This should be impossible because of my postfix > 'reject_non_fqdn_sender'; if I test this sender manually (telnet) I can see: > -- -- -- > 504 5.5.2 : Sender address rejected: need fully-qualified > address > -- -- -- > It's surely my fault ..but I couldn't understand why some quarantined > messages have those Return-Path if that envelope sender are not > permitted. In my /var/log/mail.log I cannot find any evidence of that > 'weird sender'. ... Scratch ... ?:-/ I don't see any "Return-Path: " in our quarantined messages. Amavisd always places the Return-Path line at the top of a quarantined message. Are you seeing the "Return-Path: " as the first line, or somewhere later down in a quarantined message? Btw, what type of quarantining do you use (one file per message, one mbox file for all quarantined messages, sql, ...)? If it is indeed the first line, I suggest to find the corresponding log entry made by postfix to see what the MTA got as a sender address. Amavisd does not turn a null sender address into a "MAILER-DAEMON". The only exception is when quarantining to a unix mbox file (one file for all quarantined messages), where the mbox separator line would look like "From MAILER-DAEMON ...timestamp..." for a null sender address, but this does not affect the Return-Path line. Mark From bajodel at gmail.com Thu Feb 2 18:16:22 2012 From: bajodel at gmail.com (Amedeo Rinaldo) Date: Thu, 02 Feb 2012 18:16:22 +0100 Subject: 'X-Envelope-From' missing in 2.7.0 ? In-Reply-To: <201202021726.33182.Mark.Martinec+amavis@ijs.si> References: <4F2385B8.4020201@gmail.com> <201201281345.38553.Mark.Martinec+amavis@ijs.si> <4F29D10F.7040700@gmail.com> <201202021726.33182.Mark.Martinec+amavis@ijs.si> Message-ID: <4F2AC4E6.7070806@gmail.com> Il 02/02/2012 17:26, Mark Martinec ha scritto: > Amavisd always places the Return-Path line at the top of a quarantined > message. Are you seeing the "Return-Path:" as the > first line, or somewhere later down in a quarantined message? First line.. (snip) -- -- -- Return-Path: Delivered-To: honeypot+quarantine at example.com Received: from localhost (localhost [127.0.0.1]) by mx20.example.com (Postfix) with ESMTP id C275540781 for ; Fri, 27 Jan 2012 02:41:15 +0100 (CET) X-Envelope-To: X-Envelope-To-Blocked: X-Quarantine-ID: X-Spam-Level: ******** .. .. -- -- -- And now, following the 'C275540781' postfix queue id.. (snip) -- -- -- Jan 27 02:41:15 mx20 postfix/qmgr[24052]: C275540781: from=<>, size=4793, nrcpt=1 (queue active) -- -- -- So, now we know the sender is the "null sender". > Btw, what type of quarantining do you use (one file per message, > one mbox file for all quarantined messages, sql, ...)? one file per message > If it is indeed the first line, I suggest to find the corresponding log entry > made by postfix to see what the MTA got as a sender address. > Amavisd does not turn a null sender address into a "MAILER-DAEMON". so ..who ? :-) > The only exception is when quarantining to a unix mbox file > (one file for all quarantined messages), where the mbox separator > line would look like "From MAILER-DAEMON ...timestamp..." for a > null sender address, but this does not affect the Return-Path line. > Mark so, not my scenario. any ideas ? Amedeo From Mark.Martinec+amavis at ijs.si Thu Feb 2 20:05:44 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Thu, 2 Feb 2012 20:05:44 +0100 Subject: 'X-Envelope-From' missing in 2.7.0 ? In-Reply-To: <4F2AC4E6.7070806@gmail.com> References: <4F2385B8.4020201@gmail.com> <201202021726.33182.Mark.Martinec+amavis@ijs.si> <4F2AC4E6.7070806@gmail.com> Message-ID: <201202022005.44152.Mark.Martinec+amavis@ijs.si> Amedeo, > > Amavisd always places the Return-Path line at the top of a quarantined > > message. Are you seeing the "Return-Path:" as the > > first line, or somewhere later down in a quarantined message? > > First line.. > > (snip) > -- -- -- > Return-Path: > Delivered-To: honeypot+quarantine at example.com > Received: from localhost (localhost [127.0.0.1]) > by mx20.example.com (Postfix) with ESMTP id C275540781 > for ; Fri, 27 Jan 2012 02:41:15 > +0100 (CET) > X-Envelope-To: > X-Envelope-To-Blocked: > X-Quarantine-ID: > X-Spam-Level: ******** > -- -- -- > > And now, following the 'C275540781' postfix queue id.. > Jan 27 02:41:15 mx20 postfix/qmgr[24052]: C275540781: from=<>, > size=4793, nrcpt=1 (queue active) > > So, now we know the sender is the "null sender". > > > Btw, what type of quarantining do you use (one file per message, > > one mbox file for all quarantined messages, sql, ...)? > > one file per message > > > If it is indeed the first line, I suggest to find the corresponding log > > entry made by postfix to see what the MTA got as a sender address. > > Amavisd does not turn a null sender address into a "MAILER-DAEMON". > > so ..who ? :-) Running out of ideas. Please send me a log (at log level 5) of such event, if you can capture one. Mark From Mark.Martinec+amavis at ijs.si Thu Feb 2 20:17:20 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Thu, 2 Feb 2012 20:17:20 +0100 Subject: 'X-Envelope-From' missing in 2.7.0 ? In-Reply-To: <201202022005.44152.Mark.Martinec+amavis@ijs.si> References: <4F2385B8.4020201@gmail.com> <4F2AC4E6.7070806@gmail.com> <201202022005.44152.Mark.Martinec+amavis@ijs.si> Message-ID: <201202022017.20958.Mark.Martinec+amavis@ijs.si> Amedeo, > Running out of ideas. > Please send me a log (at log level 5) of such event, if you can capture > one. Or search by yourself (assuming $log_level = 5): fgrep 'ESMTP< MAIL FROM:' /var/log/amavisd-debug.log Mark From bajodel at gmail.com Thu Feb 2 23:44:04 2012 From: bajodel at gmail.com (Amedeo Rinaldo) Date: Thu, 02 Feb 2012 23:44:04 +0100 Subject: 'X-Envelope-From' missing in 2.7.0 ? In-Reply-To: <201202022017.20958.Mark.Martinec+amavis@ijs.si> References: <4F2385B8.4020201@gmail.com> <4F2AC4E6.7070806@gmail.com> <201202022005.44152.Mark.Martinec+amavis@ijs.si> <201202022017.20958.Mark.Martinec+amavis@ijs.si> Message-ID: <4F2B11B4.9080400@gmail.com> Il 02/02/2012 20:17, Mark Martinec ha scritto: >> Running out of ideas. >> Please send me a log (at log level 5) of such event, if you can capture >> one. > > Or search by yourself (assuming $log_level = 5): > fgrep 'ESMTP< MAIL FROM:' /var/log/amavisd-debug.log > Mark First of all ..thanks of your time.. The issue came out because randomly I've had the need to release a quarantine message ..you can easily understand that the release process (custom script) could not invent a different sender. :-) Before I've always used the 'X-Envelope-From' header line without any problems (this is why I was asking for it in this thread). Anyway, I've already edited my little release-script in order to use 'Return-Path' line.. and furthermore.. maybe this is not the real source of my little issue. Mark.. I can try, but first I need to find a better way to capture it without "log-level5-debug" all traffic for weeks! :) .. stay tuned ;-) Amedeo. From Mark.Martinec+amavis at ijs.si Fri Feb 3 02:24:00 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Fri, 3 Feb 2012 02:24:00 +0100 Subject: 'X-Envelope-From' missing in 2.7.0 ? In-Reply-To: <4F2B11B4.9080400@gmail.com> References: <4F2385B8.4020201@gmail.com> <201202022017.20958.Mark.Martinec+amavis@ijs.si> <4F2B11B4.9080400@gmail.com> Message-ID: <201202030224.00537.Mark.Martinec+amavis@ijs.si> Amedeo, Actually, the envelope sender address is also reflected in the main log entry. So what was the amavisd log line corresponding to the problem message - was it a 'Blocked ... <> -> ...' or a 'Blocked ... -> ...' ? > I can try, but first I need to find a better way to capture it > without "log-level5-debug" all traffic for weeks! :) > .. stay tuned ;-) The 2.8.0-pre* brings just such a feature - assuming the trouble can be detected some time before the end of processing of a message. It may be an overkill for your purpose, but is perhaps worth considering. Release notes say (slightly rephrased from the last -pre): NEW FEATURES - as a debugging aid it is now possible that a late event triggers full logging of earlier events that occurred during processing of a current mail message. This is implemented by writing all log events to a temporary file regardless of their log level and of the current $log_level setting. A later event can cause the captured temporary log to be copied to a regular log. Each child process keeps its own temporary log file open all the time, the file is rewound and truncated after each mail message processing and reused for the next capture, so its size rarely exceeds about 50 kB. Maintaining a temporary capture log is enabled by setting a configuration variable $enable_log_capture to true: $enable_log_capture = 1; Enabling a log capture costs a little bit of resources as amavisd needs to assemble and format all log messages regardless of their log level, not benefiting from early pruning of log entries not reaching the $log_level. Nevertheless the small overhead is quite acceptable when troubleshooting some rarely occurring problem and keeping $log_level permanently at the max is not acceptable due to sheer volume of debug logging. The captured log is read from a temporary file and copied to a regular log as log level 1 entries (i.e. at LOG_INFO syslog priority) if a dynamic variable $enable_log_capture_dump is true by the end of mail message processing. A chunk of captured log entries is preceded/ended by a log line: CAPTURED DEBUG LOG DUMP BEGINS CAPTURED DEBUG LOG DUMP ENDS and each such log entry has a prepended timestamp (hours, minutes, seconds with milliseconds) of a capture time. The $enable_log_capture_dump variable can be turned on directly by some debugging patch code, but is more conveniently loaded by activating a policy bank, e.g.: $policy_bank{'SLOW'} = { enable_log_capture_dump => 1, }; $policy_bank{'GOTCHA'} = { enable_log_capture_dump => 1, }; which can be loaded for example by a custom hook, e.g.: sub after_send { my($self,$conn,$msginfo) = @_; if (Time::HiRes::time - $msginfo->rx_time > 5.5) { Amavis::load_policy_bank('SLOW', $msginfo); } # or perhaps: if ($msginfo->sender =~ /some-regexp/) { Amavis::load_policy_bank('GOTCHA', $msginfo); } } Btw, the only purpose of having two different policy banks in the example is to be able to see at a glance in the log which one was activated. Mark From bajodel at gmail.com Fri Feb 3 09:54:14 2012 From: bajodel at gmail.com (Amedeo Rinaldo) Date: Fri, 03 Feb 2012 09:54:14 +0100 Subject: 'X-Envelope-From' missing in 2.7.0 ? In-Reply-To: <201202030224.00537.Mark.Martinec+amavis@ijs.si> References: <4F2385B8.4020201@gmail.com> <201202022017.20958.Mark.Martinec+amavis@ijs.si> <4F2B11B4.9080400@gmail.com> <201202030224.00537.Mark.Martinec+amavis@ijs.si> Message-ID: <4F2BA0B6.8040706@gmail.com> Il 03/02/2012 02:24, Mark Martinec ha scritto: > Actually, the envelope sender address is also reflected in the > main log entry. So what was the amavisd log line corresponding > to the problem message - was it a 'Blocked ...<> -> ...' > or a 'Blocked ... -> ...' ? you are right! (as always) ;) (snip) -- -- -- Jan 27 02:41:15 ..[cut].. Blocked SPAM ..[cut].. <> -> -- -- -- Is this (main log '<>') address exactly_the/corresponding_to 'Return-Path' macro (when quarantined)? Anyway.. scanning my quarantine (14d) I've found only 4 items of [ Return-Path: ] (first line). On that host I've actually about 10k quarantined messages, ..nothing to be really worry about :-) >> I can try, but first I need to find a better way to capture it >> without "log-level5-debug" all traffic for weeks! :) >> .. stay tuned ;-) > > The 2.8.0-pre* brings just such a feature - assuming the trouble can > be detected some time before the end of processing of a message. It may > be an overkill for your purpose, but is perhaps worth considering. >..[cut].. > Mark Good tools for debugging are always welcome!! ;-) Thanks. Amedeo. From maf at eurotux.com Fri Feb 3 15:42:54 2012 From: maf at eurotux.com (Miguel Fernandes) Date: Fri, 03 Feb 2012 14:42:54 +0000 Subject: Sophos - savid: Client terminated connection early Message-ID: <4F2BF26E.1050303@eurotux.com> Hi all! I wondered if anyone has come across this: I'm trying to use the new Sophos-SSSP implementation using the savdid daemon: my amavis.conf: ### http://www.sophos.com/ ['Sophos-SSSP', \&ask_daemon, ["{}", 'sssp:/var/run/savdi/sssp.sock'], qr/^DONE OK\b/m, qr/^VIRUS\b/m, qr/^VIRUS\s*(\S*)/m ], All looks good, until first connect: savid: 120202:174217 [4F2AC9BD] 00038402 New session 120202:174227 [4F2AC9BD] C000460B Client terminated connection early 120202:174227 [4F2AC9BD] 00038403 Session ended amavis: Feb 2 16:19:48 etmx-v3-pre amavis[14700]: (14700-01) (!)Sophos-SSSP av-scanner FAILED: run_av error: timed out\n Tryed various alternatives, using Unix sockets, TCP and I always get the timeout message on the savid daemon... Looks like savid is waiting for amavis to send information and gives after some time... (did a strace on the savid daemon) Is there something I'm forgetting? For now I will be still using SOPHIE, that is working fine. Thanks! -- Cumprimentos, Miguel Fernandes -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mark.Martinec+amavis at ijs.si Fri Feb 3 17:00:23 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Fri, 3 Feb 2012 17:00:23 +0100 Subject: Sophos - savid: Client terminated connection early In-Reply-To: <4F2BF26E.1050303@eurotux.com> References: <4F2BF26E.1050303@eurotux.com> Message-ID: <201202031700.23796.Mark.Martinec+amavis@ijs.si> Miguel, > I wondered if anyone has come across this: > I'm trying to use the new Sophos-SSSP implementation using the savdid > daemon: > > my amavis.conf: > ### http://www.sophos.com/ > ['Sophos-SSSP', > \&ask_daemon, ["{}", 'sssp:/var/run/savdi/sssp.sock'], > qr/^DONE OK\b/m, qr/^VIRUS\b/m, qr/^VIRUS\s*(\S*)/m ], > > All looks good, until first connect: > > savid: > 120202:174217 [4F2AC9BD] 00038402 New session > 120202:174227 [4F2AC9BD] C000460B Client terminated connection early > 120202:174227 [4F2AC9BD] 00038403 Session ended > > amavis: > Feb 2 16:19:48 etmx-v3-pre amavis[14700]: (14700-01) (!)Sophos-SSSP > av-scanner FAILED: run_av error: timed out\n > > Tryed various alternatives, using Unix sockets, TCP and I always get the > timeout message on the savid daemon... > Looks like savid is waiting for amavis to send information and gives > after some time... (did a strace on the savid daemon) > > Is there something I'm forgetting? > > For now I will be still using SOPHIE, that is working fine. Try it manually first. The protocol is pretty straightforward, best to test it over a TCP connection, e.g: $ telnet 127.0.0.1 4010 OK SSSP/1.0 SSSP/1.0 SCANDIRR /tmp ACC 4F2C037F/1 [...] DONE FAIL 0210 Could not open item passed to SAVI for scanning BYE BYE Connection closed by foreign host. Mark From cantrell at chara.gsu.edu Wed Feb 8 16:33:20 2012 From: cantrell at chara.gsu.edu (Justin Cantrell) Date: Wed, 08 Feb 2012 10:33:20 -0500 Subject: invalid header: all-whitespace header field Message-ID: <4F3295C0.2080506@chara.gsu.edu> I have a Debian Squeeze mail server that was recently updated from Etch > Lenny > Squeeze. After the updates amavis is sending messages to quarantine. I can't seem to figure out why! Error: INVALID HEADER: FOLDED HEADER FIELD MADE UP ENTIRELY OF WHITESPACE My /etc/amavis/conf.d/50-user has $final_bad_header_destiny = D_PASS; $bad_header_quarantine_to = undef; So it SHOULD be passing them. Any idea on what is going on here? Other info. Running Postfix/Dovecot with Amavis/Clam/Spamassassin. From Mark.Martinec+amavis at ijs.si Thu Feb 9 17:09:22 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Thu, 9 Feb 2012 17:09:22 +0100 Subject: Warn virus recipients only for non spam mail? In-Reply-To: <4F2918B1.20109@atlas-elektronik.com> References: <4F2918B1.20109@atlas-elektronik.com> Message-ID: <201202091709.22174.Mark.Martinec+amavis@ijs.si> Michael, > is it possible just to warn recipients of virus mails only if SA score > is below $sa_kill_level_deflt?? > > $warnvirusrecip =1 send a notify on all virus mails to recipient. This isn't possible off-the-shelf, some hack is needed. Try the attached patch for 2.7.0. Amavis normally considers infected mail as the top reason to block it. Spam scanning is not even performed for infected mail, partly to save resources, and partly to offer some small protection against malware to spam scanner(s). So the attached patch needs to overcome the above performance+security measure, and then disable $warnrecip_maps_by_ccat{&CC_VIRUS} when CC_SPAM is also detected as a contents category. This patch will *not* find its way into the next version, as I think it goes against the purpose of recipient notifications (if these still have any purpose at all). I think there may be better ways of dealing with infected spam. Note that a spam scanner often recognize infected mail as spam, which probably invalidates the intention of having virus recipient notifications enabled. Perhaps the reason behind your wish is that some virus scanners declare spam as infection. There is a better way to deal with this, by using @virus_name_to_spam_score_maps . Mark -------------- next part -------------- A non-text attachment was scrubbed... Name: 0.patch Type: text/x-patch Size: 888 bytes Desc: not available URL: From Mark.Martinec+amavis at ijs.si Thu Feb 9 17:20:56 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Thu, 9 Feb 2012 17:20:56 +0100 Subject: invalid header: all-whitespace header field In-Reply-To: <4F3295C0.2080506@chara.gsu.edu> References: <4F3295C0.2080506@chara.gsu.edu> Message-ID: <201202091720.56266.Mark.Martinec+amavis@ijs.si> Justin, > Running Postfix/Dovecot with Amavis/Clam/Spamassassin. > I have a Debian Squeeze mail server that was recently updated from > Etch > Lenny > Squeeze. After the updates amavis is sending messages > to quarantine. I can't seem to figure out why! > Error: INVALID HEADER: FOLDED HEADER FIELD MADE UP ENTIRELY OF WHITESPACE > My /etc/amavis/conf.d/50-user has > $final_bad_header_destiny = D_PASS; > $bad_header_quarantine_to = undef; > > So it SHOULD be passing them. Any idea on what is going on here? If you find these message in a quarantine, this does not automatically mean they were not also passed on. Check the log. Are you sure the message was not forwarded to a recipient? Indeed the $bad_header_quarantine_to=undef should have disabled quarantining of bad header messages - unless you have some of its cousin settings configured otherwise (@bad_header_quarantine_to_maps, %quarantine_method_by_ccat, SQL field 'bad_header_quarantine_to' or its LDAP counterpart). Or perhaps the message was also spam or infected, on top of having a bad header, in which case other settings for a more relevant contents category apply. The definive answer lies in your log at level 5 when the event happens. Check it by yourself, or mail it to me, or post it someplace. Mark From eric at techsoft3d.com Thu Feb 9 21:20:09 2012 From: eric at techsoft3d.com (Eric Smith) Date: Thu, 9 Feb 2012 20:20:09 +0000 Subject: segfault amavis Message-ID: Hi All, Have this segfault 4 times Monday. amavisd-new[10969] general protection ip:7fb92996f547 sp:7fff784a26c8 error:0 in libdb-4.8.so[7fb9298be000+16a000] It zombies amavis, have to kill -9 all amavisd's, restart postfix and amavis from init.d scripts. No particular email in the log seems to be the problem, amavis just dies. As a work around I have changed this today: $enable_db = 0; # enable use of BerkeleyDB/libdb (SNMP and nanny) Here is the setup: ubuntu 10.4 LTS with standard packages amavisd-new 1:2.6.4-1ubuntu5 libdb4.8 4.8.24-1ubuntu1 spamassassin 3.3.1-1 clamav 0.96.5+dfsg-1ubuntu1.10.04.2 pyzor 1:0.5.0-0ubuntu2 razor 1:2.85-3 postfix 2.8.1-1~lucid1 General thoughts? No changes on the server that may have cause this, corrupt DB? Or is this indicative of something else? Thanks eric Eric Smith Senior Network Administrator | Tech Soft 3D http://www.techsoft3d.com skype: eric_ae_smith phone: 510-333-1729 Build with the Best From quanah at zimbra.com Thu Feb 9 21:39:55 2012 From: quanah at zimbra.com (Quanah Gibson-Mount) Date: Thu, 09 Feb 2012 12:39:55 -0800 Subject: segfault amavis In-Reply-To: References: Message-ID: <7B7343A1A562FC5F91AA3FC1@[192.168.1.30]> --On Thursday, February 09, 2012 8:20 PM +0000 Eric Smith wrote: > Hi All, > > Have this segfault 4 times Monday. > > amavisd-new[10969] general protection ip:7fb92996f547 sp:7fff784a26c8 > error:0 in libdb-4.8.so[7fb9298be000+16a000] > > It zombies amavis, have to kill -9 all amavisd's, restart postfix and > amavis from init.d scripts. No particular email in the log seems to be > the problem, amavis just dies. > Sounds like there is a problem with your Perl BDB module. --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration From eric at techsoft3d.com Thu Feb 9 22:50:09 2012 From: eric at techsoft3d.com (Eric Smith) Date: Thu, 9 Feb 2012 21:50:09 +0000 Subject: segfault amavis In-Reply-To: <7B7343A1A562FC5F91AA3FC1@[192.168.1.30]> References: <7B7343A1A562FC5F91AA3FC1@[192.168.1.30]> Message-ID: <2A4FFD10-F455-4FF7-AC37-0CA021102693@techsoft3d.com> humm, yes that looks like a candidate but no change in that either and no new package as well. Why monday and not the many months before? I know, too dang general of a question to ask, when I have the time I will try to get it to fail while I am looking at it with logs set to debug. In the mean time I will keep the setting $enable_db set to 0. Question, what downside is there not using berkeleyDB? amavisd_nanny will not work, is there any other performance issues that I should worry about? Thank you for your time! eric Eric Smith Senior Network Administrator | Tech Soft 3D http://www.techsoft3d.com skype: eric_ae_smith phone: 510-333-1729 Build with the Best On Feb 9, 2012, at 12:39 PM, Quanah Gibson-Mount wrote: > --On Thursday, February 09, 2012 8:20 PM +0000 Eric Smith wrote: > >> Hi All, >> >> Have this segfault 4 times Monday. >> >> amavisd-new[10969] general protection ip:7fb92996f547 sp:7fff784a26c8 >> error:0 in libdb-4.8.so[7fb9298be000+16a000] >> >> It zombies amavis, have to kill -9 all amavisd's, restart postfix and >> amavis from init.d scripts. No particular email in the log seems to be >> the problem, amavis just dies. >> > > Sounds like there is a problem with your Perl BDB module. > > --Quanah > > -- > > Quanah Gibson-Mount > Sr. Member of Technical Staff > Zimbra, Inc > A Division of VMware, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration From p at state-of-mind.de Thu Feb 9 23:07:06 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Thu, 9 Feb 2012 23:07:06 +0100 Subject: segfault amavis In-Reply-To: <2A4FFD10-F455-4FF7-AC37-0CA021102693@techsoft3d.com> References: <7B7343A1A562FC5F91AA3FC1@[192.168.1.30]> <2A4FFD10-F455-4FF7-AC37-0CA021102693@techsoft3d.com> Message-ID: <20120209220706.GJ2264@state-of-mind.de> * Eric Smith : > humm, yes that looks like a candidate but no change in that either and no > new package as well. Why monday and not the many months before? I know, too > dang general of a question to ask, when I have the time I will try to get it > to fail while I am looking at it with logs set to debug. In the mean time I > will keep the setting $enable_db set to 0. > > Question, what downside is there not using berkeleyDB? > > amavisd_nanny will not work, is there any other performance issues that I > should worry about? Not with recent amavis. Marc faded out some knowledge sharing between amavis child processes over a berkeleyDB in 2.7, because it has become ineffetive over time. There's no other feature coming to my mind - at least mine! p at rick -- All technical questions asked privately will be automatically answered on the list and archived for public access unless privacy is explicitely required and justified. saslfinger (debugging SMTP AUTH): From Mark.Martinec+amavis at ijs.si Thu Feb 9 23:14:52 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Thu, 9 Feb 2012 23:14:52 +0100 Subject: segfault amavis In-Reply-To: <2A4FFD10-F455-4FF7-AC37-0CA021102693@techsoft3d.com> References: <7B7343A1A562FC5F91AA3FC1@[192.168.1.30]> <2A4FFD10-F455-4FF7-AC37-0CA021102693@techsoft3d.com> Message-ID: <201202092314.52360.Mark.Martinec+amavis@ijs.si> Eric, > humm, yes that looks like a candidate but no change in that either and no > new package as well. Why monday and not the many months before? I know, > too dang general of a question to ask, when I have the time I will try to > get it to fail while I am looking at it with logs set to debug. In the > mean time I will keep the setting $enable_db set to 0. Or try upgrading libdb to a more recent version - we are now at 5.2.36 (not to forget reinstalling BerkeleyDB perl module when libdb is changed). > Question, what downside is there not using berkeleyDB? > > amavisd_nanny will not work, is there any other performance issues that > I should worry about? None other than amavisd_nanny, amavisd-agent and amavisd-snmp-subagent will not be able to get their information. Btw, I'm working on an alternative to bdb - the ZeroMQ message passing library. My primary motive was to avoid lock contention on updating a database, which has serious performance implications when there are lots of amavisd child processes running, especially when spam checks are disabled. Looks like a way to go, it's very fast! Mark From quanah at zimbra.com Thu Feb 9 23:21:32 2012 From: quanah at zimbra.com (Quanah Gibson-Mount) Date: Thu, 09 Feb 2012 14:21:32 -0800 Subject: segfault amavis In-Reply-To: <201202092314.52360.Mark.Martinec+amavis@ijs.si> References: <7B7343A1A562FC5F91AA3FC1@[192.168.1.30]> <2A4FFD10-F455-4FF7-AC37-0CA021102693@techsoft3d.com> <201202092314.52360.Mark.Martinec+amavis@ijs.si> Message-ID: <7BA25EB686E8F7BD0782F3E0@[192.168.1.30]> --On Thursday, February 09, 2012 11:14 PM +0100 Mark Martinec wrote: > Eric, > >> humm, yes that looks like a candidate but no change in that either and no >> new package as well. Why monday and not the many months before? I know, >> too dang general of a question to ask, when I have the time I will try to >> get it to fail while I am looking at it with logs set to debug. In the >> mean time I will keep the setting $enable_db set to 0. > > Or try upgrading libdb to a more recent version - we are now at 5.2.36 > (not to forget reinstalling BerkeleyDB perl module when libdb is changed). > >> Question, what downside is there not using berkeleyDB? >> >> amavisd_nanny will not work, is there any other performance issues that >> I should worry about? > > None other than amavisd_nanny, amavisd-agent and amavisd-snmp-subagent > will not be able to get their information. Btw, I'm working on an > alternative to bdb - the ZeroMQ message passing library. My primary > motive was to avoid lock contention on updating a database, which > has serious performance implications when there are lots of amavisd > child processes running, especially when spam checks are disabled. > Looks like a way to go, it's very fast! Hi Mark, That's great news. Please let me know if there's anything I can do in helping to test that. We would like to remove all dependencies on BDB from our product, and amavis is the last thing we use that requires it. That's why I was asking about OpenLDAP's MDB database as an alternative earlier. ;) --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration From eric at techsoft3d.com Fri Feb 10 00:11:37 2012 From: eric at techsoft3d.com (Eric Smith) Date: Thu, 9 Feb 2012 23:11:37 +0000 Subject: segfault amavis In-Reply-To: <201202092314.52360.Mark.Martinec+amavis@ijs.si> References: <7B7343A1A562FC5F91AA3FC1@[192.168.1.30]> <2A4FFD10-F455-4FF7-AC37-0CA021102693@techsoft3d.com> <201202092314.52360.Mark.Martinec+amavis@ijs.si> Message-ID: <4F6D7B61-B055-40B5-B440-2C0C5FDD3E39@techsoft3d.com> thanks, will let it be the time being. Great program BTH, been using it for years and this is the first issue. best eric Eric Smith Senior Network Administrator | Tech Soft 3D http://www.techsoft3d.com skype: eric_ae_smith phone: 510-333-1729 Build with the Best On Feb 9, 2012, at 2:14 PM, Mark Martinec wrote: > Eric, > >> humm, yes that looks like a candidate but no change in that either and no >> new package as well. Why monday and not the many months before? I know, >> too dang general of a question to ask, when I have the time I will try to >> get it to fail while I am looking at it with logs set to debug. In the >> mean time I will keep the setting $enable_db set to 0. > > Or try upgrading libdb to a more recent version - we are now at 5.2.36 > (not to forget reinstalling BerkeleyDB perl module when libdb is changed). > >> Question, what downside is there not using berkeleyDB? >> >> amavisd_nanny will not work, is there any other performance issues that >> I should worry about? > > None other than amavisd_nanny, amavisd-agent and amavisd-snmp-subagent > will not be able to get their information. Btw, I'm working on an > alternative to bdb - the ZeroMQ message passing library. My primary > motive was to avoid lock contention on updating a database, which > has serious performance implications when there are lots of amavisd > child processes running, especially when spam checks are disabled. > Looks like a way to go, it's very fast! > > Mark From maf at eurotux.com Fri Feb 10 17:19:59 2012 From: maf at eurotux.com (Miguel Fernandes) Date: Fri, 10 Feb 2012 16:19:59 +0000 Subject: Priority on white/black lists In-Reply-To: <4EF0BDD6.5050907@eurotux.com> References: <4EEA13F0.6000204@eurotux.com> <4EEAAA49.3010506@secnap.com> <4EEB5BA6.8030906@eurotux.com> <201112162053.57730.Mark.Martinec+amavis@ijs.si> <4EF0BDD6.5050907@eurotux.com> Message-ID: <4F3543AF.6040800@eurotux.com> Hi! Just to correct myself: (...) preceded with +(*BLACKLISTING*) or - (*WHITELISTING*) And not the other way around, sorry. On 12/20/2011 04:54 PM, Miguel Fernandes wrote: > Hi Mark! > > Replacing the wblist.wb with numeric values preceded with > +(whitelisting) or - (blacklisting), will work for me :) > > Thanks. > > > > On 12/16/2011 07:53 PM, Mark Martinec wrote: >> Miguel, >> >>> I need to have some sort of priority in black/white lists, but >>> currently >>> this field is in the mailaddr table..., >>> so I can't have different priorities per recipient right? >> I'm not sure I understand the problem. A wblist entry corresponds >> to a sender& recipient pair. Why would you want to have both >> a 'W' and a 'B' entry for the same exact pair? >> >> Note that both the sender and the recipient can be either an exact >> match, or any generalization on a domain and subdomain. >> These are sorted by priority, so a more specific match should override >> a more general match. In this sense you can have even now >> multiple wblist entries matching a sender and a recipient, at different >> levels of loose matching - the most specific match wins, assuming >> the priority field is used as suggested in the examples. >> >> >> README.lookups: >> >> SQL lookups (e.g. for user+foo at example.com) are performed in order >> which is usually specified by 'ORDER BY...DESC' in the SELECT statement; >> otherwise the order is unspecified, which is only useful if just >> specific >> entries exist in a database (e.g. full address always, not domain >> part only >> or mailbox parts only). >> >> The following order (implemented by sorting on the 'priority' field >> in DESCending order, zero is low priority) is recommended, to follow >> the same specific-to-general principle as in other lookup tables; >> the first column is a suggested priority (the exact value does not >> matter >> as long as the order is maintained): >> >> 9 - lookup for user+foo at sub.example.com >> 8 - lookup for user at sub.example.com (only if $recipient_delimiter is >> '+') >> 7 - lookup for user+foo (only if domain part is local) >> 6 - lookup for user (only local; only if $recipient_delimiter is >> '+') >> 5 - lookup for @sub.example.com >> 3 - lookup for @.sub.example.com >> 2 - lookup for @.example.com >> 1 - lookup for @.com >> 0 - lookup for @. (catchall) >> >> >>> I'm thinking on creating a priority field in the table wblist, to >>> have a >>> priority per wblist entry, mainly to prioritize white lists. >>> Can this this be achieved by some other way? >> Sure, if you want. Adjust the $sql_select_white_black_list accordingly. >> I just don't see a point in doing so. >> >>> How does a domain w/b list affect that domain's recipients? >> wblist entries are per-recipient (i.e. keyed by a sender& recipient >> pair) >> >>> How can we know the priority order in the case we have both >>> domain w/b lists and recipient's w/b lists? >> They are sorted by mailaddr.priority, the most specific address >> should have the highest priority. >> >> >> Michael Scheidell writes: >>> actually, the entry in the wb field doesn't need to be a 'w' or a 'b' >>> (if I remember correctly) >>> you can set it as a -100 for whitelist, and, say, +50 in b. >> Correct. Hard w/b-listing is turned into a soft-wblisting this way. >> >> >>> I've been digging a litle more and: >>> >>> on /usr/sbin/amavisd >>> >>> $sql_select_white_black_list = >>> 'SELECT wb FROM wblist JOIN mailaddr ON wblist.sid=mailaddr.id'. >>> ' WHERE wblist.rid=? AND mailaddr.email IN (%k)'. >>> ' ORDER BY mailaddr.priority DESC'; >>> >>> In only uses mailaddr.priority for sorting, so I think there wblist.wb >>> is not expecting a numeric value... >> amavisd-new-20040701 / amavisd-new-2.0 release notes : >> >> - extended semantics of SQL field wblist.wb, which can hold a score >> value >> boost, which is interpreted as soft black/white-listing (the same >> semantics >> as the value in @score_sender_maps); >> >> >> Mark > > -- Cumprimentos, Miguel Fernandes -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdavies at sdc.com.au Sun Feb 12 07:46:41 2012 From: sdavies at sdc.com.au (Stephen Davies) Date: Sun, 12 Feb 2012 17:16:41 +1030 Subject: Virus notification question Message-ID: <201202121716.41933.sdavies@sdc.com.au> Following is an excerpt from my mail log when a virus is detected by amavisd- new-2.7.0 and amavisd-milter-1.5.0. What might be wrong with my config? Where should I look in the doco? (Google does not know.) My config includes: $inet_socket_port = 10026; # listen on this local TCP port(s) $notify_method = 'smtp:[127.0.0.1]:10026'; Cheers and thanks, Stephen Feb 12 12:22:28 mustang sendmail[12522]: q1C1qKLZ012522: from=, size=403571, class=0, nrcpts=1, msgid=<20120210184619.2400.qmail at web.hostek.ru>, proto=ESMTP, daemon=MTA, relay=cp.hostek.ru [81.176.77.242] Feb 12 12:22:28 mustang amavis[9071]: (09071) Request: AM.PDP /var/amavis/tmp/afq1C1qKLZ012522: -> Feb 12 12:22:28 mustang amavis[9071]: (09071) Checking: cM4dH5-zzumM AM.PDP- SOCK [81.176.77.242] -> Feb 12 12:22:28 mustang amavis[9071]: (09071) WARN: MIME::Parser error: part did not end with expected boundary Feb 12 12:22:28 mustang amavis[9071]: (09071) p.path BANNED:1 sdc at sdc.com.au: "P=p003,L=1,M=multipart/mixed | P=p002,L=1/2,M=application/octet- stream,T=zip,N=FedEx_Invoice.zip | P=p004,L=1/2/1,T=exe,T=exe- ms,N=FedEx_Invoice.exe", matching_key="(?-xism:^\\.(exe-ms|dll)$)" Feb 12 12:22:28 mustang clamd[1731]: /var/amavis/tmp/afq1C1qKLZ012522/parts/p005: Email.Trojan.GZC FOUND Feb 12 12:22:28 mustang amavis[9071]: (09071) local delivery: <> -> virus- quarantine, mbx=/var/virusmails/virus-cM4dH5-zzumM Feb 12 12:22:28 mustang amavis[9070]: (09070-04) ESMTP::10026 /var/amavis/tmp/amavis-20120212T122228-09070-52jpecv4: -> ENVID=AM.09071.20120212T015228Z at mustang.sdc.com.au Received: from localhost ([127.0.0.1]) by localhost (mustang.sdc.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP for ; Sun, 12 Feb 2012 12:22:28 +1030 (CST) Feb 12 12:22:28 mustang amavis[9070]: (09070-04) Checking: BciVU7H-sYYI ORIGINATING -> Feb 12 12:22:29 mustang amavis[9070]: (09070-04) Passed CLEAN {AcceptedInternal}, ORIGINATING -> , Message-ID: , mail_id: BciVU7H-sYYI, Hits: 0, size: 2410, 868 ms Feb 12 12:22:29 mustang amavis[9070]: (09070-04) (!!)TROUBLE in process_request: NOT ALL RECIPIENTS DONE, EMPTY DELIVERY_METHOD! at (eval 109) line 971, line 78. -- ============================================================================= Stephen Davies Consulting P/L Voice: 08-8177 1595 Adelaide, South Australia. Fax : 08-8177 0133 Records & Collections Management. Mobile:040 304 0583 From andreas.schulze at datev.de Mon Feb 13 07:06:58 2012 From: andreas.schulze at datev.de (Andreas Schulze) Date: Mon, 13 Feb 2012 07:06:58 +0100 Subject: Virus notification question In-Reply-To: <201202121716.41933.sdavies@sdc.com.au> References: <201202121716.41933.sdavies@sdc.com.au> Message-ID: <20120213060658.GA18108@spider.services.datevnet.de> Am 12.02.2012 17:16 schrieb Stephen Davies: > What might be wrong with my config? looks like you check the amavis generated alert mail with amavis !? > $inet_socket_port = 10026; # listen on this local TCP port(s) > $notify_method = 'smtp:[127.0.0.1]:10026'; with postfix as MTA add "-o smtpd_milters=" to the smtp server at port 10026 -- Andreas Schulze Internetdienste | P252 DATEV eG 90329 N?rnberg | Telefon +49 911 319-0 | Telefax +49 911 319-3196 E-Mail info @datev.de | Internet www.datev.de Sitz: 90429 N?rnberg, Paumgartnerstr. 6-14 | Registergericht N?rnberg, GenReg Nr.70 Vorstand Prof. Dieter Kempf (Vorsitzender) Dipl.-Kfm. Wolfgang Stegmann (stellvertretender Vorsitzender) Dipl.-Kfm. Michael Leistenschneider Dipl.-Kfm. Dr. Robert Mayr J?rg Rabe v. Pappenheim Dipl.-Vw. Eckhard Schwarzer Vorsitzender des Aufsichtsrates: Reinhard Verholen From sdavies at sdc.com.au Mon Feb 13 07:31:11 2012 From: sdavies at sdc.com.au (Stephen Davies) Date: Mon, 13 Feb 2012 17:01:11 +1030 Subject: Virus notification question In-Reply-To: <20120213060658.GA18108@spider.services.datevnet.de> References: <201202121716.41933.sdavies@sdc.com.au> <20120213060658.GA18108@spider.services.datevnet.de> Message-ID: <201202131701.11301.sdavies@sdc.com.au> Thanks Andreas but I am using sendmail rather than postfix. The "smtp server" at 10026 is amavisd. Cheers, Stephen On Mon, 13 Feb 2012 04:36:58 PM Andreas Schulze wrote: > Am 12.02.2012 17:16 schrieb Stephen Davies: > > What might be wrong with my config? > > looks like you check the amavis generated alert mail with amavis !? > > > $inet_socket_port = 10026; # listen on this local TCP port(s) > > $notify_method = 'smtp:[127.0.0.1]:10026'; > > with postfix as MTA add "-o smtpd_milters=" to the smtp server at port > 10026 -- ============================================================================= Stephen Davies Consulting P/L Voice: 08-8177 1595 Adelaide, South Australia. Fax : 08-8177 0133 Records & Collections Management. Mobile:040 304 0583 From Mark.Martinec+amavis at ijs.si Mon Feb 13 16:39:42 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Mon, 13 Feb 2012 16:39:42 +0100 Subject: invalid header: all-whitespace header field In-Reply-To: <4F359105.5050208@chara.gsu.edu> References: <4F3295C0.2080506@chara.gsu.edu> <201202091720.56266.Mark.Martinec+amavis@ijs.si> <4F359105.5050208@chara.gsu.edu> Message-ID: <201202131639.42795.Mark.Martinec+amavis@ijs.si> Justin, > >> Running Postfix/Dovecot with Amavis/Clam/Spamassassin. > >> I have a Debian Squeeze mail server that was recently updated from > >> Etch > Lenny > Squeeze. After the updates amavis is sending messages > >> to quarantine. I can't seem to figure out why! Which version of amavisd is that? A 2.6.4 ? > I have tried to sanitize my logs. myuser at mymail.somewhere.com is my > user. sender at somewhere.com is the sender who's mail is being blocked. > Here is a loglevel 5 event: Thanks. This doesn't indicate the message was quarantined. But it does tell the message was bounced due to bad header (empty line in a header section). I don't see why the *destiny was D_BOUNCE, I can only guess it is set to D_BOUNCE somewhere in you configuration files. Check for final_destiny_by_ccat and final_bad_header_destiny. Check also for the setting of $warnbadhsender. $ egrep 'destiny|warnbadh|warnsender' \ /usr/share/amavis/conf.d/* /etc/amavis/conf.d/* Mark From Mark.Martinec+amavis at ijs.si Mon Feb 13 17:43:24 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Mon, 13 Feb 2012 17:43:24 +0100 Subject: invalid header: all-whitespace header field In-Reply-To: References: <4F3295C0.2080506@chara.gsu.edu> <201202131639.42795.Mark.Martinec+amavis@ijs.si> Message-ID: <201202131743.24045.Mark.Martinec+amavis@ijs.si> Justin, > > I don't see why the *destiny was D_BOUNCE, I can only guess > > it is set to D_BOUNCE somewhere in you configuration files. > > Check for final_destiny_by_ccat and final_bad_header_destiny. > > Check also for the setting of $warnbadhsender. > > This may be the issue? I can't find the documentation of these CC_BADH > settings, but I can only assume I should have them as D_PASS? > > %final_destiny_by_ccat = ( > CC_VIRUS, D_DISCARD, > CC_SPAM, D_DISCARD, > CC_BANNED, D_BOUNCE, > CC_OVERSIZED, D_BOUNCE, > CC_BADH.',4', D_BOUNCE, > CC_BADH.',3', D_BOUNCE, > CC_BADH, D_PASS, > CC_UNCHECKED, D_PASS, > CC_CLEAN, D_PASS, > CC_CATCHALL, D_PASS, > ); Exactly, the: CC_BADH.',4', D_BOUNCE entry is the culprit, it tells that a bad header due to a minor contents category 4 (which is an 'empty line in header') should be bounced. Similarly for minor ccat 3, which is a 'NULL or CR character in header'. You probably don't want these. Just remove the CC_BADH.',4' and CC_BADH.',3' entries from %final_destiny_by_ccat. I wonder where these came from. Mark From mike at cappella.us Tue Feb 14 00:06:07 2012 From: mike at cappella.us (Mike Cappella) Date: Mon, 13 Feb 2012 15:06:07 -0800 Subject: Spam-tag log line Message-ID: <4F39975F.8040306@cappella.us> > Stefan, > >> > If you are referring to a debug log line "Spam-tag, ...", it is now >> > logged at log level 3 starting with 2.7.0. Previously it was a >> > "SPAM-TAG, ...", logged at level 2. I suggest not to bother with this >> > debug line. >> >> But the missing "Spam-tag, ..." line breaks some reporting tools, like >> amavis- logwatch and amavislogsumm (in log_level 2), which depend on the >> information in that line. If I change the default log template, they won't >> recognize the SpamAssassin test results. >> >> What was the reason to not log the line on log_level 2 any longer? > > The reason was twofold: I tried to avoid a potential trademark issue > with a spelling "SPAM" (hence lowercasing it); and while at it, I dropped Are other log lines affected by change in case (such as SPAM log lines, or the word SPAM elsewhere)? Note: in case this response doesn't thread properly, the context for this response is at the top of: http://lists.amavis.org/pipermail/amavis-users/2012-January/thread.html > its log level from 2 to 3 to reduce the redundant clutter at level 2, > as the list of SA tests can be (and often is) already included in the > main log line (at log level 0), avoiding the need to log at 2. > >> And is there a way to have the Spam-tag line in log_level 2 again? > > Try the attached trivial patch (on 2.7.0). It brings it back to level 2, > but also further lowercases the word. Do the log analyzer still > recognize it even if fully lowercased? If so, I guess I can put it back > in this form. > > Mark > Thanks for the patch. The lowercased 'spam-tag' works without change with > amavislogsumm. amavis-logwatch will need a small patch, but I'm positive that > Mike will accept it. > >> If so, I guess I can put it back in this form. > > If that's possible, that would be great. > > Thanks a lot. > Stefan I've made the necessary changes, and will post an update, pending a response to the above question. Mike From Mark.Martinec+amavis at ijs.si Tue Feb 14 12:50:56 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Tue, 14 Feb 2012 12:50:56 +0100 Subject: Spam-tag log line In-Reply-To: <4F39975F.8040306@cappella.us> References: <4F39975F.8040306@cappella.us> Message-ID: <201202141250.56720.Mark.Martinec+amavis@ijs.si> Mike, > Are other log lines affected by change in case (such as SPAM log lines, > or the word SPAM elsewhere)? I think there is only one other visible change (ignoring changes in comments) regarding lowercasing of a word 'spam', namely the SMTP reject message, which may also end up in the log at higher log levels: %smtp_reason_by_ccat | < CC_SPAM, "id=%n - SPAM", | < CC_SPAMMY.',1', "id=%n - SPAMMY (tag3)", | < CC_SPAMMY, "id=%n - SPAMMY", |--- | > CC_SPAM, 'id=%n - spam', | > CC_SPAMMY.',1', 'id=%n - spammy (tag3)', | > CC_SPAMMY, 'id=%n - spammy', But I'm not promising other deeper log-level messages are not changing, there may be small changes in these through versions. > > Try the attached trivial patch (on 2.7.0). It brings it back to level 2, > > but also further lowercases the word. Do the log analyzer still > > recognize it even if fully lowercased? If so, I guess I can put it back > > in this form. > > Mark > > > > Thanks for the patch. The lowercased 'spam-tag' works without change with > > amavislogsumm. amavis-logwatch will need a small patch, but I'm positive > > that Mike will accept it. > > > >> If so, I guess I can put it back in this form. Mike wrote: > I've made the necessary changes, and will post an update, pending a > response to the above question. Thanks, I'm glad you're still around! Mark From Mark.Martinec+amavis at ijs.si Tue Feb 14 16:32:49 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Tue, 14 Feb 2012 16:32:49 +0100 Subject: Virus notification question In-Reply-To: <201202131701.11301.sdavies@sdc.com.au> References: <201202121716.41933.sdavies@sdc.com.au> <20120213060658.GA18108@spider.services.datevnet.de> <201202131701.11301.sdavies@sdc.com.au> Message-ID: <201202141632.49756.Mark.Martinec+amavis@ijs.si> Stephen, > Following is an excerpt from my mail log when a virus is detected > by amavisd-new-2.7.0 and amavisd-milter-1.5.0. > What might be wrong with my config? [...] > $inet_socket_port = 10026; # listen on this local TCP port(s) > $notify_method = 'smtp:[127.0.0.1]:10026'; [...] > Feb 12 12:22:28 mustang amavis[9071]: (09071) Request: AM.PDP > /var/amavis/tmp/afq1C1qKLZ012522: > -> > Feb 12 12:22:28 mustang amavis[9071]: (09071) > Checking: cM4dH5-zzumM AM.PDP-SOCK [81.176.77.242] > -> [...] > Feb 12 12:22:28 mustang amavis[9070]: (09070-04) ESMTP::10026 > /var/amavis/tmp/amavis-20120212T122228-09070-52jpecv4: > -> > ENVID=AM.09071.20120212T015228Z at mustang.sdc.com.au [...] > Feb 12 12:22:29 mustang amavis[9070]: (09070-04) Passed CLEAN > {AcceptedInternal}, ORIGINATING > -> , > Message-ID: , > mail_id: BciVU7H-sYYI, Hits: 0, size: 2410, 868 ms > Feb 12 12:22:29 mustang amavis[9070]: (09070-04) (!!)TROUBLE in > process_request: NOT ALL RECIPIENTS DONE, EMPTY DELIVERY_METHOD! > > The "smtp server" at 10026 is amavisd. This isn't right: $inet_socket_port = 10026; $notify_method = 'smtp:[127.0.0.1]:10026'; Notifications as generated by amavisd are not supposed to be fed back to itself. They should be fed to an MTA, preferably on a port where content filtering is disabled, otherwise one runs a risk of blocking own notifications. In case of Postfix this is achieved by having a dedicated smtpd service (often on port 10025) which has content filtering disabled, e.g. by cleaning these two options on a service: -o smtpd_milters= -o content_filter= I'm not sure what is the most convenient way to do so with sendmail. One clean way is to have two MTA instances, where the front-end instance has content filtering enabled, while the back-end instance has it disabled. The $notify_method should then point to the second MTA instance. What happened in your case was the notification was fed via SMTP protocol back to amavisd, where a policy bank sitting on that port did not have $forward_method configured, so amavisd did not know how to forward the message - which should explain the message you received. Mark From Mark.Martinec+amavis at ijs.si Tue Feb 14 16:57:55 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Tue, 14 Feb 2012 16:57:55 +0100 Subject: Amavisd with bogofilter In-Reply-To: <201202012031.12053.sdavies@sdc.com.au> References: <201202012031.12053.sdavies@sdc.com.au> Message-ID: <201202141657.56005.Mark.Martinec+amavis@ijs.si> Stephen, > The attached diff against amavisd-new-2.7.0 replaces Spam Assassin in > amavisd with Bogofilter. > > The resulting amavisd has been running here for over twelve hours and seems > to work perfectly. > > The only other changes were to the configuration file entries for: > $sa_tag_level_deflt = 0; # add spam info headers if at, or above that > level $sa_tag2_level_deflt = 3; # add 'spam detected' headers at that > level $sa_kill_level_deflt = 5; # triggers spam evasive actions (e.g. > blocks mail) > > The first because I like to see spam info headers and the last to force > quarantine of all spam as detected by bogofilter. Thanks! I've finally got around and test it with Bogofilter here. Made some small adjustments, the attached patch will find its way into 2.8.0. Some of your changes better belong to a config file, so I left them out. Mark -------------- next part -------------- A non-text attachment was scrubbed... Name: 0.patch Type: text/x-patch Size: 2280 bytes Desc: not available URL: From sdavies at sdc.com.au Thu Feb 16 01:54:42 2012 From: sdavies at sdc.com.au (Stephen Davies) Date: Thu, 16 Feb 2012 11:24:42 +1030 Subject: Virus notification question In-Reply-To: <201202141632.49756.Mark.Martinec+amavis@ijs.si> References: <201202121716.41933.sdavies@sdc.com.au> <201202131701.11301.sdavies@sdc.com.au> <201202141632.49756.Mark.Martinec+amavis@ijs.si> Message-ID: <201202161124.42162.sdavies@sdc.com.au> Thanks for the feedback Mark. I now have a better understanding of this side of amavisd. I am using Petr Rohar's amavisd-milter (I didn't know you had one). Does the $inet_socket_port have any relevance at all in this context? My config has: $policy_bank{'AM.PDP-SOCK'} = { protocol => 'AM.PDP', auth_required_release => 0, # do not require secret_id for amavisd-release final_spam_destiny => D_DISCARD }; And I have changed $notify_method to: $notify_method = 'smtp:[127.0.0.1]:587'; (I can live with the unlikely reject). $final_virus_destiny = D_DISCARD; Are there any other settings relevant to the milter context? Cheers and thanks, Stephen PS My bogofilter version of amavisd seems to be working perfectly. On Wed, 15 Feb 2012 02:02:49 AM Mark Martinec wrote: > Stephen, > > > Following is an excerpt from my mail log when a virus is detected > > by amavisd-new-2.7.0 and amavisd-milter-1.5.0. > > What might be wrong with my config? > > [...] > > > $inet_socket_port = 10026; # listen on this local TCP port(s) > > $notify_method = 'smtp:[127.0.0.1]:10026'; > > [...] > > > Feb 12 12:22:28 mustang amavis[9071]: (09071) Request: AM.PDP > > > > /var/amavis/tmp/afq1C1qKLZ012522: > > -> > > > > Feb 12 12:22:28 mustang amavis[9071]: (09071) > > > > Checking: cM4dH5-zzumM AM.PDP-SOCK [81.176.77.242] > > -> > > [...] > > > Feb 12 12:22:28 mustang amavis[9070]: (09070-04) ESMTP::10026 > > > > /var/amavis/tmp/amavis-20120212T122228-09070-52jpecv4: > > -> > > ENVID=AM.09071.20120212T015228Z at mustang.sdc.com.au > > [...] > > > Feb 12 12:22:29 mustang amavis[9070]: (09070-04) Passed CLEAN > > > > {AcceptedInternal}, ORIGINATING > > -> , > > Message-ID: , > > mail_id: BciVU7H-sYYI, Hits: 0, size: 2410, 868 ms > > > > Feb 12 12:22:29 mustang amavis[9070]: (09070-04) (!!)TROUBLE in > > process_request: NOT ALL RECIPIENTS DONE, EMPTY DELIVERY_METHOD! > > > > The "smtp server" at 10026 is amavisd. > > This isn't right: > > $inet_socket_port = 10026; > $notify_method = 'smtp:[127.0.0.1]:10026'; > > Notifications as generated by amavisd are not supposed to be > fed back to itself. They should be fed to an MTA, preferably on a > port where content filtering is disabled, otherwise one runs a risk > of blocking own notifications. > > In case of Postfix this is achieved by having a dedicated smtpd service > (often on port 10025) which has content filtering disabled, e.g. by > cleaning these two options on a service: > -o smtpd_milters= > -o content_filter= > > I'm not sure what is the most convenient way to do so with sendmail. > One clean way is to have two MTA instances, where the front-end > instance has content filtering enabled, while the back-end instance > has it disabled. The $notify_method should then point to the second > MTA instance. > > What happened in your case was the notification was fed via SMTP > protocol back to amavisd, where a policy bank sitting on that > port did not have $forward_method configured, so amavisd > did not know how to forward the message - which should explain > the message you received. > > Mark -- ============================================================================= Stephen Davies Consulting P/L Voice: 08-8177 1595 Adelaide, South Australia. Fax : 08-8177 0133 Records & Collections Management. Mobile:040 304 0583 From Mark.Martinec+amavis at ijs.si Thu Feb 16 17:44:39 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Thu, 16 Feb 2012 17:44:39 +0100 Subject: Virus notification question In-Reply-To: <201202161124.42162.sdavies@sdc.com.au> References: <201202121716.41933.sdavies@sdc.com.au> <201202141632.49756.Mark.Martinec+amavis@ijs.si> <201202161124.42162.sdavies@sdc.com.au> Message-ID: <201202161744.39890.Mark.Martinec+amavis@ijs.si> Stephen, > Thanks for the feedback Mark. > I now have a better understanding of this side of amavisd. > > I am using Petr Rohar's amavisd-milter > (I didn't know you had one). Yes, Petr's is the only milter interface to amavisd nowadays. There used to be an equivalent of this in the amavisd-new package, but I dropped it, as Petr's milter supports the new AM.PDP protocol and is better maintained. > Does the $inet_socket_port have any relevance at all in this context? If you are accepting AM.PDP protocol requests over a Unix socket ($unix_socketname or its equivalent in @listen_sockets), and you don't have any other needs for other inet connections (like from amavisd-release, which can use either a Unix socket or an inet socket to contact amavisd), then the $inet_socket_port has no relevance for your setup. Btw, starting with 2.7.0, the @listen_sockets is the generic config setting for all listening sockets. The $inet_socket_bind, $inet_socket_port, and $unix_socketname are only still there for compatibility: - @listen_sockets setting offers a unified configuration of listening sockets; it may be configured directly, or the traditional way: the $inet_socket_port, $unix_socketname and $inet_socket_bind just add their entries to the @listen_sockets list; > My config has: > > $policy_bank{'AM.PDP-SOCK'} = { > protocol => 'AM.PDP', > auth_required_release => 0, > final_spam_destiny => D_DISCARD > }; Ok. This applies to connections coming in over the $unix_socketname. > And I have changed $notify_method to: > $notify_method = 'smtp:[127.0.0.1]:587'; > (I can live with the unlikely reject). Ok. Keep in mind that a notification or DSN generated by amavisd and submitted to the $notify_method will need to be accepted and processed by MTA in order for the current mail processing can continue. If the MTA will invoke milter/amavisd for this submission too, there needs to be a spare amavisd process available to handle this, otherwise we end up in a deadlock. In practice this may be acceptable with sufficient number of child processes. However if the $max_servers is small (like 2, as may be a default), the likelihood of such a lockup becomes real. > $final_virus_destiny = D_DISCARD; > > Are there any other settings relevant to the milter context? Nothing comes to mind. There are some differences between a milter approach and the SMTP approach. With milter, amavisd can't split the passed message, so if a message has multiple recipients and they differ in their requirements (e.g. defanging, header edits), these individual needs won't be met. > PS My bogofilter version of amavisd seems to be working perfectly. Good. I left it running with my modified patch here for a couple of days too, just to see that it works. Mark From Mark.Martinec+amavis at ijs.si Fri Feb 17 00:56:00 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Fri, 17 Feb 2012 00:56:00 +0100 Subject: defang_spam not working In-Reply-To: References: Message-ID: <201202170056.00691.Mark.Martinec+amavis@ijs.si> Steve, > I'm having problems with my amavis not defang'ing spam. Messages are having > their subject rewritten and X-SPAM-Headers but are not defang'd. > > Could anyone shed any light on which settings I need to be paying attention > to? I've gone over my amavisd.conf with a find tooth comb to no avail. Sorry for delay. While investigating what could potentially be the reason, I found two problems regarding defanging in 2.7.0, and I'm attaching a patch to fix these. The bug can only manifest itself under certain conditions, and you are not saying neither which version of amavisd and perl are you using, nor the defanging method of choice (like whether altermime is installed and enabled, or whether Anomy::Sanitizer is to be used for defanging). Here is the description of the two problems addressed by the patch: - fixed defanging by mimedefang, it was failing with perl 5.10 or later due to an unhandled "Insecure dependency in sprintf" while logging the result if the $log_level was 2 or higher, or when debugging was enabled; - fixed defanging by Anomy::Sanitizer, it was failing with an error message: "mangling by anomy failed: replacement size 0, mail will pass unmodified" > I'm using SQL backup with default policies. Recipients access is associated > with the "Default Policy" (ie *_lover's all N, bypass_*_checks all N, > spam_modifies_subj = Y, all over fields NULL). > > $defang_virus = 1; > $defang_banned = 1; > $defang_spam = 1; > $defang_bad_header = 1; > $defang_undecipherable = 1; > $defang_all = 1; //for testing purposes only > > $sa_tag_level_deflt = -9999; //add spam headers to all messages > $sa_tag2_level_deflt = 5.0; //anything >= 5 considered > $sa_kill_level_deflt = 10.0;//anything >= 10 is quarantined > $sa_quarantine_cutoff_level = 25;//anything >= 25 is discarded completely. > > $final_virus_destiny = D_DISCARD; > $final_banned_destiny = D_BOUNCE; > $final_spam_destiny = D_DISCARD; > ##$final_bad_header_destiny = D_REJECT; > $final_bad_header_destiny = D_PASS; > > While testing I checked to see if messages marked with a bad header are > being defang'd by sending a malformed email with two Subject: headers. > They also aren't being defang'd. > Is there any change I'm missing a perl module required to defang or is it > definitely a configuration issue? Mail to local recipients with a bad header, or spam with score between tag2 and kill levels, should have been defanged, unless you are meeting the bug conditions above (perl >= 5.10, altermime installed, log level >= 2). If you have altermime installed, try disabling it ($altermime = undef), or apply the patch. If the problem persists, I'd like to see the full log of the event (at $log_level=5). Mark -------------- next part -------------- A non-text attachment was scrubbed... Name: 0.patch Type: text/x-patch Size: 2005 bytes Desc: not available URL: From amavis-users at spectrumcs.net Sun Feb 19 15:41:48 2012 From: amavis-users at spectrumcs.net (Steve Scotter) Date: Sun, 19 Feb 2012 14:41:48 +0000 Subject: (SCANNED)Re: defang_spam not working Message-ID: Hi Mark, Thanks for your reply. I replied a about an hour later to my own post basically saying that setting $altermime = undef 'fixed' my issue, and abandoned trying to use altermine. ------------------------------------------------ Sorry for the school boy error of not providing version information... # altermime --version alterMIME v0.3.11 (November-2008) by Paul L Daniels - http://www.pldaniels.com/altermime # amavisd -V amavisd-new-2.7.0 (20110701) # perl -v This is perl 5, version 14, subversion 2 (v5.14.2) built for amd64-freebsd # uname -a FreeBSD untrustedhost.example.com 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012 root at farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 ------------------------------------------------ To try and help resolve this issue I've applied your attached patch, re-enabled altermime and sent a couple of test messages though but unfortunately the problem persists. The email arrives but is not defanged when altermime is enabled. Please find attached sanitized amavisd logs of two "/usr/local/sbin/amavisd debug" sessions (one with altermime enabled, one with altermime disabled) and the emails that arrived. If you need any more information or someone to test any further patches I'm happy to help. Regards Stephen Scotter Systems Consultant T. +44 (0) 7970 463925 -------- Original Message -------- Subject: (SCANNED)Re: defang_spam not working (16-Feb-2012 23:56) From: Mark Martinec To: amavis-users at spectrumcs.net > Steve, > > > I'm having problems with my amavis not defang'ing spam. Messages are having > > their subject rewritten and X-SPAM-Headers but are not defang'd. > > > > Could anyone shed any light on which settings I need to be paying attention > > to? I've gone over my amavisd.conf with a find tooth comb to no avail. > > Sorry for delay. While investigating what could potentially be the reason, > I found two problems regarding defanging in 2.7.0, and I'm attaching > a patch to fix these. The bug can only manifest itself under certain > conditions, and you are not saying neither which version of amavisd and > perl are you using, nor the defanging method of choice (like whether > altermime is installed and enabled, or whether Anomy::Sanitizer is > to be used for defanging). > > Here is the description of the two problems addressed by the patch: > > - fixed defanging by mimedefang, it was failing with perl 5.10 or later > due to an unhandled "Insecure dependency in sprintf" while logging the > result if the $log_level was 2 or higher, or when debugging was enabled; > > - fixed defanging by Anomy::Sanitizer, it was failing with an error message: > > "mangling by anomy failed: replacement size 0, mail will pass unmodified" > > > > I'm using SQL backup with default policies. Recipients access is associated > > with the "Default Policy" (ie *_lover's all N, bypass_*_checks all N, > > spam_modifies_subj = Y, all over fields NULL). > > > > $defang_virus = 1; > > $defang_banned = 1; > > $defang_spam = 1; > > $defang_bad_header = 1; > > $defang_undecipherable = 1; > > $defang_all = 1; //for testing purposes only > > > > $sa_tag_level_deflt = -9999; //add spam headers to all messages > > $sa_tag2_level_deflt = 5.0; //anything >= 5 considered > > $sa_kill_level_deflt = 10.0;//anything >= 10 is quarantined > > $sa_quarantine_cutoff_level = 25;//anything >= 25 is discarded completely. > > > > $final_virus_destiny = D_DISCARD; > > $final_banned_destiny = D_BOUNCE; > > $final_spam_destiny = D_DISCARD; > > ##$final_bad_header_destiny = D_REJECT; > > $final_bad_header_destiny = D_PASS; > > > > While testing I checked to see if messages marked with a bad header are > > being defang'd by sending a malformed email with two Subject: headers. > > They also aren't being defang'd. > > Is there any change I'm missing a perl module required to defang or is it > > definitely a configuration issue? > > Mail to local recipients with a bad header, or spam with score between > tag2 and kill levels, should have been defanged, unless you are meeting the > bug conditions above (perl >= 5.10, altermime installed, log level >= 2). > > If you have altermime installed, try disabling it ($altermime = undef), > or apply the patch. If the problem persists, I'd like to see the full > log of the event (at $log_level=5). > > Mark > > > > To: amavis-users at amavis.org > Cc: amavis-users at spectrumcs.net To: Mark.Martinec+amavis at ijs.si amavis-users at amavis.org DISCLAIMER This email is for the use of the intended recipient(s) only. If you have received this email in error, please notify the sender immediately and then delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the authors prior permission. We have taken precautions to minimise the risk of transmitting software viruses, but we advise you to carry out your own virus checks on any attachment to this message. We cannot accept liability for any loss or damage caused by software viruses. The information contained in this communication may be confidential and may be subject to the attorney-client privilege. If you are the intended recipient and you do not wish to receive similar electronic messages from us in future then please respond to the sender to this effect. -------------- next part -------------- A non-text attachment was scrubbed... Name: 0.patch Type: application/octet-stream Size: 2041 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: amavisd-altermime-disabled.log Type: application/octet-stream Size: 110008 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: amavisd-altermime-enabled.log Type: application/octet-stream Size: 109065 bytes Desc: not available URL: -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: no subject Date: no date Size: 6057 URL: -------------- next part -------------- An embedded message was scrubbed... From: unknown sender Subject: no subject Date: no date Size: 817 URL: From Mark.Martinec+amavis at ijs.si Mon Feb 20 15:15:49 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Mon, 20 Feb 2012 15:15:49 +0100 Subject: defang_spam not working In-Reply-To: References: Message-ID: <201202201515.49331.Mark.Martinec+amavis@ijs.si> Steve, > alterMIME v0.3.11 (November-2008) by Paul L Daniels - > amavisd-new-2.7.0 (20110701) > This is perl 5, version 14, subversion 2 (v5.14.2) built for amd64-freebsd > FreeBSD untrustedhost.example.com 9.0-RELEASE FreeBSD 9.0-RELEASE > > To try and help resolve this issue I've applied your attached patch, > re-enabled altermime and sent a couple of test messages though but > unfortunately the problem persists. The email arrives but is not defanged > when altermime is enabled. > > Please find attached sanitized amavisd logs of two "/usr/local/sbin/amavisd > debug" sessions (one with altermime enabled, one with altermime disabled) > and the emails that arrived. Thanks for applying the patch and for the log. This now indicates that altermime was invoked successfully: (21760-01) mangling by: 1, (21760-01) run_command: [21792] /usr/local/bin/altermime --input=/var/amavis/tmp/amavis-20120../email-repl.txt --verbose --removeall &1 (21760-01) collect_results from [21792] (/usr/local/bin/altermime), 0 bytes, (limit 16384) (21760-01) mangling by altermime (1) done, new size: 613, orig 626 bytes (21760-01) mail body mangling in effect, So altermime did what it was told (--removeall = Remove all attachments). In case of this simple test message with no attachments, the result was no different from the original message. Altermime has some other command line options, you can tweak the @altermime_args_defang config setting, its default is: @altermime_args_defang = qw(--verbose --removeall); It seems you were expecting the defanging to insert the SpamAssassin report into a spam message, and push it to an attachment. This is what the simple defanging does ($defang_spam = 1; or the more explicit: $defang_spam = 'attach' ). Altermime and Anomy sanitizer have their own ideas on what constitutes 'defanging', and they do not mimic the simple 'attach' method. Mark From anirudha0012 at gmail.com Tue Feb 21 09:53:33 2012 From: anirudha0012 at gmail.com (Anirudha Patil) Date: Tue, 21 Feb 2012 14:23:33 +0530 Subject: Scan timeout value for amavis running clamav Message-ID: Hello Folks, I have Postfix 2.5.5 running with Amavis [amavisd-new-2.6.4 (20090625)] integrated with ClamAV 0.96.1 I need to know if there is any default *Scan Timeout* or *Timeout *for amavis process. I can see the below limit options in amavisd.conf but no options for how long will amavis process would scan a given email before timing out if any ======== $MAXLEVELS = 14; $MAXFILES = 1500; $MIN_EXPANSION_QUOTA = 100*1024; # bytes (default undef, not enforced) $MAX_EXPANSION_QUOTA = 300*1024*1024; # bytes (default undef, not enforced) ======== Is there any child_timeout value associated with amavis, if yes, what is it and if it has any default value ? Please find my sample amavid.conf file. -- With Regards Anirudha Patil -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: amavisd.conf Type: application/octet-stream Size: 35971 bytes Desc: not available URL: From Mark.Martinec+amavis at ijs.si Tue Feb 21 14:25:27 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Tue, 21 Feb 2012 14:25:27 +0100 Subject: Scan timeout value for amavis running clamav In-Reply-To: References: Message-ID: <201202211425.27825.Mark.Martinec+amavis@ijs.si> Anirudha Patil, > I have Postfix 2.5.5 running with Amavis [amavisd-new-2.6.4 (20090625)] > integrated with ClamAV 0.96.1 > > I need to know if there is any default *Scan Timeout* or *Timeout *for > amavis process. I can see the below limit options in amavisd.conf but no > options for how long will amavis process would scan a given email before > timing out if any > [...] > Is there any child_timeout value associated with amavis, if yes, what is it > and if it has any default value ? > > Please find my sample amavid.conf file. There is a $child_timeout setting, with a default value of 8*60 seconds, i.e. 8 minutes. It limits the total time for one mail message processing spent in amavisd. The sublimits (virus, spam, ... scans) are derived dynamically from $child_timeout and actual times spent so far. With your version 2.6.4 the time limits are rather coarse. The situation in this respect is significantly improved with 2.7.0 : Release notes: - improved support for pre-queue content filtering setups: reorganized time limiting on processing to obey more strictly a deadline time, which is the sum of $child_timeout and a timestamp at the moment of a reception of a complete message (SMTP data-end time). The deadline time is also passed to SpamAssassin, which since version 3.3.0 supports a 'master_deadline' option and can gracefully terminate its processing on a time limit, while still providing results collected so far. The setting $sa_timeout is now retired: the variable is still declared for backward compatibility, but has no effect. Instead, the time available for spam scanning is automatically determined from $child_timeout, taking into consideration the actual time left till the deadline; Mark From Mark.Martinec+amavis at ijs.si Tue Feb 21 15:12:20 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Tue, 21 Feb 2012 15:12:20 +0100 Subject: defang_spam not working In-Reply-To: References: Message-ID: <201202211512.20087.Mark.Martinec+amavis@ijs.si> Stephen, > You are correct I was indeed hoping to replicate $defang_spam = 'attach' > with altermime. I like receiving the spam report with spammy messages, but > I'd also like to be able to add a disclaimer to outgoing (and possibly > some incoming messages) messages as well. Are these two features mutually > exclusive? > > I've just had a quick look at amavisd.conf-default it would seem to me that > $altermime is only globally configurable and not a policy configurable, so > I can't just enable altermime on our outbound policy, or can I? You can have both. The $defang_* settings can contain one of the strings: 'attach', 'altermime', 'anomy', 'disclaimer'. Any other value (such as 1) is interpreted as 'anomy' if $enable_anomy_sanitizer is true, or as 'altermime' if $altermime proogram is available, or as 'attach' otherwise, for compatibility reasons. So the idea is to set it to 'disclaimer' with a policy bank which is accepting mail from inside or roaming authenticated users (typically named 'MYNETS' or 'ORIGINATING'), and use a setting 'attach' for everything else, i.e. as a global default. You can even have $defang_spam='attach' and $defang_virus='altermime' For example: $defang_spam='attach'; $defang_virus='altermime'; # list all you internal networks here, public and private addresses @mynetworks = qw( 127.0.0.0/8 [::1] [FE80::]/10 [FEC0::]/10 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 169.254.0.0/16 ); $policy_bank{'MYNETS'} = { originating => 1, allow_disclaimers => 1, defang_maps_by_ccat => { REPLACE => 1, &CC_CATCHALL => 'disclaimer' }, } amavisd-new-2.5.0 release notes: - provided interface code to allow mangling/defanging/sanitation to be performed by an external utility, either by directly calling a Perl module Anomy Sanitizer (within the same process, avoiding startup cost), or by invoking a program 'altermime' (or by internal defanging code as before). Mail body defanging is only allowed for local recipients (those matching @local_domains_maps), i.e. for inbound and internal-to-internal mail. If there is more than one mangling code option available, the result of a %defang_maps_by_ccat can choose between them by returning one of the following strings, the selection can depend on mail content type and on by-recipient lookups if needed: 'anomy' chooses Anomy Sanitizer (if $enable_anomy_sanitizer is true); 'altermime' chooses a program whose path is $altermime (if found); 'attach' chooses the traditional amavisd-new defanging method which pushes an original mail message to an attachment; 'null' for testing purposes - doesn't modify mail body, but pretends it does (in logging and mail header); other non-empty and non-zero value automatically choose one of the above options depending on what is available; at least the 'attach' is always available; an empty, zero or undef value disables mail body modifications; Controls: $enable_anomy_sanitizer, @anomy_sanitizer_args, and: $altermime, @altermime_args_defang; Typical use: # with altermime: $altermime = '/usr/local/bin/altermime'; @altermime_args_defang = qw(--verbose --removeall); # with Anomy Sanitizer: $enable_anomy_sanitizer = 1; @anomy_sanitizer_args = qw( /usr/local/etc/sanitizer.cfg ); $defang_spam = 1; # old style, applies the first available mangler # to all spam-loving local recipients # unnecessarily complicated example of selective choices: $defang_maps_by_ccat{+CC_BANNED} = [ 'altermime', # use altermime for everybody (a 'constant' lookup table) ]; $defang_maps_by_ccat{+CC_SPAM} = [ { # a per-recipient hash lookup table 'user at example.com' => 1, # old style, auto-selects a mangler 'user-a at example.com' => 'anomy', 'user-m at example.com' => 'altermime', 'user-t at example.com' => 'attach', '.example.net' => 0, # no mangling }, $defang_spam, # fallback to old style setting if no match above ]; - a special case of mangling is adding a disclaimer, by invoking an external program 'altermime' (if available and enabled). This differs from mangling inbound mail in two details: * uses a separately configurable list of arguments to altermime: @altermime_args_disclaimer; and * it applies only to mail submitted from internal networks or roaming users (as recognized through a policy bank which sets: allow_disclaimers => 1), and where any of the following addresses matches local domains: author (2822.From) or sender (2822.Sender) or return path (2821.mail_from); Typically the $allow_disclaimers should be set by a policy bank which also sets the $originating flag. In addition to strings that may be returned by %defang_maps_by_ccat as described above, there are two more, only taken into account when $allow_disclaimers is true: 'disclaimer' invokes $altermime program for outgoing mail with arguments as given in @altermime_args_disclaimer; 'nulldisclaimer' for testing purposes - doesn't modify mail body, but pretends it does (in logging and mail header); Typical use: $altermime = '/usr/local/bin/altermime'; @altermime_args_disclaimer = qw(--verbose --disclaimer=/etc/altermime-disclaimer.txt); $defang_maps_by_ccat{+CC_CATCHALL} = [ 'disclaimer' ]; @mynetworks = qw( ... ); $policy_bank{'MYNETS'} = { # mail originating from our networks originating => 1, allow_disclaimers => 1, } For the moment there is one limitation: there can only be one mangler in effect at a time, it is not currently possible to both defang and to append a disclaimer on the same message: for internal-to-internal mail inserting a disclaimer takes precedence. To make it possible to provide different disclaimer texts when hosting multiple domains, there is an experimental additional configuration variable available: the @disclaimer_options_bysender_maps. It is a list of lookup tables, looked up by a sender address. The sender address is chosen from the following list, first match wins: * 'Sender:' header field, if its domain matches @local_domains_maps; * 'From:' header field, if its domain matches @local_domains_maps; * envelope sender address, if its domain matches @local_domains_maps; We already know that at least one of the above will match, otherwise adding disclaimers would be skipped at an earlier stage. The result of lookups should be one simple string, which replaces a string '_OPTION_' anywhere in @altermime_args_disclaimer elements. Typical use: @altermime_args_disclaimer = qw(--disclaimer=/etc/_OPTION_.txt); @disclaimer_options_bysender_maps = ( { 'host1.example.com' => 'altermime-disclaimer-host1', 'boss at example.net' => 'altermime-disclaimer-boss', '.example.net' => 'altermime-disclaimer-net', '.' => 'altermime-disclaimer-default' }, ); It is currently not possible to disable adding disclaimers through @disclaimer_options_bysender_maps results. This needs to be improved. The exact interpretation of the @disclaimer_options_bysender_maps lookup result may change in the future (which is why I call it 'experimental'). Note that disclaimers are pretty much useless legally. If you can help it at all, please avoid the pollution. See: http://www.goldmark.org/jeff/stupid-disclaimers/ Mark From anirudha0012 at gmail.com Wed Feb 22 13:22:58 2012 From: anirudha0012 at gmail.com (Anirudha Patil) Date: Wed, 22 Feb 2012 17:52:58 +0530 Subject: Scan timeout value for amavis running clamav Message-ID: ---------- Forwarded message ---------- From: Mark Martinec To: amavis-users at amavis.org Cc: Date: Tue, 21 Feb 2012 14:25:27 +0100 Subject: Re: Scan timeout value for amavis running clamav Anirudha Patil, > I have Postfix 2.5.5 running with Amavis [amavisd-new-2.6.4 (20090625)] > integrated with ClamAV 0.96.1 > > I need to know if there is any default *Scan Timeout* or *Timeout *for > amavis process. I can see the below limit options in amavisd.conf but no > options for how long will amavis process would scan a given email before > timing out if any > [...] > Is there any child_timeout value associated with amavis, if yes, what is it > and if it has any default value ? > > Please find my sample amavid.conf file. >There is a $child_timeout setting, with a default value of 8*60 seconds, i.e. 8 minutes. It limits the total time for one mail message processing spent in amavisd. The sublimits (virus, spam, ... scans) are derived dynamically from $child_timeout and actual times spent so far. *So does this means, a particular email would be devoted a total time of 8 mins by amavis or would it be more?* >With your version 2.6.4 the time limits are rather coarse. >The situation in this respect is significantly improved with 2.7.0 : > >Release notes: > >- improved support for pre-queue content filtering setups: reorganized time > limiting on processing to obey more strictly a deadline time, which is the > sum of $child_timeout and a timestamp at the moment of a reception of a > complete message (SMTP data-end time). The deadline time is also passed > to SpamAssassin, which since version 3.3.0 supports a 'master_deadline' > option and can gracefully terminate its processing on a time limit, while > still providing results collected so far. > > The setting $sa_timeout is now retired: the variable is still declared > for backward compatibility, but has no effect. Instead, the time available > for spam scanning is automatically determined from $child_timeout, taking > into consideration the actual time left till the deadline; > > Mark Anirudha -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mark.Martinec+amavis at ijs.si Wed Feb 22 14:10:29 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Wed, 22 Feb 2012 14:10:29 +0100 Subject: Scan timeout value for amavis running clamav In-Reply-To: References: Message-ID: <201202221410.29697.Mark.Martinec+amavis@ijs.si> Anirudha, > >There is a $child_timeout setting, with a default value of 8*60 seconds, > > i.e. 8 minutes. It limits the total time for one mail message processing > > spent in amavisd. The sublimits (virus, spam, ... scans) are derived >> dynamically from $child_timeout and actual times spent so far. > > *So does this means, a particular email would be devoted a total time of 8 > mins by amavis or would it be more?* Yes, processing should abort after $child_timeout seconds or slightly beyond that. This limit is much more precisely adhered to since 2.7.0, which makes it suitable for a pre-queue filtering setup (setting the $child_timeout to something like 45 seconds). With 2.6.* this is more like a best effort / advisory time limit. Still, even with 2.7.0, there may be cases (like evaluating a regexp of a SpamAssassin rule in some degenerate case), where the limit may be exceeded. SpamAssassin 3.3.* or 3.4.0 are much better behaved in this respect compared to SpamAssassin 3.2. Mark From npf-mlists at eurotux.com Thu Feb 23 12:44:35 2012 From: npf-mlists at eurotux.com (Nuno Fernandes) Date: Thu, 23 Feb 2012 11:44:35 +0000 Subject: Cacti Template Message-ID: <201202231144.35297.npf-mlists@eurotux.com> Hello, I already have amavis-agent working: # snmpwalk -v2c -c xxxxxx localhost AMAVIS-MIB::sysUpTime.0 AMAVIS-MIB::sysUpTime.0 = Timeticks: (5757380) 15:59:33.80 I would like to create some graphs of it using cacti. For what i saw in http://www.amavis.org/amavis-2011.pdf (page 96-99) there is a possibility of that.. Does anyone have any cacti amavis template that can be used? Thanks, Nuno Fernandes From agutierr at gmail.com Fri Feb 24 11:37:06 2012 From: agutierr at gmail.com (=?ISO-8859-1?Q?Antonio_Guti=E9rrez_Mayoral?=) Date: Fri, 24 Feb 2012 11:37:06 +0100 Subject: Amavisd cant fork with a lot of emails Message-ID: Hi all! First of all, sorry if this is not the right place to ask this question. We are setting upan antispam server using spamassassin, maia, amavisd-new, etc. All is ok. But, once in a while, we have problems in amavis, when create a new process to check attached data in an email. We are getting an strange message on postfix logs after a while. The message is?like that: (host 127.0.0.1[127.0.0.1] said: 451 4.5.0 Error in processing, id=11968-01-10, mime_decode-1 FAILED: run_command (open pipe): Can't fork at /usr/lib/perl5/5.10.0/x86_64-linux-thread-multi/IO/File.pm line 188. at /usr/sbin/amavisd line 1657. (in reply to end of DATA command))? YYYYY foo.bar I am afraid that the problem is there isn't enough resources to allocate?a new thread. The message goes to the deferred queue. The problem is, that when its occurs, the deferred queue starts to increase quickly, and the memory goes down quickly also. For this reason I have?configured via limits.conf to increase the process on the * account,?but it doesnt works: /etc/security/limits.conf * ??????soft ???nofile ?65535 * ??????hard ???nofile ?65535 * ??????soft ???nproc ??65535 * ??????hard ???nproc ??65535 Amavis is running under the amavis user account. My server is a SLES 11 (virtual running under VMWARE), with 4 GB of ram and 1,5 GB of swap. When I have this problem, the free memory quickly goes down, like the swap. :((( Any ideas? So much thanks. Antonio. -- -- Antonio Guti?rrez Mayoral From Mark.Martinec+amavis at ijs.si Fri Feb 24 12:20:20 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Fri, 24 Feb 2012 12:20:20 +0100 Subject: Amavisd cant fork with a lot of emails In-Reply-To: References: Message-ID: <201202241220.20492.Mark.Martinec+amavis@ijs.si> Antonio, > First of all, sorry if this is not the right place to ask this > question. We are setting upan antispam server using spamassassin, > maia, amavisd-new, etc. All is ok. But, once in a while, we have > problems in amavis, when create a new process to check attached data > in an email. > > We are getting an strange message on postfix logs after a while. The > message is like that: > > (host 127.0.0.1[127.0.0.1] said: 451 4.5.0 Error in processing, > id=11968-01-10, mime_decode-1 FAILED: run_command (open pipe): > > Can't fork at /usr/lib/perl5/5.10.0/x86_64-linux-thread-multi/IO/File.pm > line 188. at /usr/sbin/amavisd line 1657. (in reply to end of DATA > command)) YYYYY foo.bar > > I am afraid that the problem is there isn't enough resources to > allocate a new thread. The message goes to the deferred queue. The > problem is, that when its occurs, the deferred queue starts to > increase quickly, and the memory goes down quickly also. For this > reason I have configured via limits.conf to increase the process on > the * account, but it doesnt works: > > /etc/security/limits.conf > > * soft nofile 65535 > * hard nofile 65535 > * soft nproc 65535 > * hard nproc 65535 > > Amavis is running under the amavis user account. My server is a SLES > 11 (virtual running under VMWARE), with 4 GB of ram > and 1,5 GB of swap. When I have this problem, the free memory quickly > goes down, like the swap. :((( > > Any ideas? So much thanks. See man page on fork(2). Besides nproc and MAXPROC limits, the next likely cause for failing to fork is that the system run out of swap space. My guess is that you hit the swap space exhaution, and from then on things can only get worse. Make sure to have sufficient swap space. It is common to allocate twice or four times the size of RAM to swap. Mark From agutierr at gmail.com Fri Feb 24 12:31:22 2012 From: agutierr at gmail.com (=?ISO-8859-1?Q?Antonio_Guti=E9rrez_Mayoral?=) Date: Fri, 24 Feb 2012 12:31:22 +0100 Subject: Amavisd cant fork with a lot of emails In-Reply-To: References: <201202241220.20492.Mark.Martinec+amavis@ijs.si> Message-ID: >From the logs: [1975854.842224] 51730 total pagecache pages [1975854.842226] 51672 pages in swap cache [1975854.842227] Swap cache stats: add 8314730, delete 8263058, find 134959310/135609572 [1975854.842228] Free swap = 0kB [1975854.842229] Total swap = 1534168kB [1975854.871486] 1048560 pages RAM [1975854.871488] 50144 pages reserved [1975854.871489] 27220 pages shared [1975854.871490] 962447 pages non-shared [1975854.871493] Out of memory: kill process 6695 (amavisd) score 509563 or a child [1975854.871495] Killed process 6701 (amavisd) I think that now the problem is clear, isn it? :-( Antonio. El d?a 24 de febrero de 2012 12:27, Antonio Guti?rrez Mayoral escribi?: > I think that the problem is exactly that, also. I have only 1,5 GB of swap > space, and 4 GB of ram. When this problem occurs, the RAM goes down to 1 GB > free and swap about 350 MB :-( > > Im going to try increasing the swap space and observe the server :-). Thank you > so much for your advice. > > Regards. > > Antonio. > > 2012/2/24 Mark Martinec : >> Antonio, >> >>> First of all, sorry if this is not the right place to ask this >>> question. We are setting upan antispam server using spamassassin, >>> maia, amavisd-new, etc. All is ok. But, once in a while, we have >>> problems in amavis, when create a new process to check attached data >>> in an email. >>> >>> We are getting an strange message on postfix logs after a while. The >>> message is like that: >>> >>> (host 127.0.0.1[127.0.0.1] said: 451 4.5.0 Error in processing, >>> id=11968-01-10, mime_decode-1 FAILED: run_command (open pipe): >>> >>> Can't fork at /usr/lib/perl5/5.10.0/x86_64-linux-thread-multi/IO/File.pm >>> line 188. at /usr/sbin/amavisd line 1657. (in reply to end of DATA >>> command)) ?YYYYY foo.bar >>> >>> I am afraid that the problem is there isn't enough resources to >>> allocate a new thread. The message goes to the deferred queue. The >>> problem is, that when its occurs, the deferred queue starts to >>> increase quickly, and the memory goes down quickly also. For this >>> reason I have configured via limits.conf to increase the process on >>> the * account, but it doesnt works: >>> >>> /etc/security/limits.conf >>> >>> * ? ? ? soft ? ?nofile ?65535 >>> * ? ? ? hard ? ?nofile ?65535 >>> * ? ? ? soft ? ?nproc ? 65535 >>> * ? ? ? hard ? ?nproc ? 65535 >>> >>> Amavis is running under the amavis user account. My server is a SLES >>> 11 (virtual running under VMWARE), with 4 GB of ram >>> and 1,5 GB of swap. When I have this problem, the free memory quickly >>> goes down, like the swap. :((( >>> >>> Any ideas? So much thanks. >> >> See man page on fork(2). Besides nproc and MAXPROC limits, the next >> likely cause for failing to fork is that the system run out of >> swap space. My guess is that you hit the swap space exhaution, >> and from then on things can only get worse. Make sure to have >> sufficient swap space. It is common to allocate twice or four times >> the size of RAM to swap. >> >> ?Mark > > > > -- > -- > Antonio Guti?rrez Mayoral -- -- Antonio Guti?rrez Mayoral From rbgarga at gmail.com Fri Feb 24 12:33:22 2012 From: rbgarga at gmail.com (Renato Botelho) Date: Fri, 24 Feb 2012 09:33:22 -0200 Subject: Relation between max_servers and ncpu Message-ID: Hello, I run a server with postfix + amavisd-new + clamav + spamassassin, this server have 8 cores and 16Gb of RAM. We have max_servers=6 set on amavisd.conf and it works as expected, but, when a good number of emails come at the same time, it takes a long time to process the entire queue. I did a simple math average, email are taking about 3s to be processed. I'm considering increase max_servers on this environment, but i'm worried about if I set it to a number bigger than the number of current CPUs won't be a problem for the other services running in this machine. Do you have any consideration about that subject? How is the best way to define max_servers value? Best Regards -- Renato Botelho From agutierr at gmail.com Fri Feb 24 12:27:49 2012 From: agutierr at gmail.com (=?ISO-8859-1?Q?Antonio_Guti=E9rrez_Mayoral?=) Date: Fri, 24 Feb 2012 12:27:49 +0100 Subject: Amavisd cant fork with a lot of emails In-Reply-To: <201202241220.20492.Mark.Martinec+amavis@ijs.si> References: <201202241220.20492.Mark.Martinec+amavis@ijs.si> Message-ID: I think that the problem is exactly that, also. I have only 1,5 GB of swap space, and 4 GB of ram. When this problem occurs, the RAM goes down to 1 GB free and swap about 350 MB :-( Im going to try increasing the swap space and observe the server :-). Thank you so much for your advice. Regards. Antonio. 2012/2/24 Mark Martinec : > Antonio, > >> First of all, sorry if this is not the right place to ask this >> question. We are setting upan antispam server using spamassassin, >> maia, amavisd-new, etc. All is ok. But, once in a while, we have >> problems in amavis, when create a new process to check attached data >> in an email. >> >> We are getting an strange message on postfix logs after a while. The >> message is like that: >> >> (host 127.0.0.1[127.0.0.1] said: 451 4.5.0 Error in processing, >> id=11968-01-10, mime_decode-1 FAILED: run_command (open pipe): >> >> Can't fork at /usr/lib/perl5/5.10.0/x86_64-linux-thread-multi/IO/File.pm >> line 188. at /usr/sbin/amavisd line 1657. (in reply to end of DATA >> command)) ?YYYYY foo.bar >> >> I am afraid that the problem is there isn't enough resources to >> allocate a new thread. The message goes to the deferred queue. The >> problem is, that when its occurs, the deferred queue starts to >> increase quickly, and the memory goes down quickly also. For this >> reason I have configured via limits.conf to increase the process on >> the * account, but it doesnt works: >> >> /etc/security/limits.conf >> >> * ? ? ? soft ? ?nofile ?65535 >> * ? ? ? hard ? ?nofile ?65535 >> * ? ? ? soft ? ?nproc ? 65535 >> * ? ? ? hard ? ?nproc ? 65535 >> >> Amavis is running under the amavis user account. My server is a SLES >> 11 (virtual running under VMWARE), with 4 GB of ram >> and 1,5 GB of swap. When I have this problem, the free memory quickly >> goes down, like the swap. :((( >> >> Any ideas? So much thanks. > > See man page on fork(2). Besides nproc and MAXPROC limits, the next > likely cause for failing to fork is that the system run out of > swap space. My guess is that you hit the swap space exhaution, > and from then on things can only get worse. Make sure to have > sufficient swap space. It is common to allocate twice or four times > the size of RAM to swap. > > ?Mark -- -- Antonio Guti?rrez Mayoral From luc.maignan at winxpert.com Fri Feb 24 13:10:56 2012 From: luc.maignan at winxpert.com (Luc MAIGNAN) Date: Fri, 24 Feb 2012 13:10:56 +0100 Subject: Blacklist from encoding Message-ID: <4F477E50.6090807@winxpert.com> Hi, it is possible to set a mail as a spam according to its encoding ? Same question with sending country (GeoIP ?) ? Thanks for any help BR From hege at hege.li Fri Feb 24 13:15:16 2012 From: hege at hege.li (Henrik K) Date: Fri, 24 Feb 2012 14:15:16 +0200 Subject: Relation between max_servers and ncpu In-Reply-To: References: Message-ID: <20120224121515.GA28829@smtp.hege.li> On Fri, Feb 24, 2012 at 09:33:22AM -0200, Renato Botelho wrote: > Hello, > > I run a server with postfix + amavisd-new + clamav + spamassassin, this > server have 8 cores and 16Gb of RAM. We have max_servers=6 set on > amavisd.conf and it works as expected, but, when a good number of emails > come at the same time, it takes a long time to process the entire queue. > > I did a simple math average, email are taking about 3s to be processed. > > I'm considering increase max_servers on this environment, but i'm worried > about if I set it to a number bigger than the number of current CPUs won't > be a problem for the other services running in this machine. > > Do you have any consideration about that subject? How is the best way to > define max_servers value? Look, I have max_servers=20 on an ancient 4x1Ghz SPARC with no problems. One amavisd process cannot take 100% cpu for all the 3 seconds. Most of that time is waiting for DNS lookups. Just put something like 20-30 depending on how you want to prioritize other services and mail flow. If you put too many, the worst case scenario is that some processes get marginally slower (but total throughput will be great). You need to experiment. From Mark.Martinec+amavis at ijs.si Fri Feb 24 14:19:55 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Fri, 24 Feb 2012 14:19:55 +0100 Subject: Relation between max_servers and ncpu In-Reply-To: <20120224121515.GA28829@smtp.hege.li> References: <20120224121515.GA28829@smtp.hege.li> Message-ID: <201202241419.56018.Mark.Martinec+amavis@ijs.si> Renato, > > I run a server with postfix + amavisd-new + clamav + spamassassin, this > > server have 8 cores and 16Gb of RAM. We have max_servers=6 set on > > amavisd.conf and it works as expected, but, when a good number of emails > > come at the same time, it takes a long time to process the entire queue. > > > > I did a simple math average, email are taking about 3s to be processed. > > > > I'm considering increase max_servers on this environment, but i'm worried > > about if I set it to a number bigger than the number of current CPUs > > won't be a problem for the other services running in this machine. > > > > Do you have any consideration about that subject? How is the best way to > > define max_servers value? Henrik K wrote: > Look, I have max_servers=20 on an ancient 4x1Ghz SPARC with no problems. > > One amavisd process cannot take 100% cpu for all the 3 seconds. Most of > that time is waiting for DNS lookups. > > Just put something like 20-30 depending on how you want to prioritize other > services and mail flow. > > If you put too many, the worst case scenario is that some processes get > marginally slower (but total throughput will be great). You need to > experiment. I agree with Henrik. For a reasonably modern hardware with sufficient memory and a sizable mail traffic the max_servers=6 is too low. The 20-30 is about right, and even 50 may be acceptable if need be. There are plenty of latencies during processing of a mail message, especially if spam scanning is enabled. These idle times can be put to good use when running multiple child processes. Keep an eye on the RSS usage (real memory) used by amavisd processes, these should not come too close to available RAM size. Also see CPU idle time: when CPU usage approaches 90 or 100% on the average, adding more child processes brings no advantage, just wastes memory. In other words: if mail processing is falling behind the traffic rate and CPU utilization is not close to max and memory permits, increase the $max_servers and a corresponding MTA stting. If amavisd-nanny shows that amavisd processes are frequently idling, or if CPU usage is close to 100% for long periods, or if memory is tight, decrease $max_servers setting. Mark From andrea.gabellini.sc at telecomitalia.sm Sat Feb 25 10:02:58 2012 From: andrea.gabellini.sc at telecomitalia.sm (Andrea gabellini - SC) Date: Sat, 25 Feb 2012 10:02:58 +0100 Subject: bypass_decode_parts and clamd Message-ID: <4F48A3C2.90209@telecomitalia.sm> Hello, I'm migrating from Sendmail+MIMEdefang to Postfix+Amavisd (both with SpamAssassin and ClamAV). I read about enabling bypass_decode_parts if clamd is used for better performance. I don't use the banned filename checks. Is it correct? Which is the best practice? Thanks in advance, Andrea From Ralf.Hildebrandt at charite.de Sat Feb 25 19:28:35 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 25 Feb 2012 19:28:35 +0100 Subject: lha crash, but which mail caused it? Message-ID: <20120225182835.GB28743@charite.de> Today I found: Feb 25 13:39:41 mail kernel: [949050.319465] lha[2480]: segfault at bfec787d ip 0804d62a sp bfec316c error 4 in lha[8048000+d000] Since amavis is the only program to use lha I'd like to know which mail caused thus. But how do I find the mail that caused this? # fgrep amavis /var/log/mail.log |egrep " 13:3[89]:" | grep -i Content-Type Feb 25 13:38:06 mail amavis[511]: (00511-07) p001 1 Content-Type: text/plain, size: 1550 B, name: Feb 25 13:38:06 mail amavis[31924]: (31924-15) p001 1 Content-Type: text/html, size: 15668 B, name: Feb 25 13:38:09 mail amavis[32013]: (32013-06) p001 1 Content-Type: text/html, size: 28 B, name: Feb 25 13:38:09 mail amavis[31924]: (31924-16) p003 1 Content-Type: multipart/alternative Feb 25 13:38:09 mail amavis[31924]: (31924-16) p001 1/1 Content-Type: text/plain, size: 1135 B, name: Feb 25 13:38:09 mail amavis[31924]: (31924-16) p002 1/2 Content-Type: text/html, size: 30681 B, name: Feb 25 13:38:18 mail amavis[1485]: (01485-06) p001 1 Content-Type: text/html, size: 8712 B, name: Feb 25 13:38:23 mail amavis[31924]: (31924-17) p003 1 Content-Type: multipart/alternative Feb 25 13:38:23 mail amavis[31924]: (31924-17) p001 1/1 Content-Type: text/plain, size: 1138 B, name: Feb 25 13:38:23 mail amavis[31924]: (31924-17) p004 1/2 Content-Type: multipart/related Feb 25 13:38:23 mail amavis[31924]: (31924-17) p002 1/2/1 Content-Type: text/html, size: 7470 B, name: Feb 25 13:38:27 mail amavis[30833]: (30833-19) p003 1 Content-Type: multipart/mixed Feb 25 13:38:27 mail amavis[30833]: (30833-19) p004 1/1 Content-Type: multipart/alternative Feb 25 13:38:27 mail amavis[30833]: (30833-19) p001 1/1/1 Content-Type: text/plain, size: 9810 B, name: Feb 25 13:38:27 mail amavis[30833]: (30833-19) p002 1/1/2 Content-Type: text/html, size: 105514 B, name: Feb 25 13:38:31 mail amavis[1485]: (01485-07) p003 1 Content-Type: multipart/alternative Feb 25 13:38:31 mail amavis[1485]: (01485-07) p001 1/1 Content-Type: text/plain, size: 1139 B, name: Feb 25 13:38:31 mail amavis[1485]: (01485-07) p002 1/2 Content-Type: text/html, size: 30685 B, name: Feb 25 13:38:33 mail amavis[30817]: (30817-14) p001 1 Content-Type: text/plain, size: 3202 B, name: Feb 25 13:38:47 mail amavis[26705]: (26705-15) p001 1 Content-Type: text/plain, size: 3583 B, name: Feb 25 13:38:50 mail amavis[30833]: (30833-20) p003 1 Content-Type: multipart/alternative Feb 25 13:38:50 mail amavis[30833]: (30833-20) p001 1/1 Content-Type: text/plain, size: 756 B, name: Feb 25 13:38:50 mail amavis[30833]: (30833-20) p002 1/2 Content-Type: text/html, size: 22982 B, name: Feb 25 13:38:51 mail amavis[32257]: (32257-10) p010 1 Content-Type: multipart/related Feb 25 13:38:51 mail amavis[32257]: (32257-10) p011 1/1 Content-Type: multipart/alternative Feb 25 13:38:51 mail amavis[32257]: (32257-10) p001 1/1/1 Content-Type: text/plain, size: 4366 B, name: Feb 25 13:38:51 mail amavis[32257]: (32257-10) p002 1/1/2 Content-Type: text/html, size: 83048 B, name: Feb 25 13:38:51 mail amavis[32257]: (32257-10) p003 1/2 Content-Type: image/jpeg, size: 7732 B, name: clip_image001[2].jpg Feb 25 13:38:51 mail amavis[32257]: (32257-10) p004 1/3 Content-Type: image/jpeg, size: 6113 B, name: clip_image002[2].jpg Feb 25 13:38:51 mail amavis[32257]: (32257-10) p005 1/4 Content-Type: image/png, size: 335531 B, name: clip_image004[2].png Feb 25 13:38:51 mail amavis[32257]: (32257-10) p006 1/5 Content-Type: image/png, size: 567252 B, name: clip_image006[2].png Feb 25 13:38:51 mail amavis[32257]: (32257-10) p007 1/6 Content-Type: image/png, size: 494832 B, name: clip_image008[2].png Feb 25 13:38:51 mail amavis[32257]: (32257-10) p008 1/7 Content-Type: image/png, size: 678546 B, name: clip_image010[2].png Feb 25 13:38:51 mail amavis[32257]: (32257-10) p009 1/8 Content-Type: image/png, size: 533920 B, name: clip_image012[2].png Feb 25 13:39:03 mail amavis[2072]: (02072-01) p004 1 Content-Type: multipart/mixed Feb 25 13:39:03 mail amavis[2072]: (02072-01) p005 1/1 Content-Type: multipart/alternative Feb 25 13:39:03 mail amavis[2072]: (02072-01) p001 1/1/1 Content-Type: text/plain, size: 260 B, name: Feb 25 13:39:03 mail amavis[2072]: (02072-01) p002 1/1/2 Content-Type: text/html, size: 393 B, name: Feb 25 13:39:03 mail amavis[2072]: (02072-01) p003 1/2 Content-Type: image/jpeg, size: 280694 B, name: Approba.jpg Feb 25 13:39:04 mail amavis[603]: (00603-03) p001 1 Content-Type: text/html, size: 12882 B, name: Feb 25 13:39:09 mail amavis[2072]: (02072-02) p001 1 Content-Type: text/html, size: 12858 B, name: Feb 25 13:39:21 mail amavis[603]: (00603-04) p001 1 Content-Type: text/html, size: 3121 B, name: Feb 25 13:39:26 mail amavis[30937]: (30937-15) p001 1 Content-Type: text/plain, size: 1367 B, name: Feb 25 13:39:27 mail amavis[2072]: (02072-03) p001 1 Content-Type: text/html, size: 9857 B, name: Feb 25 13:39:27 mail amavis[30937]: (30937-16) p003 1 Content-Type: multipart/alternative Feb 25 13:39:27 mail amavis[30937]: (30937-16) p001 1/1 Content-Type: text/plain, size: 189 B, name: Feb 25 13:39:27 mail amavis[30937]: (30937-16) p002 1/2 Content-Type: text/html, size: 3884 B, name: Feb 25 13:39:29 mail amavis[30817]: (30817-15) p003 1 Content-Type: multipart/alternative Feb 25 13:39:29 mail amavis[30817]: (30817-15) p001 1/1 Content-Type: text/plain, size: 14 B, name: Feb 25 13:39:29 mail amavis[30817]: (30817-15) p002 1/2 Content-Type: text/html, size: 9963 B, name: Feb 25 13:39:30 mail amavis[1485]: (01485-08) p004 1 Content-Type: multipart/mixed Feb 25 13:39:30 mail amavis[1485]: (01485-08) p001 1/1 Content-Type: text/plain, size: 402 B, name: Feb 25 13:39:30 mail amavis[1485]: (01485-08) p002 1/2 Content-Type: message/rfc822, size: 58543 B, name: Feb 25 13:39:30 mail amavis[1485]: (01485-08) p003 1/3 Content-Type: message/rfc822, size: 701 B, name: Feb 25 13:39:37 mail amavis[30937]: (30937-17) p003 1 Content-Type: multipart/mixed Feb 25 13:39:37 mail amavis[30937]: (30937-17) p001 1/1 Content-Type: text/plain, size: 1 B, name: Feb 25 13:39:37 mail amavis[30937]: (30937-17) p002 1/2 Content-Type: application/zip, size: 3897513 B, name: humanresearch.zip Feb 25 13:39:38 mail amavis[31924]: (31924-18) p001 1 Content-Type: text/plain, size: 990 B, name: Feb 25 13:39:49 mail amavis[1540]: (01540-01) p003 1 Content-Type: multipart/alternative Feb 25 13:39:49 mail amavis[1540]: (01540-01) p001 1/1 Content-Type: text/plain, size: 9250 B, name: Feb 25 13:39:49 mail amavis[1540]: (01540-01) p002 1/2 Content-Type: text/html, size: 16255 B, name: Feb 25 13:39:53 mail amavis[497]: (00497-10) p001 1 Content-Type: text/html, size: 3103 B, name: Feb 25 13:39:56 mail amavis[1540]: (01540-02) p001 1 Content-Type: text/plain, size: 958 B, name: mail:/var/amavis# fgrep amavis /var/log/mail.log |egrep -i preserving mail:/var/amavis# -- Ralf Hildebrandt Charite Universit?tsmedizin Berlin ralf.hildebrandt at charite.de Campus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Gesch?ftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155 From esavage at digitalrage.org Sat Feb 25 23:28:58 2012 From: esavage at digitalrage.org (Elijah Savage) Date: Sat, 25 Feb 2012 17:28:58 -0500 Subject: Help blocking Spam Message-ID: <43CB5730-1242-4EB1-B115-7D00E01B7641@digitalrage.org> Freebsd 9 amavisd spamassassin postfix I am so tired of seeing this magic jack email, this email is coming from multiple domains. No, score=0.516 tagged_above=-10 required=4 tests=[BAYES_50=0.8, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, LOTS_OF_MONEY=0.001, MIME_HTML_ONLY=0.723, T_RP_MATCHES_RCVD=-0.01, DSPAM:Innocent=-1.000] autolearn=no -------------- next part -------------- An HTML attachment was scrubbed... URL: From jarif at iki.fi Sun Feb 26 00:29:50 2012 From: jarif at iki.fi (Jari Fredriksson) Date: Sun, 26 Feb 2012 01:29:50 +0200 Subject: Help blocking Spam In-Reply-To: <43CB5730-1242-4EB1-B115-7D00E01B7641@digitalrage.org> References: <43CB5730-1242-4EB1-B115-7D00E01B7641@digitalrage.org> Message-ID: <4F496EEE.1070400@iki.fi> 26.2.2012 0:28, Elijah Savage kirjoitti: > Freebsd 9 > amavisd > spamassassin > postfix > > I am so tired of seeing this magic jack email, this email is coming from > multiple domains. > > No, score=0.516 tagged_above=-10 required=4 tests=[BAYES_50=0.8, > HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, LOTS_OF_MONEY=0.001, > MIME_HTML_ONLY=0.723, T_RP_MATCHES_RCVD=-0.01, DSPAM:Innocent=-1.000] > autolearn=no > Your message does not help us helping blocking spam. Put the spam to pastebin and we might have a look. Thanks for not posting it to the list. -- Fine day to work off excess energy. Steal something heavy. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From esavage at digitalrage.org Sun Feb 26 01:53:00 2012 From: esavage at digitalrage.org (Elijah Savage) Date: Sat, 25 Feb 2012 19:53:00 -0500 Subject: Help blocking Spam In-Reply-To: <4F496EEE.1070400@iki.fi> References: <43CB5730-1242-4EB1-B115-7D00E01B7641@digitalrage.org> <4F496EEE.1070400@iki.fi> Message-ID: I was hoping someone would recommend which scores they may adjust. On Feb 25, 2012, at 6:29 PM, Jari Fredriksson wrote: > 26.2.2012 0:28, Elijah Savage kirjoitti: >> Freebsd 9 >> amavisd >> spamassassin >> postfix >> >> I am so tired of seeing this magic jack email, this email is coming from >> multiple domains. >> >> No, score=0.516 tagged_above=-10 required=4 tests=[BAYES_50=0.8, >> HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, LOTS_OF_MONEY=0.001, >> MIME_HTML_ONLY=0.723, T_RP_MATCHES_RCVD=-0.01, DSPAM:Innocent=-1.000] >> autolearn=no >> > > Your message does not help us helping blocking spam. > > Put the spam to pastebin and we might have a look. Thanks for not > posting it to the list. > > -- > > Fine day to work off excess energy. Steal something heavy. > From njones at megan.vbhcs.org Sun Feb 26 02:02:04 2012 From: njones at megan.vbhcs.org (Noel Jones) Date: Sat, 25 Feb 2012 19:02:04 -0600 Subject: Help blocking Spam In-Reply-To: References: <43CB5730-1242-4EB1-B115-7D00E01B7641@digitalrage.org> <4F496EEE.1070400@iki.fi> Message-ID: <4F49848C.8000404@megan.vbhcs.org> On 2/25/2012 6:53 PM, Elijah Savage wrote: > I was hoping someone would recommend which scores they may adjust. That question is meaningless without context. Sort of like "My stew doesn't turn out right, which ingredient do I need to adjust?" Put a couple samples (including all headers) in a pastebin so we can help you find a safe and effective solution. You may mangle the recipient email address, but please leave the rest intact. From esavage at digitalrage.org Sun Feb 26 03:11:48 2012 From: esavage at digitalrage.org (Elijah Savage) Date: Sat, 25 Feb 2012 21:11:48 -0500 Subject: Help blocking Spam In-Reply-To: <43CB5730-1242-4EB1-B115-7D00E01B7641@digitalrage.org> References: <43CB5730-1242-4EB1-B115-7D00E01B7641@digitalrage.org> Message-ID: <8BD2A390-629E-4DF6-982D-0014DD9A9A23@digitalrage.org> Understood but I think I have it sorry to be of bother. On Feb 25, 2012, at 5:28 PM, Elijah Savage wrote: > Freebsd 9 > amavisd > spamassassin > postfix > > I am so tired of seeing this magic jack email, this email is coming from multiple domains. > > No, score=0.516 tagged_above=-10 required=4 tests=[BAYES_50=0.8, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, LOTS_OF_MONEY=0.001, MIME_HTML_ONLY=0.723, T_RP_MATCHES_RCVD=-0.01, DSPAM:Innocent=-1.000] autolearn=no -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at junc.org Sun Feb 26 07:49:43 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 26 Feb 2012 07:49:43 +0100 Subject: Help blocking Spam In-Reply-To: References: <43CB5730-1242-4EB1-B115-7D00E01B7641@digitalrage.org> <4F496EEE.1070400@iki.fi> Message-ID: Den 2012-02-26 01:53, Elijah Savage skrev: > I was hoping someone would recommend which scores they may adjust. that would be incorrect to adjust scores, better is to create tests in spamassassin to hit on this specific spams, simplest way is sa-learn --spam msgfile i can see you using dspam with amavis ? how do you train it ? i would like to use dspam in amavis, but so far found dovecot-antispam works with dspam From patrickdk at patrickdk.com Mon Feb 27 01:49:38 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sun, 26 Feb 2012 19:49:38 -0500 Subject: Help blocking Spam In-Reply-To: References: <43CB5730-1242-4EB1-B115-7D00E01B7641@digitalrage.org> <4F496EEE.1070400@iki.fi> Message-ID: <20120226194938.Horde.QVmtYpLnE6FPStMitUrTTHA@kishi.patrickdk.com> I personally just have a script parse my spam folder each night and send them to the learner. I also can't locate any meaningful info in those emails to id them with. The magic jack is just one of like 20-30 different variations of that spam source. They are just hacking webservers, and sending the spam from them, and looks like they move webservers that they hacked every hour or two. Just making blacklisting or tracking it basically impossible. I did setup my own blacklist, that works kind of like the mailspike.org Z list, that when a spam wave hits, they get blacklisted for 5hours, that seems to be working surpisingly well. Not sure exactly why the mailspike one isn't working as good. Maybe cause they are only basing it on spamtraps? and I'm doing it just based on spam average per ip in the last hour. Quoting Benny Pedersen : > Den 2012-02-26 01:53, Elijah Savage skrev: >> I was hoping someone would recommend which scores they may adjust. > > that would be incorrect to adjust scores, better is to create tests > in spamassassin to hit on this specific spams, simplest way is > sa-learn --spam msgfile > > i can see you using dspam with amavis ? > > how do you train it ? > > i would like to use dspam in amavis, but so far found > dovecot-antispam works with dspam From patrickdk at patrickdk.com Mon Feb 27 01:44:39 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sun, 26 Feb 2012 19:44:39 -0500 Subject: Relation between max_servers and ncpu In-Reply-To: <201202241419.56018.Mark.Martinec+amavis@ijs.si> References: <20120224121515.GA28829@smtp.hege.li> <201202241419.56018.Mark.Martinec+amavis@ijs.si> Message-ID: <20120226194439.Horde.do6DBZLnE6FPStH3mo5B8FA@kishi.patrickdk.com> Quoting Mark Martinec : > Henrik K wrote: >> Look, I have max_servers=20 on an ancient 4x1Ghz SPARC with no problems. >> >> One amavisd process cannot take 100% cpu for all the 3 seconds. Most of >> that time is waiting for DNS lookups. >> >> Just put something like 20-30 depending on how you want to prioritize other >> services and mail flow. >> >> If you put too many, the worst case scenario is that some processes get >> marginally slower (but total throughput will be great). You need to >> experiment. > > I agree with Henrik. For a reasonably modern hardware with sufficient memory > and a sizable mail traffic the max_servers=6 is too low. The 20-30 is about > right, and even 50 may be acceptable if need be. > > There are plenty of latencies during processing of a mail message, especially > if spam scanning is enabled. These idle times can be put to good use when > running multiple child processes. > > Keep an eye on the RSS usage (real memory) used by amavisd processes, > these should not come too close to available RAM size. Also see CPU idle > time: when CPU usage approaches 90 or 100% on the average, adding > more child processes brings no advantage, just wastes memory. > > In other words: if mail processing is falling behind the traffic rate > and CPU utilization is not close to max and memory permits, increase > the $max_servers and a corresponding MTA stting. > > If amavisd-nanny shows that amavisd processes are frequently idling, > or if CPU usage is close to 100% for long periods, or if memory is > tight, decrease $max_servers setting. Pretty much same results here. I found about 30 is good for me, with 6 cores and 6gigs ram. Increasing to even 60 works well, but needs more memory. At 30, I rarely ever get more than 3-4 cores used. From Mark.Martinec+amavis at ijs.si Mon Feb 27 16:11:08 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Mon, 27 Feb 2012 16:11:08 +0100 Subject: lha crash, but which mail caused it? In-Reply-To: <20120225182835.GB28743@charite.de> References: <20120225182835.GB28743@charite.de> Message-ID: <201202271611.08161.Mark.Martinec+amavis@ijs.si> Ralf, > Today I found: > Feb 25 13:39:41 mail kernel: [949050.319465] lha[2480]: segfault at > bfec787d ip 0804d62a sp bfec316c error 4 in lha[8048000+d000] > > Since amavis is the only program to use lha I'd like to know which > mail caused thus. But how do I find the mail that caused this? If you had logging level at 5, the PID of each launched process would be logged by do_log(5,"run_command: [%s] %s", $pid,$msg); > # fgrep amavis /var/log/mail.log |egrep " 13:3[89]:" | grep -i Content-Type This gives you only MIME-level types (top-level mostly). To also see nested part types a search for 'p.path ' would be more revealing (logged at log level 3 for all, and at 1 for banned). The lha decoder is launched for parts that look like in lha format, but also by do_executable(), which tries several decoders on an executable in an attampt to guess a self-extracting archives. So in your case, considering a timestamp and the fact that most other mail messages close in time to the event were text-only, the most likely culprit is: Feb 25 13:39:37 mail amavis[30937]: (30937-17) p002 1/2 Content-Type: application/zip, size: 3897513 B, name: humanresearch.zip Mark From Mark.Martinec+amavis at ijs.si Mon Feb 27 17:14:48 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Mon, 27 Feb 2012 17:14:48 +0100 Subject: bypass_decode_parts and clamd In-Reply-To: <4F48A3C2.90209@telecomitalia.sm> References: <4F48A3C2.90209@telecomitalia.sm> Message-ID: <201202271714.48579.Mark.Martinec+amavis@ijs.si> Andrea, > I'm migrating from Sendmail+MIMEdefang to Postfix+Amavisd (both with > SpamAssassin and ClamAV). > > I read about enabling bypass_decode_parts if clamd is used for better > performance. I don't use the banned filename checks. > > Is it correct? Which is the best practice? Yes, it is correct. Setting the: $bypass_decode_parts = 1; @decoders = (); makes sense if you trust your virus scanners are capable of decoding most archive formats and are capable of defenging themselves against mail bombs, *and* you have no need for information on mail structure, which is used by banned filename checks. > Which is the best practice? Hard to say, both approaches have their merits. The performance is probably the least important decision factor here, especially if you have spam scanning enabled which is typically much slower than decoding. Deciding whether to better trust decoders as used by amavisd, or the ones implemented in a virus scanner is more important in my view. Mark From Mark.Martinec+amavis at ijs.si Mon Feb 27 17:55:47 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Mon, 27 Feb 2012 17:55:47 +0100 Subject: Blacklist from encoding In-Reply-To: <4F477E50.6090807@winxpert.com> References: <4F477E50.6090807@winxpert.com> Message-ID: <201202271755.47446.Mark.Martinec+amavis@ijs.si> Luc, > it is possible to set a mail as a spam according to its encoding ? > Same question with sending country (GeoIP ?) ? There are no direct provisions for this in amavisd, such decisions are commonly delegated to SpamAssassin. Here are some of my SA rules (in file local.cf) to contribute some spam score points for character set encodings not normally seen by our users except in spam: header __L_CHARSET1 ALL =~ m{=\?(iso-8859-9|GB2312)\?}i header __L_CHARSET2 ALL =~ m{\bwindows-1254}i header __L_CHARSET3 Content-Type =~ m{\b(?:windows-1256|ISO-8859-6)\b}i full __L_CHARSET4 /^Content-Type:.*\bcharset=(?:windows-1256|ISO-8859-6)\b/mi header __L_CHARSET5 Subject =~ m{\b(?:windows-1256|ISO-8859-6)\b}i header __L_CHARSET6 From:raw =~ m{\b(?:windows-1256|ISO-8859-6)\b}i meta L_ARABIC __L_CHARSET3 || __L_CHARSET4 || __L_CHARSET5 || __L_CHARSET6 score L_ARABIC 3 meta L_CHARSET __L_CHARSET1 + __L_CHARSET2 + L_ARABIC >= 1 score L_CHARSET 1.6 Regarding the GeoIP, there is some experimental SA plugin named RelayCountry2 which uses Geo::IP, but essentially achieves similar functionality as the existing SA plugin RelayCountry, which uses an older IP::Country::Fast module. There is also an amavisd custom hook using Geo::IP, which adds an information header field about the country, but does not affect spam score. I guess it can be modified to achieve more. See amavis-users archive around 2011-05, subject: "amavisd plugin for the geolocation addresses and headers X-Anti-Abuse". Mark From Mark.Martinec+amavis at ijs.si Mon Feb 27 19:59:38 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Mon, 27 Feb 2012 19:59:38 +0100 Subject: Amavisd and clamav socket do not match In-Reply-To: <4EFC393F.8000104@schinagl.nl> References: <4EFC393F.8000104@schinagl.nl> Message-ID: <201202271959.38955.Mark.Martinec+amavis@ijs.si> Oliver, > Amavis currently uses "/var/run/clamav/clamd" as its socket in > @av_scanners. > > The default, although commented socket in clamav is /tmp/clamd.socket > > Especially to new users on new installations this may seem odd, and may > confuse users on to which default to change may be daunting. > > Gentoo uses "/var/run/clamav/clamd.sock" as it's default socket in > clamav, which seems like the perfect solution. The location makes more > sense than /tmp; and the name is more sensible then 'just' clamd. socket > might be better then sock, but sock is shorter and still clear ;) > > I thus suggest the default configuration stanza's to be updated to use > "/var/run/clamav/clamd.sock" in both amavisd.conf and clamd.conf to give > sensible defaults. Agreed, thanks for the suggestion. Will be changed for 2.7.1 and 2.8.0: - ClamAV-clamd and ClamAV-clamd-stream av scanners: changed socket name in a sample configuration file amavisd.conf to /var/run/clamav/clamd.sock (previously the socket name was /var/run/clamav/clamd); this makes it compatible with a default socket name under at least Gentoo and FreeBSD; Mark From ckoeber at gmail.com Mon Feb 27 20:19:55 2012 From: ckoeber at gmail.com (Christopher Kurtis Koeber) Date: Mon, 27 Feb 2012 14:19:55 -0500 Subject: Adding a subject tag to emails with bad headers ... Message-ID: <00dc01ccf584$ca58b0c0$5f0a1240$@gmail.com> Hello, I have a Postfix+Amavisd+Spamassasin+ClamV setup that works great for filtering and I would just like to add one feature for my clients: Is it possible to add a tag like "***spam***" or something for emails that have bad headers? We get a lot of wishy/washy emails that, while all of them are not spam, all of them have bad headers which are correctly flagged by the previously mentioned setup. I would just like to add a "***spam***" subject line to it so clients who don't want the messages can use the rules on their email clients to decide for themselves. Thanks. Regards, Christopher Koeber -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mark.Martinec+amavis at ijs.si Mon Feb 27 21:30:00 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Mon, 27 Feb 2012 21:30:00 +0100 Subject: unchecked_lovers_maps in sql READMEs In-Reply-To: <20111205160426.608194d0@spook.bendtel.com> References: <20111119211831.2c7d5d0b@sleeper.dirtymonday.net> <201111222008.32464.Mark.Martinec+amavis@ijs.si> <20111205160426.608194d0@spook.bendtel.com> Message-ID: <201202272130.00789.Mark.Martinec+amavis@ijs.si> Tim, > Here's a diff with some other changes that should make people's lives > easier with Pg. Some of these are obvious omissions, some are things > that were required for the sqhema to load correctly, and some or just > preferences or clarifications... This includes the previously > discussed fix. I finally came around to merge your changes with my current contents. Attached is my current version. I think the only significant thing I left out from your change was adding of UNIQUE to msgs.mail_id. This field is not necessarily unique. What is unique is a concatenation of partition_tag and mail_id in that table, which is already covered by CONSTRAINT and PRIMARY KEY. Thanks for the update. Mark -------------- next part -------------- A non-text attachment was scrubbed... Name: README.sql-pg Type: text/x-readme Size: 21344 bytes Desc: not available URL: From jacob at andrade.dk Tue Feb 28 10:30:36 2012 From: jacob at andrade.dk (Jacob d Andrade) Date: Tue, 28 Feb 2012 10:30:36 +0100 Subject: Clearing contents_category from a custom hook? Message-ID: <20120228091948.M9452@andrade.dk> Hello I've written a custom whitelist-hook in the custom's file, where a whitelist-file is loaded, and matched against both body and envelope from addresses. ( I know that a whitelist metodh is only supposed to match against the envelope address, but then I'd just as well use amavis' own whitelist function, but since it's an requirement from the users that they can whitelist both body and envelope addresses, I have to do it this way ) But in the case, where there is a banned file attached to an email from a whitelisted address, the email is banned anyway. I'd like to reset the contents_category to CC_CLEAN in the case where my own whitelist has cleared the email. Is that possible? I'm able to get the current status of the email from per_recip_data and contents_category, where I can see that the email has been flagged in CC_CLEAN and CC_BANNED ( 1 and 8 ). thanks for a nice piece of software :-) kind regards jacob From Mark.Martinec+amavis at ijs.si Tue Feb 28 15:21:42 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Tue, 28 Feb 2012 15:21:42 +0100 Subject: Clearing contents_category from a custom hook? In-Reply-To: <20120228091948.M9452@andrade.dk> References: <20120228091948.M9452@andrade.dk> Message-ID: <201202281521.42841.Mark.Martinec+amavis@ijs.si> Jacob, > I've written a custom whitelist-hook in the custom's file, where a > whitelist-file is loaded, and matched against both body and envelope from > addresses. > ( I know that a whitelist metodh is only supposed to match against the > envelope address, but then I'd just as well use amavis' own whitelist > function, but since it's an requirement from the users that they can > whitelist both body and envelope addresses, I have to do it this way ) The white/black-listing in amavisd was based only on envelope sender address in versions older than 2.6.0. Starting with 2.6.0, both the envelope and the author address are taken into account. amavisd-new-2.6.0 release notes: COMPATIBILITY WITH 2.5.4 - white and blacklisting now takes into account both the SMTP envelope sender address, as well as the author address from a header section (address(es) in a 'From:' header field). Note that whitelisting based only on a sender-specified address is mostly useless nowadays. For a reliable whitelisting see @author_to_policy_bank_maps below, as well as a set of whitelisting possibilities in SpamAssassin (based on DKIM, SPF, or on Received header fields); > But in the case, where there is a banned file attached to an email from a > whitelisted address, the email is banned anyway. Whitelisting based on information provided by the sender may be acceptable for low-risk decisions like spam filtering, but should not be used to bypass banning or virus checks. Use DKIM-based whitelisting for such purpose: amavisd-new-2.6.0 release notes: - loading of policy banks based on valid DKIM-signed author's address can be used for reliable whitelisting, for bypassing banned checks, etc. [...] - a new configuration variable @author_to_policy_bank_maps (also a member of policy banks) is a list of lookup tables (typically only a hash-type lookup table is used), which maps author addresses(es) (each address in a 'From:' header field - typically only one) to one or more policy bank names (a comma-separated list of names). A match can only occur if a valid DKIM author domain signature or a valid DKIM third-party signature is found, so in as much as one can trust the signing domain, loading of arbitrary policy banks can be safe, offering a flexibility of whitelisting against spam (absolute or just contributing score points), bypassing of checks (banned, virus, bad-header), using less restrictive banned rules for certain senders, by-sender routing, turning quarantining/archiving on/off, and other tricks offered by the existing policy bank loading mechanisms. When a message has a valid DKIM (or DomainKeys) author domain signature (i.e. when a 'From:' address matches a signing identity according to DKIM (RFC 4871) or DomainKeys (RFC 4870) rules), a lookup key is an unchanged author address and the usual lookup rules apply (README.lookups - hash lookups). When a valid third-party signature is found, a lookup key (author address) is extended by a '/@' and a lowercased signing domain, as shown in the example below. The semantics is very similar to a whitelist_from_dkim feature in SpamAssassin, but is more flexible as is allows any dynamic amavisd setting to be changed depending on author address, not just skipping of spam checks. A few examples of a SpamAssassin's whitelist_from_dkim (as in local.cf) along with equivalent amavisd @author_to_policy_bank_maps entries follow. To whitelist any From address with a domain example.com when a message has a valid author domain signature (i.e. a signature by the same domain): SA: whitelist_from_dkim *@example.com am: 'example.com' => 'WHITELIST', which is equivalent to a lengthy but redundant: SA: whitelist_from_dkim *@example.com example.com am: 'example.com/@example.com' => 'WHITELIST', Similar to above, but applies to subdomains of example.com carrying a valid author domain signature (i.e. signature BY THE SAME SUBDOMAIN): SA: whitelist_from_dkim *@*.example.com am: '.example.com' => 'WHITELIST', Note that in amavisd hash lookups a '.example.com' implies a parent domain 'example.com' too, while in SpamAssassin and in Postfix maps a parent domain needs its own entry if desired. To whitelist From addresses from subdomains of example.com which carry a valid third-party signature of its parent domain: SA: whitelist_from_dkim *@*.example.com example.com am: '.example.com/@example.com' => 'WHITELIST', To whitelist any From address as long as a message has a valid DKIM or DomainKeys signature by example.com, i.e. a third-party signature. Typical for mailing lists or discussion groups which sign postings. SA: whitelist_from_dkim *@* example.com am: './@example.com' => 'WHITELIST', Here is a complete example that can be included in amavisd.conf: @author_to_policy_bank_maps = ( { # 'friends.example.net' => 'WHITELIST,NOBANNEDCHECK', # 'user1 at cust.example.net' => 'WHITELIST,NOBANNEDCHECK', '.ebay.com' => 'WHITELIST', '.ebay.co.uk' => 'WHITELIST', 'members.ebay.co.uk/@ebay.co.uk' => 'WHITELIST', 'ebay.at' => 'WHITELIST', 'ebay.ca' => 'WHITELIST', 'ebay.fr' => 'WHITELIST', 'ebay.de' => 'WHITELIST', 'members.ebay.de/@ebay.de' => 'WHITELIST', '.paypal.co.uk' => 'WHITELIST', '.paypal.com' => 'WHITELIST', # author domain signatures './@paypal.com' => 'WHITELIST', # 3rd-party sign. by paypal.com 'alert.bankofamerica.com' => 'WHITELIST', 'ealerts.bankofamerica.com'=> 'WHITELIST', 'amazon.com' => 'WHITELIST', 'amazon.de' => 'WHITELIST', 'amazon.co.uk' => 'WHITELIST', 'cisco.com' => 'WHITELIST', '.cnn.com' => 'WHITELIST', 'skype.net' => 'WHITELIST', 'welcome.skype.com' => 'WHITELIST', 'cc.yahoo-inc.com' => 'WHITELIST', 'cc.yahoo-inc.com/@yahoo-inc.com' => 'WHITELIST', '.linkedin.com' => 'MILD_WHITELIST', 'google.com' => 'MILD_WHITELIST', 'googlemail.com' => 'MILD_WHITELIST', './@googlegroups.com' => 'MILD_WHITELIST', './@yahoogroups.com' => 'MILD_WHITELIST', './@yahoogroups.co.uk' => 'MILD_WHITELIST', './@yahoogroupes.fr' => 'MILD_WHITELIST', 'yousendit.com' => 'MILD_WHITELIST', 'meetup.com' => 'MILD_WHITELIST', 'dailyhoroscope at astrology.com' => 'MILD_WHITELIST', } ); $policy_bank{'MILD_WHITELIST'} = { score_sender_maps => [ { '.' => [-1.8] } ], }; $policy_bank{'WHITELIST'} = { bypass_spam_checks_maps => [1], spam_lovers_maps => [1], }; $policy_bank{'NOVIRUSCHECK'} = { bypass_decode_parts => 1, bypass_virus_checks_maps => [1], virus_lovers_maps => [1], }; $policy_bank{'NOBANNEDCHECK'} = { bypass_banned_checks_maps => [1], banned_files_lovers_maps => [1], }; > I'd like to reset the contents_category to CC_CLEAN in the case where my > own whitelist has cleared the email. Is that possible? > I'm able to get the current status of the email from per_recip_data and > contents_category, where I can see that the email has been flagged in > CC_CLEAN and CC_BANNED ( 1 and 8 ). Something like this should do: $msginfo->contents_category(undef); $msginfo->add_contents_category(CC_CLEAN,0); for my $r (@{$msginfo->per_recip_data}) { $r->contents_category(undef); $r->add_contents_category(CC_CLEAN,0); } Mark From stefan at localside.net Wed Feb 29 17:15:29 2012 From: stefan at localside.net (Stefan Jakobs) Date: Wed, 29 Feb 2012 17:15:29 +0100 Subject: [AMaViS-user] adding header to mail In-Reply-To: <201001081925.36939.Mark.Martinec+amavis@ijs.si> References: <200910022153.27024.stefan@localside.net> <200910052001.32135.stefan@localside.net> <201001081925.36939.Mark.Martinec+amavis@ijs.si> Message-ID: <2061368.XT1Da1EKXQ@nksjako1.rus.uni-stuttgart.de> Replying to an very old thread: On Friday, 8th January 2010, 19:25:36 you wrote: > Replying to an old thread, with some new information: > > On Monday 05 October 2009 20:01:29 Stefan wrote: > > > I'm using SpamAssassin with the URICountry Plugin. Now I would like to > > > add a X-URI-Country: header to the scanned message. Therefore I added > > > the > > > > > > following lines to the URICountry.pm module: > > > # Build a string of all found countries and export it as a tag > > > my $countries = ""; > > > foreach my $country (keys(%countries)) { > > > > > > $countries .= uc($country) ." "; > > > > > > } > > > chop $countries; > > > $opts->{permsgstatus}->set_tag("URICOUNTRY", $countries); > > > > > > > > > package Amavis::Custom; > > > [...] > > > sub before_send { > > > > > > my($self,$conn,$msginfo) = @_; > > > my($uri_country) = $msginfo->supplementary_info('URICOUNTRY'); > > > if (defined $uri_country && $uri_country ne '') { > > > > > > my($hdr_edits) = $msginfo->header_edits; > > > my($all_local) = !grep { !$_->recip_is_local } > > > > > > @{$msginfo-per_recip_data}; > > > > > > $hdr_edits->add_header('X-URI-Countries', > > > > > > $uri_country) if $all_local; > > > > > > } > > > > > > }; > > > > > > I can use the URICOUNTRY tag with the command 'spamassassin' (e.g. in a > > > template like 30_text_de.cf). But amavisd doesn't add a header because > > > URICOUNTRY is empty (=""). Why doesn't amavisd see the content of > > > URICOUNTRY? > > > > The concept I like to use is the same as with the RelayCountry plugin. > > I guess the problem is that the RelayCountry plugin is collecting its data > > much earlier (in the sub function extract_metadata) as the URICountry > > plugin. The URICountry plugin is collecting its data in the sub function > > parsed_metadata. Both plugins set the tag in the parsed_metadata function. > > But only the RelayCountry plugin can create an additional header (with > > put_metadata) in the extract_metadata function. > > > > So maybe the problem is that the plugin can't create an additional header > > and assign it to the tag. But on the other side is amavisd only evaluating > > the tag and not the header. > > > > So, any ideas why amavisd can not read the content of the tag URICOUNTRY? > > The $msginfo->supplementary_info gets filled in a sub call_spamassassin > from a list of 'interesting' tags. You need to add URICOUNTRY to that list: > > for my $t (qw(TESTS AUTOLEARN AUTOLEARNSCORE SC SCRULE SCTYPE > LANGUAGES RELAYCOUNTRY ASN ASNCIDR DCCB DCCR DCCREP > DKIMDOMAIN DKIMIDENTITY AWLSIGNERMEAN > CRM114STATUS CRM114SCORE CRM114CACHEID)) { > $supplementary_info{$t} = $per_msg_status->get_tag($t); > } > > > With the next release of amavisd, along with SpamAssassin 3.3.0, amavisd > will be able to 'see' added header fields as prepared by SpamAssassin > (through its 'add_header ... _URICOUNTRY_' config option), > and insert them into the mail header section, which will eliminate the > need to collect SA tags and pass them through the supplementary_info > mechanism. I use amavisd-new 2.7.0 and SpamAssassin 3.3.2, but I can't get it to work without custom hooks and the TAGs (RELAYCOUNTRY, URICOUNTRY) processed by $supplementary_info. I tried it with add_header all Relay-Countries _RELAYCOUNTRY_ add_header all URI-Countries _URICOUNTRY_ in local.cf and an empty custom-hooks.conf, but amavisd doesn't add the headers. What are the necessary steps to add these headers with amavisd? > Mark Thanks Stefan From nico_ps at yahoo.com Wed Feb 29 19:24:18 2012 From: nico_ps at yahoo.com (Nicolas) Date: Wed, 29 Feb 2012 10:24:18 -0800 (PST) Subject: spam_quarantine_to problem Message-ID: <1330539858.89506.YahooMailNeo@web112417.mail.gq1.yahoo.com> Hello, ?I installed Amavis on Debian 6.0 . ?I configure $spam_quarantine_to = "spam\@$myhostname"; and the system work ok (forward mail) . But, when I view the mailbox of spam, the header "To:" is undisclosed-recipients:; not the original. This dificult forwad false positive from the spam account mailbox. How set Amavis to preserve original header "To:" Thanks in advance. N. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Mark.Martinec+amavis at ijs.si Wed Feb 29 20:12:12 2012 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Wed, 29 Feb 2012 20:12:12 +0100 Subject: spam_quarantine_to problem In-Reply-To: <1330539858.89506.YahooMailNeo@web112417.mail.gq1.yahoo.com> References: <1330539858.89506.YahooMailNeo@web112417.mail.gq1.yahoo.com> Message-ID: <201202292012.12021.Mark.Martinec+amavis@ijs.si> Nicolas, > I installed Amavis on Debian 6.0 . > I configure $spam_quarantine_to = "spam\@$myhostname"; and the system work > ok (forward mail) . But, when I view the mailbox of spam, the header "To:" > is undisclosed-recipients:; not the original. This dificult forwad false > positive from the spam account mailbox. How set Amavis to preserve > original header "To:" This shouldn't be the case. Messages sent to a quarantine keep their orginal header field "To:" regardless of whether a quarantine is file-based, or sent with SMTP to some mailbox. I just tried it with versions 2.7.0 and 2.6.6, both behave as expected. The "To:" is also preserved when releasing from a quarantine. Which version of amavisd is that? Are you sure the "To:" header field existed in the original message? Just guessing, perhaps you are looking at some other notification message and not a quarantined message. Mark From nico_ps at yahoo.com Wed Feb 29 20:36:54 2012 From: nico_ps at yahoo.com (Nicolas) Date: Wed, 29 Feb 2012 11:36:54 -0800 (PST) Subject: spam_quarantine_to problem In-Reply-To: <201202292012.12021.Mark.Martinec+amavis@ijs.si> References: <1330539858.89506.YahooMailNeo@web112417.mail.gq1.yahoo.com> <201202292012.12021.Mark.Martinec+amavis@ijs.si> Message-ID: <1330544214.66367.YahooMailNeo@web112402.mail.gq1.yahoo.com> Mark, ?Version of Amavis: 2.6.4 Here is an example mail (from mailbox spam user): //////////////////////////////////////////////////////////////////////////////////////////////////////////// From shylaangeles at input.com? Wed Feb 29 14:19:22 2012 Return-Path: X-Original-To: spam at mtaXX Delivered-To: spam at mtaXX Received: from localhost (localhost [127.0.0.1]) ??????? by mtaXX (Postfix) with ESMTP id 08EAB80AI3 ??????? for ; Wed, 29 Feb 2012 X-Envelope-From: X-Envelope-To: X-Envelope-To-Blocked: X-Quarantine-ID: X-Spam-Flag: YES X-Spam-Score: 19.461 X-Spam-Level: ******************* X-Spam-Status: Yes, score=19.461 tag=2 tag2=5.31 kill=6.31 ??????? tests=[ALL_TRUSTED=-1, DRUGS_ERECTILE=2.221, DRUG_ED_CAPS=1.023, ??????? FB_CIALIS_LEO3=3.055, MISSING_SUBJECT=1.767, NO_PRESCRIPTION=1.102, ??????? ONLINE_PHARMACY=2.371, TVD_VISIT_PHARMA=1.196, URIBL_BLACK=1.775, ??????? URIBL_DBL_SPAM=1.7, URIBL_JP_SURBL=1.948, URIBL_SBL=0.644, ??????? URIBL_WS_SURBL=1.659] autolearn=spam Received: from mtaXX ([127.0.0.1]) ??????? by localhost (mtaXX [127.0.0.1]) (amavisd-new, port 10024) ??????? with ESMTP id BNXmyCre+d9B for ; ??????? Wed, 29 Feb 2012 Received: from pp (localhost [127.0.0.1]) ??????? by mtaXX (Postfix) with SMTP id 6BF6480A48 ??????? for ; Wed, 29 Feb 2012 Message-Id: <20120229171911.6BF6480A48 at mtaXX> Date: Wed, 29 Feb 2012 From: shylaangeles at input.com To: undisclosed-recipients:; . . . //////////////////////////////////////////////////////////////////////////////////////////////////////////// Note the content of? "To:"not match with X-Envelope-To: (original final recipient) Thanks in advance. N. ________________________________ From: Mark Martinec To: amavis-users at amavis.org Sent: Wednesday, February 29, 2012 4:12 PM Subject: Re: spam_quarantine_to problem Nicolas, > I installed Amavis on Debian 6.0 . > I configure $spam_quarantine_to = "spam\@$myhostname"; and the system work > ok (forward mail) . But, when I view the mailbox of spam, the header "To:" > is undisclosed-recipients:; not the original. This dificult forwad false > positive from the spam account mailbox. How set Amavis to preserve > original header "To:" This shouldn't be the case. Messages sent to a quarantine keep their orginal header field "To:" regardless of whether a quarantine is file-based, or sent with SMTP to some mailbox. I just tried it with versions 2.7.0 and 2.6.6, both behave as expected. The "To:" is also preserved when releasing from a quarantine. Which version of amavisd is that? Are you sure the "To:" header field existed in the original message? Just guessing, perhaps you are looking at some other notification message and not a quarantined message. ? Mark ------------ pr?xima parte ------------ Se ha borrado un adjunto en formato HTML... URL: From lstone19 at stonejongleux.com Wed Feb 29 21:03:05 2012 From: lstone19 at stonejongleux.com (Larry Stone) Date: Wed, 29 Feb 2012 14:03:05 -0600 (CST) Subject: spam_quarantine_to problem In-Reply-To: <1330544214.66367.YahooMailNeo@web112402.mail.gq1.yahoo.com> References: <1330539858.89506.YahooMailNeo@web112417.mail.gq1.yahoo.com> <201202292012.12021.Mark.Martinec+amavis@ijs.si> <1330544214.66367.YahooMailNeo@web112402.mail.gq1.yahoo.com> Message-ID: On Wed, 29 Feb 2012, Nicolas wrote: > Mark, > ?Version of Amavis: 2.6.4 > Here is an example mail (from mailbox spam user): > > /////////////////////////////////////////////////////////////////////////// > ///////////////////////////////// > From shylaangeles at input.com? Wed Feb 29 14:19:22 2012 > Return-Path: > X-Original-To: spam at mtaXX > Delivered-To: spam at mtaXX > Received: from localhost (localhost [127.0.0.1]) > ??????? by mtaXX (Postfix) with ESMTP id 08EAB80AI3 > ??????? for ; Wed, 29 Feb 2012 > X-Envelope-From: > X-Envelope-To: > X-Envelope-To-Blocked: > X-Quarantine-ID: > X-Spam-Flag: YES > X-Spam-Score: 19.461 > X-Spam-Level: ******************* > X-Spam-Status: Yes, score=19.461 tag=2 tag2=5.31 kill=6.31 > ??????? tests=[ALL_TRUSTED=-1, DRUGS_ERECTILE=2.221, DRUG_ED_CAPS=1.023, > ??????? FB_CIALIS_LEO3=3.055, MISSING_SUBJECT=1.767, NO_PRESCRIPTION=1.102, > ??????? ONLINE_PHARMACY=2.371, TVD_VISIT_PHARMA=1.196, URIBL_BLACK=1.775, > ??????? URIBL_DBL_SPAM=1.7, URIBL_JP_SURBL=1.948, URIBL_SBL=0.644, > ??????? URIBL_WS_SURBL=1.659] autolearn=spam > Received: from mtaXX ([127.0.0.1]) > ??????? by localhost (mtaXX [127.0.0.1]) (amavisd-new, port 10024) > ??????? with ESMTP id BNXmyCre+d9B for ; > ??????? Wed, 29 Feb 2012 > Received: from pp (localhost [127.0.0.1]) > ??????? by mtaXX (Postfix) with SMTP id 6BF6480A48 > ??????? for ; Wed, 29 Feb 2012 > Message-Id: <20120229171911.6BF6480A48 at mtaXX> > Date: Wed, 29 Feb 2012 > From: shylaangeles at input.com > To: undisclosed-recipients:;. > . > . > /////////////////////////////////////////////////////////////////////////// > ///////////////////////////////// > > Note the content of? "To:"not match with X-Envelope-To: (original final > recipient) The "To:" header and the envelope reicipient (shown in that X-Envelope-To: header) are two different things. SMTP delivers mail to the envelope recipient. There is no requirement that the envelope recipient be in the To: header (it could be in the CC: header or could be a BCC: recipient). For the most part, the To: and CC: headers are just part of the message data as far as SMTP is concerned. SMTP does not make mail delivery decisions based on To: or CC: headers and only in very limited circumstances does an SMTP server add or change message headers. Some mail software, when not finding a To: header, adds the To: undisclosed-recipients:; header you're seeing (one of the limited circumstances where an SMTP server might add a message header). It just means that there was no To: header or that it was blank. Mail without a To: header is almost always spam. -- Larry Stone lstone19 at stonejongleux.com