Postfix -> Amavisd-new -> SpamClient (Spamd) - Content-Length header error

[Mark Martinec]

Wouter,

> I am running a Linux based Collax Business Server where the last updated
> forced to do the spam checking through Amavisd. Before Postfix was calling
> Spamassassin direct and the problem I am currently seeing was not there.
> 
> In the currect configuration Postfix is filtering through Amavisd-new by
> sending the incoming email by SMTP to port 10033. Amavis is fetching the
> email, calling ClamAV for the virus scanning and after that is calling
> Spamc/Spamd to do the spam filtering.
> 
> In my syslog I noted some 10-25 of the below messages each day.
> 
> Message in the log:
> 
> 2012 Jan  9 12:39:19 home spamd[29330]: spamd: bad protocol: header error:
>      (Content-Length mismatch: Expected 18668 bytes, got 18667 bytes) at
> /usr/bin/spamd line 2001, <GEN304> line 255.
> 2012 Jan  9 14:48:27 home spamd[13554]: spamd: bad protocol: header error:
>      (Content-Length mismatch: Expected 14029 bytes, got 14028 bytes) at
> /usr/bin/spamd line 2001, <GEN15> line 201.
> 2012 Jan  9 14:56:10 home spamd[13554]: spamd: bad protocol: header error:
>      (Content-Length mismatch: Expected 80004 bytes, got 80003 bytes) at
> /usr/bin/spamd line 2001, <GEN24> line 1064.
> 2012 Jan  9 16:41:18 home spamd[13554]: spamd: bad protocol: header error:
>      (Content-Length mismatch: Expected 34371 bytes, got 34369 bytes) at
> /usr/bin/spamd line 2001, <GEN51> line 659.
> 2012 Jan  9 17:19:55 home spamd[13554]: spamd: bad protocol: header error:
>      (Content-Length mismatch: Expected 54037 bytes, got 54026 bytes) at
> /usr/bin/spamd line 2001, <GEN71> line 1447.
> 
> The work load for my domain is 150-200 emails a day.
> 
> Because of this protocol error, spamd is not having the message checked,
> but is returning the error after 30 seconds and amavisd is releasing the
> message to the destination, because it has no score. Several spams have
> slipped though my filter.
> 
> I have been searching for the cause for a long time now, but last weekend I
> finaly was able to pin point the source of the error:
> 
> Postfix is dot-stuffing the message it sents to amavis as requested by RFC.
> Amavis is de-stuffing these dots and is passing the message to spamd
> without the extra dots, but the Content-Length is not decreased by the
> number of removed dots. Because of that the above errors occur.
> 
> -> Is this a known issue?

Yes, a known issue. I believe I have it fixed in my current code.
This is from a would-be release notes:

- finely adjust a message size for de-stuffed dots according to a size
  definition in RFC 1870; avoids occasional message size mismatch when
  using a SpamdClient antispam entry (used for spamd testing);

> If not, what do you need from me to help you fix it? I have separate
> tcpdump captures of the communication between Postfix and Amavis and Amavis
> and Spamd for several emails triggering this problem, as well as the excel
> sheet I used to determine what difference there was between the incoming
> and outgoing message from amavis.

I'll send you the amavisd-new-2.8.0-pre3.tar.gz in my next message,
if you are willing tro give it a try. Should be compatible with 2.7.0 config.

  Mark