rar support is broken

[Philipp Gesang]

Hi,

-<| Quoting Philipp Gesang <philipp.gesang at intra2net.com>, on Wednesday, 2017-11-22 01:29:33 PM |>-
> -<| Quoting Dmitry Melekhov <dm at belkam.com>, on Wednesday, 2017-11-22 04:09:47 PM |>-
> > 22.11.2017 14:49, Dmitry Melekhov пишет:
> > > I run ubuntu 16.04 server with amavisd-new 2.10.1, but looks like 2.11.0
> > > has the same code.*
> > > *
> > > 
> > > Today I found that amavisd-new can't check attachments using rar or
> > > unrar-nonfree.
> > > 
> > > If file has only latin characters then message arrives UNCHECKED,
> > > if it has cyrillic in utf-8 then rar or unrar exits with exit code and
> > > message passes without any warnings.
> 
> This sounds interesting. Would it be possible to send me a file
> like this off-list for testing?

thanks to Dmitry’s sample we were able to cover another corner
case in the unrar handler.

Besides malware, the archive contains filenames encoded in UTF-8
that unrar extracts fine in a UTF8-locale but not in the C
locale. Amavisd ignores the failure during extraction because the
listing succeeded earlier which it does under any locale.

See attached patch.

Best,
Philipp



/tmp/malware # LC_ALL=en_US.UTF-8 unrar x falspositive.rar 

UNRAR 5.50 freeware      Copyright (c) 1993-2017 Alexander Roshal


Extracting from falspositive.rar

Extracting  Для сверки для сверки.scr                                 OK 
All OK
/tmp/malware # LC_ALL=C unrar x falspositive.rar                       

UNRAR 5.50 freeware      Copyright (c) 1993-2017 Alexander Roshal


Extracting from falspositive.rar

Cannot create ??? ?????? ??? ??????.scr
No such file or directory
No files to extract
/tmp/malware # echo $?
10
/tmp/malware # LC_ALL=en_US.UTF-8 unrar l falspositive.rar       

UNRAR 5.50 freeware      Copyright (c) 1993-2017 Alexander Roshal

Archive: falspositive.rar
Details: RAR 5

 Attributes      Size     Date    Time   Name
----------- ---------  ---------- -----  ----
    ..A....    211968  2017-11-22 07:38  Для сверки для сверки.scr
----------- ---------  ---------- -----  ----
               211968                    1

/tmp/malware # LC_ALL=C unrar l falspositive.rar 

UNRAR 5.50 freeware      Copyright (c) 1993-2017 Alexander Roshal

Archive: falspositive.rar
Details: RAR 5

 Attributes      Size     Date    Time   Name
----------- ---------  ---------- -----  ----
    ..A....    211968  2017-11-22 07:38  ??? ?????? ??? ??????.scr
----------- ---------  ---------- -----  ----
               211968                    1

-------------- next part --------------
From 707840d398fc3efa37d53662d11b0da3f2e73cb7 Mon Sep 17 00:00:00 2001
From: Philipp Gesang <philipp.gesang at intra2net.com>
Date: Thu, 23 Nov 2017 09:40:24 +0100
Subject: [PATCH] amavisd: unrar: treat failed extraction as undecipherable

unrar may error out during archive extraction despite being able
to provide a listing. This can happen with files containing UTF-8
encoded filenames: They will extract just fine under a UTF-8
locale, but error out with status 10 under the C locale. However,
the listing succeeds irrespective of the locale, leading to a
situation where amavisd attempts and fails to extract the archive
but the user (with a different locale or OS) is still able to do
so. Archives like this must be classified as undecipherable.

Signed-off-by: Philipp Gesang <philipp.gesang at intra2net.com>
---
 amavisd | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/amavisd b/amavisd
index 8ff2f43..13f1f3d 100755
--- a/amavisd
+++ b/amavisd
@@ -32409,8 +32409,13 @@ sub do_unrar($$$;$) {
       ($proc_fh,$pid) =
         run_command(undef, '&1', $archiver, qw(x -inul -ver -o- -kb),
                     @common_rar_switches, '--', $fn, "$tempdir/parts/rar/");
-      collect_results($proc_fh,$pid,$archiver,16384,
-                      [0,1,3] );  # one of: SUCCESS, WARNING, CRC
+      my ($_void, $stat) =
+        collect_results($proc_fh,$pid,$archiver,16384,
+                        [0,1,3]);  # one of: SUCCESS, WARNING, CRC
+      if (!proc_status_ok($stat)) {
+          # listing was ok but extraction errored out -> bogus archive
+          $part->attributes_add('U');
+      }
       undef $proc_fh; undef $pid;
       my $errn = lstat("$tempdir/parts/rar") ? 0 : 0+$!;
       if ($errn != ENOENT) {
-- 
2.13.6

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.amavis.org/pipermail/amavis-users/attachments/20171123/d6bf0ebf/attachment.sig>