README.banned: Purpose?

[Mark Martinec]

2018-10-11 23:36, Patrick Ben Koetter wrote
> What's the purpose of README.banned in the README_FILES directory? It 
> contains
> log/output of message parts found by amavis.

Seeds of uncompleted documentation.

The 'p003 ... Content-type:...' contains examples of a mime type
and a mail part name, as seen by regular expressions
in $banned_filename_re (and indirectly by @banned_filename_maps).

The p.path contains example strings as seen by regular expressions
in the $banned_namepath_re (newer, more complicated and tricky,
but more powerful/expressive).

The release notes have some documentation on it (below).

Search the release notes for other mentions of @banned_filename_maps,
$banned_filename_re, $banned_namepath_re and %banned_rules


>>> 

Some un-edited notes on the new banned rules mechanism:
(wrapped log lines, and replaced \\ by \ for clarity:

| Feb 24 19:07:29 hauptpostamt amavis[29847]: (29847-04-5) p.path 
BANNED:
|
|  "P=p002,M=application/octet-stream,T=zip,N=document.zip |
|   P=p003,T=exe,T=exe-ms,N=document.htm   .scr",

part p003 is of type (file(1)) MS executable, with suggested
name "document.htm   .scr" (lots of spaces in the name)

its parent resides on temp file p002 (i.e. p003 was extracted from it),
which is of type (T) zip archive, with suggested (MIME) name
(N) "document.zip", and has a MIME type (M) "application/octet-stream".

such a component p003 lying within such p002 is considered banned
by the following regexp rule (one rule within the $banned_namepath_re 
list):


|     matching_key="(?mix-s:^ (.*\t)? N= [^\t\n]* \. [^./\t\n]* \.
|                    (exe|vbs|pif|scr|bat|cmd|com|dll) (\t.*)? $)"

which says that any component at any level must not have a name (N)
matching a pattern:
   any number of characters,
   a dot,
   any number of non-dot and non-slash characters
   a dot,
   and ending with: exe or vbs ...
(basically: double extension ending with listed extensions)

The complications such as using [^\t\n]* instead of .* are there
to keep regexp contained within fields and ancestors/descendents.


There is one detail to remember when comparing logged p.path log entries
and the actual matching rules:

- for the sake of readability the logged entry has \n (newlines) 
converted
   into ' | '. The \n is a separator between components in the tree
   from the root (the mail itself, hidden) to the leaf component
   which can not be further expanded (i.e. not an archive)


- for the sake of readability the logged entry has \t (a tab) converted
   into comma, separating information fields such as P=...  M=..  T=..  
M=..

So the above logged string:
   P=p002,M=appl...,T=zip,N=document.zip | 
P=p003,T=exe,T=exe-ms,N=document.htm.scr
is actually a single string:
   
P=p002\tM=appl...\tT=zip\tN=document.zip\nP=p003\tT=exe\tT=exe-ms\tN=document.htm.scr

and a Perl regexp is applied directly to it.

The raw string is rather unsightly, but the \n and \t were chosen
to minimize clash with valid characters within file names.

If a \n or \t is present in a name of the components, such character
is converted into a space to avoid clashing with separators.


| Feb 24 19:11:58 hauptpostamt amavis[31505]: (31505-01-5) p.path 
BANNED:
|   "P=p002,M=application/octet-stream,T=zip,N=paypal.zip |
|    P=p003,T=exe,T=exe-ms,N=paypal.scr",


a MS executable named "paypal.scr" within a zip archive "paypal.zip"

|      matching_key="(?mix-s:^ (.*\t)? N= [^\t\n]* \.
|                    (exe|vbs|pif|scr|bat|com) (\t.*)? $)"

block component at any level with a name (N) terminating
by dot followed by any of the listed extensions.



| Feb 24 19:18:25 hauptpostamt amavis[32159]: (32159-01-2) p.path 
BANNED:
|   "P=p002,M=application/octet-stream,T=zip,N=text.zip |
|    P=p003,T=exe,T=exe-ms,N=text.txt     .exe",

a MS executable named "text.txt     .exe" (with lots of spaces in the 
name)
within a zip archive named "text.zip"

|      matching_key="(?mix-s:^ (.*\t)? N= [^\t\n]* \. [^.\t\n]* \.\n
|                    (exe|vbs|pif|scr|bat|cmd|com|dll) (\t.*)? $)"

blocked by the double-extension rule.


| Feb 24 19:30:15 hauptpostamt amavis[1690]: (01690-02-8) p.path BANNED:
|   "P=p002,M=application/octet-stream,T=zip,N=jokes.zip |
|    P=p003,T=exe,T=exe-ms,N=jokes.doc   .exe",
|      matching_key="(?mix-s:^ (.*\t)? N= [^\t\n]* \. [^.\t\n]* \.\n
|                    (exe|vbs|pif|scr|bat|cmd|com|dll) (\t.*)? $)"

same thing

<<<<<


   Mark