Invalid FilterAction: discard. Treating as discard
Having set the Filter content to Yes - regardless of other setting in the "Alter Messages" tab, make a HTML message come out empty. I can replicate this on my test list.
I have tried debugging by changing each setting one-by-one and sending the user's mail, but no change makes the mail come out with a content. However setting "Filter content" to "no" makes the mail come out okay.
With "Filter content" set to "Yes", sending another HTML mail is okay, so something in the user's mail may not be right, but I wonder why setting "Filter content" to no "solves" the problem. Afte all this setting is switching all other settings on and off, so I can't just ask the user to turn it off.
May 06 11:53:32 2019 (33) ACCEPT: <250E794B36C1E646AF3D81E1F15348C79DA8CA83@P2KITMBX06WC03.unicph.domain> May 06 11:53:32 2019 (37) [LISTNAME]@lists.example.com invalid FilterAction: discard. Treating as discard May 06 11:53:32 2019 (37) <250E794B36C1E646AF3D81E1F15348C79DA8CA83@P2KITMBX06WC03.unicph.domain> discarded by "default-posting-pipeline" pipeline handler "mime-delete": After content filtering, the message was empty May 06 17:27:35 2019 (33) ACCEPT: <250E794B36C1E646AF3D81E1F15348C79DA8CAEC@P2KITMBX06WC03.unicph.domain> May 06 17:27:35 2019 (37) [LISTNAME]@lists.example.com invalid FilterAction: discard. Treating as discard May 06 17:27:35 2019 (37) <250E794B36C1E646AF3D81E1F15348C79DA8CAEC@P2KITMBX06WC03.unicph.domain> discarded by "default-posting-pipeline" pipeline handler "mime-delete": After content filtering, the message was empty
/Henrik Rasmussen
On 5/7/19 6:19 AM, Henrik Rasmussen wrote:
Having set the Filter content to Yes - regardless of other setting in the "Alter Messages" tab, make a HTML message come out empty. I can replicate this on my test list.
I don't understand the question? Are you asking why the log contains "invalid FilterAction: discard. Treating as discard"? If so, that's a bug. See <https://gitlab.com/mailman/mailman/issues/592>.
If you are asking why the message is empty after filtering and discarded, It is presumably because it is a text/html message as opposed to a multipart/alternative message with text/plain and text/html alternative parts.
I have tried debugging by changing each setting one-by-one and sending the user's mail, but no change makes the mail come out with a content. However setting "Filter content" to "no" makes the mail come out okay.
There are list attributes filter_types and pass_types (also filter_extensions and pass_extensions, but those probably aren't relevant here) which affect this and which aren't exposed in Postorius. See <https://mailman.readthedocs.io/en/latest/src/mailman/handlers/docs/filtering.html> for more info of content filtering.
Presumably, the combination of filter_types and pass_types is not allowing text/html.
You need to examine these settings in mailman shell, e.g.
list(m.pass_types)
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
I don't understand the question? Are you asking why the log contains "invalid FilterAction: discard. Treating as discard"? If so, that's a
No, that was just log messages I thought would help explain the problem.
If you are asking why the message is empty after filtering and discarded, It is presumably because it is a text/html message as opposed to a multipart/alternative message with text/plain and text/html alternative parts.
The Content-Type of the mail sent to the list is multipart/mixed
From: SENDER <sender@senderdep.example.com> To: "LISTNAME@lists.example.com" <LISTNAME@lists.example.com> Subject: =?utf-8?B?QW50cm9wb2xvZ2VyIGkgcHJha3NpczogVGlsbWVsZGluZyDDpWJlbg==?= Thread-Topic: =?utf-8?B?QW50cm9wb2xvZ2VyIGkgcHJha3NpczogVGlsbWVsZGluZyDDpWJlbg==?= Thread-Index: AdT/QIb18JA+6hcSTYycFCI+5NuYoQEwPZ/Q Date: Mon, 6 May 2019 13:53:23 +0200 Message-ID: <250E794B36C1E646AF3D81E1F15348C79DA8CA83@P2KITMBX06WC03.unicph.domain> Content-Language: en-US X-MS-Has-Attach: yes X-MS-Exchange-Organization-SCL: -1 X-MS-TNEF-Correlator: Content-Type: multipart/mixed; boundary="_006_250E794B36C1E646AF3D81E1F15348C79DA8CA83P2KITMBX06WC03u_" MIME-Version: 1.0
There are list attributes filter_types and pass_types (also filter_extensions and pass_extensions, but those probably aren't
My Python isn't that great, but this looks like i'm missing something
m.filter_content True m.pass_types <generator object pass_types at 0x7f66d407bba0> m.filter_extensions <generator object filter_extensions at 0x7f66d407bdb0> m.pass_extensions <generator object pass_extensions at 0x7f66d407be08>
Presumably, the combination of filter_types and pass_types is not allowing text/html.
I am thinking that as the "Filter content" (postorius) refers to "the settings below", this setting is needed in case list admin wants an explicit reply-to address or change other switches in Alter Messages. But this shouldn't affect the content, even it is text/html, I think, at least not emptying the mail, I think.
You need to examine these settings in mailman shell, e.g.
list(m.pass_types)
list(m.pass_types) ['multipart/mixed', 'multipart/alternative', 'text/plain']
-- Henrik Rasmussen
On 5/8/19 1:07 AM, Henrik Rasmussen wrote:
The Content-Type of the mail sent to the list is multipart/mixed
and what are the Content-Types of all the subparts?
list(m.pass_types) ['multipart/mixed', 'multipart/alternative', 'text/plain']
So these are the only types you accept. If, e.g., one of the sub-parts is multipart/related, that part and all it's sub-parts, regardless of type, will be removed.
Presumably, this list was imported from MM 2.1, and the same filtering would have applied there.
I'll post a follow-up with how to change pass_types via mailman shell when I work it out.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
and what are the Content-Types of all the subparts?
According to "attachment.htm" in "Held Messages", the content type (PDF document) is "<meta http-equiv="Content-Type" content="text/html; charset=utf-8">"
Presumably, this list was imported from MM 2.1, and the same filtering would have applied there.
That's right
I'll post a follow-up with how to change pass_types via mailman shell when I work it out.
Look forward to that, thanks.
-- Henrik Rasmussen
On 5/8/19 4:45 AM, Henrik Rasmussen wrote:
Mark Sapiro wrote:
I'll post a follow-up with how to change pass_types via mailman shell when I work it out.
Look forward to that, thanks.
OK.
in mailman shell with your list as 'm'
list(m.pass_types) ['multipart/mixed', 'multipart/alternative', 'text/plain'] m.pass_types = ['multipart', 'text/plain', 'text/html'] commit() list(m.pass_types) ['multipart', 'text/plain', 'text/html']
If you want to allow text from forwarded messages, add 'message/rfc822' in the pass_types list above.
This setting will allow inspection of all elementary parts within any multipart parts ant will accept only text/plain and text/html.
Then if collapse_alternatives is True (settable here or in Postorius), any multipart/alternative parts will by replaced by the first, presumably text/plain, sub-part.
Finally, if convert_html_to_plaintext is True (settable here or in Postorius), any remaining text/html parts will be converted to plain text by lynx or whatever is configured as html_to_plain_text_command in mailman.cfg.
This will ensure that filter_action is applied only to those messages which have no text/plain or text/html parts.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
Then if collapse_alternatives is True (settable here or in Postorius), any multipart/alternative parts will by replaced by the first, presumably text/plain, sub-part.
Finally, if convert_html_to_plaintext is True (settable here or in Postorius), any remaining text/html parts will be converted to plain text by lynx or whatever is configured as html_to_plain_text_command in mailman.cfg. This will ensure that filter_action is applied only to those messages which have no text/plain or text/html parts.
Great, that helped for the content. Thanks.
Still, attachments (PDF) is removed by Mailman. I as the list admin to send a new mail, but hold it back, to see what type it is.
If that doesn't work, we have talked about deleting the list and add it from scratch. Is the any prolems (with left overs from the old imported list), when deleting the list from Postorius?
/Henrik Rasmussen
On 6/6/19 6:02 AM, Henrik Rasmussen wrote:
Still, attachments (PDF) is removed by Mailman. I as the list admin to send a new mail, but hold it back, to see what type it is.
You have to add the MIME types for the attachments you want kept the the list's pass_types. You need to do this via mailman shell or REST as it isn't currently exposed in Postorius.
Here's an example from a list that wants to accept PDFs as well as word processing and spreadsheet documents. Note that nappdf and x-pdf are uncommon PDF subtypes that I have seen.
['multipart', 'message/rfc822', 'text/plain', 'text/html', 'application/pdf', 'application/nappdf', 'application/x-pdf', 'application/rtf', 'application/vnd.oasis.opendocument.spreadsheet', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'application/vnd.ms-excel', 'application/vnd.oasis.opendocument.text', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 'application/msword' ]
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
On 6/6/19 6:02 AM, Henrik Rasmussen wrote:
Still, attachments (PDF) is removed by Mailman. I as the list admin to send a new mail, but hold it back, to see what type it is.
You have to add the MIME types for the attachments you want kept the the list's pass_types. You need to do this via mailman shell or REST as it isn't currently exposed in Postorius. Here's an example from a list that wants to accept PDFs as well as word processing and spreadsheet documents. Note that nappdf and x-pdf are uncommon PDF subtypes that I have seen. ['multipart', 'message/rfc822', 'text/plain', 'text/html', 'application/pdf', 'application/nappdf', 'application/x-pdf', 'application/rtf', 'application/vnd.oasis.opendocument.spreadsheet', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'application/vnd.ms-excel', 'application/vnd.oasis.opendocument.text', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 'application/msword' ]
Thanks, done! However, PDF's and an image is still filterd out.
On May 21st the list admin tested by setting Filter content to 'No'. As expected, the mail was received as sent, with PDF attachment, inline image and HTML intact.
Resending this mail to a newly created test list (from Outlook by selecting Resend, which does exactly just that, the mail was again received as sent, even the config of Alter Messages as the same as the list having the problem.
The difference was that list(m.pass_types) is []
Now, setting
m.pass_types = ['multipart', 'message/rfc822', 'text/plain', 'text/html', 'application/pdf', 'application/nappdf', 'application/x-pdf', 'application/rtf', 'application/vnd.oasis.opendocument.spreadsheet', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'application/vnd.ms-excel', 'application/vnd.oasis.opendocument.text', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 'application/msword']
on the new testlist and resending the mail again, the mail still came out intact.
So I guess the problem must be something complete different.
Also I wonder why a newly created list accepts the PDF in question, when list(m.pass_types) is [].
/Henrik Rasmussen
On 6/12/19 1:46 AM, Henrik Rasmussen wrote:
The difference was that list(m.pass_types) is []
Now, setting
m.pass_types = ['multipart', 'message/rfc822', 'text/plain', 'text/html', 'application/pdf', 'application/nappdf', 'application/x-pdf', 'application/rtf', 'application/vnd.oasis.opendocument.spreadsheet', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'application/vnd.ms-excel', 'application/vnd.oasis.opendocument.text', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 'application/msword']
on the new testlist and resending the mail again, the mail still came out intact.
So I guess the problem must be something complete different.
Also I wonder why a newly created list accepts the PDF in question, when list(m.pass_types) is [].
You need to look at all of pass_types, filter_types, pass_extensions and filter_extensions for both lists.
For a newly created list, these are all empty. They work as in Mailmam
2.1 and are imported from the similarly named 2.1 attributes by mailman import21
.
In brief if filter_content is True, processing is:
- If a part's MIME type is in filter_types it is removed.
- If pass_types is not empty and a part's MIME type is not in pass_types it is removed.
- If a part has a file name and the file name extension is in filter_extensions it is removed.
- If a part has a file name and pass_extensions is not empty and the part's file name extension is not in pass_extensions it is removed.
So in your case of the list that doesn't work, possibilities are:
- The PDF part's MIME type is in filter_types, or
- The PDF part has a file name whose extension is in filter_extensions, or
- pass_extensions is non-empty and The PDF part has a file name whose extension is not in pass_extensions.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Henrik Rasmussen
-
Mark Sapiro