How to identify the exact reason behind the shunting of the mails ?
Hi Team,
I am currently running Mailman 3 with version 3.3.1 and in-builtpostfix version 2.10.1-6 on RHEL 7.5 with PostgreSQL 11.7 version with default values. I am almost running 2000 lists with 128 GB RAM and 24 Cores on a single VM and on which all the mailman 3 components are running. I have been running this setup in production for the past 2 years and 6 months.
From the beginning of the above setup until recent days, Iobserved a few mails moving to the shunt queue now and then. But from recent days it is observed that many mails are moving to the shunt queue.
I browsed many websites but I could not get a proper resolutionfor identifying the exact reason behind the shunting of the mails. Because of which I had been asking the user to resend the mails instead of suggesting the user regarding the issue properly.
I in fact used the command "mailman qfile <filename.pck>" eventhen I don't understand what might be wrong with the mail composition. Most of the time I see *unknown-8bit* in To or CC fields and I don't know if it is linked with the reason for shunting or not and I don't get any clue from it either.
Is there any chance to find out the reason for shunting of themails through any means ?
-- Thanks & Regards, Shashi Kanth.K 9052671936
On 6/17/21 6:56 AM, Shashikanth Komandoor wrote:
Is there any chance to find out the reason for shunting of themails through any means ?
Look in Mailman's var/logs/mailman.log for the tracebacks associated with the shunted messages.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thank you Mark for your immediate response.
The below are the tracebacks I could find in the mailman log.
*Jun 17 19:04:08 2021 (8642) Uncaught runner exception: sequence item 0: expected str instance, Header foundJun 17 19:04:08 2021 (8642) Traceback (most recent call last): File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 173, in _one_iteration self._process_one_file(msg, msgdata) File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 266, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/var/lib/mailman/mailman/src/mailman/runners/pipeline.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/var/lib/mailman/mailman/src/mailman/core/pipelines.py", line 50, in process handler.process(mlist, msg, msgdata) File "/var/lib/mailman/mailman/src/mailman/handlers/avoid_duplicates.py", line 59, in process addrs = getaddresses(msg.get_all(header, [])) File "/usr/lib64/python3.6/email/utils.py", line 112, in getaddresses all = COMMASPACE.join(fieldvalues)TypeError: sequence item 0: expected str instance, Header foundJun 17 19:04:08 2021 (8642) SHUNTING: 1623936848.367641+4753f8127953071ddcabfe828ebe72fcf04d0ae2*
In the shunt file, I see something like this as below in CC while To is good:
*Cc: =?unknown-8bit?q?=22Support_Zimbra=22_=3Csupport-zimbra=40nic=2Ein=3E=2C?= =?unknown-8bit?b?ICLgpLbgpLDgpKYg4KSu4KS+4KSl4KWB4KSwIC0gU2hhcmFkIE1hdGh1ciIgPHN1cHBvcnQtdmFqcmFAc3VwcG9ydGdvdi5pbj4s?= =?unknown-8bit?b?ICLgpLDgpL7gpJzgpYfgpLYg4KS44KS/4KSC4KS5IC0gUmFqZXNoIFNpbmdoIiA8cmFqc0BuaWMuaW4+LA==?= =?unknown-8bit?b?ICJTYWJ5YXNhY2hpIENob3VkaHVyeSIgPHNhYnlhQG5pYy5pbj4sIG1haWxhZG1pbkBuaWMuaW4s?= =?unknown-8bit?b?ICJNUklUVU5KQVkgS1VNQVIiIDxta0BuaWMuaW4+LCAiQmhyaWd1IEt1bWFyIiA8YmhyaWd1QG5pYy5pbj4s?= =?unknown-8bit?b?ICJPbSBQcmFrYXNoIiA8b3AubmhxQG5pYy5pbj4sICJIYXJpaGFyYW4gTSIgPGhhcmloYXJhbi5tQGdvdi5pbj4s?= =?unknown-8bit?b?ICJQaGlzaGluZyBQaGlzaGluZyIgPHBoaXNoaW5nQG5pYy5pbj4sICJpbmNpZGVudCIgPGluY2lkZW50QG5pYy5pbj4s?= =?unknown-8bit?b?ICJNYW5hdiBSYWtoZWphIiA8bWFuYXYucmFraGVqYUBuaWMuaW4+?=*
Also, I came to know that two of the recipients in CC block have their mail ID display name in their local language. While I am not sure, I doubt if it could be the issue of getting shunted.
Please confirm and suggest any solution.
On Thu, Jun 17, 2021 at 8:48 PM Mark Sapiro <mark@msapiro.net> wrote:
On 6/17/21 6:56 AM, Shashikanth Komandoor wrote:
Is there any chance to find out the reason for shunting of themails through any means ?
Look in Mailman's var/logs/mailman.log for the tracebacks associated with the shunted messages.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- Thanks & Regards, Shashi Kanth.K 9052671936
Dear Mark,
FYI, I observed this issue with the postfix that receives themails from the previous MTA and hands it over to the mailman. But I fixed at postfix by upgrading it to the version postfix 3.
But still Mailman is not able to understand Hindi (Indian locallanguage) in which language the display names of the email ids are there.
So, now what to do at Mailman so that it can also understandlocal languages or any work around or other alternate solutions for these sort of cases.
On Fri, Jun 18, 2021 at 4:04 PM Shashikanth Komandoor < shashikanth.komandoor@gmail.com> wrote:
Thank you Mark for your immediate response.
The below are the tracebacks I could find in the mailman log.
*Jun 17 19:04:08 2021 (8642) Uncaught runner exception: sequence item 0: expected str instance, Header foundJun 17 19:04:08 2021 (8642) Traceback (most recent call last): File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 173, in _one_iteration self._process_one_file(msg, msgdata) File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 266, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/var/lib/mailman/mailman/src/mailman/runners/pipeline.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/var/lib/mailman/mailman/src/mailman/core/pipelines.py", line 50, in process handler.process(mlist, msg, msgdata) File "/var/lib/mailman/mailman/src/mailman/handlers/avoid_duplicates.py", line 59, in process addrs = getaddresses(msg.get_all(header, [])) File "/usr/lib64/python3.6/email/utils.py", line 112, in getaddresses all = COMMASPACE.join(fieldvalues)TypeError: sequence item 0: expected str instance, Header foundJun 17 19:04:08 2021 (8642) SHUNTING: 1623936848.367641+4753f8127953071ddcabfe828ebe72fcf04d0ae2*
In the shunt file, I see something like this as below in CC while To is good:
*Cc: =?unknown-8bit?q?=22Support_Zimbra=22_=3Csupport-zimbra=40nic=2Ein=3E=2C?= =?unknown-8bit?b?ICLgpLbgpLDgpKYg4KSu4KS+4KSl4KWB4KSwIC0gU2hhcmFkIE1hdGh1ciIgPHN1cHBvcnQtdmFqcmFAc3VwcG9ydGdvdi5pbj4s?= =?unknown-8bit?b?ICLgpLDgpL7gpJzgpYfgpLYg4KS44KS/4KSC4KS5IC0gUmFqZXNoIFNpbmdoIiA8cmFqc0BuaWMuaW4+LA==?= =?unknown-8bit?b?ICJTYWJ5YXNhY2hpIENob3VkaHVyeSIgPHNhYnlhQG5pYy5pbj4sIG1haWxhZG1pbkBuaWMuaW4s?= =?unknown-8bit?b?ICJNUklUVU5KQVkgS1VNQVIiIDxta0BuaWMuaW4+LCAiQmhyaWd1IEt1bWFyIiA8YmhyaWd1QG5pYy5pbj4s?= =?unknown-8bit?b?ICJPbSBQcmFrYXNoIiA8b3AubmhxQG5pYy5pbj4sICJIYXJpaGFyYW4gTSIgPGhhcmloYXJhbi5tQGdvdi5pbj4s?= =?unknown-8bit?b?ICJQaGlzaGluZyBQaGlzaGluZyIgPHBoaXNoaW5nQG5pYy5pbj4sICJpbmNpZGVudCIgPGluY2lkZW50QG5pYy5pbj4s?= =?unknown-8bit?b?ICJNYW5hdiBSYWtoZWphIiA8bWFuYXYucmFraGVqYUBuaWMuaW4+?=*
Also, I came to know that two of the recipients in CC block have their mail ID display name in their local language. While I am not sure, I doubt if it could be the issue of getting shunted.
Please confirm and suggest any solution.
On Thu, Jun 17, 2021 at 8:48 PM Mark Sapiro <mark@msapiro.net> wrote:
On 6/17/21 6:56 AM, Shashikanth Komandoor wrote:
Is there any chance to find out the reason for shunting of themails through any means ?
Look in Mailman's var/logs/mailman.log for the tracebacks associated with the shunted messages.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- Thanks & Regards, Shashi Kanth.K 9052671936
-- Thanks & Regards, Shashi Kanth.K 9052671936
On 6/18/21 6:48 AM, Shashikanth Komandoor wrote:
Dear Mark,
FYI, I observed this issue with the postfix that receives themails from the previous MTA and hands it over to the mailman. But I fixed at postfix by upgrading it to the version postfix 3.
But still Mailman is not able to understand Hindi (Indian locallanguage) in which language the display names of the email ids are there.
So, now what to do at Mailman so that it can also understandlocal languages or any work around or other alternate solutions for these sort of cases.
As noted in my reply at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/... the issue is a Mailman bug which is fixed in the next release. It has to do with the fact that the display names are RFC 2047 encoded and not the particular encoding.
Actually, the fix at https://gitlab.com/mailman/mailman/-/merge_requests/841 depends on https://gitlab.com/mailman/mailman/-/merge_requests/652 which was released in Mailman 3.3.2. A complete patch for versions < 3.3.2 is attached.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thank you Mark,
Then I will try to upgrade the version
Thank you once again.
On Fri, Jun 18, 2021 at 8:01 PM Mark Sapiro <mark@msapiro.net> wrote:
On 6/18/21 6:48 AM, Shashikanth Komandoor wrote:
Dear Mark,
FYI, I observed this issue with the postfix that receives themails from the previous MTA and hands it over to the mailman. But I fixed at postfix by upgrading it to the version postfix 3.
But still Mailman is not able to understand Hindi (Indianlocal language) in which language the display names of the email ids are there.
So, now what to do at Mailman so that it can also understandlocal languages or any work around or other alternate solutions for these sort of cases.
As noted in my reply at
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/... the issue is a Mailman bug which is fixed in the next release. It has to do with the fact that the display names are RFC 2047 encoded and not the particular encoding.
Actually, the fix at https://gitlab.com/mailman/mailman/-/merge_requests/841 depends on https://gitlab.com/mailman/mailman/-/merge_requests/652 which was released in Mailman 3.3.2. A complete patch for versions < 3.3.2 is attached.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- Thanks & Regards, Shashi Kanth.K 9052671936
On 6/18/21 3:34 AM, Shashikanth Komandoor wrote:
Thank you Mark for your immediate response.
The below are the tracebacks I could find in the mailman log.
*Jun 17 19:04:08 2021 (8642) Uncaught runner exception: sequence item 0: expected str instance, Header foundJun 17 19:04:08 2021 (8642) Traceback (most recent call last): File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 173, in _one_iteration self._process_one_file(msg, msgdata) File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 266, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/var/lib/mailman/mailman/src/mailman/runners/pipeline.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/var/lib/mailman/mailman/src/mailman/core/pipelines.py", line 50, in process handler.process(mlist, msg, msgdata) File "/var/lib/mailman/mailman/src/mailman/handlers/avoid_duplicates.py", line 59, in process addrs = getaddresses(msg.get_all(header, [])) File "/usr/lib64/python3.6/email/utils.py", line 112, in getaddresses all = COMMASPACE.join(fieldvalues)TypeError: sequence item 0: expected str instance, Header foundJun 17 19:04:08 2021 (8642) SHUNTING: 1623936848.367641+4753f8127953071ddcabfe828ebe72fcf04d0ae2*
This is https://gitlab.com/mailman/mailman/-/issues/881 which is fixed by https://gitlab.com/mailman/mailman/-/merge_requests/841 which will be in Mailman 3.3.5 when it's released.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Mark Sapiro -
Shashikanth Komandoor