From myroughbox@yahoo.com Tue Sep 1 17:28:30 2020 From: Jane Anara To: mailman-users@mailman3.org Subject: [MM3-users] Please help: Mailaman3 - Date: Tue, 01 Sep 2020 16:55:07 +0000 Message-ID: <159897930771.18624.2252054531841494359@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4150388610697068925==" --===============4150388610697068925== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello everyone, I hope my questions don't sound ridiculous. In my team, I have been tasked wi= th moving our mailman installations from version 1:2.1 to the current version= , mailman3. I followed the documentations online and installed all the compon= ents, created a Django project and configured settings.py and url.py. However= , when I run the development server, I receive the following on the body of P= ostorius main page: Page not found This page either doesn't exist, or it moved somewhere else. Clicking on the "Lists" menu link, I receive the following: HTTP Error 404: exa= mple.com =

Page not found

This page either doesn't exist, or it moved somewhere else.
=20 > src=3D"https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.11.0/umd/popper.m= in.js" integrity=3D"sha384-b/U6ypiBEHpOf/4+1nzFpr53nxSS+GLCkfwBdFNTxtclqqenIS= fwAzpKaMNFNmj4" crossorigin=3D"anonymous"> src=3D"/static/postorius/js/script.js"> >=20 >=20 > So far, the Postorious component is where there seem to be issues, all=20 > other link or component produce no errors thus far. Please advice on=20 > how I can troubleshoot this. I herein include the relevant sessions of=20 > my settings.py file and url.py which I have tweaked with various=20 > configurations to no avail. >=20 > settings.py: >=20 > # Application definition >=20 > INSTALLED_APPS =3D [ > # Apps added for mailman components=20 > 'hyperkitty', > 'postorius', > 'django_mailman3', > 'mailmanclient', > # Other useful apps > 'rest_framework', > 'django_gravatar', > 'compressor', > 'haystack', > 'django_extensions', > 'django_q', > 'allauth', > 'allauth.account', > 'allauth.socialaccount', > # end other useful apps > 'django.contrib.sites', > 'django.contrib.admin', > 'django.contrib.auth', > 'django.contrib.contenttypes', > 'django.contrib.sessions', > 'django.contrib.messages', > 'django.contrib.staticfiles', > ] >=20 > url.py: >=20 > urlpatterns =3D [ > #Django admin > path('admin/', admin.site.urls), > path('mailman3/', include('postorius.urls')), > path('archives/', include('hyperkitty.urls')), > path('', include('django_mailman3.urls')), > path('accounts/', include('allauth.urls')), > #path('postorius', include('postorius.urls')), > #path('lists/', list_index), > path('search/', include('haystack.urls')), > #url(r'^$', list_index), > ] >=20 > Note that when I uncomment the lines relating to list_index or use=20 > configurations similar to that on mailman-suit, I get: >=20 > NameError: name 'list_index' is not defined. You need to use the view function with `reverse_lazy` to actually get the=20 named `reverse_lazy` view. Did you try using the exact urls.py from mailman-suite project[1]? It has been tested to work locally and other place. [1]: https://gitlab.com/mailman/mailman-suite/-/blob/master/mailman-suite_pro= ject/urls.py >=20 > Looking forward to any help anyone can render. >=20 > Thank you! >=20 > Regards, > Jane. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --=20 thanks, Abhilash Raj (maxking) --===============1946134542098324734==-- From himbeere@meine-oma.de Wed Sep 2 07:29:47 2020 From: Thomas Stein To: mailman-users@mailman3.org Subject: [MM3-users] adding imprint / private policy link to postorius Date: Wed, 02 Sep 2020 08:17:35 +0200 Message-ID: <5b8e0513d64d4d8449ab0763c06053fe@meine-oma.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5179356962258531949==" --===============5179356962258531949== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Hello one and all. Is it possible to add an imprint link to the postorius start page? I guess it is, but it would be nice if someone could me point to the right direction / documentation. Something like they have: https://lists.openrailwaymap.org/mailman3/postorius/lists/ cheers, t. --===============5179356962258531949==-- From brian_carpenter@emwd.com Wed Sep 2 14:32:38 2020 From: Brian Carpenter To: mailman-users@mailman3.org Subject: [MM3-users] Unsubscribe Notifications Clarification Date: Wed, 02 Sep 2020 14:32:35 +0000 Message-ID: <159905715562.18626.16826911226719987593@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6891730998894447632==" --===============6891730998894447632== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable I am trying to understand how Mailman 3 handles unsubscribe notifications. Fr= om my testing I see the following behavior: 1. If list member sends email to listname-leave(a)listname, an unsubscribe no= tification is sent to both list owner and list member if list settings have t= hem enabled. 2. If list list member is a registered user of Postorius and he uses Postoriu= s to unsubscribe, then no notification is sent out to either list owner or li= st member, even if list settings has unsubscribe notifications enabled. Is this correct? Brian --===============6891730998894447632==-- From maxking@asynchronous.in Thu Sep 3 04:58:58 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: Unsubscribe Notifications Clarification Date: Wed, 02 Sep 2020 21:58:31 -0700 Message-ID: In-Reply-To: <159905715562.18626.16826911226719987593@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0868794412708882210==" --===============0868794412708882210== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On Wed, Sep 2, 2020, at 7:32 AM, Brian Carpenter wrote: > I am trying to understand how Mailman 3 handles unsubscribe > notifications. From my testing I see the following behavior: > > 1. If list member sends email to listname-leave(a)listname, an > unsubscribe notification is sent to both list owner and list member if > list settings have them enabled. > > 2. If list list member is a registered user of Postorius and he uses > Postorius to unsubscribe, then no notification is sent out to either > list owner or list member, even if list settings has unsubscribe > notifications enabled. > > Is this correct? This is a bug in Core, for which I have opened https://gitlab.com/mailman/mailman/-/issues/759 It seems like the Unsubscriptions via the API do not follow the right codepath and just unsubscribe the user. It also does not seem to take the unsubscription policy into account. > Brian > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking) --===============0868794412708882210==-- From maxking@asynchronous.in Thu Sep 3 05:09:34 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: adding imprint / private policy link to postorius Date: Wed, 02 Sep 2020 22:09:06 -0700 Message-ID: In-Reply-To: <5b8e0513d64d4d8449ab0763c06053fe@meine-oma.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7015829710154895701==" --===============7015829710154895701== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Tue, Sep 1, 2020, at 11:17 PM, Thomas Stein wrote: >=20 > Hello one and all. >=20 > Is it possible to add an imprint link to the postorius start page? I=20 > guess it is, but it would be > nice if someone could me point to the right direction / documentation.=20 > Something like they have: >=20 > https://lists.openrailwaymap.org/mailman3/postorius/lists/ >=20 There isn't an official way to do that, but you can patch your installation to update the footer. This[1] is the location and file you want to update and you can insert simple HTML there. If you are using system packages it should be somewhere in the `/usr/lib/python3/dist-packages` directory and in `lib/site-packages` if you are using a virtualenv based installation. https://gitlab.com/mailman/postorius/-/blob/master/src/postorius/templates/po= storius/base.html#L136 > cheers, t. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --=20 thanks, Abhilash Raj (maxking) --===============7015829710154895701==-- From himbeere@meine-oma.de Thu Sep 3 07:34:57 2020 From: Thomas Stein To: mailman-users@mailman3.org Subject: [MM3-users] Re: adding imprint / private policy link to postorius Date: Thu, 03 Sep 2020 09:34:42 +0200 Message-ID: <4144a2de967de045f6cf7df6e5834d4b@meine-oma.de> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0129698796774770510==" --===============0129698796774770510== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Thanks Abhilash. Will have a look. On 2020-09-03 07:09, Abhilash Raj wrote: > On Tue, Sep 1, 2020, at 11:17 PM, Thomas Stein wrote: >>=20 >> Hello one and all. >>=20 >> Is it possible to add an imprint link to the postorius start page? I >> guess it is, but it would be >> nice if someone could me point to the right direction / documentation. >> Something like they have: >>=20 >> https://lists.openrailwaymap.org/mailman3/postorius/lists/ >>=20 >=20 > There isn't an official way to do that, but you can patch your > installation to update the footer. >=20 > This[1] is the location and file you want to update and you can insert > simple HTML there. If you are using system packages it should be > somewhere in the `/usr/lib/python3/dist-packages` directory and > in `lib/site-packages` if you are using a virtualenv based=20 > installation. >=20 >=20 > https://gitlab.com/mailman/postorius/-/blob/master/src/postorius/templates/= postorius/base.html#L136 >=20 >> cheers, t. >> _______________________________________________ >> Mailman-users mailing list -- mailman-users(a)mailman3.org >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >>=20 --===============0129698796774770510==-- From brian_carpenter@emwd.com Thu Sep 3 14:05:37 2020 From: Brian Carpenter To: mailman-users@mailman3.org Subject: [MM3-users] Re: Unsubscribe Notifications Clarification Date: Thu, 03 Sep 2020 10:05:32 -0400 Message-ID: <36093325-fdc8-2c71-8a73-90160c953332@emwd.com> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5583337262947317626==" --===============5583337262947317626== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/3/20 12:58 AM, Abhilash Raj wrote: > This is a bug in Core, for which I have opened > > https://gitlab.com/mailman/mailman/-/issues/759 > > It seems like the Unsubscriptions via the API do not follow the > right codepath and just unsubscribe the user. It also does not > seem to take the unsubscription policy into account. Thank you for confirming that. -- Please let me know if you need further assistance. Thank you for your business. We appreciate our clients. Brian Carpenter EMWD.com -- EMWD's Knowledgebase: https://clientarea.emwd.com/index.php/knowledgebase EMWD's Community Forums http://discourse.emwd.com/ --===============5583337262947317626==-- From myroughbox@yahoo.com Thu Sep 3 16:51:43 2020 From: Jane Anara To: mailman-users@mailman3.org Subject: [MM3-users] Re: Please help: Mailaman3 - Date: Wed, 02 Sep 2020 17:28:37 +0000 Message-ID: <159906771705.18626.6008341078244851460@mailman.iad1.psf.io> In-Reply-To: <5ce2965f-d034-4eca-9f12-e2e652746105@beta.fastmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2516113757747760927==" --===============2516113757747760927== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello Abhilash, Thank you for your reply. I did use the mailman-suit url.py before and got similar errors. At your sugg= estion, I again included the following in the import session: from django.urls import reverse_lazy from django.views.generic import RedirectView and the following line in the urlpatterns: url(r'^$', RedirectView.as_view(url=3Dreverse_lazy ('list_index'), per= manent=3DTrue)), The 'list_index' undefined error does not come up, however the HTTP Error 404= I posted in my last message still shows up. I abstracted the following from = the error log as a potential cause: Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.6/site-packages/django/core/handlers/ba= se.py", line 113, in _get_response response =3D wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/mailman/venv/lib/python3.6/site-packages/postorius/views/list.py= ", line 779, in list_index paginator_class=3DMailmanPaginator) File "/opt/mailman/venv/lib/python3.6/site-packages/django_mailman3/lib/pag= inator.py", line 71, in paginate objects =3D paginator.page(page_num) File "/opt/mailman/venv/lib/python3.6/site-packages/django_mailman3/lib/pag= inator.py", line 46, in page number =3D self.validate_number(number) File "/opt/mailman/venv/lib/python3.6/site-packages/django/core/paginator.p= y", line 48, in validate_number if number > self.num_pages: File "/opt/mailman/venv/lib/python3.6/site-packages/django/utils/functional= .py", line 48, in __get__ res =3D instance.__dict__[self.name] =3D self.func(instance) File "/opt/mailman/venv/lib/python3.6/site-packages/django/core/paginator.p= y", line 97, in num_pages if self.count =3D=3D 0 and not self.allow_empty_first_page: File "/opt/mailman/venv/lib/python3.6/site-packages/django/utils/functional= .py", line 48, in __get__ res =3D instance.__dict__[self.name] =3D self.func(instance) File "/opt/mailman/venv/lib/python3.6/site-packages/django_mailman3/lib/pag= inator.py", line 56, in count return self.function(count=3D0, page=3D1).total_size File "/opt/mailman/venv/lib/python3.6/site-packages/postorius/views/list.py= ", line 775, in _get_list_page advertised=3Dadvertised, mail_host=3Dmail_host, count=3Dcount, page=3Dpag= e) File "/opt/mailman/venv/lib/python3.6/site-packages/mailmanclient/client.py= ", line 181, in get_list_page return Page(self._connection, url, MailingList, count, page) File "/opt/mailman/venv/lib/python3.6/site-packages/mailmanclient/restbase/= page.py", line 37, in __init__ self._create_page() File "/opt/mailman/venv/lib/python3.6/site-packages/mailmanclient/restbase/= page.py", line 62, in _create_page response, content =3D self._connection.call(self._build_url()) File "/opt/mailman/venv/lib/python3.6/site-packages/mailmanclient/restbase/= connection.py", line 112, in call error_msg, response, None) urllib.error.HTTPError: HTTP Error 404:=20 I would appreciate any insight. Thank you. Regards, Jane --===============2516113757747760927==-- From mathias.bedu@epitech.eu Thu Sep 3 16:51:50 2020 From: mathias.bedu@epitech.eu To: mailman-users@mailman3.org Subject: [MM3-users] mailman procesing small amount of mail, then stop process them like 30 seconds, and then loop back Date: Thu, 03 Sep 2020 09:31:37 +0000 Message-ID: <159912549766.18632.2249578231145318273@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8283722190605068989==" --===============8283722190605068989== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi,=20 I am currently trying to optimize my mailman installation, which is working. = The only issue is that i see that mailman seems to process a few number of me= ssages (around 50 each times), then stopping, and then processing few next me= ssages.=20 I spend quite some times searching for eventual parameters that could slow do= wn.=20 I precise that the machine itself is largely more powerful than what mailman = is using. So my question is : could you give me some leads for potentials parameters or= stuff like this that i could modify to optimize the installation ? Best regards,=20 Mathias --===============8283722190605068989==-- From maxking@asynchronous.in Thu Sep 3 16:55:00 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: Please help: Mailaman3 - Date: Thu, 03 Sep 2020 09:54:33 -0700 Message-ID: In-Reply-To: <159906771705.18626.6008341078244851460@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6988623311759499768==" --===============6988623311759499768== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Wed, Sep 2, 2020, at 10:28 AM, Jane Anara via Mailman-users wrote: > Hello Abhilash, >=20 > Thank you for your reply. >=20 > I did use the mailman-suit url.py before and got similar errors. At=20 > your suggestion, I again included the following in the import session: >=20 > from django.urls import reverse_lazy > from django.views.generic import RedirectView >=20 > and the following line in the urlpatterns: >=20 > url(r'^$', RedirectView.as_view(url=3Dreverse_lazy ('list_index'),=20 > permanent=3DTrue)), >=20 >=20 > The 'list_index' undefined error does not come up, however the HTTP=20 > Error 404 I posted in my last message still shows up. I abstracted the=20 > following from the error log as a potential cause: >=20 > Traceback (most recent call last): > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/django/core/handlers/base.py= ", line 113, in _get_response > response =3D wrapped_callback(request, *callback_args,=20 > **callback_kwargs) > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/postorius/views/list.py", li= ne 779, in list_index > paginator_class=3DMailmanPaginator) > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/django_mailman3/lib/paginato= r.py", line 71, in paginate > objects =3D paginator.page(page_num) > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/django_mailman3/lib/paginato= r.py", line 46, in page > number =3D self.validate_number(number) > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/django/core/paginator.py", l= ine 48, in validate_number > if number > self.num_pages: > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/django/utils/functional.py",= line 48, in __get__ > res =3D instance.__dict__[self.name] =3D self.func(instance) > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/django/core/paginator.py", l= ine 97, in num_pages > if self.count =3D=3D 0 and not self.allow_empty_first_page: > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/django/utils/functional.py",= line 48, in __get__ > res =3D instance.__dict__[self.name] =3D self.func(instance) > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/django_mailman3/lib/paginato= r.py", line 56, in count > return self.function(count=3D0, page=3D1).total_size > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/postorius/views/list.py", li= ne 775, in _get_list_page > advertised=3Dadvertised, mail_host=3Dmail_host, count=3Dcount, page=3Dp= age) > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/mailmanclient/client.py", li= ne 181, in get_list_page > return Page(self._connection, url, MailingList, count, page) > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/mailmanclient/restbase/page.= py", line 37, in __init__ > self._create_page() > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/mailmanclient/restbase/page.= py", line 62, in _create_page > response, content =3D self._connection.call(self._build_url()) > File=20 > "/opt/mailman/venv/lib/python3.6/site-packages/mailmanclient/restbase/conne= ction.py", line 112, in call > error_msg, response, None) > urllib.error.HTTPError: HTTP Error 404:=20 >=20 >=20 > I would appreciate any insight. This is trying to reach Mailman core and getting a 404 response from Mailman = Core. Do you have Core running? Which version of Core are you running? >=20 > Thank you. >=20 > Regards, > Jane > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --=20 thanks, Abhilash Raj (maxking) --===============6988623311759499768==-- From skenny@scss.tcd.ie Fri Sep 4 09:36:30 2020 From: skenny@scss.tcd.ie To: mailman-users@mailman3.org Subject: [MM3-users] Re: Struggling with log rotation in Docker context Date: Fri, 04 Sep 2020 09:36:26 +0000 Message-ID: <159921218673.18632.9788456586017905771@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8962446086918667827==" --===============8962446086918667827== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Danil, Firstly, thank you so much for your reply, and I do apologize for taking so l= ong to respond to your message. I understand your approach, and it sounds reliable and sensible. I'm still a= bit confused by something though. In your example, you create the file /etc= /logrotate.d/mailman - I presume that it must live in, or be available in the= container - but which one? You've created a single file, with entries for = both the mailman-core and mailman-web logs. But the logs of -core are not av= ailable in -web and vice-versa. Did you mean to do separate "cat < /etc/logrotate.d/mailman" operations for each container? Thanks again Stephen --===============8962446086918667827==-- From r.woithe@callassoftware.com Fri Sep 4 10:23:32 2020 From: r.woithe@callassoftware.com To: mailman-users@mailman3.org Subject: [MM3-users] File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Fri, 04 Sep 2020 07:31:53 +0000 Message-ID: <159920471343.18624.9829816949185245855@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7904919598948490587==" --===============7904919598948490587== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, every night at 00:00 i get an email by mailman3: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/django_extensions/management/command= s/runjobs.py", line 34, in runjobs job().execute() File "/usr/lib/python2.7/dist-packages/hyperkitty/jobs/sync_mailman.py", li= ne 37, in execute sync_with_mailman() File "/usr/lib/python2.7/dist-packages/hyperkitty/lib/mailman.py", line 147= , in sync_with_mailman sender.set_mailman_id() File "/usr/lib/python2.7/dist-packages/hyperkitty/models/sender.py", line 5= 7, in set_mailman_id mm_user =3D client.get_user(self.address) File "/usr/lib/python2.7/dist-packages/mailmanclient/client.py", line 192, = in get_user 'users/{0}'.format(address)) File "/usr/lib/python2.7/dist-packages/mailmanclient/restbase/connection.py= ", line 99, in call response, content =3D Http().request(url, method, data_str, headers) File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1616, in= request (response, content) =3D self._request(conn, authority, uri, request_uri, = method, body, headers, redirections, cachekey) File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1358, in= _request (response, content) =3D self._conn_request(conn, request_uri, method, bod= y, headers) File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1314, in= _conn_request response =3D conn.getresponse() File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() ResponseNotReady ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty) START TRACEBACK: END TRACEBACK Mailman3 runs on Ubuntu 18.04 and is installed via the apt-get repository pac= kages. Mailman3 is working and created lists are functional. Therefor i would like to know can i neglect this message ? Or is this message= a hint for an Error i should investigate and looking for a solution ? --===============7904919598948490587==-- From danil@smirnov.la Fri Sep 4 13:10:39 2020 From: Danil Smirnov To: mailman-users@mailman3.org Subject: [MM3-users] Re: Struggling with log rotation in Docker context Date: Fri, 04 Sep 2020 16:10:23 +0300 Message-ID: In-Reply-To: <159921218673.18632.9788456586017905771@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4484770982366907128==" --===============4484770982366907128== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Hi Stephen, My example assumes that there are folders (inside of /basedir) on a Docker host machine which are mounted to mailman containers. So we run logrotate on the host machine against those mapped folders. Hence, /etc/logrotate.d/mailman file is for the host machine as well. Danil On Fri, Sep 4, 2020 at 12:36 PM wrote: > Hi Danil, > > Firstly, thank you so much for your reply, and I do apologize for taking > so long to respond to your message. > > I understand your approach, and it sounds reliable and sensible. I'm > still a bit confused by something though. In your example, you create the > file /etc/logrotate.d/mailman - I presume that it must live in, or be > available in the container - but which one? You've created a single file, > with entries for both the mailman-core and mailman-web logs. But the logs > of -core are not available in -web and vice-versa. Did you mean to do > separate > "cat < /etc/logrotate.d/mailman" operations for each container? > > Thanks again > Stephen > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --===============4484770982366907128==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Fri Sep 4 13:32:53 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Fri, 04 Sep 2020 22:32:44 +0900 Message-ID: <24402.16892.959945.138792@turnbull.sk.tsukuba.ac.jp> In-Reply-To: <159920471343.18624.9829816949185245855@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2410762728983722468==" --===============2410762728983722468== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Perhaps someone else can provide more concrete help, but you're running a version of HyperKitty that is at least 2 years old (we stopped supporting Python 2 around second quarter 2018). I strongly recommend upgrading. I understand that this may be extremely inconvenient since you're running an older Ubuntu LTS, but not only have a lot of features been added since then, a lot of bugs have been fixed. r.woithe(a)callassoftware.com writes: > every night at 00:00 i get an email by mailman3: >=20 > Traceback (most recent call last): > File "/usr/lib/python2.7/dist-packages/django_extensions/management/comm= ands/runjobs.py", line 34, in runjobs > job().execute() > File "/usr/lib/python2.7/dist-packages/hyperkitty/jobs/sync_mailman.py",= line 37, in execute > sync_with_mailman() I don't recognize this issue, but this looks dangerous. I believe that this process keeps HyperKitty's database synced with Mailman's, and if this process is consistently failing, they could become unsynced and HyperKitty's archive could become corrupt or report incorrect information. --===============2410762728983722468==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Fri Sep 4 13:35:50 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] mailman procesing small amount of mail, then stop process them like 30 seconds, and then loop back Date: Fri, 04 Sep 2020 22:35:42 +0900 Message-ID: <24402.17070.757139.504663@turnbull.sk.tsukuba.ac.jp> In-Reply-To: <159912549766.18632.2249578231145318273@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2485866959386910051==" --===============2485866959386910051== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit mathias.bedu(a)epitech.eu writes: > I am currently trying to optimize my mailman installation, which is > working. The only issue is that i see that mailman seems to process > a few number of messages (around 50 each times), then stopping, and > then processing few next messages. What makes you think that this is Mailman stopping? Mailman's default behavior is to send messages to the MTA as fast as it will accept them, and let the MTA take care of any throttling or optimization. Mailman even tries to parallelize this process. So if you're seeing delays I believe that is more likely to be the MTA delaying acceptance rather than Mailman. The other possibility is lists that have DMARC mitigation on only for p=reject domains. Then there may be delays due to DNS lookups to check for the From domain's DMARC policy. > I precise that the machine itself is largely more powerful than > what mailman is using. This is probably true, but CPU and memory are not usually what limits mail delivery. Aside from MTA and DNS delays, and very underpowered hosts, there's no reason why Mailman would be be stopping for more than a second or two even if some of the "be nice to the MTA" parameters below are given restrictive values. > So my question is : could you give me some leads for potentials > parameters or stuff like this that i could modify to optimize the > installation ? You might look at these parameters in your configuration file, or in the definition file schema.cfg (do not change this file because it is overwritten by upgrades). The defaults are set pretty much as fast as possible. # Ceiling on the number of recipients that can be specified in a single SMTP # transaction. Set to 0 to submit the entire recipient list in one # transaction. max_recipients: 500 # Ceiling on the number of SMTP sessions to perform on a single socket # connection. Some MTAs have limits. Set this to 0 to do as many as we like # (i.e. your MTA has no limits). Set this to some number great than 0 and # Mailman will close the SMTP connection and re-open it after this number of # consecutive sessions. max_sessions_per_connection: 0 # Maximum number of simultaneous subthreads that will be used for SMTP # delivery. After the recipients list is chunked according to max_recipients, # each chunk is handed off to the SMTP server by a separate such thread. If # your Python interpreter was not built for threads, this feature is disabled. # You can explicitly disable it in all cases by setting max_delivery_threads # to 0. max_delivery_threads: 0 --===============2485866959386910051==-- From mathias.bedu@epitech.eu Fri Sep 4 13:52:27 2020 From: mathias.bedu@epitech.eu To: mailman-users@mailman3.org Subject: [MM3-users] Re: mailman procesing small amount of mail, then stop process them like 30 seconds, and then loop back Date: Fri, 04 Sep 2020 13:52:24 +0000 Message-ID: <159922754423.18632.15782782234946218113@mailman.iad1.psf.io> In-Reply-To: <24402.17070.757139.504663@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3439428456780140969==" --===============3439428456780140969== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable It was actually the MTA which was not dealing fast enought, problem solved, i= tested with another, and it was fine. thanks very much. --===============3439428456780140969==-- From myroughbox@yahoo.com Fri Sep 4 15:35:01 2020 From: Jane Anara To: mailman-users@mailman3.org Subject: [MM3-users] Re: Please help: Mailaman3 - Date: Fri, 04 Sep 2020 15:34:58 +0000 Message-ID: <159923369852.18626.10404591105871455547@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0032303700173830396==" --===============0032303700173830396== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Thank you Abhilash for your reply. I installed Mailman Core version 3.3.1. After your message, I ran `mailman start` and then ran the development server= . It complained of the port already in used and exited. That is what I got i= n the past as well. This time, I changed the CORE REST API port on django set= tings.py from port 8000 to 8001 and ran the development server (which I often= ran on 8001) on port 8006. The error has gone away. Thank you for your help. Regards, Jane --===============0032303700173830396==-- From skenny@scss.tcd.ie Fri Sep 4 20:48:55 2020 From: skenny@scss.tcd.ie To: mailman-users@mailman3.org Subject: [MM3-users] Re: Struggling with log rotation in Docker context Date: Fri, 04 Sep 2020 20:48:52 +0000 Message-ID: <159925253286.18626.14147477653680823576@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7345498560261202369==" --===============7345498560261202369== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable OK, but inside say the mailman-web container, doesn't mailman still have a fi= le descriptor open and pointing to the old (now rotated/renamed) log file? -= I don't see how doing the rotation on the Docker host aloneis going to lea= d to mailman logging to a new log file. Sorry if I'm being a bit dumb here. Any enlightenment welcome. Stephen --===============7345498560261202369==-- From danil@smirnov.la Sat Sep 5 06:04:44 2020 From: Danil Smirnov To: mailman-users@mailman3.org Subject: [MM3-users] Re: Struggling with log rotation in Docker context Date: Sat, 05 Sep 2020 09:04:27 +0300 Message-ID: In-Reply-To: <159925253286.18626.14147477653680823576@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0205121925296136318==" --===============0205121925296136318== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable For an answer, please refer to my earlier email in this thread: https://lists.mailman3.org/archives/list/mailman-users(a)mailman3.org/message= /GCG3754A245YA3RNM3GV5DQUAWEJTTZF/ On Fri, Sep 4, 2020 at 11:49 PM wrote: > OK, but inside say the mailman-web container, doesn't mailman still have a > file descriptor open and pointing to the old (now rotated/renamed) log > file? - I don't see how doing the rotation on the Docker host aloneis > going to lead to mailman logging to a new log file. > > Sorry if I'm being a bit dumb here. Any enlightenment welcome. > > Stephen > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --===============0205121925296136318==-- From simon.handfield@sl.nsw.gov.au Sun Sep 6 22:40:04 2020 From: Simon Handfield To: mailman-users@mailman3.org Subject: [MM3-users] Re: Mailman3 throws 500 error (unicode encode) when loading held messages in Postorius Date: Sun, 06 Sep 2020 22:39:50 +0000 Message-ID: In-Reply-To: <24397.127.508049.828766@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6866838531369544271==" --===============6866838531369544271== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Thanks for your advice, Stephen. As I've removed the file from the shunt queue I don't believe that file is ca= using the 500 error any more. I've confirmed the error is not cached in the browser. If the issue does not reside in the shunt queue (no other files indicate pars= e errors when checking via command line) are there any other places I should = look for data that is addressed by that page? Cheers, Simon. ________________________________ From: Stephen J. Turnbull Sent: Monday, 31 August 2020 23:51 To: Simon Handfield Cc: mailman-users(a)mailman3.org Subject: [MM3-users] Re: Mailman3 throws 500 error (unicode encode) when load= ing held messages in Postorius Simon Handfield via Mailman-users writes: > I have been able to identify a file in the shunt queue that threw a > matching error when I performed mailman qfile on it. > I have run a loop over all remaining files in the shunt queue and > all the remaining files can be accessed by mailman qfile without > error. Good. But there are still a bunch of problems with those files since they got shunted. > I moved this file out of the queue in the hope that I would then be > able to load the held_messages page successfully. Unfortunately I > am still seeing a 500 error on loading the page. If this is a Mailman problem, I don't understand it. The shunt queue is not the moderation queue. The shunt queue is for messages that Mailman cannot handle without throwing an exception. Once a message is sent to the shunt queue, Mailman ignores it, so it shouldn't cause problems for the moderation page. Have you tried restarting Mailman since moving the qfile? If that doesn't work, and if it's the same error message and traceback, the easy suspect is browser caching (although we should be disabling that, I hope). Try a manual refresh (usually Ctrl-R or in the menu). If that doesn't work, try a private browsing window or another browser if you don't want to clear the cache or restart the browser. > I temporarily reenabled debug logging to confirm that the same > error was being thrown. > > Assuming that file was the cause are there other steps I need to > perform in addition to removing the file from the shunt queue > directory? The only one I can think of is the check for browser-side caching. Unfortunately, Mark's out of town (like, on top of a mountain with no bars on his phone), and I am not going to be able to do much with this until Thursday at the earliest. If you do get a new traceback, post it here (somebody else may have an idea). If you like you may send me the qfile at the address in From. I'll see if I can reproduce the bug, and I'm pretty sure I can tell you what's in the file so at least you'll know whether it's important to somebody who isn't a scammer/spammer. Steve _ [State Library of NSW] Simon Handfield Desktop and Infrastructure Leader simon.handfield(a)sl.nsw.gov.au +61 2 9273 1535 Macquarie Street, Sydney NSW 2000, Australia www.sl.nsw.gov.au This message is intended for the addressee named and may contain confidential= information. If you are not the intended recipient, please delete it and not= ify the sender. Views expressed in this message are those of the individual s= ender, and are not necessarily the views of their organisation. --===============6866838531369544271==-- From maxking@asynchronous.in Sun Sep 6 22:58:40 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: Please help: Mailaman3 - Date: Sun, 06 Sep 2020 15:58:14 -0700 Message-ID: <991b91ee-d6e1-4869-8b4b-fc324c82c822@www.fastmail.com> In-Reply-To: <159923369852.18626.10404591105871455547@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0196787575233431030==" --===============0196787575233431030== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On Fri, Sep 4, 2020, at 8:34 AM, Jane Anara via Mailman-users wrote: > Thank you Abhilash for your reply. I installed Mailman Core version 3.3.1. > > After your message, I ran `mailman start` and then ran the development > server. It complained of the port already in used and exited. That is > what I got in the past as well. This time, I changed the CORE REST API > port on django settings.py from port 8000 to 8001 and ran the > development server (which I often ran on 8001) on port 8006. The error > has gone away. By default, Core's REST API runs on Port 8001 and Django uses 8000. If you were using 8000 for the Core's Port in settings.py, that would mean Postorius was trying to talk to itself and hence the 404 errors with the URLs. > > Thank you for your help. > > Regards, > Jane > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking) --===============0196787575233431030==-- From r.woithe@callassoftware.com Mon Sep 7 20:57:26 2020 From: roberto To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Mon, 07 Sep 2020 09:09:29 +0200 Message-ID: In-Reply-To: <24402.16892.959945.138792@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0678346183840569236==" --===============0678346183840569236== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, thanks for the fast reply. I will try to upgrade the ubuntu packages, if=20 this will not solve the problem i will upgrade to ubuntu 20.04 ... Am 04.09.20 um 15:32 schrieb Stephen J. Turnbull: > Perhaps someone else can provide more concrete help, but you're > running a version of HyperKitty that is at least 2 years old (we > stopped supporting Python 2 around second quarter 2018). I strongly > recommend upgrading. I understand that this may be extremely > inconvenient since you're running an older Ubuntu LTS, but not only > have a lot of features been added since then, a lot of bugs have been > fixed. > > r.woithe(a)callassoftware.com writes: > > every night at 00:00 i get an email by mailman3: > > > > Traceback (most recent call last): > > File "/usr/lib/python2.7/dist-packages/django_extensions/management/c= ommands/runjobs.py", line 34, in runjobs > > job().execute() > > File "/usr/lib/python2.7/dist-packages/hyperkitty/jobs/sync_mailman.p= y", line 37, in execute > > sync_with_mailman() > > I don't recognize this issue, but this looks dangerous. I believe > that this process keeps HyperKitty's database synced with Mailman's, > and if this process is consistently failing, they could become > unsynced and HyperKitty's archive could become corrupt or report > incorrect information. > > > --=20 Roberto Woithe | IT-Systemadministrator callas software GmbH | Schoenhauser Allee 6/7 | 10119 Berlin | Germany Tel +49.30.4439031-0 | Fax +49.30.4416402 | www.callassoftware.com Amtsgericht Charlottenburg, HRB 59615 Gesch=C3=A4ftsf=C3=BChrung: Olaf Dr=C3=BCmmer, Ulrich Frotscher, Dietrich von= Seggern --===============0678346183840569236==-- From r.woithe@callassoftware.com Mon Sep 7 20:57:32 2020 From: roberto To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Mon, 07 Sep 2020 12:23:48 +0200 Message-ID: In-Reply-To: <24402.16892.959945.138792@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0433697049864651867==" --===============0433697049864651867== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, tried to upgrade ubuntu 18.04 to 20.04 but the packages mailman3=20 (full,web) bringing a lot of problems. If someone have the same problems like me, better don't try to upgrade=20 cause it brings a lot of trouble, maybe it is better to use a new=20 installation of ubuntu 20.04 and install the mailman3 packages after=20 with apt. Am 04.09.20 um 15:32 schrieb Stephen J. Turnbull: > Perhaps someone else can provide more concrete help, but you're > running a version of HyperKitty that is at least 2 years old (we > stopped supporting Python 2 around second quarter 2018). I strongly > recommend upgrading. I understand that this may be extremely > inconvenient since you're running an older Ubuntu LTS, but not only > have a lot of features been added since then, a lot of bugs have been > fixed. > > r.woithe(a)callassoftware.com writes: > > every night at 00:00 i get an email by mailman3: > > > > Traceback (most recent call last): > > File "/usr/lib/python2.7/dist-packages/django_extensions/management/c= ommands/runjobs.py", line 34, in runjobs > > job().execute() > > File "/usr/lib/python2.7/dist-packages/hyperkitty/jobs/sync_mailman.p= y", line 37, in execute > > sync_with_mailman() > > I don't recognize this issue, but this looks dangerous. I believe > that this process keeps HyperKitty's database synced with Mailman's, > and if this process is consistently failing, they could become > unsynced and HyperKitty's archive could become corrupt or report > incorrect information. > > > --=20 Roberto Woithe | IT-Systemadministrator callas software GmbH | Schoenhauser Allee 6/7 | 10119 Berlin | Germany Tel +49.30.4439031-0 | Fax +49.30.4416402 | www.callassoftware.com Amtsgericht Charlottenburg, HRB 59615 Gesch=C3=A4ftsf=C3=BChrung: Olaf Dr=C3=BCmmer, Ulrich Frotscher, Dietrich von= Seggern --===============0433697049864651867==-- From r.woithe@callassoftware.com Tue Sep 8 12:31:02 2020 From: r.woithe@callassoftware.com To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Tue, 08 Sep 2020 12:30:59 +0000 Message-ID: <159956825971.18625.7659058701044057408@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3695181239475905116==" --===============3695181239475905116== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable UPDATE Still trying to repair the upgrade of ubuntu 20.04. the command apt-get upgrade did give the hint: mailman3-web (0+20180916-10) wird eingerichtet ... dbconfig-common: writing config to /etc/dbconfig-common/mailman3-web.conf dbconfig-common: flushing administrative password CommandError: Offline compression is disabled. Set COMPRESS_OFFLINE or use th= e --force to override. dpkg: Fehler beim Bearbeiten des Paketes mailman3-web (--configure): =C2=BBinstalliertes mailman3-web-Skript des Paketes post-installation=C2=AB-= Unterprozess gab den Fehlerwert 1 zur=C3=BCck dpkg: Abh=C3=A4ngigkeitsprobleme verhindern Konfiguration von mailman3-full: mailman3-full h=C3=A4ngt ab von mailman3-web; aber: Paket mailman3-web ist noch nicht konfiguriert. Therefor i set in mailman3-conf.py: COMPRESS_OFFLINE=3DTRUE Now the system is at the point: Compressing... done Compressed 2 block(s) from 63 template(s) for 1 context(s). Rebuilding indexes database for python3 version. This may take some time. There are almost 100 list included in this postgresql-database, therefor i ab= solutely guess it will take some time. Maybe somebody of u know what happening and am i on the right way? --===============3695181239475905116==-- From accounts-mailman@holbrook.no Wed Sep 9 08:23:56 2020 From: accounts-mailman@holbrook.no To: mailman-users@mailman3.org Subject: [MM3-users] Re-send subscription response and file permission error in queue Date: Wed, 09 Sep 2020 05:56:08 +0000 Message-ID: <159963096883.18624.1418458365591904001@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4809680994422549637==" --===============4809680994422549637== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable A user sent subscription request to the `..-join@` address of a list. Email d= omain wasn't set up properly, so response got rejected. Now, if the user trie= s to send another request, no reponse comes back. In the log I find a python = stack trace saying subscription already pending. ``` Sep 08 10:32:44 2020 (19918) Traceback (most recent call last): File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 171, in = _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 264, in = _process_one_file keepqueued =3D self._dispose(mlist, msg, msgdata) File "/usr/lib/python3/dist-packages/mailman/runners/command.py", line 196,= in _dispose mlist, msg, msgdata, parts, results) File "/usr/lib/python3/dist-packages/mailman/commands/eml_membership.py", l= ine 104, in process ISubscriptionManager(mlist).register(subscriber) File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line 52= 0, in register list(workflow) File "/usr/lib/python3/dist-packages/mailman/app/workflow.py", line 69, in = __next__ return step() File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line 22= 7, in _step_sanity_checks raise SubscriptionPendingError(self.mlist, self.address.email) mailman.interfaces.subscriptions.SubscriptionPendingError ``` Furthermore I am getting a permission denied error in the attempt of shunting= the message, which I don't quite understand because the folder has user `lis= t` group `list` and `700` mode: ``` Sep 08 10:25:29 2020 (19918) SHUNTING: 1599553529.9889646+88be5afa0bf3111a83e= 68cc97ce2000723ad93fd Sep 08 10:29:49 2020 (19918) Uncaught runner exception: [Errno 13] Permission= denied: '/var/lib/mailman3/queue/command/1599553789.306307+6fee74af69263553b= 5ffd0e73cc095ab651bd855.pck' Sep 08 10:29:49 2020 (19918) Traceback (most recent call last): File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 156, in = _one_iteration msg, msgdata =3D self.switchboard.dequeue(filebase) File "/usr/lib/python3/dist-packages/mailman/core/switchboard.py", line 150= , in dequeue with open(filename, 'rb') as fp: PermissionError: [Errno 13] Permission denied: '/var/lib/mailman3/queue/comma= nd/1599553789.306307+6fee74af69263553b5ffd0e73cc095ab651bd855.pck' ``` How can I remove the pending subscription, or re-send the response email? And what could be the cause of the permission error? Using mailman 3.1.1 --===============4809680994422549637==-- From r.woithe@callassoftware.com Wed Sep 9 08:24:14 2020 From: r.woithe@callassoftware.com To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Wed, 09 Sep 2020 08:24:11 +0000 Message-ID: <159963985137.18624.11195841734273926354@mailman.iad1.psf.io> In-Reply-To: <159956825971.18625.7659058701044057408@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0683066888228074322==" --===============0683066888228074322== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit UPDATE The command u can see above realy did bring mailman3 to wok again. My question now is, should i set COMPRESS_OFFLINE to false again? --===============0683066888228074322==-- From r.woithe@callassoftware.com Thu Sep 10 07:13:02 2020 From: r.woithe@callassoftware.com To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Thu, 10 Sep 2020 07:12:59 +0000 Message-ID: <159972197989.18632.12329373976731132373@mailman.iad1.psf.io> In-Reply-To: <159963985137.18624.11195841734273926354@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8853506961288747041==" --===============8853506961288747041== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable UPDATE mailman3 woks now under python3 but i still get the nightly email, now with a= n other file but with the same result: File "/usr/lib/python3.8/http/client.py", line 1322, in getresponse raise ResponseNotReady(self.__state) http.client.ResponseNotReady: Request-started ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty) anybody can help ? additionally should i set COMPRESS_OFFLINE to false again? --===============8853506961288747041==-- From r.woithe@callassoftware.com Thu Sep 10 08:16:27 2020 From: r.woithe@callassoftware.com To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Thu, 10 Sep 2020 08:16:24 +0000 Message-ID: <159972578412.18624.2738235990589070674@mailman.iad1.psf.io> In-Reply-To: <159972197989.18632.12329373976731132373@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1750964911024057917==" --===============1750964911024057917== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable How u can see above: the file client.py is referenced about /usr/lib/python3.8/http but all other = files descripted in the email are referenced about /usr/lib/python3/dist-pack= ages/... But in /usr/lib/python3/ the directory and the file client.py is missing. Can this be the problem? --===============1750964911024057917==-- From danil@smirnov.la Thu Sep 10 13:43:33 2020 From: Danil Smirnov To: mailman-users@mailman3.org Subject: [MM3-users] messages go to shunt queue Date: Thu, 10 Sep 2020 16:43:19 +0300 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8928360039441800163==" --===============8928360039441800163== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Hi guys, I've faced weird Core behaviour after the last upgrade of the docker images. All the messages sent to a list now go straight to shunt queue with the following error in the logs: --- Sep 06 22:41:32 2020 (27) Uncaught runner exception: 0 Sep 06 22:41:32 2020 (27) Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/mailman/core/runner.py", line 173, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python3.8/site-packages/mailman/core/runner.py", line 266, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/python3.8/site-packages/mailman/runners/incoming.py", line 79, in _dispose process(mlist, msg, msgdata, start_chain) File "/usr/lib/python3.8/site-packages/mailman/core/chains.py", line 57, in process if link.rule.check(mlist, msg, msgdata): File "/usr/lib/python3.8/site-packages/mailman/rules/dmarc.py", line 314, in check if maybe_mitigate(mlist, address): File "/usr/lib/python3.8/site-packages/mailman/rules/dmarc.py", line 286, in maybe_mitigate answer = is_reject_or_quarantine( File "/usr/lib/python3.8/site-packages/mailman/rules/dmarc.py", line 217, in is_reject_or_quarantine for record in txt_rec.items[0].strings) KeyError: 0 Sep 06 22:41:32 2020 (27) SHUNTING: 1599432092.4326646+4cbb58a0dd5d9705946ca583835e63ec175536e3 I'm witnessing the same behaviour in two different installations for all the messages sent to list (with plain text body like "it's a test"). --- Any insight would be greatly appreciated! Danil Smirnov --===============8928360039441800163==-- From mark@msapiro.net Thu Sep 10 15:34:54 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: messages go to shunt queue Date: Thu, 10 Sep 2020 08:34:45 -0700 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4858747852379130868==" --===============4858747852379130868== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/10/20 6:43 AM, Danil Smirnov wrote: > Hi guys, > > I've faced weird Core behaviour after the last upgrade of the docker > images. All the messages sent to a list now go straight to shunt queue with > the following error in the logs: > > --- > Sep 06 22:41:32 2020 (27) Traceback (most recent call last): ... > File "/usr/lib/python3.8/site-packages/mailman/rules/dmarc.py", line 217, > in is_reject_or_quarantine > for record in txt_rec.items[0].strings) > KeyError: 0 > Sep 06 22:41:32 2020 (27) SHUNTING: > 1599432092.4326646+4cbb58a0dd5d9705946ca583835e63ec175536e3 See . -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============4858747852379130868==-- From danil@smirnov.la Thu Sep 10 15:56:48 2020 From: Danil Smirnov To: mailman-users@mailman3.org Subject: [MM3-users] Re: messages go to shunt queue Date: Thu, 10 Sep 2020 18:56:32 +0300 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7524726087589145918==" --===============7524726087589145918== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Thanks Mark! The change provided has fixed the issue. Danil On Thu, Sep 10, 2020 at 6:35 PM Mark Sapiro wrote: > On 9/10/20 6:43 AM, Danil Smirnov wrote: > > Hi guys, > > > > I've faced weird Core behaviour after the last upgrade of the docker > > images. All the messages sent to a list now go straight to shunt queue > with > > the following error in the logs: > > > > --- > > Sep 06 22:41:32 2020 (27) Traceback (most recent call last): > ... > > File "/usr/lib/python3.8/site-packages/mailman/rules/dmarc.py", line > 217, > > in is_reject_or_quarantine > > for record in txt_rec.items[0].strings) > > KeyError: 0 > > Sep 06 22:41:32 2020 (27) SHUNTING: > > 1599432092.4326646+4cbb58a0dd5d9705946ca583835e63ec175536e3 > > > See . > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --===============7524726087589145918==-- From mark@msapiro.net Fri Sep 11 00:08:52 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Messages disappearing into hole?? Date: Thu, 10 Sep 2020 17:08:43 -0700 Message-ID: In-Reply-To: <3030906f-3486-0e41-c340-cfc03d92dfad@ivimey.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4993901578529522611==" --===============4993901578529522611== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 8/30/20 6:15 AM, Ruth Ivimey-Cook wrote: >=20 > I looked around the interface and decided to set up the sender as a > non-member, as that appeared to permit them to be "special" and outside > of the normal access rules for the list. Before I did that I could send > a message from this sender and that would be sent to me as list owner > for moderation, so both incoming (from my mailserver) and outgoing (to > it) can be completed. >=20 > However, with a non-member who is auto-accepted the message is passed > from my mailserver to mailman and then disappears. I cannot find > anything in the logs to indicate what happened next. I have tried > setting the default non-member policy to both 'hold' and 'default' with > no change in that behaviour. The logs for the mailserver show the > message being sent to mailman; mailman's "smtp.log" shows an LMTP > message being received ok, and then nothing. >=20 > Any ideas how to find out what is going on? I can share config if that > helps but not sure what would be useful. First, this is a bad way to configure an announce list because anyone can post by spoofing the authorized posting address. The better way is to set a list moderator password and post using an Approved: header or pseudo header. See for details. As far as the disappearing message is concerned, it's hard to say. What is the non-member's moderation action? Unless it is 'list default' setting the list's default non-member policy has no effect. Also, if the non-member's address or a pattern matching it is in accept_these_nonmembers, that takes precedence over whatever the non-member's moderation action is. One guess about the disappearing message is content filtering may be removing all the message. What is the setting for Filter Action? > As a side-issue, at one point I added a regex matching the sender > address to the acceptable members list, but find that I cannot now > delete it (using postorius): If I delete the field and Save, it returns > on page refresh. There is an issue with Postorius. You can't empty these settings. If you clear the box or or have it contain only whitespace, Postorius won't change the prior setting. In order to remove the only entry in Postorius, you have to replace it with a bogus entry or you can remove it with `mailman shell`. I've just filed for this. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============4993901578529522611==-- From mark@msapiro.net Fri Sep 11 02:49:51 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Failed first login Date: Thu, 10 Sep 2020 19:49:41 -0700 Message-ID: <1b90bb61-23ab-d57c-2721-e55120c4c3ea@msapiro.net> In-Reply-To: <159888238876.18624.5269520416243914495@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5487646429231292967==" --===============5487646429231292967== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 8/31/20 6:59 AM, Massimo Zappal=C3=A0 wrote: > Hi everyone, > i installed mailman3-full on debian 10. > During installation, it asks me to enter the admin user credentials. > After configuration, I restart mailman3 and mailman3-web. > When I try to log in for the first time by entering the admin credentials, = a message warns me that the credentials are incorrect. > I looked at the mailman logs, but I find nothing. > It seems that it is a django authentication problem, because even in the dj= ango admin panel I can't log in. > Looking at the mailman3web db, in the auth_user table, I see the admin user. > How can I see why it does not accept the credentials? > Thank you in advance I have no idea what the Debian package is doing, but you can see the superuser(s) with the database query select * from auth_user where is_superuser; and you can change the user's password with the django admin changepassword command and/or create a superuser with the django admin createsuperuser command. Give the commands with a --help option for more info. Since you can see the user in the auth_user table, try the django admin changepassword command to change the password. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5487646429231292967==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Fri Sep 11 04:07:57 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Fri, 11 Sep 2020 13:07:49 +0900 Message-ID: <24410.63509.901648.725675@turnbull.sk.tsukuba.ac.jp> In-Reply-To: <159972578412.18624.2738235990589070674@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8400986997574232131==" --===============8400986997574232131== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit r.woithe(a)callassoftware.com writes: > How u can see above: > the file client.py is referenced about /usr/lib/python3.8/http but > all other files descripted in the email are referenced about > /usr/lib/python3/dist-packages/... > But in /usr/lib/python3/ the directory and the file client.py is missing. > Can this be the problem? No, Python has a search path, so not all code needs to be in the same place. It appears that ResponseNotReady is being raised in that file, anyway, so that's not the problem. Unfortunately, I am not familiar with that code, so that's the most I can say until I have time to study it. Perhaps somebody else will pick it up in the meantime. Steve --===============8400986997574232131==-- From brylon@oit.rutgers.edu Fri Sep 11 14:30:26 2020 From: brylon@oit.rutgers.edu To: mailman-users@mailman3.org Subject: [MM3-users] Re: New releases Date: Fri, 11 Sep 2020 14:30:23 +0000 Message-ID: <159983462342.18624.4566478373636672112@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1859220691882541553==" --===============1859220691882541553== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi all -- I'm stuck on this "compress" issue after upgrading and not sure how to remedy? After upgrading the Mailman components (core, postorius, hyperkitty) on my te= st server to the latest available in my venv I ran the following: $ ./bin/django-admin collectstatic --clear --noinput --verbosity 0 and then $ ./bin/django-admin compress --trace Compressing... Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/base.py",= line 282, in precompile mod =3D import_module(mod_name) File "/opt/mailman/mm/venv/lib64/python3.6/importlib/__init__.py", line 126= , in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 994, in _gcd_import File "", line 971, in _find_and_load File "", line 953, in _find_and_load_unlocked ModuleNotFoundError: No module named 'sass -t compressed {infile} {outfile}' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/managemen= t/commands/compress.py", line 232, in compress result =3D parser.render_node(template, context, node) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/offline/d= jango.py", line 117, in render_node return node.render(context, forced=3DTrue) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/templatet= ags/compress.py", line 143, in render return self.render_compressed(context, self.kind, self.mode, forced=3Dfor= ced) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/templatet= ags/compress.py", line 119, in render_compressed rendered_output =3D compressor.output(mode, forced=3Dforced, basename=3Df= ile_basename) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/css.py", = line 46, in output ret.append(subnode.output(*args, **kwargs)) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/css.py", = line 48, in output return super(CssCompressor, self).output(*args, **kwargs) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/base.py",= line 314, in output output =3D '\n'.join(self.filter_input(forced)) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/base.py",= line 254, in filter_input for hunk in self.hunks(forced): File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/base.py",= line 228, in hunks precompiled, value =3D self.precompile(value, **options) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/base.py",= line 287, in precompile return True, filter.input(**kwargs) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/filters/b= ase.py", line 239, in input return super(CachedCompilerFilter, self).input(**kwargs) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/filters/b= ase.py", line 206, in input raise FilterError(err) compressor.exceptions.FilterError: Error: Invalid CSS after "...lor}: #{$valu= e}": expected "{", was ";" on line 4 of /opt/mailman/mm/venv/lib/python3.6/site-packages/hyperki= tty/static/hyperkitty/libs/bootstrap/stylesheets/_root.scss from line 11 of /opt/mailman/mm/venv/lib/python3.6/site-packages/hype= rkitty/static/hyperkitty/libs/bootstrap/stylesheets/bootstrap.scss from line 4 of /opt/mailman/mm/venv/lib/python3.6/site-packages/hyper= kitty/static/hyperkitty/sass/hyperkitty.scss Use --trace for backtrace. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/mailman/mm/venv/bin/django-admin", line 8, in sys.exit(execute_from_command_line()) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/django/core/manageme= nt/__init__.py", line 381, in execute_from_command_line utility.execute() File "/opt/mailman/mm/venv/lib/python3.6/site-packages/django/core/manageme= nt/__init__.py", line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/django/core/manageme= nt/base.py", line 323, in run_from_argv self.execute(*args, **cmd_options) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/django/core/manageme= nt/base.py", line 364, in execute output =3D self.handle(*args, **options) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/managemen= t/commands/compress.py", line 277, in handle self.handle_inner(**options) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/managemen= t/commands/compress.py", line 300, in handle_inner offline_manifest, block_count, results =3D self.compress(engine, extensio= ns, verbosity, follow_links, log) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/managemen= t/commands/compress.py", line 235, in compress "%s" % (template.template_name, smart_text(e))) django.core.management.base.CommandError: An error occurred during rendering = /opt/mailman/mm/venv/lib/python3.6/site-packages/django_mailman3/templates/so= cialaccount/connections.html: Error: Invalid CSS after "...lor}: #{$value}": = expected "{", was ";" on line 4 of /opt/mailman/mm/venv/lib/python3.6/site-packages/hyperki= tty/static/hyperkitty/libs/bootstrap/stylesheets/_root.scss from line 11 of /opt/mailman/mm/venv/lib/python3.6/site-packages/hype= rkitty/static/hyperkitty/libs/bootstrap/stylesheets/bootstrap.scss from line 4 of /opt/mailman/mm/venv/lib/python3.6/site-packages/hyper= kitty/static/hyperkitty/sass/hyperkitty.scss Use --trace for backtrace. Any suggestions? The fixes described in this thread don't seem to be working= for my case? --===============1859220691882541553==-- From brian_carpenter@emwd.com Fri Sep 11 14:31:44 2020 From: Brian Carpenter To: mailman-users@mailman3.org Subject: [MM3-users] Affinity and Empathy Are Now Available Date: Fri, 11 Sep 2020 10:31:40 -0400 Message-ID: <882a9009-a747-1ec0-9705-8fc09e4585f0@emwd.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6780983702228837927==" --===============6780983702228837927== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Dear Mailman 3 Users, We have launched Affinity and Empathy and they now are part of our shared and cloud Mailman 3 hosting services. Affinity replaces Postorius and Empathy replaces Hyperkitty. You can read more at https://mailmanhost.com Please email me off-list if you have any questions. Brian Carpenter Harmony Lists --===============6780983702228837927==-- From brylon@oit.rutgers.edu Fri Sep 11 18:48:53 2020 From: Bryan Loniewski To: mailman-users@mailman3.org Subject: [MM3-users] Office365 posts shunted (authres.core.SyntaxError) Date: Fri, 11 Sep 2020 18:48:47 +0000 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5169979071003976649==" --===============5169979071003976649== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi again -- Besides my compress issue described earlier today here: https://lists.mailman3.org/archives/list/mailman-users(a)mailman3.org/message= /OAIZ5QBMMCH5SKQZLO6V333326BBC54S/ I've been dealing with another issue (and why I upgraded to the latest code) = when posting messages to my test list from our schools Office365 infrastructu= re (OWA). The post gets shunted; mailman.log has the following error: Sep 11 11:29:44 2020 (31396) ACCEPT: Sep 11 11:29:45 2020 (31401) Uncaught runner exception: Syntax error: Expecte= d "=3D" at: ; dmarc=3Dnone action=3Dnone header.from=3Do... Sep 11 11:29:45 2020 (31401) Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/core/runner.= py", line 173, in _one_iteration self._process_one_file(msg, msgdata) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/core/runner.= py", line 266, in _process_one_file keepqueued =3D self._dispose(mlist, msg, msgdata) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/runners/pipe= line.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/core/pipelin= es.py", line 50, in process handler.process(mlist, msg, msgdata) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/handlers/val= idate_authenticity.py", line 118, in process authenticate(msg, msgdata) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/utilities/re= try.py", line 44, in f_retry return f(*args, **kwargs) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/handlers/val= idate_authenticity.py", line 85, in authenticate prev =3D trusted_auth_res(msg) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/handlers/val= idate_authenticity.py", line 73, in trusted_auth_res authserv_id =3D AuthenticationResultsHeader.parse(header).authserv_id File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/__init__.py"= , line 206, in parse return authres.core.AuthenticationResultsHeader.parse(core_features(), st= ring) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/core.py", li= ne 442, in parse return self.parse_value(feature_context, string) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/core.py", li= ne 454, in parse_value header._parse() File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/core.py", li= ne 517, in _parse result =3D self._parse_resinfo() File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/core.py", li= ne 550, in _parse_resinfo method, version, result =3D self._parse_methodspec() File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/core.py", li= ne 573, in _parse_methodspec raise SyntaxError('Expected "=3D"', self._parse_text) authres.core.SyntaxError: Syntax error: Expected "=3D" at: ; dmarc=3Dnone act= ion=3Dnone header.from=3Do... Sep 11 11:29:45 2020 (31401) SHUNTING: 1599838185.4911082+9dde922f77a1d66e351= 1f2709ec4b436ab6a3ecc Debugging the issue further shows this header to be the problem: authentication-results: lists-test.rutgers.edu; dkim=3Dnone (message not sign= ed) header.d=3Dnone;lists-test.rutgers.edu; dmarc=3Dnone action=3Dnone header.from=3Doit.rutgers.edu; What is syntactically wrong with it and/or is it more than that? Note1: If I remove that header from a copy of the message and inject directly= into Mailman it will deliver as expected. Note2: Posting messages from my personal Gmail account or cmdline mail progra= m to the same test list will also deliver as expected. ________________________ Bryan Loniewski Rutgers University OIT:EAS:Messaging and Collaboration Services System Administrator --===============5169979071003976649==-- From mark@msapiro.net Fri Sep 11 20:17:10 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Mailman3 throws 500 error (unicode encode) when loading held messages in Postorius Date: Fri, 11 Sep 2020 13:17:03 -0700 Message-ID: In-Reply-To: =?utf-8?q?=3CSYCPR01MB51031058987C5B3559F535E4F22B0=40SYCPR01MB?= =?utf-8?q?5103=2Eausprd01=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7093586621239056299==" --===============7093586621239056299== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/6/20 3:39 PM, Simon Handfield via Mailman-users wrote: > Thanks for your advice, Stephen. >=20 > As I've removed the file from the shunt queue I don't believe that file is = causing the 500 error any more. >=20 > I've confirmed the error is not cached in the browser. >=20 > If the issue does not reside in the shunt queue (no other files indicate pa= rse errors when checking via command line) are there any other places I shoul= d look for data that is addressed by that page? As Steve said, messages in the shunt queue have nothing to do with the held messages view in postorius. Held messages are in Mailman's var/messages/ directory and referenced by entries in the message table in the database which in turn are referenced by entries in the pendedkeyvalue table. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============7093586621239056299==-- From mark@msapiro.net Fri Sep 11 21:18:49 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Re-send subscription response and file permission error in queue Date: Fri, 11 Sep 2020 14:18:37 -0700 Message-ID: <674b26f0-0513-405b-5fa4-05510497d234@msapiro.net> In-Reply-To: <159963096883.18624.1418458365591904001@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8692401216140116466==" --===============8692401216140116466== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/8/20 10:56 PM, accounts-mailman(a)holbrook.no wrote: > A user sent subscription request to the `..-join@` address of a list. Email= domain wasn't set up properly, so response got rejected. Now, if the user tr= ies to send another request, no reponse comes back. In the log I find a pytho= n stack trace saying subscription already pending. >=20 > ``` > Sep 08 10:32:44 2020 (19918) Traceback (most recent call last): > File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 171, i= n _one_iteration > self._process_one_file(msg, msgdata) > File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 264, i= n _process_one_file > keepqueued =3D self._dispose(mlist, msg, msgdata) > File "/usr/lib/python3/dist-packages/mailman/runners/command.py", line 19= 6, in _dispose > mlist, msg, msgdata, parts, results) > File "/usr/lib/python3/dist-packages/mailman/commands/eml_membership.py",= line 104, in process > ISubscriptionManager(mlist).register(subscriber) > File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line = 520, in register > list(workflow) > File "/usr/lib/python3/dist-packages/mailman/app/workflow.py", line 69, i= n __next__ > return step() > File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line = 227, in _step_sanity_checks > raise SubscriptionPendingError(self.mlist, self.address.email) > mailman.interfaces.subscriptions.SubscriptionPendingError > ``` I'll look at this when I get some time. I'm still catching up after being off line for 12 days. > Furthermore I am getting a permission denied error in the attempt of shunti= ng the message, which I don't quite understand because the folder has user `l= ist` group `list` and `700` mode: >=20 > ``` > Sep 08 10:25:29 2020 (19918) SHUNTING: 1599553529.9889646+88be5afa0bf3111a8= 3e68cc97ce2000723ad93fd > Sep 08 10:29:49 2020 (19918) Uncaught runner exception: [Errno 13] Permissi= on denied: '/var/lib/mailman3/queue/command/1599553789.306307+6fee74af6926355= 3b5ffd0e73cc095ab651bd855.pck' > Sep 08 10:29:49 2020 (19918) Traceback (most recent call last): > File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 156, i= n _one_iteration > msg, msgdata =3D self.switchboard.dequeue(filebase) > File "/usr/lib/python3/dist-packages/mailman/core/switchboard.py", line 1= 50, in dequeue > with open(filename, 'rb') as fp: > PermissionError: [Errno 13] Permission denied: '/var/lib/mailman3/queue/com= mand/1599553789.306307+6fee74af69263553b5ffd0e73cc095ab651bd855.pck' > ``` > > How can I remove the pending subscription, or re-send the response email? In Postorius, pull down Subscription Requests and select Pending Confirmation and delete the request. > And what could be the cause of the permission error? Mailman can't read the /var/lib/mailman3/queue/command/1599553789.306307+6fee74af69263553b5ffd0e73cc= 095ab651bd855.pck file. This is strange since it wrote it, but check the modes and user/group of all the directories in the path and the file itself. Could it be a SELinux or apparmor issue? --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============8692401216140116466==-- From maszap@tin.it Fri Sep 11 22:59:27 2020 From: Massimo =?utf-8?q?Zappal=C3=A0?= To: mailman-users@mailman3.org Subject: [MM3-users] Re: Failed first login Date: Fri, 11 Sep 2020 22:59:24 +0000 Message-ID: <159986516401.18632.8384340539602572987@mailman.iad1.psf.io> In-Reply-To: <1b90bb61-23ab-d57c-2721-e55120c4c3ea@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4887101167873067403==" --===============4887101167873067403== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Mark, I understood and solved the problem. Due to a configuration mistake on amavis= , I had not received the verification email and for this reason I could not a= uthenticate. Once amavis was fixed, the login problem solved itself. Unfortunately, the me= ssage "credentials are incorrect" led me astray. Surely, with a more precise = message I would have solved it sooner. Thanks always.Max --===============4887101167873067403==-- From mark@msapiro.net Sat Sep 12 00:25:55 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: New releases Date: Fri, 11 Sep 2020 17:25:43 -0700 Message-ID: In-Reply-To: <159983462342.18624.4566478373636672112@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3821700306333216277==" --===============3821700306333216277== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/11/20 7:30 AM, brylon(a)oit.rutgers.edu wrote: >=20 > $ ./bin/django-admin compress --trace > Compressing... Traceback (most recent call last): > File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/base.py= ", line 282, in precompile > mod =3D import_module(mod_name) > File "/opt/mailman/mm/venv/lib64/python3.6/importlib/__init__.py", line 1= 26, in import_module > return _bootstrap._gcd_import(name[level:], package, level) > File "", line 994, in _gcd_import > File "", line 971, in _find_and_load > File "", line 953, in _find_and_load_unlocked > ModuleNotFoundError: No module named 'sass -t compressed {infile} {outfile}' You have something like COMPRESS_PRECOMPILERS =3D ( ('text/x-scss', 'sass -t compressed {infile} {outfile}'), ('text/x-sass', 'sass -t compressed {infile} {outfile}'), ) in your Django settings. You either need to install ruby-sass or install sassc if it isn't already installed and for sassc change the above to COMPRESS_PRECOMPILERS =3D ( ('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'), ) --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============3821700306333216277==-- From brylon@oit.rutgers.edu Sat Sep 12 03:19:18 2020 From: brylon@oit.rutgers.edu To: mailman-users@mailman3.org Subject: [MM3-users] Re: New releases Date: Sat, 12 Sep 2020 03:19:14 +0000 Message-ID: <159988075486.18624.15614894645969025761@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8528267278858880010==" --===============8528267278858880010== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable I already have ruby-sass installed and the compress command used to work as e= xpected prior to my upgrading to the latest core, postorius, hyperkitty versi= ons using pip in my venv. =20 Does It matter at all when performing the Django-admin commands whether I've = activated the venv or not? I think I've tried running them both activated an= d deactivated to no avail, but any other suggestions? --===============8528267278858880010==-- From mark@msapiro.net Sat Sep 12 05:27:35 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: New releases Date: Fri, 11 Sep 2020 22:27:25 -0700 Message-ID: In-Reply-To: <159988075486.18624.15614894645969025761@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3856810976544144929==" --===============3856810976544144929== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/11/20 8:19 PM, brylon(a)oit.rutgers.edu wrote: > I already have ruby-sass installed and the compress command used to work as= expected prior to my upgrading to the latest core, postorius, hyperkitty ver= sions using pip in my venv. =20 >=20 > Does It matter at all when performing the Django-admin commands whether I'v= e activated the venv or not? I think I've tried running them both activated = and deactivated to no avail, but any other suggestions? I don't think that's the issue here, but the venv should be active when you run the Django-admin commands. Looking more closely at the error, I see it says ``` File "/opt/mailman/mm/venv/lib/python3.6/site-packages/compressor/base.py", line 282, in precompile mod =3D import_module(mod_name) ... ModuleNotFoundError: No module named 'sass -t compressed {infile} {outfile}' ``` Thus it appears that compressor is trying to import 'sass -t compressed {infile} {outfile}' as a module instead of executing it as a command. What version of compressor is this? It is working for me on 4 sites with django_compressor-2.4, django_compressor-2.3 and django_compressor-2.2. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============3856810976544144929==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Sat Sep 12 12:23:21 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] Office365 posts shunted (authres.core.SyntaxError) Date: Sat, 12 Sep 2020 21:23:13 +0900 Message-ID: <24412.48561.381674.708495@turnbull.sk.tsukuba.ac.jp> In-Reply-To: =?utf-8?q?=3CBL0PR14MB368179D4F85B9DF47617EF0388240=40BL0PR14MB?= =?utf-8?q?3681=2Enamprd14=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4900356367523366637==" --===============4900356367523366637== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Bryan Loniewski writes: > I've been dealing with another issue (and why I upgraded to the > latest code) when posting messages to my test list from our schools > Office365 infrastructure (OWA). The post gets shunted; mailman.log > has the following error: Bottom line: the reported Authentication-Results (A-R) field has nonconforming syntax. The problem is that the RFC defines a syntax where the authentication service ID is the first thing in the field, and only one is allowed. In the quoted field it occurs twice. There are more syntax errors, but that's the one Mailman flagged. As far as I know the only part of Mailman that uses A-R is the ARC module. Just disable it and be happy, I think. Like DKIM, DMARC, and friends, this is really an MTA function. If possible you should get your MTA to do the work. BTW, I'm not sure this is an Office365 problem. It appears that the A-R field was constructed on lists-test.rutgers.edu, which doesn't look like Office365. Is it possible that the Office365 system didn't provide one, and this A-R field was added by Mailman itself? (Don't overthink this: if so, it's our problem and I'll be looking at it. But if you know offhand it will point me in the right direction.) Excruciating detail follows, in part as notes to me. ;-) In the following, the in-reply-to quoting is omitted and the field is folded for clarity. authentication-results: # field name lists-test.rutgers.edu; # authenticator ID dkim=none (message not signed) # DKIM result info header.d=none; # DKIM property (no DKIM signature field) lists-test.rutgers.edu; # #### nonconforming (not "; key=value") dmarc=none # DMARC result info action=none # #### syntax OK, undefined property header.from=oit.rutgers.edu # DMARC property (field to check) ; # #### nonconforming (nothing to separate) So it looks like whoever wrote the code doesn't understand RFC 7601 at all. A-R starts with a string naming the authentication authority (here lists-test.rutgers.edu) and an optional number naming the version of A-R being used. These are followed either by a semicolon and the word "none", or by a sequence of results blocks. Each block consists of a semicolon followed by a sequence of key=value pairs. This funny phrase "a semicolon followed by a sequence of key=value pairs" is probably easier to understand if I say "the semicolons separate results blocks from the authenticator ID and from each other" (they are not block terminators). RFC 7601 is quite pessimistic about nonconforming A-R, and repeatedly advises they be ignored or even deleted, although completely broken syntax is not mentioned. Steve --===============4900356367523366637==-- From psd.phi@gmail.com Sat Sep 12 16:51:06 2020 From: Marcel Pietschmann To: mailman-users@mailman3.org Subject: [MM3-users] How to set the original sender to the Reply-To? Date: Sat, 12 Sep 2020 16:51:02 +0000 Message-ID: <159992946291.18632.10363321579386515421@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4528081102311040049==" --===============4528081102311040049== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi everybody, I have a settings problem. Mailman adds the original sender to the Reply-To o= r Cc in the case where there's a reply to setting in force on the list. In my case, mailman sets the sender always to Cc. How can i change it? Greets Marcel My settings: Personalize =3D None Anonymous list =3D No Reply to list =3D No Munging DMARC =3D Munge the From: header with list adress --===============4528081102311040049==-- From mark@msapiro.net Sat Sep 12 16:56:09 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Sat, 12 Sep 2020 09:56:00 -0700 Message-ID: <60e52d55-6141-109d-416d-6d27275dc519@msapiro.net> In-Reply-To: <159920471343.18624.9829816949185245855@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4775283508410452563==" --===============4775283508410452563== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/4/20 12:31 AM, r.woithe(a)callassoftware.com wrote: > Hello, > every night at 00:00 i get an email by mailman3: >=20 > Traceback (most recent call last): > File "/usr/lib/python2.7/dist-packages/django_extensions/management/comma= nds/runjobs.py", line 34, in runjobs > job().execute() > File "/usr/lib/python2.7/dist-packages/hyperkitty/jobs/sync_mailman.py", = line 37, in execute > sync_with_mailman() > File "/usr/lib/python2.7/dist-packages/hyperkitty/lib/mailman.py", line 1= 47, in sync_with_mailman > sender.set_mailman_id() > File "/usr/lib/python2.7/dist-packages/hyperkitty/models/sender.py", line= 57, in set_mailman_id > mm_user =3D client.get_user(self.address) > File "/usr/lib/python2.7/dist-packages/mailmanclient/client.py", line 192= , in get_user > 'users/{0}'.format(address)) > File "/usr/lib/python2.7/dist-packages/mailmanclient/restbase/connection.= py", line 99, in call > response, content =3D Http().request(url, method, data_str, headers) > File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1616, = in request > (response, content) =3D self._request(conn, authority, uri, request_uri= , method, body, headers, redirections, cachekey) > File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1358, = in _request > (response, content) =3D self._conn_request(conn, request_uri, method, b= ody, headers) > File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1314, = in _conn_request > response =3D conn.getresponse() > File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse > raise ResponseNotReady() > ResponseNotReady > ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty) > START TRACEBACK: > END TRACEBACK I'm partly guessing, but this may be a REST timeout. What Mailman core version is this? the version affects what the REST server is. > Therefor i would like to know can i neglect this message ? Or is this messa= ge a hint for an Error i should investigate and looking for a solution ? Archived messages have a `mailman_id` which is Mailman core's `user_id` for the user that posted the message. The job itself runs through all the archived messages and finds those which have a null `mailman_id` and attempts to get the `user-id` from core and set the `mailman_id`. Normally, I think messages with null `mailman_id` would only be messages added with hyperkitty_import as opposed to normally archived posts. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============4775283508410452563==-- From mark@msapiro.net Sat Sep 12 18:09:21 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Re-send subscription response and file permission error in queue Date: Sat, 12 Sep 2020 11:09:07 -0700 Message-ID: <1b85b28e-e93f-6bc6-2f26-6d6af762eeee@msapiro.net> In-Reply-To: <674b26f0-0513-405b-5fa4-05510497d234@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7387948091919172270==" --===============7387948091919172270== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/11/20 2:18 PM, Mark Sapiro wrote: > On 9/8/20 10:56 PM, accounts-mailman(a)holbrook.no wrote: >> A user sent subscription request to the `..-join@` address of a list. Emai= l domain wasn't set up properly, so response got rejected. Now, if the user t= ries to send another request, no reponse comes back. In the log I find a pyth= on stack trace saying subscription already pending. >> >> ``` >> Sep 08 10:32:44 2020 (19918) Traceback (most recent call last): >> File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 171, = in _one_iteration >> self._process_one_file(msg, msgdata) >> File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 264, = in _process_one_file >> keepqueued =3D self._dispose(mlist, msg, msgdata) >> File "/usr/lib/python3/dist-packages/mailman/runners/command.py", line 1= 96, in _dispose >> mlist, msg, msgdata, parts, results) >> File "/usr/lib/python3/dist-packages/mailman/commands/eml_membership.py"= , line 104, in process >> ISubscriptionManager(mlist).register(subscriber) >> File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line= 520, in register >> list(workflow) >> File "/usr/lib/python3/dist-packages/mailman/app/workflow.py", line 69, = in __next__ >> return step() >> File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line= 227, in _step_sanity_checks >> raise SubscriptionPendingError(self.mlist, self.address.email) >> mailman.interfaces.subscriptions.SubscriptionPendingError >> ``` >=20 >=20 > I'll look at this when I get some time. I'm still catching up after > being off line for 12 days. This exception should also send a response to the user's email saying "User > How can I remove the pending subscription, or re-send the response email? >=20 > In Postorius, pull down Subscription Requests and select Pending > Confirmation and delete the request. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============7387948091919172270==-- From mark@msapiro.net Sat Sep 12 18:48:28 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: How to set the original sender to the Reply-To? Date: Sat, 12 Sep 2020 11:48:17 -0700 Message-ID: <320de937-1bd1-fa94-27c0-0973e9730227@msapiro.net> In-Reply-To: <159992946291.18632.10363321579386515421@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4459211238646480626==" --===============4459211238646480626== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/12/20 9:51 AM, Marcel Pietschmann wrote: > Hi everybody, >=20 > I have a settings problem. Mailman adds the original sender to the Reply-To= or Cc in the case where there's a reply to setting in force on the list. >=20 > In my case, mailman sets the sender always to Cc. How can i change it? >=20 > My settings: > Personalize =3D None > Anonymous list =3D No > Reply to list =3D No Munging > DMARC =3D Munge the From: header with list adress When DMARC is set to Munge From, Mailman puts the poster's address in either Cc: or Reply-To: to facilitate replying to the poster. The decision of which header to add the poster's address to is made to, as closely as possible for most MUAs, make the behavior of a 'reply' match the behavior when the From: is not munged. See the comments at for discussion of the goals for Munge From. The actual decision as to which header to use is at . That says that the original From is added to Cc: whenever Reply to list is other than No Munging. You say that you have Reply to list =3D No Munging which means the original From should be added to Reply-To: and not to Cc:, so I don't understand why it's added to Cc: in your case. I have tested locally and with Reply to list =3D No Munging, the original From: is added to Reply-To: and with Reply to list =3D Reply goes to list, the original From: is added to Cc: as expected. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============4459211238646480626==-- From mark@msapiro.net Sat Sep 12 18:53:41 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Sat, 12 Sep 2020 11:53:32 -0700 Message-ID: <8bc899e0-cb79-78da-3ae9-916e8bcf482d@msapiro.net> In-Reply-To: <159972197989.18632.12329373976731132373@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0630203892378755617==" --===============0630203892378755617== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/10/20 12:12 AM, r.woithe(a)callassoftware.com wrote: > UPDATE > mailman3 woks now under python3 but i still get the nightly email, now with= an other file but with the same result: >=20 > File "/usr/lib/python3.8/http/client.py", line 1322, in getresponse > raise ResponseNotReady(self.__state) > http.client.ResponseNotReady: Request-started > ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty) >=20 > anybody can help ? See my reply at > additionally should i set COMPRESS_OFFLINE to false again? No. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============0630203892378755617==-- From brylon@oit.rutgers.edu Sat Sep 12 19:26:14 2020 From: brylon@oit.rutgers.edu To: mailman-users@mailman3.org Subject: [MM3-users] Re: New releases Date: Sat, 12 Sep 2020 19:26:12 +0000 Message-ID: <159993877205.18625.13427816843908228329@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4602866064444901450==" --===============4602866064444901450== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit $ pip show django-compressor Name: django-compressor Version: 2.4 --===============4602866064444901450==-- From mark@msapiro.net Sat Sep 12 21:39:27 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: New releases Date: Sat, 12 Sep 2020 14:39:16 -0700 Message-ID: <49905ac8-c620-0465-4819-22c3b74f8118@msapiro.net> In-Reply-To: <159993877205.18625.13427816843908228329@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5050734886257921707==" --===============5050734886257921707== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/12/20 12:26 PM, brylon(a)oit.rutgers.edu wrote: > $ pip show django-compressor > Name: django-compressor > Version: 2.4 > It works for me with django-compressor 2.4. What exactly do you have for COMPRESS_PRECOMPILERS IN YOUR Django settings? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5050734886257921707==-- From brylon@oit.rutgers.edu Sat Sep 12 22:00:26 2020 From: brylon@oit.rutgers.edu To: mailman-users@mailman3.org Subject: [MM3-users] Re: New releases Date: Sat, 12 Sep 2020 22:00:22 +0000 Message-ID: <159994802240.18625.6122438268685069253@mailman.iad1.psf.io> In-Reply-To: <49905ac8-c620-0465-4819-22c3b74f8118@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4122688409914093898==" --===============4122688409914093898== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit $ grep -i compress settings_local.py 'compressor', COMPRESS_PRECOMPILERS = ( ('text/x-scss', 'sass -t compressed {infile} {outfile}'), ('text/x-sass', 'sass -t compressed {infile} {outfile}'), COMPRESS_OFFLINE = True COMPRESS_ENABLED = True $ rpm -q rubygem-sass rubygem-sass-3.4.19-2.el7.noarch --===============4122688409914093898==-- From mark@msapiro.net Sat Sep 12 23:15:52 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: New releases Date: Sat, 12 Sep 2020 16:15:44 -0700 Message-ID: <94b3d7be-45f1-517d-9665-22fcd73d8cbb@msapiro.net> In-Reply-To: <159994802240.18625.6122438268685069253@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4638905872578700066==" --===============4638905872578700066== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/12/20 3:00 PM, brylon(a)oit.rutgers.edu wrote: > $ grep -i compress settings_local.py=20 > 'compressor', > COMPRESS_PRECOMPILERS =3D ( > ('text/x-scss', 'sass -t compressed {infile} {outfile}'), > ('text/x-sass', 'sass -t compressed {infile} {outfile}'), > COMPRESS_OFFLINE =3D True > COMPRESS_ENABLED =3D True See . For some reason, django_compress is interpreting your 'sass -t compressed {infile} {outfile}' and 'sass -t compressed {infile} {outfile}' strings as filters rather than commands. I don't know why this would be. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============4638905872578700066==-- From simon.handfield@sl.nsw.gov.au Mon Sep 14 03:08:46 2020 From: Simon Handfield To: mailman-users@mailman3.org Subject: [MM3-users] Re: Mailman3 throws 500 error (unicode encode) when loading held messages in Postorius Date: Mon, 14 Sep 2020 03:08:37 +0000 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5045811982895304087==" --===============5045811982895304087== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Mark, thanks for correcting my assumption. Based on your information I have been able to identify the suspect message fi= le in /var/lib/mailman3/messages, replace the suspect file with another file,= update the file permissions and successfully load the held messages page. Th= is allowed me to discard the suspect message and other held messages are then= available for review again. In case the steps below help someone else who is having this issue here are t= he steps I followed with my initial search string redacted: Searched all files in messages folder for matching string: cd /var/lib/mailman3/messages grep -ir [REDACTED] * Finds string in one file: Binary file HE/D7/HED7SJLGJXTJ35AESY32AZXYPXDEVUVV matches Check whether file can be parsed using mailman command: mailman qfile ./HE/D7/HED7SJLGJXTJ35AESY32AZXYPXDEVUVV Throws a matching error: [----- start pickle -----] <----- start object 1 -----> Traceback (most recent call last): File "/usr/bin/mailman", line 11, in load_entry_point('mailman=3D=3D3.1.1', 'console_scripts', 'mailman')() File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 97, in m= ain args.func(args) File "/usr/lib/python3/dist-packages/mailman/commands/cli_qfile.py", line 8= 1, in process printer.pprint(obj) File "/usr/lib/python3.6/pprint.py", line 139, in pprint self._format(object, self._stream, 0, 0, {}, 0) File "/usr/lib/python3.6/pprint.py", line 161, in _format rep =3D self._repr(object, context, level) File "/usr/lib/python3.6/pprint.py", line 393, in _repr self._depth, level) File "/usr/lib/python3.6/pprint.py", line 405, in format return _safe_repr(object, context, maxlevels, level) File "/usr/lib/python3.6/pprint.py", line 555, in _safe_repr rep =3D repr(object) File "/usr/lib/python3/dist-packages/mailman/email/message.py", line 45, in= __repr__ return self.__str__() File "/usr/lib/python3.6/email/message.py", line 135, in __str__ return self.as_string() File "/usr/lib/python3.6/email/message.py", line 158, in as_string g.flatten(self, unixfrom=3Dunixfrom) File "/usr/lib/python3.6/email/generator.py", line 116, in flatten self._write(msg) File "/usr/lib/python3.6/email/generator.py", line 181, in _write self._dispatch(msg) File "/usr/lib/python3.6/email/generator.py", line 214, in _dispatch meth(msg) File "/usr/lib/python3.6/email/generator.py", line 272, in _handle_multipart g.flatten(part, unixfrom=3DFalse, linesep=3Dself._NL) File "/usr/lib/python3.6/email/generator.py", line 116, in flatten self._write(msg) File "/usr/lib/python3.6/email/generator.py", line 181, in _write self._dispatch(msg) File "/usr/lib/python3.6/email/generator.py", line 214, in _dispatch meth(msg) File "/usr/lib/python3.6/email/generator.py", line 243, in _handle_text msg.set_payload(payload, charset) File "/usr/lib/python3.6/email/message.py", line 315, in set_payload payload =3D payload.encode(charset.output_charset) UnicodeEncodeError: 'ascii' codec can't encode characters in position 8-9: or= dinal not in range(128) Move suspicious file out of messages: mv ./HE/D7/HED7SJLGJXTJ35AESY32AZXYPXDEVUVV /root/20200914/ held_messages page still throws 500 error. Checking debug log, it is now because the database references a file that is = no longer there. Can I just copy a parseable file to the path of the bad file so I can delete = it or do I need to remove the reference from the database? Copy some other file: cp ZZ/B2/ZZB24ADTZ2BOKBWTQWI7KG4XACGR6DTJ HE/D7/HED7SJLGJXTJ35AESY32AZXYPXDEV= UVV Still throws an error but this time it's a permissions error. mailman can't a= ccess the copied file: ls -l HE/D7/ -rw-r----- 1 root root 78472 Sep 14 12:48 HED7SJLGJXTJ35AESY32AZXYPXDEVUVV chown list:list HE/D7/HED7SJLGJXTJ35AESY32AZXYPXDEVUVV chmod g+w HE/D7/HED7SJLGJXTJ35AESY32AZXYPXDEVUVV ls -l HE/D7/ -rw-rw---- 1 list list 78472 Sep 14 12:48 HED7SJLGJXTJ35AESY32AZXYPXDEVUVV held_messages now loads OK. Cheers, Simon. ________________________________ From: Mark Sapiro Sent: Saturday, 12 September 2020 06:17 To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: Mailman3 throws 500 error (unicode encode) when load= ing held messages in Postorius On 9/6/20 3:39 PM, Simon Handfield via Mailman-users wrote: > Thanks for your advice, Stephen. > > As I've removed the file from the shunt queue I don't believe that file is = causing the 500 error any more. > > I've confirmed the error is not cached in the browser. > > If the issue does not reside in the shunt queue (no other files indicate pa= rse errors when checking via command line) are there any other places I shoul= d look for data that is addressed by that page? As Steve said, messages in the shunt queue have nothing to do with the held messages view in postorius. Held messages are in Mailman's var/messages/ directory and referenced by entries in the message table in the database which in turn are referenced by entries in the pendedkeyvalue table. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://aus01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists.mai= lman3.org%2Fmailman3%2Flists%2Fmailman-users.mailman3.org%2F&data=3D02%7C= 01%7Csimon.handfield%40sl.nsw.gov.au%7Ce9589b5b18274716c5a208d8568fb17d%7C08d= 88060ca46418e97231cab37bc0749%7C1%7C0%7C637354522456950477&sdata=3Dj8nmYE= T95MmTzfeNqDBwjGIKZrpAT6VprD0I17xDIcI%3D&reserved=3D0 _ [State Library of NSW] Simon Handfield Desktop and Infrastructure Leader simon.handfield(a)sl.nsw.gov.au +61 2 9273 1535 Macquarie Street, Sydney NSW 2000, Australia www.sl.nsw.gov.au This message is intended for the addressee named and may contain confidential= information. If you are not the intended recipient, please delete it and not= ify the sender. Views expressed in this message are those of the individual s= ender, and are not necessarily the views of their organisation. --===============5045811982895304087==-- From mmasoudf@aaas.org Mon Sep 14 14:55:03 2020 From: Mohsen Masoudfar To: mailman-users@mailman3.org Subject: [MM3-users] Can creating too many Django accounts crash Mailman3 server? Date: Mon, 14 Sep 2020 14:54:54 +0000 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1689611503527922814==" --===============1689611503527922814== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, It seems anybody can go to the site: https://LISTSERVNAME/mailman3/postorius/lists/ and click on the [Sign up] on the top-right corner and create an account. Thi= s can easily be automated, even though the next step, confirming the email ad= dress, being ignored. I believe it can be used as a target for a DOS attack b= y creating so many accounts, that eventually causes an 'out of space' error. Is this a justified concern? Is there a way to manage this feature in a secure way? Best regards Mohsen Masoudfar Lead System Analyst, IT Operation American Association for the Advancement of Science 1200 New York Ave, NW, Washington, DC 20005 202-326-7087 --===============1689611503527922814==-- From brian_carpenter@emwd.com Mon Sep 14 15:03:28 2020 From: Brian Carpenter To: mailman-users@mailman3.org Subject: [MM3-users] Re: Can creating too many Django accounts crash Mailman3 server? Date: Mon, 14 Sep 2020 11:03:23 -0400 Message-ID: In-Reply-To: =?utf-8?q?=3CMN2PR20MB3245C1C40CC4244B18B1F1ACB6230=40MN2PR20MB?= =?utf-8?q?3245=2Enamprd20=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3151019716485892246==" --===============3151019716485892246== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/14/20 10:54 AM, Mohsen Masoudfar wrote: > It seems anybody can go to the site: > https://LISTSERVNAME/mailman3/postorius/lists/ > and click on the [Sign up] on the top-right corner and create an account. T= his can easily be automated, even though the next step, confirming the email = address, being ignored. I believe it can be used as a target for a DOS attack= by creating so many accounts, that eventually causes an 'out of space' error. > > Is this a justified concern? > Is there a way to manage this feature in a secure way? We no longer use Django to manage our mailman 3 user registration, but=20 the same concerns that you have are still valid for us. What we do is=20 run a cronjob that purges all unverified accounts once per day. So=20 problem solved for us. Perhaps something like that can be done for Django. --=20 Please let me know if you need further assistance. Thank you for your business. We appreciate our clients. Brian Carpenter EMWD.com -- EMWD's Knowledgebase: https://clientarea.emwd.com/index.php/knowledgebase EMWD's Community Forums http://discourse.emwd.com/ --===============3151019716485892246==-- From brylon@oit.rutgers.edu Mon Sep 14 20:10:46 2020 From: brylon@oit.rutgers.edu To: mailman-users@mailman3.org Subject: [MM3-users] Re: Office365 posts shunted (authres.core.SyntaxError) Date: Mon, 14 Sep 2020 20:10:43 +0000 Message-ID: <160011424342.16962.13469732859298089399@mailman.iad1.psf.io> In-Reply-To: <24412.48561.381674.708495@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5892921131280796135==" --===============5892921131280796135== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Steve -- Thanks for the response. You are correct in the fact that I've enabled/confi= gured the ARC module, however, I don't think Mailman is adding that (malforme= d) header considering I'm now HOLDing the test email(s) sent to our postfix i= nbound MX servers *before* they are LMTP'd to the Mailman server and its alre= ady there when I postcat it. At the same time I also can't fathom that Office365 has this wrong (still?) a= nd no one has noticed/complained? --===============5892921131280796135==-- From mm3@hovo.id.au Mon Sep 14 22:21:48 2020 From: mm3@hovo.id.au To: mailman-users@mailman3.org Subject: [MM3-users] Held Message view, JSONDecodeError: Invalid control character Date: Mon, 14 Sep 2020 12:22:30 +0000 Message-ID: <160008615059.16962.148177562734174464@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3905662477072741359==" --===============3905662477072741359== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable I am running the Debian 10 mailman3 and mailman3-web packages with mariadb. I received an email to the list from an invoicing software and now I am unabl= e to view or accept this email from the list. All other lists seem to be still working fine. I'm guessing there is a piece of data in my database that is corrupt or unabl= e to be decoded, but I am at a loss on how to find and fix it. Any help would be greatly appreciated ``` Sep 11 12:29:23 2020 (11577) 127.0.0.1 - - "GET /3.1/lists/redacted(a)redacte= d.com/held?count=3D0&page=3D1 HTTP/1.1" 200 90 Sep 11 12:29:23 2020 (11577) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3.7/wsgiref/handlers.py", line 137, in run self.result =3D application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line= 50, in wrapper rtn =3D function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 216, in= __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 215, in falcon.api.API.__call__ File "falcon/api.py", line 189, in falcon.api.API.__call__ File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line= 160, in on_get resource =3D self._make_collection(request) File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in= _make_collection for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in= for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line= 150, in _resource_as_dict resource =3D self._make_resource(request.id) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line= 70, in _make_resource resource =3D super()._make_resource(request_id) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line= 39, in _make_resource results =3D requests.get_request(request_id) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line= 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/requests.py", line 120, = in get_request result.data_hash, expunge=3DFalse) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line= 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/pending.py", line 142, i= n confirm value =3D json.loads(keyvalue.value) File "/usr/lib/python3.7/json/__init__.py", line 348, in loads return _default_decoder.decode(s) File "/usr/lib/python3.7/json/decoder.py", line 337, in decode obj, end =3D self.raw_decode(s, idx=3D_w(s, 0).end()) File "/usr/lib/python3.7/json/decoder.py", line 353, in raw_decode obj, end =3D self.scan_once(s, idx) json.decoder.JSONDecodeError: Invalid control character at: line 1 column 21 = (char 20) ``` --===============3905662477072741359==-- From mark@msapiro.net Tue Sep 15 03:20:45 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Held Message view, JSONDecodeError: Invalid control character Date: Mon, 14 Sep 2020 20:20:33 -0700 Message-ID: <991b9211-b77e-d2f4-600a-bee8abb58bad@msapiro.net> In-Reply-To: <160008615059.16962.148177562734174464@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0860063490587888228==" --===============0860063490587888228== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/14/20 5:22 AM, mm3(a)hovo.id.au wrote: > I am running the Debian 10 mailman3 and mailman3-web packages with mariadb. > I received an email to the list from an invoicing software and now I am una= ble to view or accept this email from the list. > All other lists seem to be still working fine. > I'm guessing there is a piece of data in my database that is corrupt or una= ble to be decoded, but I am at a loss on how to find and fix it. > Any help would be greatly appreciated >=20 > ``` > Sep 11 12:29:23 2020 (11577) 127.0.0.1 - - "GET /3.1/lists/redacted(a)redac= ted.com/held?count=3D0&page=3D1 HTTP/1.1" 200 90 > Sep 11 12:29:23 2020 (11577) REST request handler error: > Traceback (most recent call last): > File "/usr/lib/python3.7/wsgiref/handlers.py", line 137, in run > self.result =3D application(self.environ, self.start_response) > File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", li= ne 50, in wrapper > rtn =3D function(*args, **kws) > File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 216, = in __call__ > return super().__call__(environ, start_response) > File "falcon/api.py", line 215, in falcon.api.API.__call__ > File "falcon/api.py", line 189, in falcon.api.API.__call__ > File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", li= ne 160, in on_get > resource =3D self._make_collection(request) > File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, = in _make_collection > for resource in collection] > File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, = in > for resource in collection] > File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", li= ne 150, in _resource_as_dict > resource =3D self._make_resource(request.id) > File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", li= ne 70, in _make_resource > resource =3D super()._make_resource(request_id) > File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", li= ne 39, in _make_resource > results =3D requests.get_request(request_id) > File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", li= ne 85, in wrapper > return function(args[0], config.db.store, *args[1:], **kws) > File "/usr/lib/python3/dist-packages/mailman/model/requests.py", line 120= , in get_request > result.data_hash, expunge=3DFalse) > File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", li= ne 85, in wrapper > return function(args[0], config.db.store, *args[1:], **kws) > File "/usr/lib/python3/dist-packages/mailman/model/pending.py", line 142,= in confirm > value =3D json.loads(keyvalue.value) There is some value with a key not 'type' in the pendedkeyvalue table which is the issue here. > File "/usr/lib/python3.7/json/__init__.py", line 348, in loads > return _default_decoder.decode(s) > File "/usr/lib/python3.7/json/decoder.py", line 337, in decode > obj, end =3D self.raw_decode(s, idx=3D_w(s, 0).end()) > File "/usr/lib/python3.7/json/decoder.py", line 353, in raw_decode > obj, end =3D self.scan_once(s, idx) > json.decoder.JSONDecodeError: Invalid control character at: line 1 column 2= 1 (char 20) > ``` Examine the table looking at the 21st character of values with keys not =3D 'type', most likely keys 'email' or 'message_id'. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============0860063490587888228==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Tue Sep 15 07:08:45 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] Re: Office365 posts shunted (authres.core.SyntaxError) Date: Tue, 15 Sep 2020 16:08:37 +0900 Message-ID: <24416.26741.778037.428824@turnbull.sk.tsukuba.ac.jp> In-Reply-To: <160011424342.16962.13469732859298089399@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2330556931739405634==" --===============2330556931739405634== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit brylon(a)oit.rutgers.edu writes: > Thanks for the response. You are correct in the fact that I've > enabled/configured the ARC module, however, I don't think Mailman > is adding that (malformed) header considering I'm now HOLDing the > test email(s) sent to our postfix inbound MX servers *before* they > are LMTP'd to the Mailman server and its already there when I > postcat it. OK, thanks for that clarification. I feel better (but will still check it! just a bit less urgently ;-) What do you still need from us? I guess the easiest thing for us would be to catch the Exception in our ARC module, and ignore nonconforming Authentication-Results, so that there will be no ARC signature from Mailman on those posts. Will that do? (Note that "ignore and/or delete" is the RFC recommendation. I'd need more than "it would be nice" as a reason to try to parse non-conforming A-R. Mark's experience with parsing *conforming* (non)delivery notifications is quite a deterrent to that.) > At the same time I also can't fathom that Office365 has this wrong > (still?) and no one has noticed/complained? I guess I would say, how would we know if there were complaints? ARC is a very recent addition to Mailman, so you wouldn't necessarily see it here yet, and Office365 doesn't have a public bug database AFAIK. ;-) Microsoft (and other > 1G$ corporations) have a long history of "embrace, extend, and drive conformant implementations out of business". --===============2330556931739405634==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Tue Sep 15 07:10:00 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] Can creating too many Django accounts crash Mailman3 server? Date: Tue, 15 Sep 2020 16:09:53 +0900 Message-ID: <24416.26817.359634.565707@turnbull.sk.tsukuba.ac.jp> In-Reply-To: =?utf-8?q?=3CMN2PR20MB3245C1C40CC4244B18B1F1ACB6230=40MN2PR20MB?= =?utf-8?q?3245=2Enamprd20=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2286671836588256023==" --===============2286671836588256023== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Mohsen Masoudfar writes: > It seems anybody can go to the site: > https://LISTSERVNAME/mailman3/postorius/lists/ > and click on the [Sign up] on the top-right corner and create an > account. Yes. > This can easily be automated, even though the next step, confirming > the email address, being ignored. I believe it can be used as a > target for a DOS attack by creating so many accounts, that > eventually causes an 'out of space' error. > > Is this a justified concern? "Justified" depends on who you think the attacker is. Theoretically it's a real possibility, but the space used by an unverified account is not large. I suspect there are always much more painful DoS attacks to worry about. > Is there a way to manage this feature in a secure way? In some sense, I don't think there is. The point of the feature is to allow anybody to sign up without authentication beyond having a usable email address. Either you allow the DoS attack or anything you do is likely to deny service to legitimate users. Brian's approach (cron job purging unverified users) is generally good hygiene, but I don't think it would do much against an deliberate attack. And it shouldn't be hard to automate the confirmation process. I guess you could handle that by purging accounts with no subscriptions or other roles on any lists, but that might annoy a few legitimate users. Probably the best you could do for an open server would be to throttle user creation to say 1 per minute (adjustable to the scale of your site). I don't know if there is currently a way to require admin approval for account creation (as opposed to list subscription). Perhaps we should add that if we don't have it. Steve --===============2286671836588256023==-- From accounts-mailman@holbrook.no Tue Sep 15 12:37:16 2020 From: accounts-mailman@holbrook.no To: mailman-users@mailman3.org Subject: [MM3-users] Re: Re-send subscription response and file permission error in queue Date: Tue, 15 Sep 2020 12:37:12 +0000 Message-ID: <160017343275.16962.2478475119033423684@mailman.iad1.psf.io> In-Reply-To: <674b26f0-0513-405b-5fa4-05510497d234@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4821367335746407251==" --===============4821367335746407251== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The cause probably was that I was using with nginx, and set up nginx with def= ault www user, conflicting with the user mailman-web was running. I got lost = in the pieces that had to fit together. By the way, I never got notification about this reply by email. --===============4821367335746407251==-- From mm3@hovo.id.au Tue Sep 15 12:40:36 2020 From: mm3@hovo.id.au To: mailman-users@mailman3.org Subject: [MM3-users] Re: Held Message view, JSONDecodeError: Invalid control character Date: Tue, 15 Sep 2020 12:40:32 +0000 Message-ID: <160017363298.16960.5259485393161795591@mailman.iad1.psf.io> In-Reply-To: <991b9211-b77e-d2f4-600a-bee8abb58bad@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3950461152261208861==" --===============3950461152261208861== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Thanks Mark for pointing me to the correct table to look at, The entry turned out to be spam and not the invoice that I was trying to acce= pt so I was able to remove the spam message inside this table and the refranc= e to it in _requests. --===============3950461152261208861==-- From mark@msapiro.net Tue Sep 15 18:42:53 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Re-send subscription response and file permission error in queue Date: Tue, 15 Sep 2020 11:42:41 -0700 Message-ID: <98fbf386-6122-e195-b566-b893bb262c72@msapiro.net> In-Reply-To: <160017343275.16962.2478475119033423684@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1344553136851021985==" --===============1344553136851021985== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/15/20 5:37 AM, accounts-mailman(a)holbrook.no wrote: > > By the way, I never got notification about this reply by email. You have email delivery disabled and you read and post from HyperKitty. This is fine, but there are no push notifications from HK about activity in threads you are involved in. There are indications in the summaries that a thread has messages you haven't read, but you have to look. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============1344553136851021985==-- From shashikanth.komandoor@gmail.com Wed Sep 16 08:06:26 2020 From: Shashikanth Komandoor To: mailman-users@mailman3.org Subject: [MM3-users] My Server is unable to connect to Mailman REST API few times Date: Wed, 16 Sep 2020 13:36:09 +0530 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7655770572221835183==" --===============7655770572221835183== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Team, I am currently running Mailman 3 with version 3.3.1 and in built postfix version 2.10.1-6 on RHEL 7.5 with PostgreSQL 11.7 version with default values in production environment from the past 4 months almost. As of now, we are having around 1327 lists created on this. When I am trying to do some operations like creating the lists, deleting the lists or accessing the lists through the GUI, I am getting the below error. If I am trying to do the operations through API or through mailman shell, still I am not able to connect to the API. Something went wrong Mailman REST API not available. Please start Mailman core. But my mailman core was working fine in the background. At the same time I was able to do other operations on a few other lists. By the time I am getting the above error, the mailman.log recorded the below messages. *[2020-09-16 12:49:08 +0530] [2600] [CRITICAL] WORKER TIMEOUT (pid:20234)* *[2020-09-16 12:49:08 +0530] [20234] [INFO] Worker exiting (pid: 20234)* *[2020-09-16 12:49:09 +0530] [21042] [INFO] Booting worker with pid: 21042* My mailman.cfg configuration is as below. The timeout value is the default value. I did not set any customized value. *(venv3) [root(a)lsmgr mailman]# grep -v ^# /var/lib/mailman/mailman/var/etc/mailman.cfg* *[archiver.hyperkitty]* *class: mailman_hyperkitty.Archiver* *enable: yes* *configuration: /var/lib/mailman/mailman-hyperkitty/mailman-hyperkitty.cfg* *[mta]* *incoming: mailman.mta.postfix.LMTP* *outgoing: mailman.mta.deliver.deliver* *lmtp_host: 0.0.0.0* *lmtp_port: 8024* *smtp_host: localhost* *smtp_port: 25* *[webservice]* *workers: 10* *[database]* *class: mailman.database.postgresql.PostgreSQLDatabase* *url: postgres://:@127.0.0.1/mailman * Then as per the suggestion I got earlier I configured the [webservice] section in the above file as below. *[webservice]* *configuration: /var/lib/mailman/mailman/var/etc/gunicorn.cfg* * [root(a)lsmgr logs]# cat /var/lib/mailman/mailman/var/etc/gunicorn.cfg* *[gunicorn]* *workers =3D 10* *timeout =3D 90 * After the above configuration, I found many errors like below in mailman.log file: *sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'pended.id '"* *sqlalchemy.exc.ResourceClosedError: This result object does not return rows. It has been closed automatically.* *sqlalchemy.exc.DatabaseError: (psycopg2.DatabaseError) error with status PGRES_TUPLES_OK and no message from the libpq* The above might be few of the errors. FYI, for the creation of each list, it is taking almost around 15 to 20 minutes. After multiple times of the above error messages, at some time, I am fortunately getting the list created or some other operation getting done. I found the same scenario when the workers are from default 2 to 10. I don't understand where exactly I need to focus to solve the issue. Please let me know if I have to share any other details for you to suggest to me. --=20 Thanks & Regards, Shashi Kanth.K 9052671936 Virus-free. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> --===============7655770572221835183==-- From pgrzywaczewski@gmail.com Wed Sep 16 15:25:22 2020 From: Pawel Grzywaczewski To: mailman-users@mailman3.org Subject: [MM3-users] Problem with list creation Date: Wed, 16 Sep 2020 15:25:18 +0000 Message-ID: <160026991850.16960.3600913716239931160@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1085443881517235932==" --===============1085443881517235932== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, We want to create thousand of lists. We use a simple curl command curl --user restadmin:restpass -X POST http://localhost:8001/3.1/lists -d fq= dn_listname=3DnameOfTheList(a)example.com After creating ~2k of them we started getting errors in Mailman logs:=20 ``` [2020-09-16 17:11:10 +0200] [4564] [DEBUG] POST /3.1/lists [2020-09-16 17:11:41 +0200] [1339] [CRITICAL] WORKER TIMEOUT (pid:4564) [2020-09-16 17:11:41 +0200] [4564] [INFO] Worker exiting (pid: 4564) [2020-09-16 17:11:41 +0200] [5019] [INFO] Booting worker with pid: 5019 ``` Since then creation fails in most cases. We increased timeout in mailman.cfg ``` [gunicorn] workers: 4 timeout: 900 ``` And we increased timeout in nginx ``` http { # MODIF UNICORN proxy_connect_timeout 300s; proxy_read_timeout 300s; # MODIF UNICORN ``` Where might be the problem? How to start investigating and fix it? Thanks, Pawe=C5=82 --===============1085443881517235932==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Wed Sep 16 15:26:10 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] My Server is unable to connect to Mailman REST API few times Date: Thu, 17 Sep 2020 00:26:03 +0900 Message-ID: <24418.11915.554392.597699@turnbull.sk.tsukuba.ac.jp> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8499116946908432650==" --===============8499116946908432650== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Shashikanth Komandoor writes: > I am currently running Mailman 3 with version 3.3.1 and in > built postfix version 2.10.1-6 on RHEL 7.5 with PostgreSQL 11.7 version > with default values in production environment from the past 4 months > almost. As of now, we are having around 1327 lists created on this. Aren't you the crew that modified Mailman to interface with SMS or something like that? If so, don't worry, it doesn't invalidate your warranty, but it seems possible that it has something to do with this: > After the above configuration, I found many errors like below in > mailman.log file: > > *sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for > column 'pended.id '"* > > *sqlalchemy.exc.ResourceClosedError: This result object does not return > rows. It has been closed automatically.* > > *sqlalchemy.exc.DatabaseError: (psycopg2.DatabaseError) error with status > PGRES_TUPLES_OK and no message from the libpq* Whether you have that modified Mailman or not, the messages above suggest that there may have been a change in the schema of the database that one side (presumably SQLAlchemy) knows about but have not been propagated to the other (PostgreSQL). Have you perhaps done an upgrade but not run the database migration script? About the REST API not available error: > Something went wrong > > Mailman REST API not available. Please start Mailman core. > > But my mailman core was working fine in the background. At > the same time I was able to do other operations on a few other lists. > > By the time I am getting the above error, the mailman.log > recorded the below messages. > > *[2020-09-16 12:49:08 +0530] [2600] [CRITICAL] WORKER TIMEOUT (pid:20234)* > > *[2020-09-16 12:49:08 +0530] [20234] [INFO] Worker exiting (pid: 20234)* > > *[2020-09-16 12:49:09 +0530] [21042] [INFO] Booting worker with pid: 21042* Something is taking too long, the worker is timing out. That's probably why you get the REST API not available message. Do the sqlalchemy errors correlate with the timeouts? I suspect the exit and reboot of the worker is normal but I haven't looked at this part of the code. > My mailman.cfg configuration is as below. The timeout > value is the default value. I did not set any customized value. Thanks for sending it; sometimes it speeds up debugging. But in this case you say that other lists are working, so site configuration doesn't seem to be the problem. Is this particular list special in some way? List configuration is different? > The above might be few of the errors. FYI, for the creation of each list, > it is taking almost around 15 to 20 minutes. After multiple times of the > above error messages, at some time, I am fortunately getting the list > created or some other operation getting done. This is quite strange. I don't see why this would suddenly start working after many minutes. Does PostgreSQL have any logs with "interesting" (or scary) messages in them? Maybe somebody else has some ideas, but this is as far as I can go. --===============8499116946908432650==-- From mark@msapiro.net Wed Sep 16 15:49:06 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Problem with list creation Date: Wed, 16 Sep 2020 08:48:57 -0700 Message-ID: <54a4605e-46b4-a579-3aeb-2a1994ed72cd@msapiro.net> In-Reply-To: <160026991850.16960.3600913716239931160@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8104946606570525654==" --===============8104946606570525654== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/16/20 8:25 AM, Pawel Grzywaczewski wrote: >=20 > We increased timeout in mailman.cfg > ``` > [gunicorn] > workers: 4 > timeout: 900 > ``` This doesn't do it. In order to pass configuration settings to the REST gunicorn servers, you need to put in mailman.cfg ``` [webservice] configuration: /path/to/gunicorn.cfg ``` and then put settings like these ``` [gunicorn] workers: 4 timeout: 900 ``` in /path/to/gunicorn.cfg. Some settings, but only hostname, port and workers can be put in mailman.cfg, but in the [webservice] section, not in a [gunicorn] section. See for more detail. > And we increased timeout in nginx >=20 > ``` > http { >=20 > # MODIF UNICORN > proxy_connect_timeout 300s; > proxy_read_timeout 300s; > # MODIF UNICORN >=20 > ``` nginx is not involved. These are not proxied requests through nginx. They are requests to Mailman's REST API which use a separate gunicorn server. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============8104946606570525654==-- From brylon@oit.rutgers.edu Wed Sep 16 16:20:20 2020 From: brylon@oit.rutgers.edu To: mailman-users@mailman3.org Subject: [MM3-users] Re: Office365 posts shunted (authres.core.SyntaxError) Date: Wed, 16 Sep 2020 16:20:17 +0000 Message-ID: <160027321742.16974.4283769167770319406@mailman.iad1.psf.io> In-Reply-To: <24416.26741.778037.428824@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3751644133614908805==" --===============3751644133614908805== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Yes, I think catch and ignore in the ARC module would be acceptable. Is ther= e any additional information I can provide to help and/or any further actions= to assist with? --===============3751644133614908805==-- From pgrzywaczewski@gmail.com Thu Sep 17 07:24:15 2020 From: Pawel Grzywaczewski To: mailman-users@mailman3.org Subject: [MM3-users] Re: Problem with list creation Date: Thu, 17 Sep 2020 07:24:12 +0000 Message-ID: <160032745207.16974.362386939684847089@mailman.iad1.psf.io> In-Reply-To: <54a4605e-46b4-a579-3aeb-2a1994ed72cd@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1737556908066075835==" --===============1737556908066075835== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Hi Mark, That was it. Thank you! Best regards, Paweł --===============1737556908066075835==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Thu Sep 17 11:13:05 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] Re: Office365 posts shunted (authres.core.SyntaxError) Date: Thu, 17 Sep 2020 20:12:55 +0900 Message-ID: <24419.17591.664555.192166@turnbull.sk.tsukuba.ac.jp> In-Reply-To: <160027321742.16974.4283769167770319406@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1483327432715830503==" --===============1483327432715830503== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit brylon(a)oit.rutgers.edu writes: > Yes, I think catch and ignore in the ARC module would be > acceptable. Is there any additional information I can provide to > help and/or any further actions to assist with? I think the work needed is quite clear. How pressing is this for you? We won't have a new release for some weeks to a few months as far as I know, but we could have a patch (or a single file to replace) that you could apply more quickly. Since it would be in the ARC module, which is independent of the rest of the code, it would be a safe patch to apply, and easy to reverse. As far as my schedule, I'm very busy for the next two weeks (my heavy teaching term starts on October 1). I'm looking at working on this intermittently starting maybe this weekend but probably not finishing a mergeable patch until late October. A patch you could apply (ie, just the code, not the documentation and tests) would be done sooner. I haven't talked to others about their schedules, somebody else might pick it up and I would have time to answer emails and review changes from them, but I'm the obvious person to work on ARC. Steve --===============1483327432715830503==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Thu Sep 17 11:17:59 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] Re: Problem with list creation Date: Thu, 17 Sep 2020 20:17:53 +0900 Message-ID: <24419.17889.575827.248584@turnbull.sk.tsukuba.ac.jp> In-Reply-To: <54a4605e-46b4-a579-3aeb-2a1994ed72cd@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4600958087717754496==" --===============4600958087717754496== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Mark Sapiro writes: > On 9/16/20 8:25 AM, Pawel Grzywaczewski wrote: > > We increased timeout in mailman.cfg This looks like the issue in Shashikanth's post, too: https://lists.mailman3.org/archives/list/mailman-users(a)mailman3.org/message= /2MYQDZMCURRSHLJKF24727FMRKL6RIA4/ They are also creating more than 1000 lists. Do we know why these timeouts are happening? (Search for "list creation times out" returns no results in either core or Postorius issues.) --===============4600958087717754496==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Thu Sep 17 11:52:35 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] My Server is unable to connect to Mailman REST API few times Date: Thu, 17 Sep 2020 20:52:28 +0900 Message-ID: <24419.19964.416207.590843@turnbull.sk.tsukuba.ac.jp> In-Reply-To: <24418.11915.554392.597699@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1617643105088185867==" --===============1617643105088185867== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Shashikanth, It appears I may have sent you on a wild goose chase. You have my apologies. There's a very similar report today, so this appears to be a Mailman 3 issue. For now, pleasee see Mark's post for the correct way to increase timeout: https://lists.mailman3.org/archives/list/mailman-users(a)mailman3.org/message= /66G5B27J4UZDFWWNKJLOATMVSLLF6D6H/ I am working with Mark to diagnose the underlying problem. Steve Stephen J. Turnbull writes: > Shashikanth Komandoor writes: >=20 > > I am currently running Mailman 3 with version 3.3.1 and in > > built postfix version 2.10.1-6 on RHEL 7.5 with PostgreSQL 11.7 version > > with default values in production environment from the past 4 months > > almost. As of now, we are having around 1327 lists created on this. >=20 > Aren't you the crew that modified Mailman to interface with SMS or > something like that? If so, don't worry, it doesn't invalidate your > warranty, but it seems possible that it has something to do with this: >=20 > > After the above configuration, I found many errors like below in > > mailman.log file: > >=20 > > *sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for > > column 'pended.id '"* > >=20 > > *sqlalchemy.exc.ResourceClosedError: This result object does not return > > rows. It has been closed automatically.* > >=20 > > *sqlalchemy.exc.DatabaseError: (psycopg2.DatabaseError) error with stat= us > > PGRES_TUPLES_OK and no message from the libpq* >=20 > Whether you have that modified Mailman or not, the messages above > suggest that there may have been a change in the schema of the > database that one side (presumably SQLAlchemy) knows about but have > not been propagated to the other (PostgreSQL). Have you perhaps done > an upgrade but not run the database migration script? >=20 > About the REST API not available error: >=20 > > Something went wrong > >=20 > > Mailman REST API not available. Please start Mailman core. > >=20 > > But my mailman core was working fine in the background. At > > the same time I was able to do other operations on a few other lists. > >=20 > > By the time I am getting the above error, the mailman.log > > recorded the below messages. > >=20 > > *[2020-09-16 12:49:08 +0530] [2600] [CRITICAL] WORKER TIMEOUT (pid:2023= 4)* > >=20 > > *[2020-09-16 12:49:08 +0530] [20234] [INFO] Worker exiting (pid: 20234)* > >=20 > > *[2020-09-16 12:49:09 +0530] [21042] [INFO] Booting worker with pid: 21= 042* >=20 > Something is taking too long, the worker is timing out. That's > probably why you get the REST API not available message. Do the > sqlalchemy errors correlate with the timeouts? >=20 > I suspect the exit and reboot of the worker is normal but I haven't > looked at this part of the code. >=20 > > My mailman.cfg configuration is as below. The timeout > > value is the default value. I did not set any customized value. >=20 > Thanks for sending it; sometimes it speeds up debugging. But in this > case you say that other lists are working, so site configuration > doesn't seem to be the problem. Is this particular list special in > some way? List configuration is different? >=20 > > The above might be few of the errors. FYI, for the creation of each lis= t, > > it is taking almost around 15 to 20 minutes. After multiple times of the > > above error messages, at some time, I am fortunately getting the list > > created or some other operation getting done. >=20 > This is quite strange. I don't see why this would suddenly start > working after many minutes. >=20 > Does PostgreSQL have any logs with "interesting" (or scary) messages > in them? >=20 > Maybe somebody else has some ideas, but this is as far as I can go. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >=20 >=20 --===============1617643105088185867==-- From pgrzywaczewski@gmail.com Thu Sep 17 15:31:57 2020 From: Pawel Grzywaczewski To: mailman-users@mailman3.org Subject: [MM3-users] Redundant and load balanced Mailman3 Date: Thu, 17 Sep 2020 15:31:54 +0000 Message-ID: <160035671427.16961.4228285617823670746@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7676865710557531496==" --===============7676865710557531496== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi,=20 Can someone share some hints about how to deploy redundant and load balanced = Mailman3? What are your recommendations and best practices? For example what components to put on the same VM? Can we have 2VM with Mailm= an core/Postorious/Hyperkitty/django-mailman/mailmanclient connecting to one = (the same) database? How to load balance mail routing (incoming)? Thanks! Pawe=C5=82 --===============7676865710557531496==-- From seth@tofutavern.com Thu Sep 17 16:33:49 2020 From: Seth Seeger To: mailman-users@mailman3.org Subject: [MM3-users] Template path Date: Thu, 17 Sep 2020 12:33:41 -0400 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4618821124214774377==" --===============4618821124214774377== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello everyone, I=E2=80=99m running Mailman from Docker containers, without much customizatio= n. (Postorius 1.3.3, Core 3.3.1, API 3.1, Python 3.8.5). I=E2=80=99ve creat= ed a list:user:notice:welcome template, and when I try to confirm a subscript= ion, I get an error. =20 I followed previous emails on this list which told me to set POSTORIUS_TEMPLA= TE_BASE_URL in /opt/mailman/web/settings_local.py, but that doesn=E2=80=99t s= eem to make any difference. (I set it to a different value and the error mes= sage didn=E2=80=99t change. I also tried setting it in /opt/mailman/core/set= tings_local.py to no effect. After each change, I restarted mailman-web and = mailman-core containers.) >From within mailman-web and mailman-core containers, I can reach =E2=80=9Cmai= lman-web:8000=E2=80=9D. In the webpage, I see: Mailman REST API not available. Please start Mailman = core. In /opt/mailman/core/var/logs/mailman.log, I get: [17/Sep/2020:16:23:29 +0000] "GET /3.1/lists/xxx.example.com HTTP/1.1" 200 41= 6 "-" "GNU Mailman REST client v3.3.1" Sep 17 16:23:35 2020 (48) deque: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426= , in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421= , in _make_request httplib_response =3D conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 307, in begin version, status, reason =3D self._read_status() File "/usr/lib/python3.8/http/client.py", line 268, in _read_status line =3D str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.8/socket.py", line 669, in readinto return self._sock.recv_into(b) socket.timeout: timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in = send resp =3D conn.urlopen( File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724= , in urlopen retries =3D retries.increment( File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 403, in= increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, = in reraise raise value File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670= , in urlopen httplib_response =3D self._make_request( File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 428= , in _make_request self._raise_timeout(err=3De, url=3Durl, timeout_value=3Dread_timeout) File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 335= , in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host=3D'mailman-web',= port=3D8000): Read timed out. (read timeout=3D5) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/mailman/app/workflow.py", line 69, i= n __next__ return step() File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py", line = 332, in _step_do_subscription self.member =3D self.mlist.subscribe( File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", li= ne 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3.8/site-packages/mailman/model/mailinglist.py", line = 505, in subscribe notify(SubscriptionEvent( File "/usr/lib/python3.8/site-packages/zope/event/__init__.py", line 32, in= notify subscriber(event) File "/usr/lib/python3.8/site-packages/mailman/app/membership.py", line 176= , in handle_SubscriptionEvent send_welcome_message(mlist, member, member.preferred_language) File "/usr/lib/python3.8/site-packages/mailman/app/notifications.py", line = 51, in send_welcome_message welcome_message =3D wrap(getUtility(ITemplateLoader).get( File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 188= , in get contents =3D getUtility(ITemplateManager).get( File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", li= ne 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 110= , in get contents =3D protocols.get(actual_uri, **auth) File "/usr/lib/python3.8/site-packages/mailman/utilities/protocols.py", lin= e 38, in get response =3D requests.get(url, timeout=3DREQUEST_TIMEOUT, **kws) File "/usr/lib/python3.8/site-packages/requests/api.py", line 76, in get return request('get', url, params=3Dparams, **kwargs) File "/usr/lib/python3.8/site-packages/requests/api.py", line 61, in request return session.request(method=3Dmethod, url=3Durl, **kwargs) File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 530, in = request resp =3D self.send(prep, **send_kwargs) File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 643, in = send r =3D adapter.send(request, **kwargs) File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 529, in = send raise ReadTimeout(e, request=3Drequest) requests.exceptions.ReadTimeout: HTTPConnectionPool(host=3D'mailman-web', por= t=3D8000): Read timed out. (read timeout=3D5) [2020-09-17 16:23:35 +0000] [48] [ERROR] Socket error processing request. Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426= , in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421= , in _make_request httplib_response =3D conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 307, in begin version, status, reason =3D self._read_status() File "/usr/lib/python3.8/http/client.py", line 268, in _read_status line =3D str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.8/socket.py", line 669, in readinto return self._sock.recv_into(b) socket.timeout: timed out Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in = send resp =3D conn.urlopen( File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724= , in urlopen retries =3D retries.increment( File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 403, in= increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, = in reraise raise value File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670= , in urlopen httplib_response =3D self._make_request( File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 428= , in _make_request self._raise_timeout(err=3De, url=3Durl, timeout_value=3Dread_timeout) File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 335= , in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host=3D'mailman-web',= port=3D8000): Read timed out. (read timeout=3D5) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 135,= in handle self.handle_request(listener, req, client, addr) File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 191,= in handle_request six.reraise(*sys.exc_info()) File "/usr/lib/python3.8/site-packages/gunicorn/six.py", line 625, in rerai= se raise value File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 176,= in handle_request respiter =3D self.wsgi(environ, resp.start_response) File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", li= ne 50, in wrapper rtn =3D function(*args, **kws) File "/usr/lib/python3.8/site-packages/mailman/rest/wsgiapp.py", line 193, = in __call__ return super().__call__(environ, start_response) File "/usr/lib/python3.8/site-packages/falcon/api.py", line 269, in __call__ responder(req, resp, **params) File "/usr/lib/python3.8/site-packages/mailman/rest/sub_moderation.py", lin= e 88, in on_post self._registrar.confirm(self._token) File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py", line = 559, in confirm list(workflow) File "/usr/lib/python3.8/site-packages/mailman/app/workflow.py", line 69, i= n __next__ return step() File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py", line = 332, in _step_do_subscription self.member =3D self.mlist.subscribe( File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", li= ne 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3.8/site-packages/mailman/model/mailinglist.py", line = 505, in subscribe notify(SubscriptionEvent( File "/usr/lib/python3.8/site-packages/zope/event/__init__.py", line 32, in= notify subscriber(event) File "/usr/lib/python3.8/site-packages/mailman/app/membership.py", line 176= , in handle_SubscriptionEvent send_welcome_message(mlist, member, member.preferred_language) File "/usr/lib/python3.8/site-packages/mailman/app/notifications.py", line = 51, in send_welcome_message welcome_message =3D wrap(getUtility(ITemplateLoader).get( File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 188= , in get contents =3D getUtility(ITemplateManager).get( File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", li= ne 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 110= , in get contents =3D protocols.get(actual_uri, **auth) File "/usr/lib/python3.8/site-packages/mailman/utilities/protocols.py", lin= e 38, in get response =3D requests.get(url, timeout=3DREQUEST_TIMEOUT, **kws) File "/usr/lib/python3.8/site-packages/requests/api.py", line 76, in get return request('get', url, params=3Dparams, **kwargs) File "/usr/lib/python3.8/site-packages/requests/api.py", line 61, in request return session.request(method=3Dmethod, url=3Durl, **kwargs) File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 530, in = request resp =3D self.send(prep, **send_kwargs) File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 643, in = send r =3D adapter.send(request, **kwargs) File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 529, in = send raise ReadTimeout(e, request=3Drequest) requests.exceptions.ReadTimeout: HTTPConnectionPool(host=3D'mailman-web', por= t=3D8000): Read timed out. (read timeout=3D5) What=E2=80=99s the next step to get templates working? Thank you, Seth --===============4618821124214774377==-- From mark@msapiro.net Thu Sep 17 16:54:55 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Template path Date: Thu, 17 Sep 2020 09:54:42 -0700 Message-ID: <00fdef30-da40-41fa-b3dd-0c2105cfd725@msapiro.net> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5701308401068017098==" --===============5701308401068017098== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/17/20 9:33 AM, Seth Seeger wrote: > Hello everyone, >=20 > I=E2=80=99m running Mailman from Docker containers, without much customizat= ion. (Postorius 1.3.3, Core 3.3.1, API 3.1, Python 3.8.5). I=E2=80=99ve cre= ated a list:user:notice:welcome template, and when I try to confirm a subscri= ption, I get an error. =20 >=20 > I followed previous emails on this list which told me to set POSTORIUS_TEMP= LATE_BASE_URL in /opt/mailman/web/settings_local.py, but that doesn=E2=80=99t= seem to make any difference. (I set it to a different value and the error m= essage didn=E2=80=99t change. I also tried setting it in /opt/mailman/core/s= ettings_local.py to no effect. After each change, I restarted mailman-web an= d mailman-core containers.) How did you create the template? In Postorius or by manually creating a file somewhere? If by creating a file, POSTORIUS_TEMPLATE_BASE_URL is not relevant, but ensure that the file is findable and readable by Mailman. > From within mailman-web and mailman-core containers, I can reach =E2=80=9Cm= ailman-web:8000=E2=80=9D. >=20 > In the webpage, I see: Mailman REST API not available. Please start Mailma= n core. The REST APT is normally exposed on port 8001, not 8000 (See `port` in `mailman conf -s webservice`. What is your setting for MAILMAN_REST_API_URL? does the port match? > In /opt/mailman/core/var/logs/mailman.log, I get: >=20 > [17/Sep/2020:16:23:29 +0000] "GET /3.1/lists/xxx.example.com HTTP/1.1" 200 = 416 "-" "GNU Mailman REST client v3.3.1" It appears you are connecting to the REST API, but ... ... > urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host=3D'mailman-web= ', port=3D8000): Read timed out. (read timeout=3D5) Something appears to be trying to connect to REST on port 8000 --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5701308401068017098==-- From seth@tofutavern.com Thu Sep 17 17:21:26 2020 From: Seth Seeger To: mailman-users@mailman3.org Subject: [MM3-users] Re: Template path Date: Thu, 17 Sep 2020 13:21:20 -0400 Message-ID: <1F27D8AA-90B4-4D1F-836D-49E6F90D4170@tofutavern.com> In-Reply-To: <00fdef30-da40-41fa-b3dd-0c2105cfd725@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7075909569992748813==" --===============7075909569992748813== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable > On Sep 17, 2020, at 12:54 PM, Mark Sapiro wrote: >=20 >> I followed previous emails on this list which told me to set POSTORIUS_TEM= PLATE_BASE_URL in /opt/mailman/web/settings_local.py, but that doesn=E2=80=99= t seem to make any difference. (I set it to a different value and the error = message didn=E2=80=99t change. I also tried setting it in /opt/mailman/core/= settings_local.py to no effect. After each change, I restarted mailman-web a= nd mailman-core containers.) >=20 >=20 > How did you create the template? In Postorius or by manually creating a > file somewhere? If by creating a file, POSTORIUS_TEMPLATE_BASE_URL is > not relevant, but ensure that the file is findable and readable by Mailman. I created the template in Postorius. >> From within mailman-web and mailman-core containers, I can reach =E2=80=9C= mailman-web:8000=E2=80=9D. >>=20 >> In the webpage, I see: Mailman REST API not available. Please start Mailm= an core. >=20 >=20 > The REST APT is normally exposed on port 8001, not 8000 (See `port` in > `mailman conf -s webservice`. Yes, it=E2=80=99s 8001: [webservice] hostname: 172.19.199.2 [webservice] port: 8001 > What is your setting for MAILMAN_REST_API_URL? does the port match? I=E2=80=99m not overriding it: ./postorius/mailman-web/settings.py:MAILMAN_REST_API_URL =3D os.environ.get('= MAILMAN_REST_URL', 'http://mailman-core:8001') >=20 >> In /opt/mailman/core/var/logs/mailman.log, I get: >>=20 >> [17/Sep/2020:16:23:29 +0000] "GET /3.1/lists/xxx.example.com HTTP/1.1" 200= 416 "-" "GNU Mailman REST client v3.3.1" >=20 > It appears you are connecting to the REST API, but ... > ... >> urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host=3D'mailman-we= b', port=3D8000): Read timed out. (read timeout=3D5) >=20 >=20 > Something appears to be trying to connect to REST on port 8000 Is this a red herring? Is something else wrong? Setting POSTORIUS_TEMPLATE_= BASE_URL in settings_local.py doesn=E2=80=99t seem to make any difference. Having set the POSTORIUS_TEMPLATE_BASE_URL, I deleted the template and recrea= ted it. Now, when I approve a pending subscription it goes through, but the = welcome email is blank. Seth --===============7075909569992748813==-- From seth@tofutavern.com Thu Sep 17 19:19:33 2020 From: Seth Seeger To: mailman-users@mailman3.org Subject: [MM3-users] Re: Template path Date: Thu, 17 Sep 2020 15:19:24 -0400 Message-ID: In-Reply-To: <00fdef30-da40-41fa-b3dd-0c2105cfd725@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6201284360465422390==" --===============6201284360465422390== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable > On Sep 17, 2020, at 12:54 PM, Mark Sapiro wrote: >=20 > Something appears to be trying to connect to REST on port 8000 I found this (older) email: https://lists.mailman3.org/archives/list/mailman-users(a)mailman3.org/thread/= TJPXCCOFBRI33I3YDX5KKDMWTX6CBGBZ/ that does say that the POSTORIUS_TEMPLATE_BASE_URL should be =E2=80=9Chttp://= mailman-web:8000=E2=80=9D Seth --===============6201284360465422390==-- From mark@msapiro.net Thu Sep 17 21:40:22 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Template path Date: Thu, 17 Sep 2020 14:40:14 -0700 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3387289510533742923==" --===============3387289510533742923== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/17/20 12:19 PM, Seth Seeger wrote: >=20 > I found this (older) email: >=20 > https://lists.mailman3.org/archives/list/mailman-users(a)mailman3.org/threa= d/TJPXCCOFBRI33I3YDX5KKDMWTX6CBGBZ/ >=20 > that does say that the POSTORIUS_TEMPLATE_BASE_URL should be =E2=80=9Chttp:= //mailman-web:8000=E2=80=9D You've exhausted my very limited knowledge of the Docker install. Hopefully Abhilash (@maxking) or someone else will chime in. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============3387289510533742923==-- From brylon@oit.rutgers.edu Thu Sep 17 21:42:00 2020 From: brylon@oit.rutgers.edu To: mailman-users@mailman3.org Subject: [MM3-users] Re: Office365 posts shunted (authres.core.SyntaxError) Date: Thu, 17 Sep 2020 21:41:57 +0000 Message-ID: <160037891751.16962.15775443048014944423@mailman.iad1.psf.io> In-Reply-To: <24419.17591.664555.192166@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7346878872379665184==" --===============7346878872379665184== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Okay. Well the current plan/goal is to have a production setup/rollout in a = few months time (i.e., 12/31/2020), so a knowledgeable[1] patch I could apply= while still testing would be fantastic. We currently host ~4000 lists on MM= 2 (although that number will be closer to ~2500 migrating to our in the works= MM3 deployment after we audit MM2 in a month or so). Thanks for devoting an= y time you can find to this! [1] I currently applied this hack to get around the syntax error... $ diff -u venv/lib/python3.6/site-packages/mailman/handlers/validate_authenti= city.py.ORIG venv/lib/python3.6/site-packages/mailman/handlers/validate_authe= nticity.py --- venv/lib/python3.6/site-packages/mailman/handlers/validate_authenticity.p= y.ORIG 2020-09-16 14:23:03.565482574 -0400 +++ venv/lib/python3.6/site-packages/mailman/handlers/validate_authenticity.p= y 2020-09-17 13:17:09.296409626 -0400 @@ -70,10 +70,12 @@ =20 if config.arc.trusted_authserv_ids and (AUTH_RESULT_HEADER in msg): header =3D '{}: {}'.format(AUTH_RESULT_HEADER, msg[AUTH_RESULT_HEADE= R]) - authserv_id =3D AuthenticationResultsHeader.parse(header).authserv_id - if authserv_id in config.arc.trusted_authserv_ids: - return header - + try: + authserv_id =3D AuthenticationResultsHeader.parse(header).authse= rv_id + if authserv_id in config.arc.trusted_authserv_ids: + return header + except: + log.exception('AR header failed to parse for message: %s', msg['= Message-ID']) =20 @retry(Timeout, NUM_TIMEOUT_RETRIES) def authenticate(msg, msgdata): --===============7346878872379665184==-- From mark@msapiro.net Thu Sep 17 21:55:36 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Office365 posts shunted (authres.core.SyntaxError) Date: Thu, 17 Sep 2020 14:55:27 -0700 Message-ID: In-Reply-To: <160037891751.16962.15775443048014944423@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3763218905297404135==" --===============3763218905297404135== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/17/20 2:41 PM, brylon(a)oit.rutgers.edu wrote: >=20 > [1] I currently applied this hack to get around the syntax error... >=20 > $ diff -u venv/lib/python3.6/site-packages/mailman/handlers/validate_authen= ticity.py.ORIG venv/lib/python3.6/site-packages/mailman/handlers/validate_aut= henticity.py > --- venv/lib/python3.6/site-packages/mailman/handlers/validate_authenticity= .py.ORIG 2020-09-16 14:23:03.565482574 -0400 > +++ venv/lib/python3.6/site-packages/mailman/handlers/validate_authenticity= .py 2020-09-17 13:17:09.296409626 -0400 > @@ -70,10 +70,12 @@ > =20 > if config.arc.trusted_authserv_ids and (AUTH_RESULT_HEADER in msg): > header =3D '{}: {}'.format(AUTH_RESULT_HEADER, msg[AUTH_RESULT_HEA= DER]) > - authserv_id =3D AuthenticationResultsHeader.parse(header).authserv= _id > - if authserv_id in config.arc.trusted_authserv_ids: > - return header > - > + try: > + authserv_id =3D AuthenticationResultsHeader.parse(header).auth= serv_id > + if authserv_id in config.arc.trusted_authserv_ids: > + return header > + except: > + log.exception('AR header failed to parse for message: %s', msg= ['Message-ID']) > =20 > @retry(Timeout, NUM_TIMEOUT_RETRIES) > def authenticate(msg, msgdata): I think that's fine. Also, you should look at . --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============3763218905297404135==-- From mmasoudf@aaas.org Fri Sep 18 19:32:43 2020 From: Mohsen Masoudfar To: mailman-users@mailman3.org Subject: [MM3-users] The message is larger than the 40 KB maximum size Date: Fri, 18 Sep 2020 19:32:37 +0000 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5678356388518028924==" --===============5678356388518028924== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, I am trying to find how I may change this. I have an attachment of 800 KB, an= d when I post it to the list, I do not get any response and nothing is hold. = It looks like the message is just dropped. Best regards Mohsen Masoudfar Lead System Analyst, IT Operation American Association for the Advancement of Science 1200 New York Ave, NW, Washington, DC 20005 202-326-7087 --===============5678356388518028924==-- From phils@caerllewys.net Fri Sep 18 19:48:44 2020 From: Phil Stracchino To: mailman-users@mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size Date: Fri, 18 Sep 2020 15:48:34 -0400 Message-ID: <6845128e-b1b3-dcce-2693-ef98c116a962@caerllewys.net> In-Reply-To: =?utf-8?q?=3CMN2PR20MB3245B9BFD5A872A2751263FFB63F0=40MN2PR20MB?= =?utf-8?q?3245=2Enamprd20=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7496783871989494685==" --===============7496783871989494685== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 2020-09-18 15:32, Mohsen Masoudfar wrote: > Hi, >=20 > I am trying to find how I may change this. I have an attachment of 800 KB, = and when I post it to the list, I do not get any response and nothing is hold= . It looks like the message is just dropped. Assuming that this works roughly the same in mm3 as it does in mm2, you can set max_message_size in your list configuration. It defaults to 40; units are KB. --=20 Phil Stracchino Babylon Communications phils(a)caerllewys.net phil(a)co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 --===============7496783871989494685==-- From mark@msapiro.net Fri Sep 18 19:55:19 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size Date: Fri, 18 Sep 2020 12:55:06 -0700 Message-ID: In-Reply-To: =?utf-8?q?=3CMN2PR20MB3245B9BFD5A872A2751263FFB63F0=40MN2PR20MB?= =?utf-8?q?3245=2Enamprd20=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5692629544736451779==" --===============5692629544736451779== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/18/20 12:32 PM, Mohsen Masoudfar wrote: > Hi, >=20 > I am trying to find how I may change this. I have an attachment of 800 KB, = and when I post it to the list, I do not get any response and nothing is hold= . It looks like the message is just dropped. >=20 Is the message in Mailman's var/queue/shunt queue? Is there an error in var/log/mailman.log? --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5692629544736451779==-- From mark@msapiro.net Fri Sep 18 20:00:14 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size Date: Fri, 18 Sep 2020 13:00:03 -0700 Message-ID: In-Reply-To: <6845128e-b1b3-dcce-2693-ef98c116a962@caerllewys.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3748219668213143190==" --===============3748219668213143190== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/18/20 12:48 PM, Phil Stracchino wrote: > On 2020-09-18 15:32, Mohsen Masoudfar wrote: >> Hi, >> >> I am trying to find how I may change this. I have an attachment of 800 KB,= and when I post it to the list, I do not get any response and nothing is hol= d. It looks like the message is just dropped. >=20 >=20 > Assuming that this works roughly the same in mm3 as it does in mm2, you > can set max_message_size in your list configuration. It defaults to 40; > units are KB. Yes, and you can set it to zero for "unlimited", but even if that is a satisfactory workaround, the much more interesting question, at least to me, is why does the message "disappear". --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============3748219668213143190==-- From shashikanth.komandoor@gmail.com Sat Sep 19 11:32:30 2020 From: Shashikanth Komandoor To: mailman-users@mailman3.org Subject: [MM3-users] Re: My Server is unable to connect to Mailman REST API few times Date: Sat, 19 Sep 2020 17:02:13 +0530 Message-ID: In-Reply-To: <24419.19964.416207.590843@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2600594949091070011==" --===============2600594949091070011== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Thank you a lot Stephen for your attempt to help me. In fact I got some info from your response. Aren't you the crew that modified Mailman to interface with SMS or > something like that? If so, don't worry, it doesn't invalidate your > warranty, but it seems possible that it has something to do with this: Yes. We have taken the source code of mailman and customised the software to send SMS notifications along with the mail notifications when any mail is waiting for the pending approval. But not sure if it is creating a problem. Whether you have that modified Mailman or not, the messages above > suggest that there may have been a change in the schema of the > database that one side (presumably SQLAlchemy) knows about but have > not been propagated to the other (PostgreSQL). Have you perhaps done > an upgrade but not run the database migration script? I have not done any upgrade of any component of the mailman. Something is taking too long, the worker is timing out. That's > probably why you get the REST API not available message. Do the > sqlalchemy errors correlate with the timeouts? I am unable to state if both are correlating with each other. Thanks for sending it; sometimes it speeds up debugging. But in this > case you say that other lists are working, so site configuration > doesn't seem to be the problem. Is this particular list special in > some way? List configuration is different? No. All the lists on the server are similarly configured. May be with different settings of list or list members. > > Does PostgreSQL have any logs with "interesting" (or scary) messages > in them? No, I don't find any messages in postgresql logs. It appears I may have sent you on a wild goose chase. You have my > apologies. I am very happy you spent your precious time working on my issue. I am working with Mark to diagnose the underlying problem. Thank you both for this. But I am trying to delete a list on an urgent basis. Because of which I am getting the above said errors. I tried to delete through the interface, through curl command and through "mailman shell" but nothing led me to a positive result. Is there any work around to delete the list ? Virus-free. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Thu, Sep 17, 2020 at 5:22 PM Stephen J. Turnbull < turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote: > Hi Shashikanth, > > It appears I may have sent you on a wild goose chase. You have my > apologies. There's a very similar report today, so this appears to be > a Mailman 3 issue. For now, pleasee see Mark's post for the correct > way to increase timeout: > > https://lists.mailman3.org/archives/list/mailman-users(a)mailman3.org/messa= ge/66G5B27J4UZDFWWNKJLOATMVSLLF6D6H/ > > I am working with Mark to diagnose the underlying problem. > > Steve > > > Stephen J. Turnbull writes: > > Shashikanth Komandoor writes: > > > > > I am currently running Mailman 3 with version 3.3.1 and > in > > > built postfix version 2.10.1-6 on RHEL 7.5 with PostgreSQL 11.7 > version > > > with default values in production environment from the past 4 months > > > almost. As of now, we are having around 1327 lists created on this. > > > > Aren't you the crew that modified Mailman to interface with SMS or > > something like that? If so, don't worry, it doesn't invalidate your > > warranty, but it seems possible that it has something to do with this: > > > > > After the above configuration, I found many errors like below in > > > mailman.log file: > > > > > > *sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row > for > > > column 'pended.id '"* > > > > > > *sqlalchemy.exc.ResourceClosedError: This result object does not > return > > > rows. It has been closed automatically.* > > > > > > *sqlalchemy.exc.DatabaseError: (psycopg2.DatabaseError) error with > status > > > PGRES_TUPLES_OK and no message from the libpq* > > > > Whether you have that modified Mailman or not, the messages above > > suggest that there may have been a change in the schema of the > > database that one side (presumably SQLAlchemy) knows about but have > > not been propagated to the other (PostgreSQL). Have you perhaps done > > an upgrade but not run the database migration script? > > > > About the REST API not available error: > > > > > Something went wrong > > > > > > Mailman REST API not available. Please start Mailman core. > > > > > > But my mailman core was working fine in the > background. At > > > the same time I was able to do other operations on a few other lists. > > > > > > By the time I am getting the above error, the > mailman.log > > > recorded the below messages. > > > > > > *[2020-09-16 12:49:08 +0530] [2600] [CRITICAL] WORKER TIMEOUT > (pid:20234)* > > > > > > *[2020-09-16 12:49:08 +0530] [20234] [INFO] Worker exiting (pid: > 20234)* > > > > > > *[2020-09-16 12:49:09 +0530] [21042] [INFO] Booting worker with pid: > 21042* > > > > Something is taking too long, the worker is timing out. That's > > probably why you get the REST API not available message. Do the > > sqlalchemy errors correlate with the timeouts? > > > > I suspect the exit and reboot of the worker is normal but I haven't > > looked at this part of the code. > > > > > My mailman.cfg configuration is as below. The timeout > > > value is the default value. I did not set any customized value. > > > > Thanks for sending it; sometimes it speeds up debugging. But in this > > case you say that other lists are working, so site configuration > > doesn't seem to be the problem. Is this particular list special in > > some way? List configuration is different? > > > > > The above might be few of the errors. FYI, for the creation of each > list, > > > it is taking almost around 15 to 20 minutes. After multiple times of > the > > > above error messages, at some time, I am fortunately getting the list > > > created or some other operation getting done. > > > > This is quite strange. I don't see why this would suddenly start > > working after many minutes. > > > > Does PostgreSQL have any logs with "interesting" (or scary) messages > > in them? > > > > Maybe somebody else has some ideas, but this is as far as I can go. > > _______________________________________________ > > Mailman-users mailing list -- mailman-users(a)mailman3.org > > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > > > > --=20 Thanks & Regards, Shashi Kanth.K 9052671936 Virus-free. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> --===============2600594949091070011==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Sat Sep 19 12:33:18 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] Re: Office365 posts shunted (authres.core.SyntaxError) Date: Sat, 19 Sep 2020 21:33:12 +0900 Message-ID: <24421.64136.753444.60243@turnbull.sk.tsukuba.ac.jp> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7664572137479775788==" --===============7664572137479775788== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Mark Sapiro writes: > I think that's fine. Also, you should look at > . I agree with Mark. I was pretty sure something like that would work, but I haven't looked at the code in a while, so didn't know exactly where to patch. I'll take a look at it tomorrow, and see if I can spot any edge cases, but that's probably as simple as it looks. Steve --===============7664572137479775788==-- From mark@msapiro.net Sat Sep 19 15:25:35 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: My Server is unable to connect to Mailman REST API few times Date: Sat, 19 Sep 2020 08:25:26 -0700 Message-ID: <384941ed-23d7-8d47-ed33-ec1cf044305e@msapiro.net> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5123237176161623174==" --===============5123237176161623174== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/19/20 4:32 AM, Shashikanth Komandoor wrote: > > But I am trying to delete a list on an urgent basis. Because of which I am > getting the above said errors. I tried to delete > through the interface, through curl command and through "mailman shell" but > nothing led me to a > positive result. Is there any work around to delete the list ? What have you tried in Mailman shell? This should do it. $ mailman shell -l list.example.com Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. The variable 'm' is the list.example.com mailing list >>> getUtility(IListManager).delete(m) >>> commit() So should just $ mailman remove list.example.com What happens if you do those? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5123237176161623174==-- From maszap69@gmail.com Sat Sep 19 17:34:01 2020 From: Massimo =?utf-8?q?Zappal=C3=A0?= To: mailman-users@mailman3.org Subject: [MM3-users] Bad Signature with Gmail Date: Sat, 19 Sep 2020 17:11:29 +0000 Message-ID: <160053548912.12978.11246499763267029184@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6159521059850429736==" --===============6159521059850429736== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi everyone, if I write a my lists from other email address I get the correct message: opendkim[23516]: 8EEE440FFD: DKIM-Signature field added (s=3Dmailselector, d= =3Dcatania.linux.it). While if I write to a my lists from email address Gmail, on opendkim I get th= e following messages: opendkim[23516]: BA95241952: no signing table match for 'maszap69(a)gmail.com' opendkim[23516]: BA95241952: s=3D20161025 d=3Dgmail.com SSL error:04091068:rs= a routines:int_rsa_verify:bad signature opendkim[23516]: BA95241952: bad signature data The message is received regularly by all members of the list. I also receive an email at superuser: [Django] ERROR (EXTERNAL IP): Internal Server Error: /hyperkitty/api/mailman/= archive=20 Internal Server Error: /hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\xB1\\xF0\\x9F...' for colum= n `mailman3web`.`hyperkitty_email`.`content` at row 1") ... If I check with mail-tester I get 10/10 If I check with dmarcanalyzer I get "This seems to be a valid DKIM Record." I don't understand where the problem is. Could anyone tell me how to solve? Thank you in advance Max --===============6159521059850429736==-- From mark@msapiro.net Sat Sep 19 18:32:56 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Bad Signature with Gmail Date: Sat, 19 Sep 2020 11:32:45 -0700 Message-ID: <18a7f260-9aaa-d35f-1bb6-3e66ac6df7ec@msapiro.net> In-Reply-To: <160053548912.12978.11246499763267029184@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1498910593183384103==" --===============1498910593183384103== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/19/20 10:11 AM, Massimo Zappal=C3=A0 wrote: > Hi everyone, > if I write a my lists from other email address I get the correct message: > opendkim[23516]: 8EEE440FFD: DKIM-Signature field added (s=3Dmailselector, = d=3Dcatania.linux.it). >=20 > While if I write to a my lists from email address Gmail, on opendkim I get = the following messages: > opendkim[23516]: BA95241952: no signing table match for 'maszap69(a)gmail.c= om' > opendkim[23516]: BA95241952: s=3D20161025 d=3Dgmail.com SSL error:04091068:= rsa routines:int_rsa_verify:bad signature > opendkim[23516]: BA95241952: bad signature data OpenDKIM on your server is not signing the message because it is not configured to sign messages From: 'maszap69(a)gmail.com' What's in your opendkim SigningTable and what's in your opendkim.conf for SenderHeaders. The recommend config is SenderHeaders List-Post,Sender,From to use the domain of the list for signing list mail and then to include that domain in SigningTable. > I also receive an email at superuser: >=20 > [Django] ERROR (EXTERNAL IP): Internal Server Error: /hyperkitty/api/mailma= n/archive=20 >=20 > Internal Server Error: /hyperkitty/api/mailman/archive > OperationalError at /hyperkitty/api/mailman/archive > (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\xB1\\xF0\\x9F...' for col= umn `mailman3web`.`hyperkitty_email`.`content` at row 1") > ... This appears to be unrelated. HyperKitty is trying to store the message in the database hyperkitty_email table and gets this error. To say more, I'd need to see the complete raw message which you may be able to find in Mailman's var/archives/prototype/ The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============1498910593183384103==-- From maxking@asynchronous.in Sat Sep 19 19:03:05 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: Template path Date: Sat, 19 Sep 2020 12:02:38 -0700 Message-ID: <9f4ade1b-a4b3-4bb8-a05f-1953503fa1e4@www.fastmail.com> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4035864457729165350==" --===============4035864457729165350== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Thu, Sep 17, 2020, at 9:33 AM, Seth Seeger wrote: > Hello everyone, >=20 > I=E2=80=99m running Mailman from Docker containers, without much customizat= ion.=20 > (Postorius 1.3.3, Core 3.3.1, API 3.1, Python 3.8.5). I=E2=80=99ve create= d a=20 > list:user:notice:welcome template, and when I try to confirm a=20 > subscription, I get an error. =20 >=20 > I followed previous emails on this list which told me to set=20 > POSTORIUS_TEMPLATE_BASE_URL in /opt/mailman/web/settings_local.py, but=20 > that doesn=E2=80=99t seem to make any difference. (I set it to a different= =20 > value and the error message didn=E2=80=99t change. I also tried setting it= in=20 > /opt/mailman/core/settings_local.py to no effect. After each change, I=20 > restarted mailman-web and mailman-core containers.) >=20 > From within mailman-web and mailman-core containers, I can reach=20 > =E2=80=9Cmailman-web:8000=E2=80=9D. >=20 > In the webpage, I see: Mailman REST API not available. Please start=20 > Mailman core. >=20 > In /opt/mailman/core/var/logs/mailman.log, I get: >=20 > [17/Sep/2020:16:23:29 +0000] "GET /3.1/lists/xxx.example.com HTTP/1.1"=20 > 200 416 "-" "GNU Mailman REST client v3.3.1" > Sep 17 16:23:35 2020 (48) deque: > Traceback (most recent call last): > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 426, in _make_request > six.raise_from(e, None) > File "", line 3, in raise_from > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 421, in _make_request > httplib_response =3D conn.getresponse() > File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse > response.begin() > File "/usr/lib/python3.8/http/client.py", line 307, in begin > version, status, reason =3D self._read_status() > File "/usr/lib/python3.8/http/client.py", line 268, in _read_status > line =3D str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") > File "/usr/lib/python3.8/socket.py", line 669, in readinto > return self._sock.recv_into(b) > socket.timeout: timed out >=20 > During handling of the above exception, another exception occurred: >=20 > Traceback (most recent call last): > File "/usr/lib/python3.8/site-packages/requests/adapters.py", line=20 > 439, in send > resp =3D conn.urlopen( > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 724, in urlopen > retries =3D retries.increment( > File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line=20 > 403, in increment > raise six.reraise(type(error), error, _stacktrace) > File "/usr/lib/python3.8/site-packages/urllib3/packages/six.py", line=20 > 735, in reraise > raise value > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 670, in urlopen > httplib_response =3D self._make_request( > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 428, in _make_request > self._raise_timeout(err=3De, url=3Durl, timeout_value=3Dread_timeout) > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 335, in _raise_timeout > raise ReadTimeoutError( > urllib3.exceptions.ReadTimeoutError:=20 > HTTPConnectionPool(host=3D'mailman-web', port=3D8000): Read timed out.=20 > (read timeout=3D5) >=20 > During handling of the above exception, another exception occurred: >=20 > Traceback (most recent call last): > File "/usr/lib/python3.8/site-packages/mailman/app/workflow.py", line=20 > 69, in __next__ > return step() > File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py",=20 > line 332, in _step_do_subscription > self.member =3D self.mlist.subscribe( > File=20 > "/usr/lib/python3.8/site-packages/mailman/database/transaction.py",=20 > line 85, in wrapper > return function(args[0], config.db.store, *args[1:], **kws) > File "/usr/lib/python3.8/site-packages/mailman/model/mailinglist.py",=20 > line 505, in subscribe > notify(SubscriptionEvent( > File "/usr/lib/python3.8/site-packages/zope/event/__init__.py", line=20 > 32, in notify > subscriber(event) > File "/usr/lib/python3.8/site-packages/mailman/app/membership.py",=20 > line 176, in handle_SubscriptionEvent > send_welcome_message(mlist, member, member.preferred_language) > File "/usr/lib/python3.8/site-packages/mailman/app/notifications.py",=20 > line 51, in send_welcome_message > welcome_message =3D wrap(getUtility(ITemplateLoader).get( > File "/usr/lib/python3.8/site-packages/mailman/model/template.py",=20 > line 188, in get > contents =3D getUtility(ITemplateManager).get( > File=20 > "/usr/lib/python3.8/site-packages/mailman/database/transaction.py",=20 > line 85, in wrapper > return function(args[0], config.db.store, *args[1:], **kws) > File "/usr/lib/python3.8/site-packages/mailman/model/template.py",=20 > line 110, in get > contents =3D protocols.get(actual_uri, **auth) > File=20 > "/usr/lib/python3.8/site-packages/mailman/utilities/protocols.py", line=20 > 38, in get > response =3D requests.get(url, timeout=3DREQUEST_TIMEOUT, **kws) > File "/usr/lib/python3.8/site-packages/requests/api.py", line 76, in=20 > get > return request('get', url, params=3Dparams, **kwargs) > File "/usr/lib/python3.8/site-packages/requests/api.py", line 61, in=20 > request > return session.request(method=3Dmethod, url=3Durl, **kwargs) > File "/usr/lib/python3.8/site-packages/requests/sessions.py", line=20 > 530, in request > resp =3D self.send(prep, **send_kwargs) > File "/usr/lib/python3.8/site-packages/requests/sessions.py", line=20 > 643, in send > r =3D adapter.send(request, **kwargs) > File "/usr/lib/python3.8/site-packages/requests/adapters.py", line=20 > 529, in send > raise ReadTimeout(e, request=3Drequest) > requests.exceptions.ReadTimeout: HTTPConnectionPool(host=3D'mailman-web',=20 > port=3D8000): Read timed out. (read timeout=3D5) > [2020-09-17 16:23:35 +0000] [48] [ERROR] Socket error processing=20 > request. > Traceback (most recent call last): > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 426, in _make_request > six.raise_from(e, None) > File "", line 3, in raise_from > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 421, in _make_request > httplib_response =3D conn.getresponse() > File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse > response.begin() > File "/usr/lib/python3.8/http/client.py", line 307, in begin > version, status, reason =3D self._read_status() > File "/usr/lib/python3.8/http/client.py", line 268, in _read_status > line =3D str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") > File "/usr/lib/python3.8/socket.py", line 669, in readinto > return self._sock.recv_into(b) > socket.timeout: timed out >=20 > Traceback (most recent call last): > File "/usr/lib/python3.8/site-packages/requests/adapters.py", line=20 > 439, in send > resp =3D conn.urlopen( > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 724, in urlopen > retries =3D retries.increment( > File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line=20 > 403, in increment > raise six.reraise(type(error), error, _stacktrace) > File "/usr/lib/python3.8/site-packages/urllib3/packages/six.py", line=20 > 735, in reraise > raise value > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 670, in urlopen > httplib_response =3D self._make_request( > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 428, in _make_request > self._raise_timeout(err=3De, url=3Durl, timeout_value=3Dread_timeout) > File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py",=20 > line 335, in _raise_timeout > raise ReadTimeoutError( > urllib3.exceptions.ReadTimeoutError:=20 > HTTPConnectionPool(host=3D'mailman-web', port=3D8000): Read timed out.=20 > (read timeout=3D5) >=20 > During handling of the above exception, another exception occurred: >=20 > Traceback (most recent call last): > File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py",=20 > line 135, in handle > self.handle_request(listener, req, client, addr) > File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py",=20 > line 191, in handle_request > six.reraise(*sys.exc_info()) > File "/usr/lib/python3.8/site-packages/gunicorn/six.py", line 625, in=20 > reraise > raise value > File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py",=20 > line 176, in handle_request > respiter =3D self.wsgi(environ, resp.start_response) > File=20 > "/usr/lib/python3.8/site-packages/mailman/database/transaction.py",=20 > line 50, in wrapper > rtn =3D function(*args, **kws) > File "/usr/lib/python3.8/site-packages/mailman/rest/wsgiapp.py", line=20 > 193, in __call__ > return super().__call__(environ, start_response) > File "/usr/lib/python3.8/site-packages/falcon/api.py", line 269, in=20 > __call__ > responder(req, resp, **params) > File=20 > "/usr/lib/python3.8/site-packages/mailman/rest/sub_moderation.py", line=20 > 88, in on_post > self._registrar.confirm(self._token) > File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py",=20 > line 559, in confirm > list(workflow) > File "/usr/lib/python3.8/site-packages/mailman/app/workflow.py", line=20 > 69, in __next__ > return step() > File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py",=20 > line 332, in _step_do_subscription > self.member =3D self.mlist.subscribe( > File=20 > "/usr/lib/python3.8/site-packages/mailman/database/transaction.py",=20 > line 85, in wrapper > return function(args[0], config.db.store, *args[1:], **kws) > File "/usr/lib/python3.8/site-packages/mailman/model/mailinglist.py",=20 > line 505, in subscribe > notify(SubscriptionEvent( > File "/usr/lib/python3.8/site-packages/zope/event/__init__.py", line=20 > 32, in notify > subscriber(event) > File "/usr/lib/python3.8/site-packages/mailman/app/membership.py",=20 > line 176, in handle_SubscriptionEvent > send_welcome_message(mlist, member, member.preferred_language) > File "/usr/lib/python3.8/site-packages/mailman/app/notifications.py",=20 > line 51, in send_welcome_message > welcome_message =3D wrap(getUtility(ITemplateLoader).get( > File "/usr/lib/python3.8/site-packages/mailman/model/template.py",=20 > line 188, in get > contents =3D getUtility(ITemplateManager).get( > File=20 > "/usr/lib/python3.8/site-packages/mailman/database/transaction.py",=20 > line 85, in wrapper > return function(args[0], config.db.store, *args[1:], **kws) > File "/usr/lib/python3.8/site-packages/mailman/model/template.py",=20 > line 110, in get > contents =3D protocols.get(actual_uri, **auth) > File=20 > "/usr/lib/python3.8/site-packages/mailman/utilities/protocols.py", line=20 > 38, in get > response =3D requests.get(url, timeout=3DREQUEST_TIMEOUT, **kws) > File "/usr/lib/python3.8/site-packages/requests/api.py", line 76, in=20 > get > return request('get', url, params=3Dparams, **kwargs) > File "/usr/lib/python3.8/site-packages/requests/api.py", line 61, in=20 > request > return session.request(method=3Dmethod, url=3Durl, **kwargs) > File "/usr/lib/python3.8/site-packages/requests/sessions.py", line=20 > 530, in request > resp =3D self.send(prep, **send_kwargs) > File "/usr/lib/python3.8/site-packages/requests/sessions.py", line=20 > 643, in send > r =3D adapter.send(request, **kwargs) > File "/usr/lib/python3.8/site-packages/requests/adapters.py", line=20 > 529, in send > raise ReadTimeout(e, request=3Drequest) > requests.exceptions.ReadTimeout: HTTPConnectionPool(host=3D'mailman-web',=20 > port=3D8000): Read timed out. (read timeout=3D5) >=20 >=20 >=20 > What=E2=80=99s the next step to get templates working? Are you using the provided docker-compose file or are you using something els= e like Kubernetes to deploy the container images? It seems like your mailman-core container isn't able to talk to Postorius on = the default dns name, mailman-web, in the provided docker-compose file. These= dns names are setup by default if you are using the docker-compose.yaml prov= ided with the repo, but would otherwise needs either changing the value of PO= STORIUS_TEMPLATES_BASE_URL or manually setting up the mailman-web DNS in the = `/etc/host` of the mailman-core container. > Thank you, > Seth >=20 >=20 > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --=20 thanks, Abhilash Raj (maxking) --===============4035864457729165350==-- From maxking@asynchronous.in Sat Sep 19 19:11:29 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: Can creating too many Django accounts crash Mailman3 server? Date: Sat, 19 Sep 2020 12:11:05 -0700 Message-ID: In-Reply-To: <24416.26817.359634.565707@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2844889024476102676==" --===============2844889024476102676== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Tue, Sep 15, 2020, at 12:09 AM, Stephen J. Turnbull wrote: > Mohsen Masoudfar writes: >=20 > > It seems anybody can go to the site: > > https://LISTSERVNAME/mailman3/postorius/lists/ > > and click on the [Sign up] on the top-right corner and create an > > account. >=20 > Yes. >=20 > > This can easily be automated, even though the next step, confirming > > the email address, being ignored. I believe it can be used as a > > target for a DOS attack by creating so many accounts, that > > eventually causes an 'out of space' error. > >=20 > > Is this a justified concern? >=20 > "Justified" depends on who you think the attacker is. Theoretically > it's a real possibility, but the space used by an unverified account > is not large. I suspect there are always much more painful DoS > attacks to worry about. >=20 > > Is there a way to manage this feature in a secure way? >=20 > In some sense, I don't think there is. The point of the feature is to > allow anybody to sign up without authentication beyond having a usable > email address. Either you allow the DoS attack or anything you do is > likely to deny service to legitimate users. >=20 > Brian's approach (cron job purging unverified users) is generally good > hygiene, but I don't think it would do much against an deliberate > attack. And it shouldn't be hard to automate the confirmation > process. I guess you could handle that by purging accounts with no > subscriptions or other roles on any lists, but that might annoy a few > legitimate users. >=20 > Probably the best you could do for an open server would be to throttle > user creation to say 1 per minute (adjustable to the scale of your > site). >=20 > I don't know if there is currently a way to require admin approval for > account creation (as opposed to list subscription). Perhaps we should > add that if we don't have it. I checked django-allauth, it doesn't have a feature like that. We could imple= ment something on-top of that to optionally allow for admin approval but some other details would need to be figured out and would be a sizeable task involving a= new view for account approvals. We potentially also would have to modify the models used in django-allauth to add the flag for "approved", which I am not sure would be an easy task to fig= ure out.=20 Perhaps, it might be easier to make a feature request and implement it in django-allauth though. >=20 > Steve > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --=20 thanks, Abhilash Raj (maxking) --===============2844889024476102676==-- From maxking@asynchronous.in Sat Sep 19 19:28:07 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: Redundant and load balanced Mailman3 Date: Sat, 19 Sep 2020 12:27:41 -0700 Message-ID: <06cbc194-b440-4f3d-98bc-0a031cdea0b0@www.fastmail.com> In-Reply-To: <160035671427.16961.4228285617823670746@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7025532469297956757==" --===============7025532469297956757== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit On Thu, Sep 17, 2020, at 8:31 AM, Pawel Grzywaczewski wrote: > Hi, > > Can someone share some hints about how to deploy redundant and load > balanced Mailman3? What are your recommendations and best practices? > For example what components to put on the same VM? Can we have 2VM with > Mailman core/Postorious/Hyperkitty/django-mailman/mailmanclient > connecting to one (the same) database? How to load balance mail routing > (incoming)? The current implementation of Mailman is good for vertical scaling but not that great for horizontal scaling and redundancy. The web frontend, Postorius & Hyperkitty, are pretty much stateless and can be horizontally scaled and redundantly deployed easily. I am not going to recommend how to handle redundancy for database, because I am not the best person for that and I honestly don't know anything beyond using a "cloud managed" database from cloud providers for HA. Mailman Core is a bit hard to run redundantly mostly because of it's reliance on filesystem for some data like email queues, held messages, shunted messages etc. You can potentially have two running instances, one on standby and failover to other instance. You'd need to synchronize the var directory between them using *some* mechanism, rsync or NFS or something. Although, you want to make sure that they both aren't sending out emails at the same time. > Thanks! > Paweł > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking) --===============7025532469297956757==-- From maszap69@gmail.com Sat Sep 19 19:33:49 2020 From: Massimo =?utf-8?q?Zappal=C3=A0?= To: mailman-users@mailman3.org Subject: [MM3-users] Re: Bad Signature with Gmail Date: Sat, 19 Sep 2020 19:33:45 +0000 Message-ID: <160054402599.12963.3732650319346930403@mailman.iad1.psf.io> In-Reply-To: <18a7f260-9aaa-d35f-1bb6-3e66ac6df7ec@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8045191267498485560==" --===============8045191267498485560== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Mark, thanks for the reply. my /etc/opendkim.conf AutoRestart Yes AutoRestartRate 10/1h UMask 002 Syslog yes SyslogSuccess Yes LogWhy Yes Canonicalization relaxed/simple ExternalIgnoreList refile:/etc/opendkim/TrustedHosts InternalHosts refile:/etc/opendkim/TrustedHosts KeyTable refile:/etc/opendkim/KeyTable SigningTable refile:/etc/opendkim/SigningTable Mode sv PidFile /var/run/opendkim/opendkim.pid SignatureAlgorithm rsa-sha256 UserID opendkim:opendkim Socket inet:12301(a)localhost my /etc/opendkim/SigningTable *@catania.linux.it mailselector._domainkey.catania.linux.it full email django error Internal Server Error: /hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\xB1\\xF0\\x9F...' for colum= n `mailman3web`.`hyperkitty_email`.`content` at row 1") Request Method: POST Request URL: https://lists.catania.linux.it/hyperkitty/api/mailman/archive?ke= y=3Dmykey Django Version: 1.11.29 Python Executable: /usr/bin/uwsgi-core Python Version: 3.7.3 Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/l= ib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/li= b/python3/dist-packages'] Server time: Sab, 19 Set 2020 20:05:41 +0200 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Traceback: =20 File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in exe= cute 101. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 250. self.errorhandler(self, exc, value) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in defaulterrorh= andler 50. raise errorvalue File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 247. res =3D self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 411. rowcount =3D self._do_query(q) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query 374. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 292. _mysql.connection.query(self, query) =20 =20 The above exception ((1366, "Incorrect string value: '\\xF0\\x9F\\x98\\= xB1\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at ro= w 1")) was the direct cause of the following exception: =20 =20 File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in in= ner 41. response =3D get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_re= sponse 187. response =3D self.process_exception_by_middleware(e, r= equest) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_re= sponse 185. response =3D wrapped_callback(request, *callback_args,= **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py" in inner 40. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in _decorat= or 69. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py" in wrap= ped_view 58. return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in archive 115. add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py" in add_to_li= st 151. email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save 808. force_update=3Dforce_update, update_fields=3Dup= date_fields) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save_base 838. updated =3D self._save_table(raw, cls, force_insert, force= _update, using, update_fields) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _save_table 924. result =3D self._do_insert(cls._base_manager, using, field= s, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _do_insert 963. using=3Dusing, raw=3Draw) File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_= method 85. return getattr(self.get_queryset(), name)(*args, **kwar= gs) File "/usr/lib/python3/dist-packages/django/db/models/query.py" in _insert 1079. return query.get_compiler(using=3Dusing).execute_sql(return_i= d) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py" in exe= cute_sql 1112. cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py" in __exit__ 94. six.reraise(dj_exc_type, dj_exc_value, traceback) File "/usr/lib/python3/dist-packages/django/utils/six.py" in reraise 685. raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in exe= cute 101. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 250. self.errorhandler(self, exc, value) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in defaulterrorh= andler 50. raise errorvalue File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 247. res =3D self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 411. rowcount =3D self._do_query(q) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query 374. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 292. _mysql.connection.query(self, query) Exception Type: OperationalError at /hyperkitty/api/mailman/archive Exception Value: (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\xB1\\xF0\\= x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1") Request information: USER: AnonymousUser GET: key =3D 'mykey' POST: mlist =3D 'lug(a)catania.linux.it' FILES: message =3D COOKIES: No cookie data META: CONTENT_LENGTH =3D '2485' CONTENT_TYPE =3D 'multipart/form-data; boundary=3D3fc2cf34ecf58287e2b148ab43b= 4b111' DOCUMENT_ROOT =3D '/usr/share/nginx/html' HTTPS =3D 'on' HTTP_ACCEPT =3D '*/*' HTTP_ACCEPT_ENCODING =3D 'gzip, deflate' HTTP_CONNECTION =3D 'keep-alive' HTTP_CONTENT_LENGTH =3D '2485' HTTP_CONTENT_TYPE =3D 'multipart/form-data; boundary=3D3fc2cf34ecf58287e2b148= ab43b4b111' HTTP_HOST =3D 'lists.catania.linux.it' HTTP_USER_AGENT =3D 'python-requests/2.21.0' PATH_INFO =3D '/hyperkitty/api/mailman/archive' QUERY_STRING =3D 'key=3Dmykey' REMOTE_ADDR =3D '95.216.191.209' REMOTE_PORT =3D '35022' REQUEST_METHOD =3D 'POST' REQUEST_SCHEME =3D 'https' REQUEST_URI =3D '/hyperkitty/api/mailman/archive?key=3Dmykey' SCRIPT_NAME =3D '' SERVER_NAME =3D 'lists.catania.linux.it' SERVER_PORT =3D '443' SERVER_PROTOCOL =3D 'HTTP/1.1' uwsgi.core =3D 0 uwsgi.node =3D b'glugct.catania.linux.it' uwsgi.version =3D b'2.0.18-debian' wsgi.errors =3D <_io.TextIOWrapper name=3D2 mode=3D'w' encoding=3D'UTF-8'> wsgi.file_wrapper =3D '' wsgi.input =3D wsgi.multiprocess =3D False wsgi.multithread =3D True wsgi.run_once =3D False wsgi.url_scheme =3D 'https' wsgi.version =3D '(1, 0)' Settings: Using settings module settings ABSOLUTE_URL_OVERRIDES =3D {} ACCOUNT_AUTHENTICATION_METHOD =3D 'username_email' ACCOUNT_DEFAULT_HTTP_PROTOCOL =3D 'https' ACCOUNT_EMAIL_REQUIRED =3D True ACCOUNT_EMAIL_VERIFICATION =3D 'mandatory' ACCOUNT_UNIQUE_EMAIL =3D True ADMINS =3D "(('Mailman Suite Admin', 'admin(a)catania.linux.it'),)" ALLOWED_HOSTS =3D ['*'] APPEND_SLASH =3D True AUTHENTICATION_BACKENDS =3D "('django.contrib.auth.backends.ModelBackend', 'a= llauth.account.auth_backends.AuthenticationBackend')" AUTH_PASSWORD_VALIDATORS =3D '********************' AUTH_USER_MODEL =3D 'auth.User' BASE_DIR =3D '/usr/share/mailman3-web' CACHES =3D {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemC= ache'}} CACHE_MIDDLEWARE_ALIAS =3D 'default' CACHE_MIDDLEWARE_KEY_PREFIX =3D '********************' CACHE_MIDDLEWARE_SECONDS =3D 600 COMPRESS_CACHEABLE_PRECOMPILERS =3D '()' COMPRESS_CACHE_BACKEND =3D 'default' COMPRESS_CACHE_KEY_FUNCTION =3D '********************' COMPRESS_CLEAN_CSS_ARGUMENTS =3D '' COMPRESS_CLEAN_CSS_BINARY =3D 'cleancss' COMPRESS_CLOSURE_COMPILER_ARGUMENTS =3D '' COMPRESS_CLOSURE_COMPILER_BINARY =3D 'java -jar compiler.jar' COMPRESS_CSS_COMPRESSOR =3D 'compressor.css.CssCompressor' COMPRESS_CSS_FILTERS =3D ['compressor.filters.css_default.CssAbsoluteFilter'] COMPRESS_CSS_HASHING_METHOD =3D 'mtime' COMPRESS_DATA_URI_MAX_SIZE =3D 1024 COMPRESS_DEBUG_TOGGLE =3D None COMPRESS_ENABLED =3D True COMPRESS_JINJA2_GET_ENVIRONMENT =3D COMPRESS_JS_COMPRESSOR =3D 'compressor.js.JsCompressor' COMPRESS_JS_FILTERS =3D ['compressor.filters.jsmin.JSMinFilter'] COMPRESS_MINT_DELAY =3D 30 COMPRESS_MTIME_DELAY =3D 10 COMPRESS_OFFLINE =3D True COMPRESS_OFFLINE_CONTEXT =3D {'STATIC_URL': '/mailman3/static/'} COMPRESS_OFFLINE_MANIFEST =3D 'manifest.json' COMPRESS_OFFLINE_TIMEOUT =3D 31536000 COMPRESS_OUTPUT_DIR =3D 'CACHE' COMPRESS_PARSER =3D 'compressor.parser.AutoSelectParser' COMPRESS_PRECOMPILERS =3D "(('text/less', 'lessc {infile} {outfile}'), ('text= /x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -= t compressed {infile} {outfile}'))" COMPRESS_REBUILD_TIMEOUT =3D 2592000 COMPRESS_ROOT =3D '/var/lib/mailman3/web/static' COMPRESS_STORAGE =3D 'compressor.storage.CompressorFileStorage' COMPRESS_TEMPLATE_FILTER_CONTEXT =3D {'STATIC_URL': '/mailman3/static/'} COMPRESS_URL =3D '/mailman3/static/' COMPRESS_URL_PLACEHOLDER =3D '/__compressor_url_placeholder__/' COMPRESS_VERBOSE =3D False COMPRESS_YUGLIFY_BINARY =3D 'yuglify' COMPRESS_YUGLIFY_CSS_ARGUMENTS =3D '--terminal' COMPRESS_YUGLIFY_JS_ARGUMENTS =3D '--terminal' COMPRESS_YUI_BINARY =3D 'java -jar yuicompressor.jar' COMPRESS_YUI_CSS_ARGUMENTS =3D '' COMPRESS_YUI_JS_ARGUMENTS =3D '' CSRF_COOKIE_AGE =3D 31449600 CSRF_COOKIE_DOMAIN =3D None CSRF_COOKIE_HTTPONLY =3D False CSRF_COOKIE_NAME =3D 'csrftoken' CSRF_COOKIE_PATH =3D '/' CSRF_COOKIE_SECURE =3D False CSRF_FAILURE_VIEW =3D 'django.views.csrf.csrf_failure' CSRF_HEADER_NAME =3D 'HTTP_X_CSRFTOKEN' CSRF_TRUSTED_ORIGINS =3D [] CSRF_USE_SESSIONS =3D False DATABASES =3D {'default': {'ENGINE': 'django.db.backends.mysql', 'NAME': 'mai= lman3web', 'USER': 'mailman3web', 'PASSWORD': '********************', 'HOST':= 'localhost', 'PORT': '', 'OPTIONS': {'init_command': "SET sql_mode=3D'STRICT= _TRANS_TABLES'"}, 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE= ': 0, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME':= None, 'MIRROR': None}}} DATABASE_ROUTERS =3D [] DATA_UPLOAD_MAX_MEMORY_SIZE =3D 2621440 DATA_UPLOAD_MAX_NUMBER_FIELDS =3D 1000 DATETIME_FORMAT =3D 'N j, Y, P' DATETIME_INPUT_FORMATS =3D ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-= %m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d= /%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%= y %H:%M', '%m/%d/%y'] DATE_FORMAT =3D 'N j, Y' DATE_INPUT_FORMATS =3D ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %= d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B,= %Y'] DEBUG =3D False DEBUG_PROPAGATE_EXCEPTIONS =3D False DECIMAL_SEPARATOR =3D '.' DEFAULT_CHARSET =3D 'utf-8' DEFAULT_CONTENT_TYPE =3D 'text/html' DEFAULT_EXCEPTION_REPORTER_FILTER =3D 'django.views.debug.SafeExceptionReport= erFilter' DEFAULT_FILE_STORAGE =3D 'django.core.files.storage.FileSystemStorage' DEFAULT_FROM_EMAIL =3D 'postorius(a)lists.catania.linux.it' DEFAULT_INDEX_TABLESPACE =3D '' DEFAULT_TABLESPACE =3D '' DISALLOWED_USER_AGENTS =3D [] EMAILNAME =3D 'lists.catania.linux.it' EMAIL_BACKEND =3D 'django.core.mail.backends.smtp.EmailBackend' EMAIL_HOST =3D 'localhost' EMAIL_HOST_PASSWORD =3D '********************' EMAIL_HOST_USER =3D '' EMAIL_PORT =3D 25 EMAIL_SSL_CERTFILE =3D None EMAIL_SSL_KEYFILE =3D '********************' EMAIL_SUBJECT_PREFIX =3D '[Django] ' EMAIL_TIMEOUT =3D None EMAIL_USE_LOCALTIME =3D False EMAIL_USE_SSL =3D False EMAIL_USE_TLS =3D False FILE_CHARSET =3D 'utf-8' FILE_UPLOAD_DIRECTORY_PERMISSIONS =3D None FILE_UPLOAD_HANDLERS =3D ['django.core.files.uploadhandler.MemoryFileUploadHa= ndler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler'] FILE_UPLOAD_MAX_MEMORY_SIZE =3D 2621440 FILE_UPLOAD_PERMISSIONS =3D None FILE_UPLOAD_TEMP_DIR =3D None FILTER_VHOST =3D False FIRST_DAY_OF_WEEK =3D 0 FIXTURE_DIRS =3D [] FORCE_SCRIPT_NAME =3D None FORMAT_MODULE_PATH =3D None FORM_RENDERER =3D 'django.forms.renderers.DjangoTemplates' HAYSTACK_CONNECTIONS =3D {'default': {'ENGINE': 'haystack.backends.whoosh_bac= kend.WhooshEngine', 'PATH': '/var/lib/mailman3/web/fulltext_index'}} HOSTNAME =3D 'localhost.local' IGNORABLE_404_URLS =3D [] INSTALLED_APPS =3D "('hyperkitty', 'postorius', 'django_mailman3', 'django.co= ntrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.c= ontrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.= contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'hay= stack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allau= th.socialaccount')" INTERNAL_IPS =3D [] LANGUAGES =3D [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('= az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Benga= li'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech')= , ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian= '), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en= -gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', '= Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spa= nish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('e= t', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr= ', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), = ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('= hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('ia', 'Interlingua'), ('id', 'I= ndonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'J= apanese'), ('ka', 'Georgian'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kan= nada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv= ', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian')= , ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokm=C3=A5l'), ('ne= ', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic')= , ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazi= lian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), (= 'sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbi= an Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Te= lugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt')= , ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('vi', 'Vietnamese'), ('zh-hans', 'Sim= plified Chinese'), ('zh-hant', 'Traditional Chinese')] LANGUAGES_BIDI =3D ['he', 'ar', 'fa', 'ur'] LANGUAGE_CODE =3D 'it-IT' LANGUAGE_COOKIE_AGE =3D None LANGUAGE_COOKIE_DOMAIN =3D None LANGUAGE_COOKIE_NAME =3D 'django_language' LANGUAGE_COOKIE_PATH =3D '/' LOCALE_PATHS =3D [] LOGGING =3D {'version': 1, 'disable_existing_loggers': False, 'filters': {'re= quire_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers'= : {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'cla= ss': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class'= : 'logging.handlers.RotatingFileHandler', 'filename': '/var/log/mailman3/web/= mailman-web.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.Stre= amHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers= ': ['mail_admins', 'file'], 'level': 'INFO', 'propagate': True}, 'django': {'= handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'hyperkitty': {'han= dlers': ['file'], 'level': 'INFO', 'propagate': True}, 'postorius': {'handler= s': ['file'], 'level': 'INFO', 'propagate': True}}, 'formatters': {'verbose':= {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 's= imple': {'format': '%(levelname)s %(message)s'}}} LOGGING_CONFIG =3D 'logging.config.dictConfig' LOGIN_REDIRECT_URL =3D 'list_index' LOGIN_URL =3D 'account_login' LOGOUT_REDIRECT_URL =3D None LOGOUT_URL =3D 'account_logout' MAILMAN_ARCHIVER_FROM =3D "('127.0.0.1', '::1', '::ffff:127.0.0.1', '95.216.1= 91.209')" MAILMAN_ARCHIVER_KEY =3D '********************' MAILMAN_REST_API_PASS =3D '********************' MAILMAN_REST_API_URL =3D '********************' MAILMAN_REST_API_USER =3D '********************' MANAGERS =3D [] MEDIA_ROOT =3D '' MEDIA_URL =3D '' MESSAGE_STORAGE =3D 'django.contrib.messages.storage.fallback.FallbackStorage' MESSAGE_TAGS =3D {40: 'danger'} MIDDLEWARE =3D "('django.contrib.sessions.middleware.SessionMiddleware', 'dja= ngo.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMidd= leware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.mi= ddleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.Messa= geMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'dja= ngo.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.Time= zoneMiddleware', 'postorius.middleware.PostoriusMiddleware')" MIDDLEWARE_CLASSES =3D ['django.middleware.common.CommonMiddleware', 'django.= middleware.csrf.CsrfViewMiddleware'] MIGRATION_MODULES =3D {} MONTH_DAY_FORMAT =3D 'F j' NUMBER_GROUPING =3D 0 PASSWORD_HASHERS =3D '********************' PASSWORD_RESET_TIMEOUT_DAYS =3D '********************' POSTORIUS_TEMPLATE_BASE_URL =3D 'http://localhost/mailman3/' PREPEND_WWW =3D False Q_CLUSTER =3D {'timeout': 300, 'save_limit': 100, 'orm': 'default', 'poll': 5} ROOT_URLCONF =3D 'urls' SECRET_KEY =3D '********************' SECURE_BROWSER_XSS_FILTER =3D False SECURE_CONTENT_TYPE_NOSNIFF =3D False SECURE_HSTS_INCLUDE_SUBDOMAINS =3D False SECURE_HSTS_PRELOAD =3D False SECURE_HSTS_SECONDS =3D 0 SECURE_PROXY_SSL_HEADER =3D None SECURE_REDIRECT_EXEMPT =3D [] SECURE_SSL_HOST =3D None SECURE_SSL_REDIRECT =3D False SERVER_EMAIL =3D 'root(a)lists.catania.linux.it' SESSION_CACHE_ALIAS =3D 'default' SESSION_COOKIE_AGE =3D 1209600 SESSION_COOKIE_DOMAIN =3D None SESSION_COOKIE_HTTPONLY =3D True SESSION_COOKIE_NAME =3D 'sessionid' SESSION_COOKIE_PATH =3D '/' SESSION_COOKIE_SECURE =3D False SESSION_ENGINE =3D 'django.contrib.sessions.backends.db' SESSION_EXPIRE_AT_BROWSER_CLOSE =3D False SESSION_FILE_PATH =3D None SESSION_SAVE_EVERY_REQUEST =3D False SESSION_SERIALIZER =3D 'django.contrib.sessions.serializers.PickleSerializer' SETTINGS_MODULE =3D 'settings' SHORT_DATETIME_FORMAT =3D 'm/d/Y P' SHORT_DATE_FORMAT =3D 'm/d/Y' SIGNING_BACKEND =3D 'django.core.signing.TimestampSigner' SILENCED_SYSTEM_CHECKS =3D [] SITE_ID =3D 1 SOCIALACCOUNT_PROVIDERS =3D {} STATICFILES_DIRS =3D '()' STATICFILES_FINDERS =3D "('django.contrib.staticfiles.finders.FileSystemFinde= r', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.fi= nders.CompressorFinder')" STATICFILES_STORAGE =3D 'django.contrib.staticfiles.storage.StaticFilesStorag= e' STATIC_ROOT =3D '/var/lib/mailman3/web/static' STATIC_URL =3D '/mailman3/static/' TEMPLATES =3D [{'BACKEND': 'django.template.backends.django.DjangoTemplates',= 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.tem= plate.context_processors.debug', 'django.template.context_processors.i18n', '= django.template.context_processors.media', 'django.template.context_processor= s.static', 'django.template.context_processors.tz', 'django.template.context_= processors.csrf', 'django.template.context_processors.request', 'django.contr= ib.auth.context_processors.auth', 'django.contrib.messages.context_processors= .messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_= processors.common', 'postorius.context_processors.postorius']}}] TEST_NON_SERIALIZED_APPS =3D [] TEST_RUNNER =3D 'django.test.runner.DiscoverRunner' THOUSAND_SEPARATOR =3D ',' TIME_FORMAT =3D 'P' TIME_INPUT_FORMATS =3D ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M'] TIME_ZONE =3D 'Europe/Rome' USE_ETAGS =3D False USE_I18N =3D True USE_L10N =3D True USE_THOUSAND_SEPARATOR =3D False USE_TZ =3D True USE_X_FORWARDED_HOST =3D True USE_X_FORWARDED_PORT =3D False WSGI_APPLICATION =3D 'wsgi.application' X_FRAME_OPTIONS =3D 'SAMEORIGIN' YEAR_MONTH_FORMAT =3D 'F Y' Max --===============8045191267498485560==-- From maxking@asynchronous.in Sat Sep 19 19:59:03 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: Problem with list creation Date: Sat, 19 Sep 2020 12:58:36 -0700 Message-ID: <76158b21-671e-437f-a75b-3727884ba04e@www.fastmail.com> In-Reply-To: <24419.17889.575827.248584@turnbull.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2828716314191051082==" --===============2828716314191051082== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Thu, Sep 17, 2020, at 4:17 AM, Stephen J. Turnbull wrote: > Mark Sapiro writes: > > On 9/16/20 8:25 AM, Pawel Grzywaczewski wrote: >=20 > > > We increased timeout in mailman.cfg >=20 > This looks like the issue in Shashikanth's post, too: > https://lists.mailman3.org/archives/list/mailman-users(a)mailman3.org/messa= ge/2MYQDZMCURRSHLJKF24727FMRKL6RIA4/ > They are also creating more than 1000 lists. >=20 > Do we know why these timeouts are happening? (Search for "list > creation times out" returns no results in either core or Postorius > issues.) This is the first time I am actually seeing this. I tried hammering a local server with default number of workers, 2, and created 5000 lists in a loop and went through pretty smooth. I am probably not replicating the exact situation, I am using the sqlite database (which I presumed would be slow and choke up faster but...it didn't).=20 Creating list isn't one of the slow operations AFAIK, so I am actually surprised that it needs increasing timeout. It could be the interaction with database layer maybe, but I can't be sure without more debugging. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --=20 thanks, Abhilash Raj (maxking) --===============2828716314191051082==-- From mark@msapiro.net Sat Sep 19 20:54:29 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Bad Signature with Gmail Date: Sat, 19 Sep 2020 13:54:18 -0700 Message-ID: <9c993c22-d066-cd63-957a-20dae8a408af@msapiro.net> In-Reply-To: <160054402599.12963.3732650319346930403@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5234355916835334755==" --===============5234355916835334755== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/19/20 12:33 PM, Massimo Zappal=C3=A0 wrote: > Hi Mark, thanks for the reply. >=20 > my /etc/opendkim.conf > AutoRestart Yes > AutoRestartRate 10/1h > UMask 002 > Syslog yes > SyslogSuccess Yes > LogWhy Yes > Canonicalization relaxed/simple > ExternalIgnoreList refile:/etc/opendkim/TrustedHosts > InternalHosts refile:/etc/opendkim/TrustedHosts > KeyTable refile:/etc/opendkim/KeyTable > SigningTable refile:/etc/opendkim/SigningTable > Mode sv > PidFile /var/run/opendkim/opendkim.pid > SignatureAlgorithm rsa-sha256 > UserID opendkim:opendkim > Socket inet:12301(a)localhost >=20 >=20 > my /etc/opendkim/SigningTable > *@catania.linux.it mailselector._domainkey.catania.linux.it Assuminng your list's domain is catania.linux.it, what you need to do to enable DKIM signing of this message is add SenderHeaders List-Post,Sender,From to /etc/opendkim.conf > full email django error > Internal Server Error: /hyperkitty/api/mailman/archive >=20 > OperationalError at /hyperkitty/api/mailman/archive > (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\xB1\\xF0\\x9F...' for col= umn `mailman3web`.`hyperkitty_email`.`content` at row 1") Unfortunately, the rest of the detail doesn't help. What we need to see is the raw message. You should be able to find it in Mailman's var/archives/prototype/ The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5234355916835334755==-- From pgrzywaczewski@gmail.com Sat Sep 19 21:37:21 2020 From: Pawel Grzywaczewski To: mailman-users@mailman3.org Subject: [MM3-users] Re: Redundant and load balanced Mailman3 Date: Sat, 19 Sep 2020 21:37:16 +0000 Message-ID: <160055143694.12976.2911693188613420467@mailman.iad1.psf.io> In-Reply-To: <06cbc194-b440-4f3d-98bc-0a031cdea0b0@www.fastmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3425687380043970953==" --===============3425687380043970953== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Thanks Abhilash,=20 I'm a bit confused about dependency on local files? You mean files that are c= reated when an e-mail is processed? So where is the problem with redundancy? = If this host is gone there will be another Mailman deployment able to deal wi= th the load.=20 What about having 3 VM where all components of Mailman are installed. Those V= Ms connect to 1 database on a remote host. We deploy HAProxy in front of 3VMs= . So as the result all requests (HTTP/SMTP) are distributed to 1 VM (out of 3= ). A given VM handles a request HTTP or SMTP, and if there is a problem (or v= ery high load of SMTP) there are still 2 other VM that can do the job. So per= haps the question is: can we plug 3 Mailman instances into 1 DB? Is Mailman s= oftware developed taking into account that several Mailman instance writes to= the same database at the same time? Cheers, Pawe=C5=82 --===============3425687380043970953==-- From pgrzywaczewski@gmail.com Sat Sep 19 21:39:16 2020 From: Pawel Grzywaczewski To: mailman-users@mailman3.org Subject: [MM3-users] Re: Problem with list creation Date: Sat, 19 Sep 2020 21:39:13 +0000 Message-ID: <160055155336.12978.9963755669397922480@mailman.iad1.psf.io> In-Reply-To: <76158b21-671e-437f-a75b-3727884ba04e@www.fastmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5557295668315892925==" --===============5557295668315892925== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit We use Postgres but DB is on remote host. Cheers, Paweł --===============5557295668315892925==-- From mailinglisten@wbock.de Sat Sep 19 21:47:26 2020 From: Wolfgang Bock To: mailman-users@mailman3.org Subject: [MM3-users] including a template failed Date: Sat, 19 Sep 2020 23:47:15 +0200 Message-ID: <000001d68ece$7161f2a0$5425d7e0$@wbock.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2004101465952232847==" --===============2004101465952232847== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Hi, I created a new mailinglist and got during the test-process the following errormessage in my mailman.log after: ... Sep 19 22:55:08 2020 (1433) ACCEPT: <000001d68ec7$2553e6e0$6ffbb4a0$@domain-01.de> Errors: Sep 19 22:55:09 2020 (1436) Cannot connect to SMTP server localhost on port 25 HTTPSConnectionPool(host='webmail.domain-02.de', port=443): Max retries exceeded with url: /mailman3/postorius/api/templates/list/mylist.lists.domain-03.de/list:member :regular:footer (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known')) ... domain02.de isn't involved in mailman3 either as a user domain nor as a list-domain. After deleting the template mailman3/postorius/api/templates/list/mylist.lists.domain-03.de/list:member: regular:footer the mail to the list is processed without problems: Sep 19 22:56:22 2020 (1436) <000001d68ec7$2553e6e0$6ffbb4a0$@domain-01.de> smtp to mylist(a)list.domain-03.de for 1 recips, completed in 0.21389174461364746 seconds Sep 19 22:56:22 2020 (1436) <000001d68ec7$2553e6e0$6ffbb4a0$@wbock.de> post to mylist(a)list.domain-03.de from user(a)domain-01.de, 1331 bytes Please help to solve this problem? Thanks and regards Wolfgang --===============2004101465952232847==-- From mailinglisten@wbock.de Sat Sep 19 22:05:36 2020 From: Wolfgang Bock To: mailman-users@mailman3.org Subject: [MM3-users] Missing whitespace in postfix_vmap after mailman genaliases Date: Sun, 20 Sep 2020 00:05:25 +0200 Message-ID: <000101d68ed0$fb853080$f28f9180$@wbock.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5301653521343267184==" --===============5301653521343267184== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Hi, after creating a new list I got the following errormessage: mailman aliases -> postmap: warning: /var/lib/mailman3/data/postfix_vmap, line 17: expected format: key whitespace value /var/lib/mailman3/data/postfix_vmap # AUTOMATICALLY GENERATED BY MAILMAN ON 2020-09-19 07:44:34 .... xxxxxxxxxxx(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-bounces(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-bounces(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-confirm(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-confirm(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-join(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-join(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-leave(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-leave(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-owner(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-owner(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-request(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-request(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-subscribe(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-subscribe(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-unsubscribe(a)lists.xxxxxxxxxxxxxxxx.dexxxxxxxxxxx-unsubscribe@xxx xxxxxxxxxx.de The last line in the left part contains 48 chars. I solved the problem by adding a whitespace in the last line and I made a new postmap for postfix_vmap. But it would be better to solve the problem by changing the genaliases process in the next update. Regards Wolfgang --===============5301653521343267184==-- From mark@msapiro.net Sat Sep 19 22:43:23 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Missing whitespace in postfix_vmap after mailman genaliases Date: Sat, 19 Sep 2020 15:43:11 -0700 Message-ID: In-Reply-To: <000101d68ed0$fb853080$f28f9180$@wbock.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7692753160480092053==" --===============7692753160480092053== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/19/20 3:05 PM, Wolfgang Bock via Mailman-users wrote: >=20 > after creating a new list I got the following errormessage: >=20 > mailman aliases -> postmap: warning: /var/lib/mailman3/data/postfix_vmap, > line 17: expected format: key whitespace value >=20 > /var/lib/mailman3/data/postfix_vmap > # AUTOMATICALLY GENERATED BY MAILMAN ON 2020-09-19 07:44:34 > .... > xxxxxxxxxxx(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-bounces(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-bounces(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-confirm(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-confirm(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-join(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-join(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-leave(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-leave(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-owner(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-owner(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-request(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-request(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-subscribe(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-subscribe(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-unsubscribe(a)lists.xxxxxxxxxxxxxxxx.dexxxxxxxxxxx-unsubscribe@= xxx > xxxxxxxxxx.de >=20 > The last line in the left part contains 48 chars. It isn't clear what you really have, but I'm guessing that in each case except the last, what you show as two lines is actually a single line with whitespace between and the last line has no whitespace. This is , fixed in Mailman 3.3.1 by . > I solved the problem by adding a whitespathe last line has no whitespace.ce= in the last line and I made a > new postmap for postfix_vmap. >=20 > But it would be better to solve the problem by changing the genaliases > process in the next update. Already fixed and released in the last update. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============7692753160480092053==-- From mark@msapiro.net Sat Sep 19 23:05:22 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: including a template failed Date: Sat, 19 Sep 2020 16:05:09 -0700 Message-ID: <900c7815-1ca0-4af7-da30-57a00f89c415@msapiro.net> In-Reply-To: <000001d68ece$7161f2a0$5425d7e0$@wbock.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4743818357553646235==" --===============4743818357553646235== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/19/20 2:47 PM, Wolfgang Bock via Mailman-users wrote: > Hi, >=20 > I created a new mailinglist and got during the test-process the following > errormessage in my mailman.log after: > ... > Sep 19 22:55:08 2020 (1433) ACCEPT: > <000001d68ec7$2553e6e0$6ffbb4a0$@domain-01.de> >=20 > Errors: >=20 > Sep 19 22:55:09 2020 (1436) Cannot connect to SMTP server localhost on port > 25 > HTTPSConnectionPool(host=3D'webmail.domain-02.de', port=3D443): Max retries > exceeded with url: > /mailman3/postorius/api/templates/list/mylist.lists.domain-03.de/list:member > :regular:footer (Caused by NewConnectionError(' tion.VerifiedHTTPSConnection object at 0x7fc8380f1748>: Failed to establish > a new connection: [Errno -2] Name or service not known')) > ... This is wierd. It looks like we are trying to establish a connection to host=3D'webmail.domain-02.de', port=3D443 in order to GET /mailman3/postorius/api/templates/list/mylist.lists.domain-03.de/list:member:= regular:footer, but the error says "Cannot connect to SMTP server localhost on port 25". I have no idea how port 25 is coming into it. Can you get https://webmail.domain-02.de/mailman3/postorius/api/templates/list/mylist.lis= ts.domain-03.de/list:member:regular:footer on the Mailman machine? What is your setting for POSTORIUS_TEMPLATE_BASE_URL? --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============4743818357553646235==-- From seth@tofutavern.com Sat Sep 19 23:20:08 2020 From: Seth Seeger To: mailman-users@mailman3.org Subject: [MM3-users] Re: Template path Date: Sat, 19 Sep 2020 19:19:59 -0400 Message-ID: In-Reply-To: <9f4ade1b-a4b3-4bb8-a05f-1953503fa1e4@www.fastmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3084149294016236461==" --===============3084149294016236461== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable > On Sep 19, 2020, at 3:02 PM, Abhilash Raj wrote: >=20 > Are you using the provided docker-compose file or are you using something e= lse like Kubernetes to deploy the container images? I am using the provided docker-compose file. > It seems like your mailman-core container isn't able to talk to Postorius o= n the default dns name, mailman-web, in the provided docker-compose file. The= se dns names are setup by default if you are using the docker-compose.yaml pr= ovided with the repo, but would otherwise needs either changing the value of = POSTORIUS_TEMPLATES_BASE_URL or manually setting up the mailman-web DNS in th= e `/etc/host` of the mailman-core container. Telnet isn=E2=80=99t included in the container, so I tested with curl: $ sudo docker exec -it -u mailman mailman-core bash bash-5.0$ grep mailman-web /etc/hosts bash-5.0$ curl http://mailman-web:8000 bash-5.0$=20 mailman-web does not seem to be in /etc/hosts of mailman-core, but it does re= solve and port 8000 is reachable. (I tested with a bad hostname and a differ= ent port and did get error messages.) Is there a URL I can pull that will sh= ow that it=E2=80=99s working properly? My original (slightly censored) docker-compose.yaml file is below. Thanks, Seth version: '2' services: mailman-core: image: maxking/mailman-core:latest container_name: mailman-core hostname: mailman-core restart: always volumes: - /opt/mailman/core:/opt/mailman/ #stop_grace_period: 30s links: - database:database depends_on: - database environment: - DATABASE_URL=3Dpostgres://xxx:xxx(a)database/mailmandb - DATABASE_TYPE=3Dpostgres - DATABASE_CLASS=3Dmailman.database.postgresql.PostgreSQLDatabase - HYPERKITTY_API_KEY=3Dxxxxxx networks: mailman: ipv4_address: 172.19.199.2 mailman-web: image: maxking/mailman-web:latest container_name: mailman-web hostname: mailman-web restart: always depends_on: - database links: - mailman-core:mailman-core - database:database volumes: - /opt/mailman/web:/opt/mailman-web-data environment: - DATABASE_TYPE=3Dpostgres - DATABASE_URL=3Dpostgres://xxx:xxx(a)database/mailmandb - HYPERKITTY_API_KEY=3Dxxxx - SERVE_FROM_DOMAIN=3Dlists.example.com - DJANGO_ALLOWED_HOSTS=3Dlists.example2.com - MAILMAN_ADMIN_USER=3Dxxx - MAILMAN_ADMIN_EMAIL=3Dxxx(a)xxxx - SECRET_KEY=3Dxxxxxx networks: mailman: ipv4_address: 172.19.199.3 database: environment: POSTGRES_DB: mailmandb POSTGRES_USER: xxx POSTGRES_PASSWORD: xxx restart: always image: postgres:9.6-alpine volumes: - /opt/mailman/database:/var/lib/postgresql/data networks: mailman: ipv4_address: 172.19.199.4 networks: mailman: driver: bridge ipam: driver: default config: - subnet: 172.19.199.0/24 --===============3084149294016236461==-- From maszap69@gmail.com Sat Sep 19 23:57:22 2020 From: Massimo =?utf-8?q?Zappal=C3=A0?= To: mailman-users@mailman3.org Subject: [MM3-users] Re: Bad Signature with Gmail Date: Sat, 19 Sep 2020 23:57:19 +0000 Message-ID: <160055983933.12977.11788127438118881892@mailman.iad1.psf.io> In-Reply-To: <9c993c22-d066-cd63-957a-20dae8a408af@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8845562504421527577==" --===============8845562504421527577== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Mark, For opendkim, I solved as you indicated Add SenderHeaders List-Post,Sender,From to /etc/opendkim.conf For Hyperkitty I solved editing my /etc/mailman3/mailman-web.py and append 'c= harset': 'utf8mb4', DATABASES =3D { 'default': { # Use 'sqlite3', 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle= '. #'ENGINE': 'django.db.backends.sqlite3', #'ENGINE': 'django.db.backends.postgresql_psycopg2', 'ENGINE': 'django.db.backends.mysql', # DB name or path to database file if using sqlite3. 'NAME': 'mailman3web', # The following settings are not used with sqlite3: 'USER': 'mailman3web', 'PASSWORD': 'mypass', # HOST: empty for localhost through domain sockets or '127.0.0.1' for # localhost through TCP. 'HOST': 'localhost', # PORT: set to empty string for default. 'PORT': '', # OPTIONS: Extra parameters to use when connecting to the database. 'OPTIONS': { # Set sql_mode to 'STRICT_TRANS_TABLES' for MySQL. See # https://docs.djangoproject.com/en/1.11/ref/ # databases/#setting-sql-mode 'init_command': "SET sql_mode=3D'STRICT_TRANS_TABLES'", 'charset': 'utf8mb4', }, } } Thanks Max --===============8845562504421527577==-- From turnbull.stephen.fw@u.tsukuba.ac.jp Sun Sep 20 09:24:22 2020 From: "Stephen J. Turnbull" To: mailman-users@mailman3.org Subject: [MM3-users] Re: Redundant and load balanced Mailman3 Date: Sun, 20 Sep 2020 18:24:17 +0900 Message-ID: <24423.8129.84191.85884@turnbull.sk.tsukuba.ac.jp> In-Reply-To: <160055143694.12976.2911693188613420467@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1235551846164175531==" --===============1235551846164175531== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Bottom line up front: 1. Mailman was designed in the context of a single host, and most installations do work that way; we at Mailman don't know a lot about multihost setups and performance yet. This can give you some pretty large installations (eg, mail.python.org is a single host, I believe). Maybe Mark will chime in on this point. 2. You might get better information from postmasters at the largest sites who may have experience with distributed setups. fedora.org might be a good candidate because their people put a ton of work into optimizing HyperKitty for their situation (that's why HyperKitty has all the options for redis caching etc). Pawel Grzywaczewski writes: > I'm a bit confused about dependency on local files? You mean files > that are created when an e-mail is processed? Yes. > So where is the problem with redundancy? If this host is gone there > will be another Mailman deployment able to deal with the load. Where does the file go if the host goes down? If it's local and the host goes down, you lose or delay mail -- that should not happen if your Mailman system is up. That's why Abhilash mentions NFS, which allows another "core" host to take over if a "core" host goes down -- but that's not necessarily a good solution either. (I'm not saying it's *bad*, just that NFS has known problems for *some* applications, and I don't know if Mailman is one.) And of course the NFS server becomes a SPOF (single point of failure). > What about having 3 VM where all components of Mailman are > installed. Those VMs connect to 1 database on a remote host. But they don't connect to 1 database. Each component has its own database, and HyperKitty and Postorius also connects indirectly to core's database via core. They *could* all connect to one database server, but that server becomes a SPOF. > A given VM handles a request HTTP or SMTP, and if there is a > problem (or very high load of SMTP) Mailman doesn't handle SMTP requests, I assume you mean LMTP. I don't think that LMTP handling is very strenuous (could be wrong). The aiosmtp module can handle quite high loads, spending most time waiting for the payload (during which it handles other connections), then it dumps a file to disk, also quite fast. As I understand it, the bottleneck is the database stuff. Mailman does a lot of database accesses to handle a post. > there are still 2 other VM that can do the job. So perhaps the > question is: can we plug 3 Mailman instances into 1 DB? Is Mailman > software developed taking into account that several Mailman > instance writes to the same database at the same time? I don't see why that would be an ACID problem -- the database takes care of that. I do think it would make the database bottleneck worse for performance. Note that we're currently seeing reports of long delays (enough to cause database connection timeouts, so on the order of minutes) in creating lists when there are already a lot of lists. If I had to put money on it, I would bet that there is some procedure iterating over incremental requests in Python that should be doing "one big request" to the database. But I don't know that that's true. I do know that there are some very big sites (such as fedora.org) that use Mailman 3. But they seem to have rather complex setups, with powerful databases and caches to speed up even that. Finally, Mailman 3 was originally designed and tested with a single host doing all the work in mind. The most obvious symptom is the lack of authentication on the REST API. (Of course you can simulate that model with an appropriate firewall and subnetting setup.) --===============1235551846164175531==-- From maxking@asynchronous.in Sun Sep 20 16:15:10 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: Problem with list creation Date: Sun, 20 Sep 2020 09:14:25 -0700 Message-ID: <5f8ec314-4a84-4cc5-8111-99357a3642d1@www.fastmail.com> In-Reply-To: <160055155336.12978.9963755669397922480@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8239626160932729289==" --===============8239626160932729289== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit On Sat, Sep 19, 2020, at 2:39 PM, Pawel Grzywaczewski wrote: > We use Postgres but DB is on remote host. Do you have some sort of instrumentation in the database layer to get the latency for the requests? Also, you mentioned that you are creating lists via the curl command, are you doing in sequence via a loop or parallely via multiple threads at a specific requests/second to the Mailman server? > > Cheers, > Paweł > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking) --===============8239626160932729289==-- From maxking@asynchronous.in Sun Sep 20 16:42:51 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: Template path Date: Sun, 20 Sep 2020 09:42:28 -0700 Message-ID: <4479a99c-a1fa-4e1b-b838-6b53dc4dce0a@www.fastmail.com> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1756740124420186627==" --===============1756740124420186627== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Sat, Sep 19, 2020, at 4:19 PM, Seth Seeger wrote: >=20 >> On Sep 19, 2020, at 3:02 PM, Abhilash Raj wrot= e: >>=20 >> Are you using the provided docker-compose file or are you using something = else like Kubernetes to deploy the container images? >=20 > I am using the provided docker-compose file. >=20 >=20 >> It seems like your mailman-core container isn't able to talk to Postorius = on the default dns name, mailman-web, in the provided docker-compose file. Th= ese dns names are setup by default if you are using the docker-compose.yaml p= rovided with the repo, but would otherwise needs either changing the value of= POSTORIUS_TEMPLATES_BASE_URL or manually setting up the mailman-web DNS in t= he `/etc/host` of the mailman-core container. > Telnet isn=E2=80=99t included in the container, so I tested with curl: >=20 >> $ sudo docker exec -it -u mailman mailman-core bash >> bash-5.0$ grep mailman-web /etc/hosts >> bash-5.0$ curl http://mailman-web:8000 >> bash-5.0$=20 >=20 > mailman-web does not seem to be in /etc/hosts of mailman-core, but it does = resolve and port 8000 is reachable. (I tested with a bad hostname and a diff= erent port and did get error messages.) Is there a URL I can pull that will = show that it=E2=80=99s working properly? Can you run this command (replacing the mailing list's id and name of the tem= plate correctly) from mailman-core container and see if you get the template'= s text as the right response: $ curl -L http://mailman-web:8000/postorius/api/templates/list/users.example.= com/list:user:action:subscribe >=20 > My original (slightly censored) docker-compose.yaml file is below. Yeah, the file looks pretty much same as the one in the repo, nothing jumps o= ut to me. Abhilash >=20 > Thanks, > Seth >=20 >=20 >=20 >=20 > version: '2' >=20 > services: > mailman-core: > image: maxking/mailman-core:latest > container_name: mailman-core > hostname: mailman-core > restart: always > volumes: > - /opt/mailman/core:/opt/mailman/ > #stop_grace_period: 30s > links: > - database:database > depends_on: > - database > environment: > - DATABASE_URL=3Dpostgres://xxx:xxx(a)database/mailmandb > - DATABASE_TYPE=3Dpostgres > - DATABASE_CLASS=3Dmailman.database.postgresql.PostgreSQLDatabase > - HYPERKITTY_API_KEY=3Dxxxxxx > networks: > mailman: > ipv4_address: 172.19.199.2 >=20 > mailman-web: > image: maxking/mailman-web:latest > container_name: mailman-web > hostname: mailman-web > restart: always > depends_on: > - database > links: > - mailman-core:mailman-core > - database:database > volumes: > - /opt/mailman/web:/opt/mailman-web-data > environment: > - DATABASE_TYPE=3Dpostgres > - DATABASE_URL=3Dpostgres://xxx:xxx(a)database/mailmandb > - HYPERKITTY_API_KEY=3Dxxxx > - SERVE_FROM_DOMAIN=3Dlists.example.com > - DJANGO_ALLOWED_HOSTS=3Dlists.example2.com > - MAILMAN_ADMIN_USER=3Dxxx > - MAILMAN_ADMIN_EMAIL=3Dxxx(a)xxxx > - SECRET_KEY=3Dxxxxxx > networks: > mailman: > ipv4_address: 172.19.199.3 >=20 > database: > environment: > POSTGRES_DB: mailmandb > POSTGRES_USER: xxx > POSTGRES_PASSWORD: xxx > restart: always > image: postgres:9.6-alpine > volumes: > - /opt/mailman/database:/var/lib/postgresql/data > networks: > mailman: > ipv4_address: 172.19.199.4 >=20 > networks: > mailman: > driver: bridge > ipam: > driver: default > config: > - > subnet: 172.19.199.0/24 >=20 -- thanks, Abhilash Raj (maxking) =20 --===============1756740124420186627==-- From seth@tofutavern.com Mon Sep 21 11:30:33 2020 From: Seth Seeger To: mailman-users@mailman3.org Subject: [MM3-users] Re: Template path Date: Mon, 21 Sep 2020 07:30:24 -0400 Message-ID: <25A1F722-EAC8-4E1F-9F91-ED8D9A57A6B4@tofutavern.com> In-Reply-To: <4479a99c-a1fa-4e1b-b838-6b53dc4dce0a@www.fastmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2597690362545272954==" --===============2597690362545272954== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable > On Sep 20, 2020, at 12:42 PM, Abhilash Raj wrot= e: >=20 > Can you run this command (replacing the mailing list's id and name of the t= emplate correctly) from mailman-core container and see if you get the templat= e's text as the right response: >=20 > $ curl -L http://mailman-web:8000/postorius/api/templates/list/users.exampl= e.com/list:user:action:subscribe >=20 The correct text does come through. Seth --===============2597690362545272954==-- From r.woithe@callassoftware.com Mon Sep 21 13:21:38 2020 From: r.woithe@callassoftware.com To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Mon, 21 Sep 2020 13:21:35 +0000 Message-ID: <160069449571.12976.2662880236844851975@mailman.iad1.psf.io> In-Reply-To: <60e52d55-6141-109d-416d-6d27275dc519@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0891690184885173165==" --===============0891690184885173165== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, Version: Mailman Core Version GNU Mailman 3.2.2 (La Villa Strangiato) Mailman Core API Version 3.0 Mailman Core Python Version 3.8.2 (default, Jul 16 2020, 14:00:26) [GCC 9.3.= 0] But the Error was there allready before the update have been done. That means= the Mailman-Core changed but the error persisted. mfG --===============0891690184885173165==-- From mmasoudf@aaas.org Mon Sep 21 16:23:49 2020 From: Mohsen Masoudfar To: mailman-users@mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size Date: Mon, 21 Sep 2020 16:23:44 +0000 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0242037474697795933==" --===============0242037474697795933== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Mark, Thank you Mark and Phil for your help. It seems I got stuck! 1- I tried to set up the max_message_size in WUI under: listname > settings >= Message Acceptance > Maximum message size to 0.=20 After hitting save button, I got the error message: " An error occured: Unkno= wn attribute: max_message_size" 2- I checked the python files and found this file, where, it seems, the defau= lt value is defined:=20 /usr/lib/python3/dist-packages/mailman/styles/base.py : mlist.max_mess= age_size =3D 40 I changed this value to 0, saved the file, restarted the mailman and postgres= ql (not sure if needed) and sent an email with a large attachment to my test = list with just two members. I got the response email: The message is being held because: The message is larger than the 40 KB ma= ximum size 3- I totally agree that the most important issue is the fact that the message= vanished with the production list.=20 I repeated the same with a (test) list which has two members, I got the email= mentioning the 40 KB limit and I could release it. This is how I am running = my tests now. I repeated it with the original list (with 197 members) the mes= sage disappeared again. I can send an email to the list with 197 members, wit= hout this attachment and it works. 4- I checked the folders, under /var/lib/mailman3/queue/, all folders are emp= ty: archive/ bad/ bounces/ command/ digest/ in/ nntp/ out/ pipeline/ = retry/ shunt/ virgin/ Thanks Mohsen -----Original Message----- From: Mark Sapiro =20 Sent: Friday, September 18, 2020 4:00 PM To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size [EXTERNAL EMAIL] On 9/18/20 12:48 PM, Phil Stracchino wrote: > On 2020-09-18 15:32, Mohsen Masoudfar wrote: >> Hi, >> >> I am trying to find how I may change this. I have an attachment of 800 KB,= and when I post it to the list, I do not get any response and nothing is hol= d. It looks like the message is just dropped. > > > Assuming that this works roughly the same in mm3 as it does in mm2,=20 > you can set max_message_size in your list configuration. It defaults=20 > to 40; units are KB. Yes, and you can set it to zero for "unlimited", but even if that is a satisf= actory workaround, the much more interesting question, at least to me, is why= does the message "disappear". -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe sen= d an email to mailman-users-leave(a)mailman3.org https://nam12.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists.mai= lman3.org%2Fmailman3%2Flists%2Fmailman-users.mailman3.org%2F&data=3D01%7C= 01%7Cmmasoudf%40aaas.org%7C2c2bd90370a84ffaceb208d85c0d8b5a%7C2eebd8ff9ed140f= 0a15638e5dfb3bc56%7C0&sdata=3DF%2Fw%2FylysGI83aXf2PDV5zCugbxFVV6qvps4hSzG= 4PAM%3D&reserved=3D0 --===============0242037474697795933==-- From mmasoudf@aaas.org Mon Sep 21 22:01:21 2020 From: Mohsen Masoudfar To: mailman-users@mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size Date: Mon, 21 Sep 2020 22:01:16 +0000 Message-ID: In-Reply-To: =?utf-8?q?=3CMN2PR20MB3245FBACB8B1830E808207FEB63A0=40MN2PR20MB?= =?utf-8?q?3245=2Enamprd20=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3817291853221996588==" --===============3817291853221996588== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable It seems something is fundamentally wrong with Hyperkitty: >From /var/log/mailman3/mailman.log --- 49 Sep 21 21:49:25 2020 (24925) HyperKitty failure on http://localhost/mail= man3/hyperkitty/api/mailman/urls: Auth required 50

Authorization Required

(401) 51 Sep 21 21:49:25 2020 (24925) HyperKitty failure on http://localhost/mail= man3/hyperkitty/api/mailman/urls: Auth required 52

Authorization Required

(401) 53 Sep 21 21:49:25 2020 (24925) HyperKitty failure on http://localhost/mail= man3/hyperkitty/api/mailman/urls: Auth required 54

Authorization Required

(401) 55 Sep 21 21:49:26 2020 (24918) HyperKitty failure on http://localhost/mail= man3/hyperkitty/api/mailman/archive: 56 57 503 Service Unavailable 58 59

Service Unavailable

60

The server is temporarily unable to service your 61 request due to maintenance downtime or capacity 62 problems. Please try again later.

63
64
Apache/2.4.29 (Ubuntu) Server at localhost Port 80
65 66 (503) 67 Sep 21 21:49:26 2020 (24918) Exception in the HyperKitty archiver: 68 69 503 Service Unavailable 70 71

Service Unavailable

72

The server is temporarily unable to service your 73 request due to maintenance downtime or capacity 74 problems. Please try again later.

75
76
Apache/2.4.29 (Ubuntu) Server at localhost Port 80
77 78 Sep 21 21:49:26 2020 (24918) Traceback (most recent call last): 79 File "/usr/lib/python3/dist-packages/mailman_hyperkitty/__init__.py", = line 154, in _archive_message 80 url =3D self._send_message(mlist, msg) 81 File "/usr/lib/python3/dist-packages/mailman_hyperkitty/__init__.py", = line 210, in _send_message 82 raise ValueError(result.text) 83 ValueError: 84 85 503 Service Unavailable 86 87

Service Unavailable

88

The server is temporarily unable to service your 89 request due to maintenance downtime or capacity 90 problems. Please try again later.

--- Thanks Mohsen -----Original Message----- From: Mohsen Masoudfar =20 Sent: Monday, September 21, 2020 12:24 PM To: Mark Sapiro ; mailman-users(a)mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size [EXTERNAL EMAIL] Hi Mark, Thank you Mark and Phil for your help. It seems I got stuck! 1- I tried to set up the max_message_size in WUI under: listname > settings >= Message Acceptance > Maximum message size to 0. After hitting save button, I got the error message: " An error occured: Unkno= wn attribute: max_message_size" 2- I checked the python files and found this file, where, it seems, the defau= lt value is defined: /usr/lib/python3/dist-packages/mailman/styles/base.py : mlist.max_mess= age_size =3D 40 I changed this value to 0, saved the file, restarted the mailman and postgres= ql (not sure if needed) and sent an email with a large attachment to my test = list with just two members. I got the response email: The message is being held because: The message is larger than the 40 KB ma= ximum size 3- I totally agree that the most important issue is the fact that the message= vanished with the production list. I repeated the same with a (test) list which has two members, I got the email= mentioning the 40 KB limit and I could release it. This is how I am running = my tests now. I repeated it with the original list (with 197 members) the mes= sage disappeared again. I can send an email to the list with 197 members, wit= hout this attachment and it works. 4- I checked the folders, under /var/lib/mailman3/queue/, all folders are emp= ty: archive/ bad/ bounces/ command/ digest/ in/ nntp/ out/ pipeline/ = retry/ shunt/ virgin/ Thanks Mohsen -----Original Message----- From: Mark Sapiro Sent: Friday, September 18, 2020 4:00 PM To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size [EXTERNAL EMAIL] On 9/18/20 12:48 PM, Phil Stracchino wrote: > On 2020-09-18 15:32, Mohsen Masoudfar wrote: >> Hi, >> >> I am trying to find how I may change this. I have an attachment of 800 KB,= and when I post it to the list, I do not get any response and nothing is hol= d. It looks like the message is just dropped. > > > Assuming that this works roughly the same in mm3 as it does in mm2,=20 > you can set max_message_size in your list configuration. It defaults=20 > to 40; units are KB. Yes, and you can set it to zero for "unlimited", but even if that is a satisf= actory workaround, the much more interesting question, at least to me, is why= does the message "disappear". -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe sen= d an email to mailman-users-leave(a)mailman3.org https://nam12.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists.mai= lman3.org%2Fmailman3%2Flists%2Fmailman-users.mailman3.org%2F&data=3D01%7C= 01%7Cmmasoudf%40aaas.org%7C509263f415924065a53e08d85e4abe6c%7C2eebd8ff9ed140f= 0a15638e5dfb3bc56%7C0&sdata=3DY2h%2BO5%2FCSuEFhCASCgSpv8ILBRxjJhkRxb%2FrE= 8jEPL4%3D&reserved=3D0 _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe sen= d an email to mailman-users-leave(a)mailman3.org https://nam12.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists.mai= lman3.org%2Fmailman3%2Flists%2Fmailman-users.mailman3.org%2F&data=3D01%7C= 01%7Cmmasoudf%40aaas.org%7C509263f415924065a53e08d85e4abe6c%7C2eebd8ff9ed140f= 0a15638e5dfb3bc56%7C0&sdata=3DY2h%2BO5%2FCSuEFhCASCgSpv8ILBRxjJhkRxb%2FrE= 8jEPL4%3D&reserved=3D0 --===============3817291853221996588==-- From peterbw@gmail.com Mon Sep 21 22:32:04 2020 From: peterbw@gmail.com To: mailman-users@mailman3.org Subject: [MM3-users] cc'd e-mail addresses rewritten with @invalid.domain Date: Mon, 21 Sep 2020 17:49:04 +0000 Message-ID: <160071054473.12963.2449939238036528672@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3739710842480757232==" --===============3739710842480757232== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, We've recently noticed an issue where users who are cc'd in a message where a= mailing list is also within the cc list are getting rewritten a user(a)inval= id.domain. This issue looks like it may be related to Outlook and how user names are dis= played specifically. Here is an example: To: test_user(a)gmail.com cc: listserv(a)lists.test.org; Williams, Peter; "Williams, Peter" maps to a user e-mail in the Outlook Global Address book. W= hen the message is sent the list will send the message with the first name an= d last name separated, for example the message will appear: to: test_user(a)gmail.com cc: listserv(a)lists.test.org, Williams(a)invalid.domain, Peter(a)invalid.dom= ain This only seems to happen with users who are cc'd who are also members of the= list that is being cc'd. For example in the scenario above my e-mail that is= cc'd is also a member of listserv(a)lists.test.org. This issue does not occur if the user e-mail address is listed before the mai= ling list in the cc field. So for example, if the cc in the scenario above co= ntained Williams, Peter; listserv(a)lists.test.org; it would route properly. Do you know what may be causing this? Please let me know if there is any addi= tional information I may provide regarding this and I will try to respond as = soon as possible. Thanks and have a nice day! Peter --===============3739710842480757232==-- From mark@msapiro.net Tue Sep 22 00:56:06 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Mon, 21 Sep 2020 17:55:53 -0700 Message-ID: In-Reply-To: <160069449571.12976.2662880236844851975@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5868594169634925234==" --===============5868594169634925234== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/21/20 6:21 AM, r.woithe(a)callassoftware.com wrote: > Hello, > Version: > Mailman Core Version GNU Mailman 3.2.2 (La Villa Strangiato) > Mailman Core API Version 3.0 > Mailman Core Python Version 3.8.2 (default, Jul 16 2020, 14:00:26) [GCC 9.= 3.0] >=20 > But the Error was there allready before the update have been done. That mea= ns the Mailman-Core changed but the error persisted. What is the full traceback from this error? Mailman 3 has not supported Python 2 since 3.0 beta4. It is not clear what is invoking /usr/lib/python2.7/httplib.py, but if it is Mailman core, something is seriously amiss in the way it's installed. Even Postorius and HyperKitty have not supported Python 2 for quit some time. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5868594169634925234==-- From mark@msapiro.net Tue Sep 22 01:41:57 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size Date: Mon, 21 Sep 2020 18:41:46 -0700 Message-ID: <3795ce3d-9ee4-499f-da95-9fb4bd353c84@msapiro.net> In-Reply-To: =?utf-8?q?=3CMN2PR20MB3245FBACB8B1830E808207FEB63A0=40MN2PR20MB?= =?utf-8?q?3245=2Enamprd20=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7415737719443804561==" --===============7415737719443804561== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/21/20 9:23 AM, Mohsen Masoudfar wrote: >=20 > 1- I tried to set up the max_message_size in WUI under: listname > settings= > Message Acceptance > Maximum message size to 0.=20 > After hitting save button, I got the error message: " An error occured: Unk= nown attribute: max_message_size" This is fixed by in Mailman core 3.2.0. Either upgrade Mailman core or set it via `mailman shell` > 2- I checked the python files and found this file, where, it seems, the def= ault value is defined:=20 > /usr/lib/python3/dist-packages/mailman/styles/base.py : mlist.max_me= ssage_size =3D 40 >=20 > I changed this value to 0, saved the file, restarted the mailman and postgr= esql (not sure if needed) and sent an email with a large attachment to my tes= t list with just two members. I got the response email: > The message is being held because: The message is larger than the 40 KB = maximum size Changing that file only affects the default setting for new lists. It doesn't affect any existing list. To change this you can either update Mailman's database to set max_message_size =3D 0 in the entry for the list in the mailinglist table or do something like $ /opt/mailman/mm/bin/mailman shell -l list.example.com Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. The variable 'm' is the list.example.com mailing list >>> m.max_message_size 40 >>> m.max_message_size =3D 0 >>> commit() > 3- I totally agree that the most important issue is the fact that the messa= ge vanished with the production list.=20 > I repeated the same with a (test) list which has two members, I got the ema= il mentioning the 40 KB limit and I could release it. This is how I am runnin= g my tests now. I repeated it with the original list (with 197 members) the m= essage disappeared again. I can send an email to the list with 197 members, w= ithout this attachment and it works. >=20 > 4- I checked the folders, under /var/lib/mailman3/queue/, all folders are e= mpty: archive/ bad/ bounces/ command/ digest/ in/ nntp/ out/ pipeline= / retry/ shunt/ virgin/ OK, but what if anything is in Mailman's mailman.log? --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============7415737719443804561==-- From mark@msapiro.net Tue Sep 22 01:50:39 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size Date: Mon, 21 Sep 2020 18:50:31 -0700 Message-ID: In-Reply-To: =?utf-8?q?=3CMN2PR20MB3245B5F4507E31DF00516A54B63A0=40MN2PR20MB?= =?utf-8?q?3245=2Enamprd20=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8603261036879243647==" --===============8603261036879243647== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/21/20 3:01 PM, Mohsen Masoudfar wrote: > It seems something is fundamentally wrong with Hyperkitty: >=20 >>From /var/log/mailman3/mailman.log >=20 > --- > 49 Sep 21 21:49:25 2020 (24925) HyperKitty failure on http://localhost/ma= ilman3/hyperkitty/api/mailman/urls: Auth required > 50

Authorization Required

(401) > 51 Sep 21 21:49:25 2020 (24925) HyperKitty failure on http://localhost/ma= ilman3/hyperkitty/api/mailman/urls: Auth required > 52

Authorization Required

(401) > 53 Sep 21 21:49:25 2020 (24925) HyperKitty failure on http://localhost/ma= ilman3/hyperkitty/api/mailman/urls: Auth required > 54

Authorization Required

(401) > 55 Sep 21 21:49:26 2020 (24918) HyperKitty failure on http://localhost/ma= ilman3/hyperkitty/api/mailman/archive: ... I doubt this has anything to do with the missing approved message, but what's in you mailman-hyperkitty.cfg file for api_key and does that match your settings_local.py setting for MAILMAN_ARCHIVER_KEY except for being quoted in settings_local.py and not in mailman-hyperkitty.cfg. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============8603261036879243647==-- From mark@msapiro.net Tue Sep 22 03:59:29 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: cc'd e-mail addresses rewritten with @invalid.domain Date: Mon, 21 Sep 2020 20:59:18 -0700 Message-ID: In-Reply-To: <160071054473.12963.2449939238036528672@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0856894847246867111==" --===============0856894847246867111== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/21/20 10:49 AM, peterbw(a)gmail.com wrote: > Hi, >=20 > We've recently noticed an issue where users who are cc'd in a message where= a mailing list is also within the cc list are getting rewritten a user(a)inv= alid.domain. >=20 > This issue looks like it may be related to Outlook and how user names are d= isplayed specifically. Here is an example: >=20 > To: test_user(a)gmail.com > cc: listserv(a)lists.test.org; Williams, Peter; >=20 > "Williams, Peter" maps to a user e-mail in the Outlook Global Address book.= When the message is sent the list will send the message with the first name = and last name separated, for example the message will appear: >=20 > to: test_user(a)gmail.com > cc: listserv(a)lists.test.org, Williams(a)invalid.domain, Peter(a)invalid.d= omain Because apparently Outlook is sending the message with the literal > Cc: listserv(a)lists.test.org; Williams, Peter; instead of something like > Cc: listserv(a)lists.test.org, "Williams, Peter" Then some MTA in the chain is seeing the unqualified addresses Williams and Peter and appending its own local domain. > This only seems to happen with users who are cc'd who are also members of t= he list that is being cc'd. For example in the scenario above my e-mail that = is cc'd is also a member of listserv(a)lists.test.org. In this case, where do you see this? i.e., do other list members see it in the message they receive from the list? I don't know why that would be. I'm certain that Mailman is not involved in this and the Williams(a)invalid.domain and Peter(a)invalid.domain addresses are in the Cc: as received by Mailman. > This issue does not occur if the user e-mail address is listed before the m= ailing list in the cc field. So for example, if the cc in the scenario above = contained Williams, Peter; listserv(a)lists.test.org; it would route properly. Again, I think this is Outlook. What's in the Cc: of the mail that test_user(a)gmail.com receives directly? What happens if you send mail in this way with To: test_user(a)gmail.com cc: other_user(a)gmail.com; Williams, Peter; Again, what's in the Cc: of the mail that test_user(a)gmail.com and other_user(a)gmail.com receive? Also, please see . --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============0856894847246867111==-- From r.woithe@callassoftware.com Tue Sep 22 07:21:02 2020 From: r.woithe@callassoftware.com To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Tue, 22 Sep 2020 07:20:57 +0000 Message-ID: <160075925765.15206.16893573586727225587@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3593403647343673965==" --===============3593403647343673965== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, this is the complete email we get every night: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django_extensions/management/commands/= runjobs.py", line 36, in runjobs job().execute() File "/usr/lib/python3/dist-packages/hyperkitty/jobs/sync_mailman.py", line= 36, in execute sync_with_mailman() File "/usr/lib/python3/dist-packages/hyperkitty/lib/mailman.py", line 146, = in sync_with_mailman sender.set_mailman_id() File "/usr/lib/python3/dist-packages/hyperkitty/models/sender.py", line 55,= in set_mailman_id mm_user =3D client.get_user(self.address) File "/usr/lib/python3/dist-packages/mailmanclient/client.py", line 320, in= get_user response, content =3D self._connection.call( File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py",= line 95, in call response, content =3D Http().request(url, method, data_str, headers) File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1948, in r= equest (response, content) =3D self._request( File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1621, in _= request (response, content) =3D self._conn_request( File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1560, in _= conn_request response =3D conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1322, in getresponse raise ResponseNotReady(self.__state) http.client.ResponseNotReady: Request-started ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty) START TRACEBACK: END TRACEBACK --===============3593403647343673965==-- From pgrzywaczewski@gmail.com Tue Sep 22 09:30:43 2020 From: Pawel Grzywaczewski To: mailman-users@mailman3.org Subject: [MM3-users] Re: Problem with list creation Date: Tue, 22 Sep 2020 09:30:40 +0000 Message-ID: <160076704096.15207.5298507319316696044@mailman.iad1.psf.io> In-Reply-To: <5f8ec314-4a84-4cc5-8111-99357a3642d1@www.fastmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4141784189976520960==" --===============4141784189976520960== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Not sure about getting latency for the requests. I will ask. It's a simple loop (sequence). Cheers, Paweł --===============4141784189976520960==-- From shashikanth.komandoor@gmail.com Tue Sep 22 17:00:12 2020 From: Shashikanth Komandoor To: mailman-users@mailman3.org Subject: [MM3-users] Re: My Server is unable to connect to Mailman REST API few times Date: Tue, 22 Sep 2020 22:29:55 +0530 Message-ID: In-Reply-To: <384941ed-23d7-8d47-ed33-ec1cf044305e@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0816864849875960755==" --===============0816864849875960755== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Thank you Mark a lot. It worked awesome. You don't know how much you helped me. Thank you once again. Virus-free. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Sat, Sep 19, 2020 at 8:56 PM Mark Sapiro wrote: > On 9/19/20 4:32 AM, Shashikanth Komandoor wrote: > > > > But I am trying to delete a list on an urgent basis. Because of which I > am > > getting the above said errors. I tried to delete > > through the interface, through curl command and through "mailman shell" > but > > nothing led me to a > > positive result. Is there any work around to delete the list ? > > What have you tried in Mailman shell? This should do it. > > $ mailman shell -l list.example.com > Welcome to the GNU Mailman shell > Use commit() to commit changes. > Use abort() to discard changes since the last commit. > Exit with ctrl+D does an implicit commit() but exit() does not. > The variable 'm' is the list.example.com mailing list > >>> getUtility(IListManager).delete(m) > >>> commit() > > So should just > > $ mailman remove list.example.com > > What happens if you do those? > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --=20 Thanks & Regards, Shashi Kanth.K 9052671936 --===============0816864849875960755==-- From mark@msapiro.net Tue Sep 22 17:38:46 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Tue, 22 Sep 2020 10:38:34 -0700 Message-ID: In-Reply-To: <160075925765.15206.16893573586727225587@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7879647053017380414==" --===============7879647053017380414== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/22/20 12:20 AM, r.woithe(a)callassoftware.com wrote: > Hello, > this is the complete email we get every night: >=20 > Traceback (most recent call last): > File "/usr/lib/python3/dist-packages/django_extensions/management/command= s/runjobs.py", line 36, in runjobs > job().execute() > File "/usr/lib/python3/dist-packages/hyperkitty/jobs/sync_mailman.py", li= ne 36, in execute > sync_with_mailman() > File "/usr/lib/python3/dist-packages/hyperkitty/lib/mailman.py", line 146= , in sync_with_mailman > sender.set_mailman_id() > File "/usr/lib/python3/dist-packages/hyperkitty/models/sender.py", line 5= 5, in set_mailman_id > mm_user =3D client.get_user(self.address) > File "/usr/lib/python3/dist-packages/mailmanclient/client.py", line 320, = in get_user > response, content =3D self._connection.call( > File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py= ", line 95, in call > response, content =3D Http().request(url, method, data_str, headers) > File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1948, in= request > (response, content) =3D self._request( > File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1621, in= _request > (response, content) =3D self._conn_request( > File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1560, in= _conn_request > response =3D conn.getresponse() > File "/usr/lib/python3.8/http/client.py", line 1322, in getresponse > raise ResponseNotReady(self.__state) > http.client.ResponseNotReady: Request-started > ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty) > START TRACEBACK: > END TRACEBACK OK. So python2.7 in the Subject: is not relevant. The issue is probably a timeout in mailmanclient's REST request. See for info on how to increase it. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============7879647053017380414==-- From mmasoudf@aaas.org Tue Sep 22 18:17:01 2020 From: Mohsen Masoudfar To: mailman-users@mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size Date: Tue, 22 Sep 2020 18:16:55 +0000 Message-ID: In-Reply-To: <3795ce3d-9ee4-499f-da95-9fb4bd353c84@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2649705270178125786==" --===============2649705270178125786== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Thank you so much Mark. I appreciate your help! I dropped the list and recreated it, the default value worked for the new lis= t. Using Mailman shell is very helpful. I checked the mailman.log and found the = issue.=20 Thanks Mohsen -----Original Message----- From: Mark Sapiro =20 Sent: Monday, September 21, 2020 9:42 PM To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size [EXTERNAL EMAIL] On 9/21/20 9:23 AM, Mohsen Masoudfar wrote: > > 1- I tried to set up the max_message_size in WUI under: listname > settings= > Message Acceptance > Maximum message size to 0. > After hitting save button, I got the error message: " An error occured: Unk= nown attribute: max_message_size" This is fixed by in Mailman core 3.2.0. Either upgrade Mailman core or set it via `mailman shell` > 2- I checked the python files and found this file, where, it seems, the def= ault value is defined: > /usr/lib/python3/dist-packages/mailman/styles/base.py : mlist.max_me= ssage_size =3D 40 > > I changed this value to 0, saved the file, restarted the mailman and postgr= esql (not sure if needed) and sent an email with a large attachment to my tes= t list with just two members. I got the response email: > The message is being held because: The message is larger than the 40 KB = maximum size Changing that file only affects the default setting for new lists. It doesn't= affect any existing list. To change this you can either update Mailman's database to set max_message_si= ze =3D 0 in the entry for the list in the mailinglist table or do something l= ike $ /opt/mailman/mm/bin/mailman shell -l list.example.com Welcome to the GNU Ma= ilman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. The variable 'm' is the list.example.com mailing list >>> m.max_message_size 40 >>> m.max_message_size =3D 0 >>> commit() > 3- I totally agree that the most important issue is the fact that the messa= ge vanished with the production list. > I repeated the same with a (test) list which has two members, I got the ema= il mentioning the 40 KB limit and I could release it. This is how I am runnin= g my tests now. I repeated it with the original list (with 197 members) the m= essage disappeared again. I can send an email to the list with 197 members, w= ithout this attachment and it works. > > 4- I checked the folders, under /var/lib/mailman3/queue/, all folders=20 > are empty: archive/ bad/ bounces/ command/ digest/ in/ nntp/ =20 > out/ pipeline/ retry/ shunt/ virgin/ OK, but what if anything is in Mailman's mailman.log? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe sen= d an email to mailman-users-leave(a)mailman3.org https://nam12.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists.mai= lman3.org%2Fmailman3%2Flists%2Fmailman-users.mailman3.org%2F&data=3D01%7C= 01%7Cmmasoudf%40aaas.org%7Cb4f23b106bf343d57ac908d85e98c830%7C2eebd8ff9ed140f= 0a15638e5dfb3bc56%7C0&sdata=3DKd6yoUjxlJJhLjF7Lsz42KC7jSWDTxauuQJYFdilYYg= %3D&reserved=3D0 --===============2649705270178125786==-- From mark@msapiro.net Tue Sep 22 18:52:01 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: The message is larger than the 40 KB maximum size Date: Tue, 22 Sep 2020 11:51:51 -0700 Message-ID: <531ba2a5-c32f-5b37-f8ab-6fd99a4459cb@msapiro.net> In-Reply-To: =?utf-8?q?=3CMN2PR20MB32458464891043ECF482748DB63B0=40MN2PR20MB?= =?utf-8?q?3245=2Enamprd20=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2968727809491272430==" --===============2968727809491272430== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/22/20 11:16 AM, Mohsen Masoudfar wrote: > I checked the mailman.log and found the issue. I assume you mean why the message disappeared after approval. If so, what was the issue? I.e., was it something unique to your installation such as permissions or configuration, or was it something we need to fix? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============2968727809491272430==-- From seth@tofutavern.com Tue Sep 22 19:45:11 2020 From: Seth Seeger To: mailman-users@mailman3.org Subject: [MM3-users] API timeout for one list Date: Tue, 22 Sep 2020 15:45:04 -0400 Message-ID: <3CCA0DA1-C731-4D70-9E24-4B47A5A59AFF@tofutavern.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6855683426339088672==" --===============6855683426339088672== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable I apologize, because I think this was talked about before, but I could not fi= nd it in the archives. For one list, the REST API times out. Doesn=E2=80=99t seem to matter which p= age I try to go to. The uwsgi-error.log entry: --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 96, in inn= er return func(*args, **kwargs) psycopg2.ProgrammingError: no results to fetch The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 332, in p= usher task_set =3D broker.dequeue() File "/usr/lib/python3.8/site-packages/django_q/brokers/orm.py", line 67, i= n dequeue if tasks: File "/usr/lib/python3.8/site-packages/django/db/models/query.py", line 278= , in __bool__ self._fetch_all() File "/usr/lib/python3.8/site-packages/django/db/models/query.py", line 124= 2, in _fetch_all self._result_cache =3D list(self._iterable_class(self)) File "/usr/lib/python3.8/site-packages/django/db/models/query.py", line 55,= in __iter__ results =3D compiler.execute_sql(chunked_fetch=3Dself.chunked_fetch, chun= k_size=3Dself.chunk_size) File "/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py", l= ine 1175, in execute_sql return list(result) File "/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py", l= ine 1554, in cursor_iter for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel): File "/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py", l= ine 1554, in for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel): File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 96, in inn= er return func(*args, **kwargs) File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 89, in __e= xit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 96, in inn= er return func(*args, **kwargs) django.db.utils.ProgrammingError: no results to fetch During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.8/logging/__init__.py", line 1081, in emit msg =3D self.format(record) File "/usr/lib/python3.8/logging/__init__.py", line 925, in format return fmt.format(record) File "/usr/lib/python3.8/logging/__init__.py", line 664, in format record.message =3D record.getMessage() File "/usr/lib/python3.8/logging/__init__.py", line 369, in getMessage msg =3D msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "./manage.py", line 10, in execute_from_command_line(sys.argv) File "/usr/lib/python3.8/site-packages/django/core/management/__init__.py",= line 381, in execute_from_command_line utility.execute() File "/usr/lib/python3.8/site-packages/django/core/management/__init__.py",= line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/usr/lib/python3.8/site-packages/django/core/management/base.py", lin= e 323, in run_from_argv self.execute(*args, **cmd_options) File "/usr/lib/python3.8/site-packages/django/core/management/base.py", lin= e 364, in execute output =3D self.handle(*args, **options) File "/usr/lib/python3.8/site-packages/django_q/management/commands/qcluste= r.py", line 22, in handle q.start() File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 67, in st= art self.sentinel.start() File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen =3D self._Popen(self) File "/usr/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init= __ self._launch(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch code =3D process_obj._bootstrap(parent_sentinel=3Dchild_r) File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstr= ap self.run() File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 157, in _= _init__ self.start() File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 162, in s= tart self.guard() File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 268, in g= uard self.reincarnate(self.pusher) File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 210, in r= eincarnate self.pusher =3D self.spawn_pusher() File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 190, in s= pawn_pusher return self.spawn_process(pusher, self.task_queue, self.event_out, self.b= roker) File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 186, in s= pawn_process p.start() File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen =3D self._Popen(self) File "/usr/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init= __ self._launch(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch code =3D process_obj._bootstrap(parent_sentinel=3Dchild_r) File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstr= ap self.run() File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 334, in p= usher logger.error(e, traceback.format_exc()) Message: ProgrammingError('no results to fetch') Arguments: ('Traceback (most recent call last):\n File "/usr/lib/python3.8/s= ite-packages/django/db/utils.py", line 96, in inner\n return func(*args, *= *kwargs)\npsycopg2.ProgrammingError: no results to fetch\n \nThe above exception was the direct cause of the following exception:\n\nTra= ceback (most recent call last):\n File "/usr/lib/python3.8/site-packages/dja= ngo_q/cluster.py", line 332, in pusher\n task_set =3D bro ker.dequeue()\n File "/usr/lib/python3.8/site-packages/django_q/brokers/orm.= py", line 67, in dequeue\n if tasks:\n File "/usr/lib/python3.8/site-pack= ages/django/db/models/query.py", line 278, in __bool__\n self._fetch_all()\n File "/usr/lib/python3.8/site-packages/django/db/model= s/query.py", line 1242, in _fetch_all\n self._result_cache =3D list(self._= iterable_class(self))\n File "/usr/lib/python3.8/site-packa ges/django/db/models/query.py", line 55, in __iter__\n results =3D compile= r.execute_sql(chunked_fetch=3Dself.chunked_fetch, chunk_size=3Dself.chunk_siz= e)\n File "/usr/lib/python3.8/site-packages/django/db/models/sq l/compiler.py", line 1175, in execute_sql\n return list(result)\n File "/= usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1554,= in cursor_iter\n for rows in iter((lambda: cursor.fetc hmany(itersize)), sentinel):\n File "/usr/lib/python3.8/site-packages/django= /db/models/sql/compiler.py", line 1554, in \n for rows in iter((la= mbda: cursor.fetchmany(itersize)), sentinel):\n File "/us r/lib/python3.8/site-packages/django/db/utils.py", line 96, in inner\n ret= urn func(*args, **kwargs)\n File "/usr/lib/python3.8/site-packages/django/db= /utils.py", line 89, in __exit__\n raise dj_exc_value.w ith_traceback(traceback) from exc_value\n File "/usr/lib/python3.8/site-pack= ages/django/db/utils.py", line 96, in inner\n return func(*args, **kwargs)= \ndjango.db.utils.ProgrammingError: no results to fetch\n' ,) The other lists work fine. Thank you, Seth --===============6855683426339088672==-- From mark@msapiro.net Tue Sep 22 20:31:00 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: API timeout for one list Date: Tue, 22 Sep 2020 13:30:50 -0700 Message-ID: In-Reply-To: <3CCA0DA1-C731-4D70-9E24-4B47A5A59AFF@tofutavern.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3483232993516441040==" --===============3483232993516441040== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/22/20 12:45 PM, Seth Seeger wrote: > I apologize, because I think this was talked about before, but I could not = find it in the archives. >=20 > For one list, the REST API times out. Doesn=E2=80=99t seem to matter which= page I try to go to. The uwsgi-error.log entry: >=20 ... > The other lists work fine. This can happen on lists with lots of members. Try increasing the timeout. See --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============3483232993516441040==-- From r.woithe@callassoftware.com Wed Sep 23 11:33:06 2020 From: r.woithe@callassoftware.com To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Wed, 23 Sep 2020 11:33:02 +0000 Message-ID: <160086078219.5141.5244210221451664323@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3300849183158228256==" --===============3300849183158228256== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Thank u for the hint ! The only one is that Ubuntu uses uWSGI and not gunicorn. I set the para socket-timeout = XXX. I hope thats the right parameter. --===============3300849183158228256==-- From shashikanth.komandoor@gmail.com Wed Sep 23 13:32:28 2020 From: Shashikanth Komandoor To: mailman-users@mailman3.org Subject: [MM3-users] Re: My Server is unable to connect to Mailman REST API few times Date: Wed, 23 Sep 2020 19:02:11 +0530 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============9090579635174678776==" --===============9090579635174678776== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Dear Mark, As I already said, your suggestion worked well on one list. But when I tried to implement the same trick on another list, I found the below error messages in the mailman.log. > * Sep 23 18:31:31 2020 (22951) Uncaught runner exception: [Errno > 2] No such file or directory: > '/var/lib/mailman/mailman/var/lists/press.dpr-pb.lsmgr.nic.in/digest.mmdf > '* > > > *Sep 23 18:31:31 2020 (22951) Traceback (most recent call last):* > * File "/usr/lib64/python3.6/mailbox.py", line 581, in __init__* > * f =3D open(self._path, 'rb+')* > *FileNotFoundError: [Errno 2] No such file or directory: > '/var/lib/mailman/mailman/var/lists/press.dpr-pb.lsmgr.nic.in/digest.mmdf > '* > *During handling of the above exception, another exception occurred:* > *Traceback (most recent call last):* > * File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 173, > in _one_iteration* > * self._process_one_file(msg, msgdata)* > * File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 266, > in _process_one_file* > * keepqueued =3D self._dispose(mlist, msg, msgdata)* > * File "/var/lib/mailman/mailman/src/mailman/runners/pipeline.py", line > 37, in _dispose* > * process(mlist, msg, msgdata, pipeline)* > * File "/var/lib/mailman/mailman/src/mailman/core/pipelines.py", line 50, > in process* > * handler.process(mlist, msg, msgdata)* > * File "/var/lib/mailman/mailman/src/mailman/handlers/to_digest.py", line > 47, in process* > * with Mailbox(mailbox_path, create=3DTrue) as mbox:* > * File "/usr/lib64/python3.6/mailbox.py", line 893, in __init__* > * _mboxMMDF.__init__(self, path, factory, create)* > * File "/usr/lib64/python3.6/mailbox.py", line 585, in __init__* > * f =3D open(self._path, 'wb+')**FileNotFoundError: [Errno 2] No such > file or directory: > '/var/lib/mailman/mailman/var/lists/press.dpr-pb.lsmgr.nic.in/digest.mmdf > '* > > I ran in both the ways you suggested i.e. through "mailman shell" and "mailman remove". But no luck. Could you please suggest me at this point ? Virus-free. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Tue, Sep 22, 2020 at 10:29 PM Shashikanth Komandoor < shashikanth.komandoor(a)gmail.com> wrote: > > Thank you Mark a lot. > > It worked awesome. You don't know how much you helped me. > > Thank you once again. > > > Virus-free. > www.avast.com > > <#m_8502129290396208734_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > > On Sat, Sep 19, 2020 at 8:56 PM Mark Sapiro wrote: > >> On 9/19/20 4:32 AM, Shashikanth Komandoor wrote: >> > >> > But I am trying to delete a list on an urgent basis. Because of which I >> am >> > getting the above said errors. I tried to delete >> > through the interface, through curl command and through "mailman shell" >> but >> > nothing led me to a >> > positive result. Is there any work around to delete the list ? >> >> What have you tried in Mailman shell? This should do it. >> >> $ mailman shell -l list.example.com >> Welcome to the GNU Mailman shell >> Use commit() to commit changes. >> Use abort() to discard changes since the last commit. >> Exit with ctrl+D does an implicit commit() but exit() does not. >> The variable 'm' is the list.example.com mailing list >> >>> getUtility(IListManager).delete(m) >> >>> commit() >> >> So should just >> >> $ mailman remove list.example.com >> >> What happens if you do those? >> >> -- >> Mark Sapiro The highway is for gamblers, >> San Francisco Bay Area, California better use your sense - B. Dylan >> _______________________________________________ >> Mailman-users mailing list -- mailman-users(a)mailman3.org >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> > > > -- > Thanks & Regards, > Shashi Kanth.K > 9052671936 > > > > --=20 Thanks & Regards, Shashi Kanth.K 9052671936 --===============9090579635174678776==-- From seth@tofutavern.com Wed Sep 23 13:33:23 2020 From: Seth Seeger To: mailman-users@mailman3.org Subject: [MM3-users] Re: API timeout for one list Date: Wed, 23 Sep 2020 09:33:16 -0400 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4415529521505147263==" --===============4415529521505147263== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable > On Sep 22, 2020, at 4:30 PM, Mark Sapiro wrote: >=20 > On 9/22/20 12:45 PM, Seth Seeger wrote: >> I apologize, because I think this was talked about before, but I could not= find it in the archives. >>=20 >> For one list, the REST API times out. Doesn=E2=80=99t seem to matter whic= h page I try to go to. The uwsgi-error.log entry: >>=20 > ... >> The other lists work fine. >=20 > This can happen on lists with lots of members. >=20 > Try increasing the timeout. See Thank you - that did solve my timeout problem. =20 However, each page for this list takes between 40 and 80 seconds to load. Th= e list has 465 members, and it=E2=80=99s running (via Docker) on a 2 core, 4 = GB RAM Linode VM. I=E2=80=99ve watch htop during the load - one CPU is arou= nd 20%, and the other hovers around 80%, mostly with the process =E2=80=9Crun= ner=E2=80=9D. Is there a way to optimize this list? There are no held messages, no subscri= ption requests, no templates. The next biggest list has 438 members, and all those pages load very quickly = (0-2 seconds). I can=E2=80=99t figure out what the difference between the tw= o lists. Thanks, Seth --===============4415529521505147263==-- From brian_carpenter@emwd.com Wed Sep 23 14:02:33 2020 From: Brian Carpenter To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Wed, 23 Sep 2020 10:02:29 -0400 Message-ID: <2b7d65b2-ea38-be3c-d309-fdd25001dd14@emwd.com> In-Reply-To: <160086078219.5141.5244210221451664323@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7917640330302352819==" --===============7917640330302352819== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/23/20 7:33 AM, r.woithe(a)callassoftware.com wrote: > Thank u for the hint ! > The only one is that Ubuntu uses uWSGI and not gunicorn. > I set the para socket-timeout = XXX. > I hope thats the right parameter. > _________________________________________ I think Mailman 3 core still has its own gunicorn process that it runs. -- Brian Carpenter Harmonylists.com Emwd.com --===============7917640330302352819==-- From maxking@asynchronous.in Wed Sep 23 15:46:25 2020 From: Abhilash Raj To: mailman-users@mailman3.org Subject: [MM3-users] Re: API timeout for one list Date: Wed, 23 Sep 2020 08:45:54 -0700 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2082363955002908417==" --===============2082363955002908417== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Wed, Sep 23, 2020, at 6:33 AM, Seth Seeger wrote: >=20 > > On Sep 22, 2020, at 4:30 PM, Mark Sapiro wrote: > >=20 > > On 9/22/20 12:45 PM, Seth Seeger wrote: > >> I apologize, because I think this was talked about before, but I could n= ot find it in the archives. > >>=20 > >> For one list, the REST API times out. Doesn=E2=80=99t seem to matter wh= ich page I try to go to. The uwsgi-error.log entry: > >>=20 > > ... > >> The other lists work fine. > >=20 > > This can happen on lists with lots of members. > >=20 > > Try increasing the timeout. See >=20 > Thank you - that did solve my timeout problem. =20 >=20 > However, each page for this list takes between 40 and 80 seconds to=20 > load. The list has 465 members, and it=E2=80=99s running (via Docker) on a= 2=20 > core, 4 GB RAM Linode VM. I=E2=80=99ve watch htop during the load - one C= PU=20 > is around 20%, and the other hovers around 80%, mostly with the process=20 > =E2=80=9Crunner=E2=80=9D. >=20 > Is there a way to optimize this list? There are no held messages, no=20 > subscription requests, no templates. Are there any subscription requests pending confirmation from the User. If yo= u go to Subscription Requests -> Pending Confirmation from list's admin page,= you'll see the list there. In the past, I've noticed this caused a significant delay in every list page.= This has since been fixed in the master branch, but yet to be released. >=20 > The next biggest list has 438 members, and all those pages load very=20 > quickly (0-2 seconds). I can=E2=80=99t figure out what the difference betw= een=20 > the two lists. >=20 > Thanks, > Seth > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --=20 thanks, Abhilash Raj (maxking) --===============2082363955002908417==-- From cristian.bianchi@usi.ch Wed Sep 23 16:14:56 2020 From: cristian.bianchi@usi.ch To: mailman-users@mailman3.org Subject: [MM3-users] Re: Mails are not archived at all -- how to check if Hyperkitty is running or not? Date: Wed, 23 Sep 2020 11:59:03 +0000 Message-ID: <160086234385.5142.15422068314320458208@mailman.iad1.psf.io> In-Reply-To: <45d65375-f93b-103c-ef4b-55dcdd54a592@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8219006707711943764==" --===============8219006707711943764== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable I have a similar problem with mailman 3.2.1 (debian package): # mailman info GNU Mailman 3.2.1 (La Villa Strangiato) Python 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0] the messages are not archived even if the key is set and hyperkitty is checke= d as active archiver. To analyze the problem I set DEBUG =3D True in /etc/mailman3/mailman-web.py. In debug mode the problem does not occur, but this second problem appears:=20 https://gitlab.com/mailman/hyperkitty/-/issues/84 How can I fix it? --===============8219006707711943764==-- From pwilliam@fredhutch.org Wed Sep 23 16:48:13 2020 From: "Williams, Peter B" To: mailman-users@mailman3.org Subject: [MM3-users] Re: cc'd e-mail addresses rewritten with @invalid.domain Date: Wed, 23 Sep 2020 16:20:38 +0000 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6308497757940113201==" --===============6308497757940113201== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Mark, Thanks for the help with this! Other list members will also see the e-mail as= coming from user(a)invalid.domain. I will cont= inue testing the other scenarios and will reply back as soon as possible. Thanks again for the help with this and have a nice day! Peter On Mon, Sep 21, 2020 at 9:00 PM Mark Sapiro wrote: On 9/21/20 10:49 AM, peterbw(a)gmail.com wrote: > Hi, > > We've recently noticed an issue where users who are cc'd in a message where= a mailing list is also within the cc list are getting rewritten a user(a)inv= alid.domain. > > This issue looks like it may be related to Outlook and how user names are d= isplayed specifically. Here is an example: > > To: test_user(a)gmail.com > cc: listserv(a)lists.test.org; Williams, = Peter; > > "Williams, Peter" maps to a user e-mail in the Outlook Global Address book.= When the message is sent the list will send the message with the first name = and last name separated, for example the message will appear: > > to: test_user(a)gmail.com > cc: listserv(a)lists.test.org, Williams(a= )invalid.domain, Peter(a)invalid.domain Because apparently Outlook is sending the message with the literal > Cc: listserv(a)lists.test.org; Williams, = Peter; instead of something like > Cc: listserv(a)lists.test.org, "Williams,= Peter" > Then some MTA in the chain is seeing the unqualified addresses Williams and Peter and appending its own local domain. > This only seems to happen with users who are cc'd who are also members of t= he list that is being cc'd. For example in the scenario above my e-mail that = is cc'd is also a member of listserv(a)lists.test.org. In this case, where do you see this? i.e., do other list members see it in the message they receive from the list? I don't know why that would be. I'm certain that Mailman is not involved in this and the Williams(a)invalid.domain and Peter(a)invalid.domain addresses are in the Cc: as received by Mailman. > This issue does not occur if the user e-mail address is listed before the m= ailing list in the cc field. So for example, if the cc in the scenario above = contained Williams, Peter; listserv(a)lists.test.org; it would route properly. Again, I think this is Outlook. What's in the Cc: of the mail that test_user(a)gmail.com receives directly? What happens if you send mail in this way with To: test_user(a)gmail.com cc: other_user(a)gmail.com; Williams, Peter; Again, what's in the Cc: of the mail that test_user(a)gmail.com and other_user(a)gmail.com receive? Also, please see . -- Mark Sapiro > The highwa= y is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ --===============6308497757940113201==-- From mark@msapiro.net Wed Sep 23 17:02:50 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: My Server is unable to connect to Mailman REST API few times Date: Wed, 23 Sep 2020 10:02:37 -0700 Message-ID: <7174ffc3-d925-ac45-2dc7-6cd1dbf4b936@msapiro.net> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2373687147998577266==" --===============2373687147998577266== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/23/20 6:32 AM, Shashikanth Komandoor wrote: > Dear Mark, > > As I already said, your suggestion worked well on one list. > > But when I tried to implement the same trick on another list, I > found the below error messages in the mailman.log. > > >> * Sep 23 18:31:31 2020 (22951) Uncaught runner exception: [Errno >> 2] No such file or directory: >> '/var/lib/mailman/mailman/var/lists/press.dpr-pb.lsmgr.nic.in/digest.mmdf >> '* >> >> >> *Sep 23 18:31:31 2020 (22951) Traceback (most recent call last):* >> * File "/usr/lib64/python3.6/mailbox.py", line 581, in __init__* >> * f = open(self._path, 'rb+')* >> *FileNotFoundError: [Errno 2] No such file or directory: >> '/var/lib/mailman/mailman/var/lists/press.dpr-pb.lsmgr.nic.in/digest.mmdf >> '* >> *During handling of the above exception, another exception occurred:* >> *Traceback (most recent call last):* >> * File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 173, >> in _one_iteration* >> * self._process_one_file(msg, msgdata)* >> * File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 266, >> in _process_one_file* >> * keepqueued = self._dispose(mlist, msg, msgdata)* >> * File "/var/lib/mailman/mailman/src/mailman/runners/pipeline.py", line >> 37, in _dispose* >> * process(mlist, msg, msgdata, pipeline)* >> * File "/var/lib/mailman/mailman/src/mailman/core/pipelines.py", line 50, >> in process* >> * handler.process(mlist, msg, msgdata)* >> * File "/var/lib/mailman/mailman/src/mailman/handlers/to_digest.py", line >> 47, in process* >> * with Mailbox(mailbox_path, create=True) as mbox:* >> * File "/usr/lib64/python3.6/mailbox.py", line 893, in __init__* >> * _mboxMMDF.__init__(self, path, factory, create)* >> * File "/usr/lib64/python3.6/mailbox.py", line 585, in __init__* >> * f = open(self._path, 'wb+')**FileNotFoundError: [Errno 2] No such >> file or directory: >> '/var/lib/mailman/mailman/var/lists/press.dpr-pb.lsmgr.nic.in/digest.mmdf >> '* >> >> I ran in both the ways you suggested i.e. through "mailman shell" > and "mailman remove". But no luck. > > Could you please suggest me at this point ? Mailman is trying to add a message to the digest of the press.dpr-pb.lsmgr.nic.in list, but the /var/lib/mailman/mailman/var/lists/press.dpr-pb.lsmgr.nic.in/ directory doesn't exist, presumably because the list has been removed. The question is why is Mailman trying to write to this list's digest.mmdf? Did the above result from unshunting something from before the list was removed? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============2373687147998577266==-- From mark@msapiro.net Wed Sep 23 17:53:00 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Wed, 23 Sep 2020 10:52:48 -0700 Message-ID: <9ca6dc68-195f-4488-13c0-8a1441ad4b96@msapiro.net> In-Reply-To: <2b7d65b2-ea38-be3c-d309-fdd25001dd14@emwd.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5243606862359815435==" --===============5243606862359815435== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/23/20 7:02 AM, Brian Carpenter wrote: > On 9/23/20 7:33 AM, r.woithe(a)callassoftware.com wrote: >> Thank u for the hint ! >> The only one is that Ubuntu uses uWSGI and not gunicorn. >> I set the para socket-timeout = XXX. >> I hope thats the right parameter. >> _________________________________________ > > I think Mailman 3 core still has its own gunicorn process that it runs. > Brian is correct. The REST API does not use the web server and uWSGI is not involved. Since Mailman core 3.3.0, the REST API is served by Gunicorn processes that have nothing to do with the web server. The time limit for these processes needs to be set as described at . Prior to 3.3.0, the REST API used the standard library wsgiref module which doesn't have timeouts. Since you said you have Mailman core 3.2.2, this may not be a REST timeout at all. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5243606862359815435==-- From seth@tofutavern.com Wed Sep 23 19:42:11 2020 From: Seth Seeger To: mailman-users@mailman3.org Subject: [MM3-users] Re: API timeout for one list Date: Wed, 23 Sep 2020 15:42:03 -0400 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7781448249442839951==" --===============7781448249442839951== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable > On Sep 23, 2020, at 11:45 AM, Abhilash Raj wrot= e: >=20 >> Is there a way to optimize this list? There are no held messages, no=20 >> subscription requests, no templates. >=20 > Are there any subscription requests pending confirmation from the User. If = you go to Subscription Requests -> Pending Confirmation from list's admin pag= e, you'll see the list there. >=20 > In the past, I've noticed this caused a significant delay in every list pag= e. This has since been fixed in the master branch, but yet to be released. Ah hah! There are 2,095. Thank you. Seth --===============7781448249442839951==-- From pwilliam@fredhutch.org Wed Sep 23 19:48:51 2020 From: pwilliam@fredhutch.org To: mailman-users@mailman3.org Subject: [MM3-users] Re: cc'd e-mail addresses rewritten with @invalid.domain Date: Wed, 23 Sep 2020 19:48:47 +0000 Message-ID: <160089052720.5141.13480977921458434838@mailman.iad1.psf.io> In-Reply-To: =?utf-8?q?=3CMWHPR11MB20145D3F55C76355A4D95B62D2380=40MWHPR11MB?= =?utf-8?q?2014=2Enamprd11=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0580280573819777619==" --===============0580280573819777619== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Mailman Users, I've done some more testing with this issue and have determined that this see= ms to be an issue with how commas within the display name in cc'd messages ar= e being processed with messages sent from the Outlook client. Here are some test scenarios. Note I set the mailing list I was testing with = to hold messages in moderation. Once the message was held I copied the inform= ation from the headers directly from Postorius held message information into = the space below: Test Scenario 1: Display name in Outlook: Williamss, Peterr C Outcome: Does not work as intended. The CC address is formatted as Williamss(= a)invalid.domain in messages received by mailing list members, oddly the "Pet= err C" part of the display name is not present anywhere in the CC addresses r= eceived by mailing list members. Header information from held message in Postorius=20 CC: "os-x-lion(a)lists.test.org" , "Williamss, Peterr C" Test Scenario 2: Display name in Outlook: Williams, Peter D Outcome: Does not work as intended. Interestingly, the CC address in the mess= age received by mailing list members is formatted as two separate addresses i= n this case: Williams(a)invalid.domain and Peter(a)invalid.domain. "D" is not= present anywhere. Could this issue be related to the number of characters? Header information from held message in Postorius: CC: "os-x-lion(a)lists.test.org" , "Williams, Peter B" Test Scenario 3: Display name in Outlook: Williamss Peterr C Outcome: Works properly. The CC address contains the correct address and disp= lay name information. The information in messages received from the mailing l= ist contain the address formatted correctly as: "Williamss Peterr C peterbw(a= )test.com" Header information from held message in Postorius: CC: "os-x-lion(a)lists.test.org" , Williamss Peterr C --===============0580280573819777619==-- From mark@msapiro.net Wed Sep 23 19:57:17 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Mails are not archived at all -- how to check if Hyperkitty is running or not? Date: Wed, 23 Sep 2020 12:57:06 -0700 Message-ID: In-Reply-To: <160086234385.5142.15422068314320458208@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5304418538448729612==" --===============5304418538448729612== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/23/20 4:59 AM, cristian.bianchi(a)usi.ch wrote: > I have a similar problem with mailman 3.2.1 (debian package): >=20 > # mailman info > GNU Mailman 3.2.1 (La Villa Strangiato) > Python 3.7.3 (default, Jul 25 2020, 13:03:44) > [GCC 8.3.0] >=20 > the messages are not archived even if the key is set and hyperkitty is chec= ked as active archiver. >=20 > To analyze the problem I set DEBUG =3D True in /etc/mailman3/mailman-web.py. > In debug mode the problem does not occur, but this second problem appears: = > https://gitlab.com/mailman/hyperkitty/-/issues/84 I don't understand. https://gitlab.com/mailman/hyperkitty/-/issues/84 is supposed to be fixed. Are you saying it isn't. > How can I fix it? Start by finding all the log messages relating to this archiving failure. If there are exceptions with tracebacks, what are they? Also, are there messages in var/archives/hyperkitty/spool/? If so, there will also be ''Exception in the HyperKitty archiver:' messages with an exception logged in mailman.log for each such message every time there is a list post. Note that HyperKitty is not a service that runs like Mailman core. The only check is can you access it via the web? --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5304418538448729612==-- From pwilliam@fredhutch.org Wed Sep 23 20:22:53 2020 From: pwilliam@fredhutch.org To: mailman-users@mailman3.org Subject: [MM3-users] Re: cc'd e-mail addresses rewritten with @invalid.domain Date: Wed, 23 Sep 2020 20:22:50 +0000 Message-ID: <160089257019.5143.16816507031511777238@mailman.iad1.psf.io> In-Reply-To: <160089052720.5141.13480977921458434838@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4622855583336284910==" --===============4622855583336284910== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Mailman Users, I apologize for sending multiple messages! I wanted to mention that this issu= e occurs whether or not the user with the malformed cc is a member of the lis= t in question. I also noticed that the issues seem to be only happening with = the addresses in quotes, and wondered if this snippet of code in avoid_duplic= ates.py gives any hints as to what the issues may be? # We've seen messages with Cc: headers folded inside a quoted string. # I.e., a message composed with several Cc addresses of the form # 'real name (dept) ', the MUA quotes # "real name (dept)" and then folds the header between 'name' and # '(dept)' resulting in a header including the entry # '"real name\r\n (dept)" ' which parses incorrec= tly, # so we "unfold" headers here. for header in ('to', 'cc', 'resent-to', 'resent-cc'): hdrs_unfolded =3D [re.sub('[\r\n]', '', value) for value in msg.get_all(header, [])] addrs =3D getaddresses(hdrs_unfolded) header_addresses =3D dict((addr, formataddr((name, addr))) for name, addr in addrs if addr) Thanks again and have a nice day! Peter --===============4622855583336284910==-- From mark@msapiro.net Wed Sep 23 20:26:11 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: cc'd e-mail addresses rewritten with @invalid.domain Date: Wed, 23 Sep 2020 13:26:02 -0700 Message-ID: <1eeff011-2a03-9808-fcf9-8ecb638ec87d@msapiro.net> In-Reply-To: <160089052720.5141.13480977921458434838@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============9077363127210541829==" --===============9077363127210541829== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/23/20 12:48 PM, pwilliam(a)fredhutch.org wrote: > Hi Mailman Users, >=20 > I've done some more testing with this issue and have determined that this s= eems to be an issue with how commas within the display name in cc'd messages = are being processed with messages sent from the Outlook client. >=20 > Here are some test scenarios. Note I set the mailing list I was testing wit= h to hold messages in moderation. Once the message was held I copied the info= rmation from the headers directly from Postorius held message information int= o the space below: >=20 > Test Scenario 1: > Display name in Outlook: Williamss, Peterr C > Outcome: Does not work as intended. The CC address is formatted as Williams= s(a)invalid.domain in messages received by mailing list members, oddly the "P= eterr C" part of the display name is not present anywhere in the CC addresses= received by mailing list members. > Header information from held message in Postorius=20 > CC: "os-x-lion(a)lists.test.org" , > "Williamss, > Peterr C" "Williamss, Peterr C" is correct for how this address spec should be represented. If in fact, it is split into two lines "Williamss, Peterr C" as it appears above, it looks to me as if the actual header should be > CC: "os-x-lion(a)lists.test.org" , "Williamss, = Peterr C" but is being folded on the whitespace between Williamss and Peterr. This is , fixed at . The fix will be in the next (3.3.2) release of Mailman core. This is really an Outlook issue. See for why folding within a quoted string is allowed, but in cases like this SHOULD not be done. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============9077363127210541829==-- From mailinglisten@wbock.de Wed Sep 23 22:03:25 2020 From: Wolfgang Bock To: mailman-users@mailman3.org Subject: [MM3-users] Re: including a template failed Date: Thu, 24 Sep 2020 00:03:09 +0200 Message-ID: <003701d691f5$532188d0$f9649a70$@wbock.de> In-Reply-To: <900c7815-1ca0-4af7-da30-57a00f89c415@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3283848221443274156==" --===============3283848221443274156== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Dear Mark, Q: Can you get https://webmail.domain-02.de/mailman3/postorius/api/templates/list/mylist.li sts.domain-03.de/list:member:regular:footer on the Mailman machine? A: No Q: What is your setting for POSTORIUS_TEMPLATE_BASE_URL? A: was: POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost/mailman3/' Changed it to: POSTORIUS_TEMPLATE_BASE_URL = 'https://localhost/mailman3/' Now another problem occurs: Sep 23 22:51:34 2020 (13492) Cannot connect to SMTP server localhost on port 25 HTTPSConnectionPool ... 'localhost' doesn't match either of 'blog.domain-04.de', 'lists.domain-04.de', 'mail.domain-04.de', 'domain-04.de', 'www.domain-04.de'"))) Port 25 entry results from the manual regarding the postfix configuration: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.ht ml e.g.: [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver lmtp_host: mail.example.com lmtp_port: 8024 smtp_host: mail.example.com smtp_port: 25 Regards Wolfgang -----Ursprüngliche Nachricht----- Von: Mark Sapiro Gesendet: Sonntag, 20. September 2020 01:05 An: mailman-users(a)mailman3.org Betreff: [MM3-users] Re: including a template failed On 9/19/20 2:47 PM, Wolfgang Bock via Mailman-users wrote: > Hi, > > I created a new mailinglist and got during the test-process the > following errormessage in my mailman.log after: > ... > Sep 19 22:55:08 2020 (1433) ACCEPT: > <000001d68ec7$2553e6e0$6ffbb4a0$@domain-01.de> > > Errors: > > Sep 19 22:55:09 2020 (1436) Cannot connect to SMTP server localhost on > port > 25 > HTTPSConnectionPool(host='webmail.domain-02.de', port=443): Max > retries exceeded with url: > /mailman3/postorius/api/templates/list/mylist.lists.domain-03.de/list: > member :regular:footer (Caused by NewConnectionError(' tion.VerifiedHTTPSConnection object at 0x7fc8380f1748>: Failed to > establish a new connection: [Errno -2] Name or service not known')) > ... This is wierd. It looks like we are trying to establish a connection to host='webmail.domain-02.de', port=443 in order to GET /mailman3/postorius/api/templates/list/mylist.lists.domain-03.de/list:member :regular:footer, but the error says "Cannot connect to SMTP server localhost on port 25". I have no idea how port 25 is coming into it. Can you get https://webmail.domain-02.de/mailman3/postorius/api/templates/list/mylist.li sts.domain-03.de/list:member:regular:footer on the Mailman machine? What is your setting for POSTORIUS_TEMPLATE_BASE_URL? -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ --===============3283848221443274156==-- From pwilliam@fredhutch.org Wed Sep 23 22:04:20 2020 From: pwilliam@fredhutch.org To: mailman-users@mailman3.org Subject: [MM3-users] Re: cc'd e-mail addresses rewritten with @invalid.domain Date: Wed, 23 Sep 2020 22:04:16 +0000 Message-ID: <160089865659.5142.16991873996183941519@mailman.iad1.psf.io> In-Reply-To: <1eeff011-2a03-9808-fcf9-8ecb638ec87d@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6476365013807450773==" --===============6476365013807450773== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit That helps a lot, thank you for the assistance with this! --===============6476365013807450773==-- From mailinglisten@wbock.de Wed Sep 23 22:08:33 2020 From: Wolfgang Bock To: mailman-users@mailman3.org Subject: [MM3-users] Re: Missing whitespace in postfix_vmap after mailman genaliases Date: Thu, 24 Sep 2020 00:08:19 +0200 Message-ID: <003801d691f6$0b78b250$226a16f0$@wbock.de> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6538761673386902581==" --===============6538761673386902581== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Dear Mark, I am running mailman3 (version 3.2.1) on debian buster. Created yesterday a Test mailinglist and faced the same problem in postmapping postfix_vmap again. Is the debian buster version of mailman3 not uptodate?? Regards Wolfgang -----Ursprüngliche Nachricht----- Von: Mark Sapiro Gesendet: Sonntag, 20. September 2020 00:43 An: mailman-users(a)mailman3.org Betreff: [MM3-users] Re: Missing whitespace in postfix_vmap after mailman genaliases On 9/19/20 3:05 PM, Wolfgang Bock via Mailman-users wrote: > > after creating a new list I got the following errormessage: > > mailman aliases -> postmap: warning: > /var/lib/mailman3/data/postfix_vmap, > line 17: expected format: key whitespace value > > /var/lib/mailman3/data/postfix_vmap > # AUTOMATICALLY GENERATED BY MAILMAN ON 2020-09-19 07:44:34 .... > xxxxxxxxxxx(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-bounces(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-bounces(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-confirm(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-confirm(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-join(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-join(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-leave(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-leave(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-owner(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-owner(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-request(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-request(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-subscribe(a)lists.xxxxxxxxxxxxxxxx.de > xxxxxxxxxxx-subscribe(a)xxxxxxxxxxxxx.de > xxxxxxxxxxx-unsubscribe(a)lists.xxxxxxxxxxxxxxxx.dexxxxxxxxxxx-unsubscri > be(a)xxx > xxxxxxxxxx.de > > The last line in the left part contains 48 chars. It isn't clear what you really have, but I'm guessing that in each case except the last, what you show as two lines is actually a single line with whitespace between and the last line has no whitespace. This is , fixed in Mailman 3.3.1 by . > I solved the problem by adding a whitespathe last line has no > whitespace.ce in the last line and I made a new postmap for postfix_vmap. > > But it would be better to solve the problem by changing the genaliases > process in the next update. Already fixed and released in the last update. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ --===============6538761673386902581==-- From mark@msapiro.net Wed Sep 23 23:46:53 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Missing whitespace in postfix_vmap after mailman genaliases Date: Wed, 23 Sep 2020 16:46:45 -0700 Message-ID: <138ad9b3-7a20-136e-64fc-a3c671742b12@msapiro.net> In-Reply-To: <003801d691f6$0b78b250$226a16f0$@wbock.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2663200079087949770==" --===============2663200079087949770== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/23/20 3:08 PM, Wolfgang Bock via Mailman-users wrote: > > Dear Mark, > > I am running mailman3 (version 3.2.1) on debian buster. ... > Is the debian buster version of mailman3 not uptodate?? That is correct. The latest release and the one with this fix is 3.3.1 released 2020-04-19. Debian Buster has 3.2.1, released 2019-02-22. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============2663200079087949770==-- From mark@msapiro.net Thu Sep 24 02:36:00 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: including a template failed Date: Wed, 23 Sep 2020 19:35:50 -0700 Message-ID: <1e515636-f351-f7e0-3b73-81b42b6b7488@msapiro.net> In-Reply-To: <003701d691f5$532188d0$f9649a70$@wbock.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5934840782304082538==" --===============5934840782304082538== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/23/20 3:03 PM, Wolfgang Bock via Mailman-users wrote: >=20 > Dear Mark, >=20 > Q: Can you get > https://webmail.domain-02.de/mailman3/postorius/api/templates/list/mylist.li > sts.domain-03.de/list:member:regular:footer on the Mailman machine? >=20 > A: No >=20 > Q: What is your setting for POSTORIUS_TEMPLATE_BASE_URL? >=20 > A: > was: POSTORIUS_TEMPLATE_BASE_URL =3D 'http://localhost/mailman3/' >=20 > Changed it to: POSTORIUS_TEMPLATE_BASE_URL =3D 'https://localhost/mailman3/' Can you get https://localhost/mailman3/postorius/api/templates/list/mylist.lists.domain-0= 3.de/list:member:regular:footer on the Mailman machine? > Now another problem occurs: >=20 > Sep 23 22:51:34 2020 (13492) Cannot connect to SMTP server localhost on port > 25 > HTTPSConnectionPool ... 'localhost' doesn't match either of > 'blog.domain-04.de', 'lists.domain-04.de', 'mail.domain-04.de', > 'domain-04.de', 'www.domain-04.de'"))) The 'HTTPSConnectionPool' message has nothing to do with the 'Cannot connect' issue. > Port 25 entry results from the manual regarding the postfix configuration: >=20 > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.ht > ml >=20 > e.g.: > [mta] > incoming: mailman.mta.postfix.LMTP > outgoing: mailman.mta.deliver.deliver > lmtp_host: mail.example.com > lmtp_port: 8024 > smtp_host: mail.example.com > smtp_port: 25 And I wonder why it's trying to connect to localhost when smtp_host is mail.example.com. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5934840782304082538==-- From shashikanth.komandoor@gmail.com Thu Sep 24 11:34:58 2020 From: Shashikanth Komandoor To: mailman-users@mailman3.org Subject: [MM3-users] Re: My Server is unable to connect to Mailman REST API few times Date: Thu, 24 Sep 2020 17:04:41 +0530 Message-ID: In-Reply-To: <7174ffc3-d925-ac45-2dc7-6cd1dbf4b936@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1678521543530849295==" --===============1678521543530849295== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Thank you Mark for your response. I am not sure. How can I check that if it is from unshunting ? Virus-free. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Wed, Sep 23, 2020 at 10:33 PM Mark Sapiro wrote: > On 9/23/20 6:32 AM, Shashikanth Komandoor wrote: > > Dear Mark, > > > > As I already said, your suggestion worked well on one list. > > > > But when I tried to implement the same trick on another list, I > > found the below error messages in the mailman.log. > > > > > >> * Sep 23 18:31:31 2020 (22951) Uncaught runner exception: > [Errno > >> 2] No such file or directory: > >> '/var/lib/mailman/mailman/var/lists/ > press.dpr-pb.lsmgr.nic.in/digest.mmdf > >> '* > >> > >> > >> *Sep 23 18:31:31 2020 (22951) Traceback (most recent call last):* > >> * File "/usr/lib64/python3.6/mailbox.py", line 581, in __init__* > >> * f =3D open(self._path, 'rb+')* > >> *FileNotFoundError: [Errno 2] No such file or directory: > >> '/var/lib/mailman/mailman/var/lists/ > press.dpr-pb.lsmgr.nic.in/digest.mmdf > >> '* > >> *During handling of the above exception, another exception occurred:* > >> *Traceback (most recent call last):* > >> * File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 173, > >> in _one_iteration* > >> * self._process_one_file(msg, msgdata)* > >> * File "/var/lib/mailman/mailman/src/mailman/core/runner.py", line 266, > >> in _process_one_file* > >> * keepqueued =3D self._dispose(mlist, msg, msgdata)* > >> * File "/var/lib/mailman/mailman/src/mailman/runners/pipeline.py", line > >> 37, in _dispose* > >> * process(mlist, msg, msgdata, pipeline)* > >> * File "/var/lib/mailman/mailman/src/mailman/core/pipelines.py", line > 50, > >> in process* > >> * handler.process(mlist, msg, msgdata)* > >> * File "/var/lib/mailman/mailman/src/mailman/handlers/to_digest.py", > line > >> 47, in process* > >> * with Mailbox(mailbox_path, create=3DTrue) as mbox:* > >> * File "/usr/lib64/python3.6/mailbox.py", line 893, in __init__* > >> * _mboxMMDF.__init__(self, path, factory, create)* > >> * File "/usr/lib64/python3.6/mailbox.py", line 585, in __init__* > >> * f =3D open(self._path, 'wb+')**FileNotFoundError: [Errno 2] No such > >> file or directory: > >> '/var/lib/mailman/mailman/var/lists/ > press.dpr-pb.lsmgr.nic.in/digest.mmdf > >> '* > >> > >> I ran in both the ways you suggested i.e. through "mailman > shell" > > and "mailman remove". But no luck. > > > > Could you please suggest me at this point ? > > > Mailman is trying to add a message to the digest of the > press.dpr-pb.lsmgr.nic.in list, but the > /var/lib/mailman/mailman/var/lists/press.dpr-pb.lsmgr.nic.in/ directory > doesn't exist, presumably because the list has been removed. > > The question is why is Mailman trying to write to this list's > digest.mmdf? Did the above result from unshunting something from before > the list was removed? > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > --=20 Thanks & Regards, Shashi Kanth.K 9052671936 Virus-free. www.avast.com <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> --===============1678521543530849295==-- From mark@msapiro.net Thu Sep 24 16:45:43 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: My Server is unable to connect to Mailman REST API few times Date: Thu, 24 Sep 2020 09:45:32 -0700 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8677396896616493250==" --===============8677396896616493250== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/24/20 4:34 AM, Shashikanth Komandoor wrote: > Thank you Mark for your response. > > I am not sure. How can I check that if it is from unshunting ? It is from unshunting if you or someone else manually ran `mailman unshunt`. I was trying to understand how Mailman would be trying to add a message for the digest to a non-existent list. One way I think is if a message to be added was shunted from the digest queue and then unshunted after the list was deleted. This however seems unlikely. Another possibility is mailman was stopped with an entry for the list in the digest queue and started again after deleting the list. This seems more likely. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============8677396896616493250==-- From w.schoener@alp.dillingen.de Fri Sep 25 08:16:47 2020 From: w.schoener@alp.dillingen.de To: mailman-users@mailman3.org Subject: [MM3-users] Discard all mail with the subject "The resutls of your email commands" Date: Fri, 25 Sep 2020 08:16:42 +0000 Message-ID: <160102180283.8841.5299445227862284520@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4732023823917564046==" --===============4732023823917564046== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, I am using mailman 3.2.1 for several mailing lists. My colleagues want me to = stop all mails with the subject "The results of your email commands". I found some contributions here, but nowhere a solution for this question. Does anybody has any suggestion? Maybe I could try to handle this outside of mailman, but then I would have to= pass this task on to someone I dont know .... :-( Thanks for all hints. :-) Wolfgang --===============4732023823917564046==-- From ralph@ml.seichter.de Fri Sep 25 16:49:58 2020 From: Ralph Seichter To: mailman-users@mailman3.org Subject: [MM3-users] Re: Discard all mail with the subject "The resutls of your email commands" Date: Fri, 25 Sep 2020 17:50:16 +0200 Message-ID: <87h7rllutj.fsf@wedjat.horus-it.com> In-Reply-To: <160102180283.8841.5299445227862284520@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1214744752683885377==" --===============1214744752683885377== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit * w. schoener: > My colleagues want me to stop all mails with the subject "The results > of your email commands". That's pretty vague. What exactly to you mean by "stop"? -Ralph --===============1214744752683885377==-- From mark@msapiro.net Fri Sep 25 17:10:54 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Discard all mail with the subject "The resutls of your email commands" Date: Fri, 25 Sep 2020 10:10:45 -0700 Message-ID: <432b3492-b249-0189-5df2-e21843328f2f@msapiro.net> In-Reply-To: <160102180283.8841.5299445227862284520@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1190401602857284216==" --===============1190401602857284216== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/25/20 1:16 AM, w.schoener(a)alp.dillingen.de wrote: > Hello, > I am using mailman 3.2.1 for several mailing lists. My colleagues want me t= o stop all mails with the subject "The results of your email commands". > I found some contributions here, but nowhere a solution for this question. > Does anybody has any suggestion? > Maybe I could try to handle this outside of mailman, but then I would have = to pass this task on to someone I dont know .... :-( > Thanks for all hints. :-) Doing it in the MTA, even though you don't want to, may be best as, with postfix for example, you can do it with header_checks. To do it in Mailman 3, patch mailman/runners/command.py and comment out or remove the last line ``` reply.send(mlist) ``` Are you really sure you want to do this at all. It means for example that one can't get a response to a `help` or the new `who` command among others sent to the -request address. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============1190401602857284216==-- From brylon@oit.rutgers.edu Fri Sep 25 21:00:05 2020 From: brylon@oit.rutgers.edu To: mailman-users@mailman3.org Subject: [MM3-users] Re: New releases Date: Fri, 25 Sep 2020 21:00:02 +0000 Message-ID: <160106760226.8841.17281141787206776225@mailman.iad1.psf.io> In-Reply-To: <94b3d7be-45f1-517d-9665-22fcd73d8cbb@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6141395953504205285==" --===============6141395953504205285== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable So the issue was resolved for me by installing sassc (and libsass) via yum an= d abandoning rubygem-sass (thereby also updating COMPRESS_PRECOMPILERS to use= 'sassc'). From what I debugged, the import module error is misleading in th= at the compressor code was trying to exec the 'sass' command but failing; if = I try exec'ing the exact command the compressor attempts i.e., /bin/sass (fro= m rubygem-sass) it fails with: $ sass -t compressed /opt/mailman/mm-take2/venv-take2/lib/python3.6/site-pack= ages/hyperkitty/static/hyperkitty/sass/hyperkitty.scss /tmp/tmpatncz9d9.css Error: Invalid CSS after "...lor}: #{$value}": expected "{", was ";" on line 4 of /opt/mailman/mm-take2/venv-take2/lib/python3.6/site-pack= ages/hyperkitty/static/hyperkitty/libs/bootstrap/stylesheets/_root.scss from line 11 of /opt/mailman/mm-take2/venv-take2/lib/python3.6/site-p= ackages/hyperkitty/static/hyperkitty/libs/bootstrap/stylesheets/bootstrap.scss from line 4 of /opt/mailman/mm-take2/venv-take2/lib/python3.6/site-pa= ckages/hyperkitty/static/hyperkitty/sass/hyperkitty.scss Use --trace for backtrace. $ echo $? 65 However, using sassc it works (from the command line and when re-running djan= go-admin compress): $ sassc -t compressed /opt/mailman/mm-take2/venv-take2/lib/python3.6/site-pac= kages/hyperkitty/static/hyperkitty/sass/hyperkitty.scss /tmp/tmpatncz9d9.css $ echo $? 0 --===============6141395953504205285==-- From mark@msapiro.net Fri Sep 25 22:46:21 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: New releases Date: Fri, 25 Sep 2020 15:46:10 -0700 Message-ID: In-Reply-To: <160106760226.8841.17281141787206776225@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3667195874253010210==" --===============3667195874253010210== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/25/20 2:00 PM, brylon(a)oit.rutgers.edu wrote: > So the issue was resolved for me by installing sassc (and libsass) via yum = and abandoning rubygem-sass (thereby also updating COMPRESS_PRECOMPILERS to u= se 'sassc'). I had forgotten that I had seen this myself. I have now filed for this issue. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============3667195874253010210==-- From r.woithe@callassoftware.com Mon Sep 28 07:58:22 2020 From: r.woithe@callassoftware.com To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Mon, 28 Sep 2020 07:58:17 +0000 Message-ID: <160127989772.4671.10070281095013586511@mailman.iad1.psf.io> In-Reply-To: <9ca6dc68-195f-4488-13c0-8a1441ad4b96@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3443560523791866528==" --===============3443560523791866528== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, i set the necessary config params like descriped at https://wiki.list.org/x/1= 7892072 with timeout =3D 9000. But we still get the nigthly email. I guess the best way would to upgrade to mailman-core 3.3.3 to avoid this err= or. Unfortunatly i can not use the ubuntu package-manager... Or is there a second way to avoid this error? --===============3443560523791866528==-- From brian_carpenter@emwd.com Mon Sep 28 13:43:17 2020 From: Brian Carpenter To: mailman-users@mailman3.org Subject: [MM3-users] Unable to remove non-member Date: Mon, 28 Sep 2020 09:43:11 -0400 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0982664835206868338==" --===============0982664835206868338== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Hi Everyone, I ran across a strange issue today. A client reported the inability to remove a non-member in the following format: z_email/domain.org.za(a)domain.org.za The reported error from Postorius was: "The user could not be removed" I used https://verifalia.com/validate-email to validate the email address and it is syntactically correct. -- Brian Carpenter Harmonylists.com Emwd.com --===============0982664835206868338==-- From w.schoener@alp.dillingen.de Mon Sep 28 14:43:46 2020 From: w.schoener@alp.dillingen.de To: mailman-users@mailman3.org Subject: [MM3-users] Re: Discard all mail with the subject "The resutls of your email commands" Date: Mon, 28 Sep 2020 14:43:43 +0000 Message-ID: <160130422381.4681.1695879911319380807@mailman.iad1.psf.io> In-Reply-To: <432b3492-b249-0189-5df2-e21843328f2f@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4078202300846444443==" --===============4078202300846444443== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello Mark, thank you for your hint. This is not my choice but is wanted by my collegues.= I discussed it already... I commented out the line in command.py and the "result of email commands" mai= l still was send. Exists there a kind of cache, or do I just have to wait some time, until this= works? Thank you so much, Wolfgang --===============4078202300846444443==-- From mark@msapiro.net Tue Sep 29 04:19:02 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Mon, 28 Sep 2020 21:18:49 -0700 Message-ID: In-Reply-To: <160127989772.4671.10070281095013586511@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3082509439438721413==" --===============3082509439438721413== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/28/20 12:58 AM, r.woithe(a)callassoftware.com wrote: > Hello, > i set the necessary config params like descriped at https://wiki.list.org/x= /17892072 with timeout =3D 9000. > But we still get the nigthly email. If you are still using Mailman core 3.2.2, those settings have no effect because the REST server doesn't use gunicorn prior to 3.3.0. See I guess the best way would to upgrade to mailman-core 3.3.3 to avoid this e= rror. > Unfortunatly i can not use the ubuntu package-manager... > Or is there a second way to avoid this error? I don't know if upgrading Mailman core would avoid the error or not. As I indicated, it's almost certainly not a REST timeout after all. We need to understand what causes the error. As I've said, the sync_mailman goes through the message senders and tries to set the senders mailman_id. If you apply the attached patch to hyperkitty/models/sender.py, it will log the error and ignore it. Then, we'll get a bit more info from the log message. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============3082509439438721413== Content-Type: text/x-patch Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="sender.patch" MIME-Version: 1.0 ZGlmZiAtLWdpdCBhL2h5cGVya2l0dHkvbW9kZWxzL3NlbmRlci5weSBiL2h5cGVya2l0dHkvbW9k ZWxzL3NlbmRlci5weQppbmRleCBjOTRkZjNiLi5kOWUxYzdhIDEwMDY0NAotLS0gYS9oeXBlcmtp dHR5L21vZGVscy9zZW5kZXIucHkKKysrIGIvaHlwZXJraXR0eS9tb2RlbHMvc2VuZGVyLnB5CkBA IC01Niw4ICs1NiwxMCBAQCBjbGFzcyBTZW5kZXIobW9kZWxzLk1vZGVsKToKICAgICAgICAgZXhj ZXB0IEhUVFBFcnJvciBhcyBlOgogICAgICAgICAgICAgaWYgZS5jb2RlID09IDQwNDoKICAgICAg ICAgICAgICAgICByZXR1cm4gICMgVXNlciBub3QgZm91bmQgaW4gTWFpbG1hbgotICAgICAgICAg ICAgIyBub3JtYWxpemUgYWxsIHBvc3NpYmxlIGVycm9yIHR5cGVzCi0gICAgICAgICAgICByYWlz ZSBNYWlsbWFuQ29ubmVjdGlvbkVycm9yKGUpCisgICAgICAgICAgICAjIFhYWCB0ZW1wb3Jhcnkg ZGVidWcgLSBsb2cgYW5kIGlnbm9yZQorICAgICAgICAgICAgbG9nZ2VyLndhcm5pbmcoCisgICAg ICAgICAgICAgICAgInNlbmRlci5zZXRfbWFpbG1hbl9pZCBIVFRQRXJyb3IgJXMgZm9yICVzIiwg ZSwgc2VsZi5hZGRyZXNzKQorICAgICAgICAgICAgcmV0dXJuCiAgICAgICAgIGV4Y2VwdCBWYWx1 ZUVycm9yOgogICAgICAgICAgICAgIyBUaGlzIHNtZWxscyBsaWtlIGEgYmFkbHkgZm9ybWF0dGVk IGVtYWlsIGFkZHJlc3MgKHNhdyBpdCBpbiB0aGUKICAgICAgICAgICAgICMgd2lsZCkK --===============3082509439438721413==-- From gordon@dickens.com Tue Sep 29 04:29:34 2020 From: gordon@dickens.com To: mailman-users@mailman3.org Subject: [MM3-users] Cron error Date: Mon, 28 Sep 2020 14:38:51 +0000 Message-ID: <160130393103.4670.13403217123036051149@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6127325266852214281==" --===============6127325266852214281== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable I recently upgraded to mailman3 on two Debian servers and I am getting a cron= error periodically sent to me as depicted below. It looks like a permission= problem, however, /var/lib/mailman3/locks and /var/lib/mailman3/* all have u= ser =3D "list" and group =3D "list" and the cron job runs as user =3D "list" = so I don't quite understand why I am getting these errors. Hopefully, someone might reply with what the problem is.=20 Thanks, Gordon Here is some additional information: "ls -ld locks" output: drwxr-xr-x 2 list list 4096 Sep 28 08:51 locks ls -l locks" output: total 12 -rw-rw---- 1 list list 80 Sep 28 00:00 'dbcreate.lck|slc.mailhub4u.com|24435|= 4056142050254595815' -rw-rw---- 2 list list 77 Sep 28 2020 master.lck -rw-rw---- 2 list list 77 Sep 28 2020 'master.lck|slc.mailhub4u.com|5291|370= 8878559945320758' Here is the error message that is emailed to me: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 216, in= lock os.link(self._claimfile, self._lockfile) FileExistsError: [Errno 17] File exists: '/var/lib/mailman3/locks/dbcreate.lc= k|slc.mailhub4u.com|24435|4056142050254595815' -> '/var/lib/mailman3/locks/db= create.lck' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/bin/mailman", line 11, in load_entry_point('mailman=3D=3D3.2.1', 'console_scripts', 'mailman')() File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3/dist-packages/click/core.py", line 716, in main with self.make_context(prog_name, args, **extra) as ctx: File "/usr/lib/python3/dist-packages/click/core.py", line 641, in make_cont= ext self.parse_args(ctx, args) File "/usr/lib/python3/dist-packages/click/core.py", line 1089, in parse_ar= gs rest =3D Command.parse_args(self, ctx, args) File "/usr/lib/python3/dist-packages/click/core.py", line 940, in parse_args value, args =3D param.handle_parse_result(ctx, opts, args) File "/usr/lib/python3/dist-packages/click/core.py", line 1477, in handle_p= arse_result self.callback, ctx, self, value) File "/usr/lib/python3/dist-packages/click/core.py", line 96, in invoke_par= am_callback return callback(ctx, param, value) File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 94, in i= nitialize_config initialize(value) File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 218,= in initialize initialize_2(propagate_logs=3Dpropagate_logs) File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 177,= in initialize_2 config.db =3D getUtility(IDatabaseFactory, utility_name).create() File "/usr/lib/python3/dist-packages/mailman/database/factory.py", line 50,= in create with Lock(os.path.join(config.LOCK_DIR, 'dbcreate.lck')): File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 334, in= __enter__ self.lock() File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 248, in= lock elif self._read() =3D=3D self._claimfile: File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 442, in= _read with open(self._lockfile) as fp: PermissionError: [Errno 13] Permission denied: '/var/lib/mailman3/locks/dbcre= ate.lck' --===============6127325266852214281==-- From mark@msapiro.net Tue Sep 29 04:40:11 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Unable to remove non-member Date: Mon, 28 Sep 2020 21:40:01 -0700 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5673938843454770960==" --===============5673938843454770960== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/28/20 6:43 AM, Brian Carpenter wrote: > Hi Everyone, > > I ran across a strange issue today. A client reported the inability to > remove a non-member in the following format: > > z_email/domain.org.za(a)domain.org.za > > The reported error from Postorius was: > > "The user could not be removed" Is that all it said? The code says it is The user could not be removed: Where is the relevant HTTP error. > I used https://verifalia.com/validate-email to validate the email > address and it is syntactically correct. Mailman core thinks it's OK too as does RFC5322. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5673938843454770960==-- From mark@msapiro.net Tue Sep 29 05:26:49 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Cron error Date: Mon, 28 Sep 2020 22:26:36 -0700 Message-ID: <6d5e176d-5e69-ce73-19cb-f4e29bba5520@msapiro.net> In-Reply-To: <160130393103.4670.13403217123036051149@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8886491670530263221==" --===============8886491670530263221== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/28/20 7:38 AM, gordon(a)dickens.com wrote: > I recently upgraded to mailman3 on two Debian servers and I am getting a cr= on error periodically sent to me as depicted below. It looks like a permissi= on problem, however, /var/lib/mailman3/locks and /var/lib/mailman3/* all have= user =3D "list" and group =3D "list" and the cron job runs as user =3D "list= " so I don't quite understand why I am getting these errors. >=20 > ls -l locks" output: > total 12 > -rw-rw---- 1 list list 80 Sep 28 00:00 'dbcreate.lck|slc.mailhub4u.com|2443= 5|4056142050254595815' Does PID 24435 exist and if so, what is it? This is strange because it says PID 24435 has claimed the lock, but there is no lock which would be a file named dbcreate.lck if there is no PID 24435 or if it isn't related to Mailman, just remove the 'dbcreate.lck|slc.mailhub4u.com|24435|4056142050254595815' file. > -rw-rw---- 2 list list 77 Sep 28 2020 master.lck > -rw-rw---- 2 list list 77 Sep 28 2020 'master.lck|slc.mailhub4u.com|5291|3= 708878559945320758' >=20 > Here is the error message that is emailed to me: What command is cron running? Although it is some `mailman` command and the sub-command isn't relevant as it never gets that far, but the -C/--config option may be. > Traceback (most recent call last): > File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 216, = in lock > os.link(self._claimfile, self._lockfile) > FileExistsError: [Errno 17] File exists: '/var/lib/mailman3/locks/dbcreate.= lck|slc.mailhub4u.com|24435|4056142050254595815' -> '/var/lib/mailman3/locks/= dbcreate.lck' >=20 > During handling of the above exception, another exception occurred: >=20 > Traceback (most recent call last): > File "/usr/bin/mailman", line 11, in > load_entry_point('mailman=3D=3D3.2.1', 'console_scripts', 'mailman')() > File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__ > return self.main(*args, **kwargs) > File "/usr/lib/python3/dist-packages/click/core.py", line 716, in main > with self.make_context(prog_name, args, **extra) as ctx: > File "/usr/lib/python3/dist-packages/click/core.py", line 641, in make_co= ntext > self.parse_args(ctx, args) > File "/usr/lib/python3/dist-packages/click/core.py", line 1089, in parse_= args > rest =3D Command.parse_args(self, ctx, args) > File "/usr/lib/python3/dist-packages/click/core.py", line 940, in parse_a= rgs > value, args =3D param.handle_parse_result(ctx, opts, args) > File "/usr/lib/python3/dist-packages/click/core.py", line 1477, in handle= _parse_result > self.callback, ctx, self, value) > File "/usr/lib/python3/dist-packages/click/core.py", line 96, in invoke_p= aram_callback > return callback(ctx, param, value) > File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 94, in= initialize_config > initialize(value) > File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 21= 8, in initialize > initialize_2(propagate_logs=3Dpropagate_logs) > File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 17= 7, in initialize_2 > config.db =3D getUtility(IDatabaseFactory, utility_name).create() > File "/usr/lib/python3/dist-packages/mailman/database/factory.py", line 5= 0, in create > with Lock(os.path.join(config.LOCK_DIR, 'dbcreate.lck')): > File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 334, = in __enter__ > self.lock() > File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 248, = in lock > elif self._read() =3D=3D self._claimfile: > File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 442, = in _read > with open(self._lockfile) as fp: > PermissionError: [Errno 13] Permission denied: '/var/lib/mailman3/locks/dbc= reate.lck' --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============8886491670530263221==-- From mark@msapiro.net Tue Sep 29 05:29:26 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Discard all mail with the subject "The resutls of your email commands" Date: Mon, 28 Sep 2020 22:29:19 -0700 Message-ID: <08e6d5e8-d608-5973-6692-59437b93cb25@msapiro.net> In-Reply-To: <160130422381.4681.1695879911319380807@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0980356375244083973==" --===============0980356375244083973== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/28/20 7:43 AM, w.schoener(a)alp.dillingen.de wrote: > Hello Mark, > thank you for your hint. This is not my choice but is wanted by my collegue= s. I discussed it already... >=20 > I commented out the line in command.py and the "result of email commands" m= ail still was send. > Exists there a kind of cache, or do I just have to wait some time, until th= is works? > Thank you so much, > Wolfgang Did you restart Mailman core? You need to do that. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============0980356375244083973==-- From w.schoener@alp.dillingen.de Tue Sep 29 06:30:44 2020 From: w.schoener@alp.dillingen.de To: mailman-users@mailman3.org Subject: [MM3-users] Re: Discard all mail with the subject "The resutls of your email commands" Date: Tue, 29 Sep 2020 06:30:41 +0000 Message-ID: <160136104146.4669.4645581667796188030@mailman.iad1.psf.io> In-Reply-To: <08e6d5e8-d608-5973-6692-59437b93cb25@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1626720673637680969==" --===============1626720673637680969== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit o god, realy stupid of me... Thank you :-) Wolfgang --===============1626720673637680969==-- From brian_carpenter@emwd.com Tue Sep 29 11:54:11 2020 From: Brian Carpenter To: mailman-users@mailman3.org Subject: [MM3-users] Re: Unable to remove non-member Date: Tue, 29 Sep 2020 07:54:06 -0400 Message-ID: <3de73358-565b-d84f-eb3f-14c0fea9f4ac@emwd.com> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2368703112063884670==" --===============2368703112063884670== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/29/20 12:40 AM, Mark Sapiro wrote: > Is that all it said? The code says it is > > The user could not be removed: > > Where is the relevant HTTP error. It says: The user could not be removed: Nothing comes after the colon. -- Brian Carpenter Harmonylists.com Emwd.com --===============2368703112063884670==-- From gordon@dickens.com Tue Sep 29 11:59:12 2020 From: Gordon Dickens To: mailman-users@mailman3.org Subject: [MM3-users] Re: Cron error Date: Tue, 29 Sep 2020 07:59:04 -0400 Message-ID: <0181fe39-41e4-bb7c-5ff4-9608fb7c889b@dickens.com> In-Reply-To: <6d5e176d-5e69-ce73-19cb-f4e29bba5520@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6709930424687438871==" --===============6709930424687438871== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Mark, Thanks for the reply! I have removed the lock file as you recommended. The command in the subject header of the error email that mailman3 sent was as follows: Cron if [ -x /usr/bin/mailman ]; then /usr/bin/mailman digests --send; fi So, I believe that cron was running the following script in /etc/cron.d/mailman3: =C3=82 =C3=82 =C3=82 # /etc/cron.d/mailman3: crontab entries for the mail= man3 package =C3=82 =C3=82 =C3=82 SHELL=3D/bin/sh =C3=82 =C3=82 =C3=82 PATH=3D/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin =C3=82 =C3=82 =C3=82 0 0 * * *=C3=82 =C3=82 =C3=82 =C3=82 =C3=82 =C3=82 = list=C3=82 =C3=82 =C3=82 if [ -x /usr/bin/mailman ]; then /usr/bin/mailman digests --send; fi Please let me know if you have any other ideas.... Thanks, Gordon --===============6709930424687438871==-- From mark@msapiro.net Tue Sep 29 17:18:42 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Unable to remove non-member Date: Tue, 29 Sep 2020 10:18:28 -0700 Message-ID: <88c427e9-4d24-63ab-36ca-7302ef947d07@msapiro.net> In-Reply-To: <3de73358-565b-d84f-eb3f-14c0fea9f4ac@emwd.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2397591803037204480==" --===============2397591803037204480== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/29/20 4:54 AM, Brian Carpenter wrote: > On 9/29/20 12:40 AM, Mark Sapiro wrote: >> Is that all it said? The code says it is >> >> The user could not be removed: >> >> Where is the relevant HTTP error. > > It says: > > The user could not be removed: > > Nothing comes after the colon. > I see the issue. when mailmanclient processes the delete request from Postorius it calls the REST API DELETE method with the URL lists///
and the address is `quoted` with urllib.parse.quote_plus, co in this case, the URL becomes lists//nonmember/z_email%2Fdomain.org.za(a)domain.org.za This is all fine except the REST API doesn't call urllib.parse.unquote_plus to change the %2F back to slash. This is now reported at . -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============2397591803037204480==-- From mark@msapiro.net Tue Sep 29 21:04:50 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Cron error Date: Tue, 29 Sep 2020 14:04:41 -0700 Message-ID: In-Reply-To: <0181fe39-41e4-bb7c-5ff4-9608fb7c889b@dickens.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7503930142055548183==" --===============7503930142055548183== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/29/20 4:59 AM, Gordon Dickens wrote: > Hi Mark, > Thanks for the reply! > I have removed the lock file as you recommended. Has the error recurred? If so, what is the content of var/lib/mailman3/locks/? What do you see if you run /usr/bin/mailman manually with no arguments? It should print some help text and a list of commands. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============7503930142055548183==-- From brian_carpenter@emwd.com Tue Sep 29 21:12:44 2020 From: Brian Carpenter To: mailman-users@mailman3.org Subject: [MM3-users] Re: Unable to remove non-member Date: Tue, 29 Sep 2020 17:12:39 -0400 Message-ID: <874d65ca-2ace-00b4-ce08-773ea3357586@emwd.com> In-Reply-To: <88c427e9-4d24-63ab-36ca-7302ef947d07@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5872748646297566624==" --===============5872748646297566624== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/29/20 1:18 PM, Mark Sapiro wrote: > I see the issue. when mailmanclient processes the delete request from > Postorius it calls the REST API DELETE method with the URL > lists///
and the address is `quoted` with > urllib.parse.quote_plus, co in this case, the URL becomes > lists//nonmember/z_email%2Fdomain.org.za(a)domain.org.za > > This is all fine except the REST API doesn't call > urllib.parse.unquote_plus to change the %2F back to slash. This is now > reported at. Thanks Mark! -- Brian Carpenter Harmonylists.com Emwd.com --===============5872748646297566624==-- From mark@msapiro.net Tue Sep 29 21:12:59 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Cron error Date: Tue, 29 Sep 2020 14:12:51 -0700 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============7415042651018163006==" --===============7415042651018163006== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/29/20 2:04 PM, Mark Sapiro wrote: > On 9/29/20 4:59 AM, Gordon Dickens wrote: >> Hi Mark, >> Thanks for the reply! >> I have removed the lock file as you recommended. > > Has the error recurred? If so, what is the content of > var/lib/mailman3/locks/? > > > What do you see if you run > > /usr/bin/mailman > > manually with no arguments? It should print some help text and a list of > commands. > If the error has not recurred, and the output from `/usr/bin/mailman` is as expected, I'm guessing that the cause of the issue may have been some other `mailman` command that was run manually and left a stale lock behind for the cron of `mailman digests --send` to trip over. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============7415042651018163006==-- From r.woithe@callassoftware.com Wed Sep 30 07:36:07 2020 From: r.woithe@callassoftware.com To: mailman-users@mailman3.org Subject: [MM3-users] Re: File "/usr/lib/python2.7/httplib.py", line 1148, in getresponse raise ResponseNotReady() Date: Wed, 30 Sep 2020 07:36:02 +0000 Message-ID: <160145136258.4671.4755954926930735187@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6095702532321111026==" --===============6095702532321111026== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, i did import the patch and will provide further informations as soon as it is= possible for me. thanks a lot --===============6095702532321111026==-- From ruth@ivimey.org Wed Sep 30 13:57:20 2020 From: Ruth Ivimey-Cook To: mailman-users@mailman3.org Subject: [MM3-users] Various patches for mm3 stable 3.3.1 Date: Wed, 30 Sep 2020 14:57:15 +0100 Message-ID: <49ea79e4-72c3-a6bc-805e-31d40b91c9a2@ivimey.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3664788302143954096==" --===============3664788302143954096== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Folks, For my own purposes I have created some patches which apply to the current mailman pip package. The first of them is for my purposes necessary, and has been discussed here before: being able to specify the address used to listen for tcp connections independently of the address the server considers it to be accessible at. A possible change to this patch would be to check whether this address was undefined but the old hostname was defined, and use that if so. For me this is not needed, and in any case it is a trivial change to make. The second patch adds a bunch of additional logging calls to various network-related parts of the package. I wrote these into the package because I was seeking the cause of template expansion causing messages to be silently dropped. I eventually found the cause was that I had an incorrect setting of the template root URL, but I regard the lack of anything in the logs as just as much of a bug. One important part of those changes was that I added a try/catch around two parts of the 'decorate' code, which is where the silent dropping occurred. If you only take one part of this patch, I encourage you to take this part. For the third patch, I discovered that the task runner was at one point busy-waiting to the tune of 100% cpu, and the cause was the way that the runner checked to see if there was work left in the queue. I am not quite sure what was the exact problem, but I think it was that there was something to do, but it was not achievable at that time, and the result was continual retry. The third patch changes the logic slightly so that this no longer occurs - retry will happen, but somewhat more gracefully. I hope you find some use in these three. Finally, I am currently experiencing problems with templates and would appreciate any thoughts: 1. If I do not set up 'individual' or 'full' personalisation, all is well and the default footer templates are expanded as expected. However setting up personalisation causes template expansion to fail completely, and messages get sent with a blank expansion. 2. I would find the postorius UI much easier to cope with if the default template was 'populated' into the UI wherever it was non-blank, rather than the UI starting off as if no templates were defined at all. Regards, Ruth --===============3664788302143954096== Content-Type: text/x-patch Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="mailman-debug.patch" MIME-Version: 1.0 ZGlmZiAtdSAtLW1pbmltYWwgLXIgJy0tZXhjbHVkZT0qLnB5YycgbGliL3B5dGhvbjMuNi9zaXRl LXBhY2thZ2VzL21haWxtYW4vaGFuZGxlcnMvZGVjb3JhdGUucHkgbGliL3B5dGhvbjMuNi9zaXRl LXBhY2thZ2VzL21haWxtYW4vaGFuZGxlcnMvZGVjb3JhdGUucHkKLS0tIGxpYi9weXRob24zLjYv c2l0ZS1wYWNrYWdlcy9tYWlsbWFuL2hhbmRsZXJzL2RlY29yYXRlLnB5CTIwMjAtMDktMjggMjE6 MzM6NTMuMjA2MDU4OTM0ICswMTAwCisrKyBsaWIvcHl0aG9uMy42L3NpdGUtcGFja2FnZXMvbWFp bG1hbi9oYW5kbGVycy9kZWNvcmF0ZS5weQkyMDIwLTA5LTAzIDIzOjA4OjU3LjkwNzMzNjgxOCAr MDEwMApAQCAtMzMsOCArMzMsOCBAQAogZnJvbSB6b3BlLmNvbXBvbmVudCBpbXBvcnQgZ2V0VXRp bGl0eQogZnJvbSB6b3BlLmludGVyZmFjZSBpbXBvcnQgaW1wbGVtZW50ZXIKIAotCi1sb2cgPSBs b2dnaW5nLmdldExvZ2dlcignbWFpbG1hbi5lcnJvcicpCitsb2cgPSBsb2dnaW5nLmdldExvZ2dl cignbWFpbG1hbi50ZW1wbGF0ZScpCitlbG9nID0gbG9nZ2luZy5nZXRMb2dnZXIoJ21haWxtYW4u ZXJyb3InKQogYWxvZyA9IGxvZ2dpbmcuZ2V0TG9nZ2VyKCdtYWlsbWFuLmFyY2hpdmVyJykKIAog CkBAIC00MiwxNzkgKzQyLDIxMSBAQAogICAgICIiIkRlY29yYXRlIHRoZSBtZXNzYWdlIHdpdGgg aGVhZGVycyBhbmQgZm9vdGVycy4iIiIKICAgICAjIERpZ2VzdHMgYW5kIE1haWxtYW4tY3JhZnQg bWVzc2FnZXMgc2hvdWxkIG5vdCBnZXQgYWRkaXRpb25hbCBoZWFkZXJzLgogICAgIGlmIG1zZ2Rh dGEuZ2V0KCdpc2RpZ2VzdCcpIG9yIG1zZ2RhdGEuZ2V0KCdub2RlY29yYXRlJyk6CisgICAgICAg IGxvZy5pbmZvKCdTa2lwIHByb2Nlc3MoKSB0ZW1wbGF0ZTogbm90IHJlcXVpcmVkJykKICAgICAg ICAgcmV0dXJuCisKICAgICAjIEtsdWRnZSB0byBub3QgZGVjb3JhdGUgbWFpbCBmb3IgTWFpbC1B cmNoaXZlLmNvbS4KICAgICBpZiAoJ3JlY2lwaWVudHMnIGluIG1zZ2RhdGEgYW5kIGxlbihtc2dk YXRhWydyZWNpcGllbnRzJ10pID09IDEgYW5kCiAgICAgICAgICAgICBsaXN0KG1zZ2RhdGFbJ3Jl Y2lwaWVudHMnXSlbMF0gPT0gTWFpbEFyY2hpdmUoKS5yZWNpcGllbnQpOgorICAgICAgICBsb2cu aW5mbygnU2tpcCBwcm9jZXNzKCkgdGVtcGxhdGU6IE1haWxBcmNoaXZlIGlzIG9ubHkgcmVjaXBp ZW50JykKICAgICAgICAgcmV0dXJuCisKICAgICBkID0ge30KLSAgICBtZW1iZXIgPSBtc2dkYXRh LmdldCgnbWVtYmVyJykKLSAgICBpZiBtZW1iZXIgaXMgbm90IE5vbmU6Ci0gICAgICAgICMgQ2Fs Y3VsYXRlIHRoZSBleHRyYSBwZXJzb25hbGl6YXRpb24gZGljdGlvbmFyeS4KLSAgICAgICAgIyBt ZW1iZXIuc3Vic2NyaWJlciBjYW4gYmUgYSBVc2VyIGluc3RhbmNlIG9yIGFuIEFkZHJlc3MgaW5z dGFuY2UsIGFuZAotICAgICAgICAjIG1lbWJlci5hZGRyZXNzIGNhbiBiZSBOb25lIGFuZCBzbyBj YW4gbWVtYmVyLl91c2VyLnByZWZlcnJlZF9hZGRyZXNzLgotICAgICAgICBpZiBtZW1iZXIuX2Fk ZHJlc3MgaXMgbm90IE5vbmU6Ci0gICAgICAgICAgICBfYWRkcmVzcyA9IG1lbWJlci5fYWRkcmVz cwotICAgICAgICBlbHNlOgotICAgICAgICAgICAgX2FkZHJlc3MgPSAobWVtYmVyLl91c2VyLnBy ZWZlcnJlZF9hZGRyZXNzIG9yCi0gICAgICAgICAgICAgICAgICAgICAgICBsaXN0KG1lbWJlci5f dXNlci5hZGRyZXNzZXMpWzBdKQotICAgICAgICByZWNpcGllbnQgPSBtc2dkYXRhLmdldCgncmVj aXBpZW50JywgX2FkZHJlc3Mub3JpZ2luYWxfZW1haWwpCi0gICAgICAgIGRbJ21lbWJlciddID0g Zm9ybWF0YWRkcigKLSAgICAgICAgICAgIChfYWRkcmVzcy5kaXNwbGF5X25hbWUsIF9hZGRyZXNz LmVtYWlsKSkKLSAgICAgICAgZFsndXNlcl9lbWFpbCddID0gcmVjaXBpZW50Ci0gICAgICAgIGRb J3VzZXJfZGVsaXZlcmVkX3RvJ10gPSBfYWRkcmVzcy5vcmlnaW5hbF9lbWFpbAotICAgICAgICBk Wyd1c2VyX2xhbmd1YWdlJ10gPSBtZW1iZXIucHJlZmVycmVkX2xhbmd1YWdlLmRlc2NyaXB0aW9u Ci0gICAgICAgIGRbJ3VzZXJfbmFtZSddID0gbWVtYmVyLmRpc3BsYXlfbmFtZQotICAgICAgICBk Wyd1c2VyX25hbWVfb3JfYWRkcmVzcyddID0gbWVtYmVyLmRpc3BsYXlfbmFtZSBvciByZWNpcGll bnQKLSAgICAgICAgIyBGb3IgYmFja3dhcmQgY29tcGF0aWJpbGl0eS4KLSAgICAgICAgZFsndXNl cl9hZGRyZXNzJ10gPSByZWNpcGllbnQKLSAgICAjIENhbGN1bGF0ZSB0aGUgYXJjaGl2ZXIgcGVy bWFsaW5rIHN1YnN0aXR1dGlvbiB2YXJpYWJsZXMuICBUaGlzIHByb3ZpZGVzCi0gICAgIyB0aGUg JDxhcmNoaXZlLW5hbWU+X3VybCBwbGFjZWhvbGRlciBmb3IgZXZlcnkgZW5hYmxlZCBhcmNoaXZl ci4KLSAgICBmb3IgYXJjaGl2ZXIgaW4gSUxpc3RBcmNoaXZlclNldChtbGlzdCkuYXJjaGl2ZXJz OgotICAgICAgICBpZiBhcmNoaXZlci5pc19lbmFibGVkOgotICAgICAgICAgICAgIyBHZXQgdGhl IHBlcm1hbGluayBvZiB0aGUgbWVzc2FnZSBmcm9tIHRoZSBhcmNoaXZlci4gIFdhdGNoIG91dAot ICAgICAgICAgICAgIyBmb3IgZXhjZXB0aW9ucyBpbiB0aGUgYXJjaGl2ZXIgcGx1Z2luLgotICAg ICAgICAgICAgdHJ5OgotICAgICAgICAgICAgICAgIGFyY2hpdmVfdXJsID0gYXJjaGl2ZXIuc3lz dGVtX2FyY2hpdmVyLnBlcm1hbGluayhtbGlzdCwgbXNnKQotICAgICAgICAgICAgZXhjZXB0IEV4 Y2VwdGlvbjoKLSAgICAgICAgICAgICAgICBhbG9nLmV4Y2VwdGlvbignRXhjZXB0aW9uIGluICJ7 fSIgYXJjaGl2ZXInLmZvcm1hdCgKLSAgICAgICAgICAgICAgICAgICAgYXJjaGl2ZXIuc3lzdGVt X2FyY2hpdmVyLm5hbWUpKQotICAgICAgICAgICAgICAgIGFyY2hpdmVfdXJsID0gTm9uZQotICAg ICAgICAgICAgaWYgYXJjaGl2ZV91cmwgaXMgbm90IE5vbmU6Ci0gICAgICAgICAgICAgICAgcGxh Y2Vob2xkZXIgPSAne31fdXJsJy5mb3JtYXQoYXJjaGl2ZXIuc3lzdGVtX2FyY2hpdmVyLm5hbWUp Ci0gICAgICAgICAgICAgICAgZFtwbGFjZWhvbGRlcl0gPSBhcmNoaXZlX3VybAotICAgICMgVGhl c2Ugc3RyaW5ncyBhcmUgZGVzY3JpcHRpdmUgZm9yIHRoZSBsb2cgZmlsZSBhbmQgc2hvdWxkbid0 IGJlIGkxOG4nZAotICAgIGQudXBkYXRlKG1zZ2RhdGEuZ2V0KCdkZWNvcmF0aW9uLWRhdGEnLCB7 fSkpCi0gICAgaGVhZGVyID0gZGVjb3JhdGUoJ2xpc3Q6bWVtYmVyOnJlZ3VsYXI6aGVhZGVyJywg bWxpc3QsIGQpCi0gICAgZm9vdGVyID0gZGVjb3JhdGUoJ2xpc3Q6bWVtYmVyOnJlZ3VsYXI6Zm9v dGVyJywgbWxpc3QsIGQpCi0gICAgIyBFc2NhcGUgaGF0Y2ggaWYgYm90aCB0aGUgZm9vdGVyIGFu ZCBoZWFkZXIgYXJlIGVtcHR5IG9yIE5vbmUuCi0gICAgaWYgbGVuKGhlYWRlcikgPT0gMCBhbmQg bGVuKGZvb3RlcikgPT0gMDoKLSAgICAgICAgcmV0dXJuCi0gICAgIyBCZSBNSU1FIHNtYXJ0IGhl cmUuICBXZSBvbmx5IGF0dGFjaCB0aGUgaGVhZGVyIGFuZCBmb290ZXIgYnkKLSAgICAjIGNvbmNh dGVuYXRpb24gd2hlbiB0aGUgbWVzc2FnZSBpcyBhIG5vbi1tdWx0aXBhcnQgb2YgdHlwZSB0ZXh0 L3BsYWluLgotICAgICMgT3RoZXJ3aXNlLCBpZiBpdCBpcyBub3QgYSBtdWx0aXBhcnQsIHdlIG1h a2UgaXQgYSBtdWx0aXBhcnQsIGFuZCB0aGVuIHdlCi0gICAgIyBhZGQgdGhlIGhlYWRlciBhbmQg Zm9vdGVyIGFzIHRleHQvcGxhaW4gcGFydHMuCi0gICAgIwotICAgICMgQkpHOiBJbiBhZGRpdGlv biwgb25seSBhZGQgdGhlIGZvb3RlciBpZiB0aGUgbWVzc2FnZSdzIGNoYXJhY3RlciBzZXQKLSAg ICAjIG1hdGNoZXMgdGhlIGNoYXJzZXQgb2YgdGhlIGxpc3QncyBwcmVmZXJyZWQgbGFuZ3VhZ2Uu ICBUaGlzIGlzIGEKLSAgICAjIHN1Ym9wdGltYWwgc29sdXRpb24sIGFuZCBzaG91bGQgYmUgc29s dmVkIGJ5IGFsbG93aW5nIGEgbGlzdCB0byBoYXZlCi0gICAgIyBtdWx0aXBsZSBoZWFkZXJzL2Zv b3RlcnMsIGZvciBlYWNoIGxhbmd1YWdlIHRoZSBsaXN0IHN1cHBvcnRzLgotICAgICMKLSAgICAj IEFsc28sIGlmIHRoZSBsaXN0J3MgcHJlZmVycmVkIGNoYXJzZXQgaXMgdXMtYXNjaWksIHdlIGNh biBhbHdheXMKLSAgICAjIHNhZmVseSBhZGQgdGhlIGhlYWRlci9mb290ZXIgdG8gYSBwbGFpbiB0 ZXh0IG1lc3NhZ2Ugc2luY2UgYWxsCi0gICAgIyBjaGFyc2V0cyBNYWlsbWFuIHN1cHBvcnRzIGFy ZSBzdHJpY3Qgc3VwZXJzZXRzIG9mIHVzLWFzY2lpIC0tCi0gICAgIyBubywgVVRGLTE2IGVtYWls cyBhcmUgbm90IHN1cHBvcnRlZCB5ZXQuCi0gICAgIwotICAgICMgVEs6IE1lc3NhZ2Ugd2l0aCAn Y2hhcnNldD0nIGNhdXNlIHRyb3VibGUuIFNvLCBpbnN0ZWFkIG9mCi0gICAgIyAgICAgbWdzLmdl dF9jb250ZW50X2NoYXJzZXQoJ3VzLWFzY2lpJykgLi4uCi0gICAgbWNzZXQgPSBtc2cuZ2V0X2Nv bnRlbnRfY2hhcnNldCgpIG9yICd1cy1hc2NpaScKLSAgICBsY3NldCA9IG1saXN0LnByZWZlcnJl ZF9sYW5ndWFnZS5jaGFyc2V0Ci0gICAgbXNndHlwZSA9IG1zZy5nZXRfY29udGVudF90eXBlKCkK LSAgICAjIEJBVzogSWYgdGhlIGNoYXJzZXRzIGRvbid0IG1hdGNoLCBzaG91bGQgd2UgYWRkIHRo ZSBoZWFkZXIgYW5kIGZvb3RlciBieQotICAgICMgTUlNRSBtdWx0aXBhcnQgY2hyb21pbmcgdGhl IG1lc3NhZ2U/Ci0gICAgd3JhcCA9IFRydWUKLSAgICBpZiBub3QgbXNnLmlzX211bHRpcGFydCgp IGFuZCBtc2d0eXBlID09ICd0ZXh0L3BsYWluJzoKLSAgICAgICAgIyBTYXZlIHRoZSBSRkMtMzY3 NiBmb3JtYXQgcGFyYW1ldGVycy4KLSAgICAgICAgZm9ybWF0X3BhcmFtID0gbXNnLmdldF9wYXJh bSgnZm9ybWF0JykKLSAgICAgICAgZGVsc3AgPSBtc2cuZ2V0X3BhcmFtKCdkZWxzcCcpCi0gICAg ICAgICMgU2F2ZSAnQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZycgaGVhZGVyIGluIGNhc2UgZGVj b3JhdGlvbiBmYWlscy4KLSAgICAgICAgY3RlID0gbXNnLmdldCgnY29udGVudC10cmFuc2Zlci1l bmNvZGluZycpCi0gICAgICAgICMgaGVhZGVyL2Zvb3RlciBpcyBub3cgaW4gdW5pY29kZS4KLSAg ICAgICAgdHJ5OgotICAgICAgICAgICAgb2xkcGF5bG9hZCA9IG1zZy5nZXRfcGF5bG9hZChkZWNv ZGU9VHJ1ZSkuZGVjb2RlKG1jc2V0KQotICAgICAgICAgICAgZGVsIG1zZ1snY29udGVudC10cmFu c2Zlci1lbmNvZGluZyddCi0gICAgICAgICAgICBmcm9udHNlcCA9IGVuZHNlcCA9ICcnCi0gICAg ICAgICAgICBpZiBsZW4oaGVhZGVyKSA+IDAgYW5kIG5vdCBoZWFkZXIuZW5kc3dpdGgoJ1xuJyk6 Ci0gICAgICAgICAgICAgICAgZnJvbnRzZXAgPSAnXG4nCi0gICAgICAgICAgICBpZiBsZW4oZm9v dGVyKSA+IDAgYW5kIG5vdCBvbGRwYXlsb2FkLmVuZHN3aXRoKCdcbicpOgotICAgICAgICAgICAg ICAgIGVuZHNlcCA9ICdcbicKLSAgICAgICAgICAgIHBheWxvYWQgPSBoZWFkZXIgKyBmcm9udHNl cCArIG9sZHBheWxvYWQgKyBlbmRzZXAgKyBmb290ZXIKLSAgICAgICAgICAgICMgV2hlbiBzZXR0 aW5nIHRoZSBwYXlsb2FkIGZvciB0aGUgbWVzc2FnZSwgdHJ5IHZhcmlvdXMgY2hhcnNldAotICAg ICAgICAgICAgIyBlbmNvZGluZ3MgdW50aWwgb25lIGRvZXMgbm90IHByb2R1Y2UgYSBVbmljb2Rl RXJyb3IuICBXZSdsbCB0cnkKLSAgICAgICAgICAgICMgY2hhcnNldHMgaW4gdGhpcyBvcmRlcjog dGhlIGxpc3QncyBjaGFyc2V0LCB0aGUgbWVzc2FnZSdzCi0gICAgICAgICAgICAjIGNoYXJzZXQs IHRoZW4gdXRmLTguICBJdCdzIG9rYXkgaWYgc29tZSBvZiB0aGVzZSBhcmUgZHVwbGljYXRlcy4K LSAgICAgICAgICAgIGZvciBjc2V0IGluIChsY3NldCwgbWNzZXQsICd1dGYtOCcpOgorICAgIHRy eToKKyAgICAgICAgbWVtYmVyID0gbXNnZGF0YS5nZXQoJ21lbWJlcicpCisgICAgICAgIGlmIG1l bWJlciBpcyBub3QgTm9uZToKKyAgICAgICAgICAgICMgQ2FsY3VsYXRlIHRoZSBleHRyYSBwZXJz b25hbGl6YXRpb24gZGljdGlvbmFyeS4KKyAgICAgICAgICAgICMgbWVtYmVyLnN1YnNjcmliZXIg Y2FuIGJlIGEgVXNlciBpbnN0YW5jZSBvciBhbiBBZGRyZXNzIGluc3RhbmNlLCBhbmQKKyAgICAg ICAgICAgICMgbWVtYmVyLmFkZHJlc3MgY2FuIGJlIE5vbmUgYW5kIHNvIGNhbiBtZW1iZXIuX3Vz ZXIucHJlZmVycmVkX2FkZHJlc3MuCisgICAgICAgICAgICBpZiBtZW1iZXIuX2FkZHJlc3MgaXMg bm90IE5vbmU6CisgICAgICAgICAgICAgICAgX2FkZHJlc3MgPSBtZW1iZXIuX2FkZHJlc3MKKyAg ICAgICAgICAgIGVsc2U6CisgICAgICAgICAgICAgICAgX2FkZHJlc3MgPSAobWVtYmVyLl91c2Vy LnByZWZlcnJlZF9hZGRyZXNzIG9yCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgbGlzdCht ZW1iZXIuX3VzZXIuYWRkcmVzc2VzKVswXSkKKworICAgICAgICAgICAgbG9nLmRlYnVnKCdnZXQg bWVtYmVyIGRpY3QgZm9yIHRlbXBsYXRlOiAlcycsIF9hZGRyZXNzKQorCisgICAgICAgICAgICBy ZWNpcGllbnQgPSBtc2dkYXRhLmdldCgncmVjaXBpZW50JywgX2FkZHJlc3Mub3JpZ2luYWxfZW1h aWwpCisgICAgICAgICAgICBkWydtZW1iZXInXSA9IGZvcm1hdGFkZHIoCisgICAgICAgICAgICAg ICAgKF9hZGRyZXNzLmRpc3BsYXlfbmFtZSwgX2FkZHJlc3MuZW1haWwpKQorICAgICAgICAgICAg ZFsndXNlcl9lbWFpbCddID0gcmVjaXBpZW50CisgICAgICAgICAgICBkWyd1c2VyX2RlbGl2ZXJl ZF90byddID0gX2FkZHJlc3Mub3JpZ2luYWxfZW1haWwKKyAgICAgICAgICAgIGRbJ3VzZXJfbGFu Z3VhZ2UnXSA9IG1lbWJlci5wcmVmZXJyZWRfbGFuZ3VhZ2UuZGVzY3JpcHRpb24KKyAgICAgICAg ICAgIGRbJ3VzZXJfbmFtZSddID0gbWVtYmVyLmRpc3BsYXlfbmFtZQorICAgICAgICAgICAgZFsn dXNlcl9uYW1lX29yX2FkZHJlc3MnXSA9IG1lbWJlci5kaXNwbGF5X25hbWUgb3IgcmVjaXBpZW50 CisgICAgICAgICAgICAjIEZvciBiYWNrd2FyZCBjb21wYXRpYmlsaXR5LgorICAgICAgICAgICAg ZFsndXNlcl9hZGRyZXNzJ10gPSByZWNpcGllbnQKKworICAgICAgICAjIENhbGN1bGF0ZSB0aGUg YXJjaGl2ZXIgcGVybWFsaW5rIHN1YnN0aXR1dGlvbiB2YXJpYWJsZXMuICBUaGlzIHByb3ZpZGVz CisgICAgICAgICMgdGhlICQ8YXJjaGl2ZS1uYW1lPl91cmwgcGxhY2Vob2xkZXIgZm9yIGV2ZXJ5 IGVuYWJsZWQgYXJjaGl2ZXIuCisgICAgICAgIGZvciBhcmNoaXZlciBpbiBJTGlzdEFyY2hpdmVy U2V0KG1saXN0KS5hcmNoaXZlcnM6CisgICAgICAgICAgICBpZiBhcmNoaXZlci5pc19lbmFibGVk OgorICAgICAgICAgICAgICAgIGxvZy5kZWJ1ZygnZ2V0ICVzIGFyY2hpdmUgcGVybWFsaW5rIGZv ciB0ZW1wbGF0ZS4nLCBhcmNoaXZlci5zeXN0ZW1fYXJjaGl2ZXIubmFtZSkKKworICAgICAgICAg ICAgICAgICMgR2V0IHRoZSBwZXJtYWxpbmsgb2YgdGhlIG1lc3NhZ2UgZnJvbSB0aGUgYXJjaGl2 ZXIuICBXYXRjaCBvdXQKKyAgICAgICAgICAgICAgICAjIGZvciBleGNlcHRpb25zIGluIHRoZSBh cmNoaXZlciBwbHVnaW4uCiAgICAgICAgICAgICAgICAgdHJ5OgotICAgICAgICAgICAgICAgICAg ICBtc2cuc2V0X3BheWxvYWQocGF5bG9hZC5lbmNvZGUoY3NldCksIGNzZXQpCi0gICAgICAgICAg ICAgICAgZXhjZXB0IFVuaWNvZGVFcnJvcjoKLSAgICAgICAgICAgICAgICAgICAgcGFzcwotICAg ICAgICAgICAgICAgIGVsc2U6Ci0gICAgICAgICAgICAgICAgICAgIGlmIGZvcm1hdF9wYXJhbToK LSAgICAgICAgICAgICAgICAgICAgICAgIG1zZy5zZXRfcGFyYW0oJ2Zvcm1hdCcsIGZvcm1hdF9w YXJhbSkKLSAgICAgICAgICAgICAgICAgICAgaWYgZGVsc3A6Ci0gICAgICAgICAgICAgICAgICAg ICAgICBtc2cuc2V0X3BhcmFtKCdkZWxzcCcsIGRlbHNwKQotICAgICAgICAgICAgICAgICAgICB3 cmFwID0gRmFsc2UKLSAgICAgICAgICAgICAgICAgICAgYnJlYWsKLSAgICAgICAgZXhjZXB0IChM b29rdXBFcnJvciwgVW5pY29kZUVycm9yKToKLSAgICAgICAgICAgIGlmIGN0ZToKLSAgICAgICAg ICAgICAgICAjIFJlc3RvcmUgdGhlIG9yaWdpbmFsIGMtdC1lLgorICAgICAgICAgICAgICAgICAg ICBhcmNoaXZlX3VybCA9IGFyY2hpdmVyLnN5c3RlbV9hcmNoaXZlci5wZXJtYWxpbmsobWxpc3Qs IG1zZykKKyAgICAgICAgICAgICAgICBleGNlcHQgRXhjZXB0aW9uOgorICAgICAgICAgICAgICAg ICAgICBhbG9nLmV4Y2VwdGlvbignRXhjZXB0aW9uIGluICJ7fSIgYXJjaGl2ZXInLmZvcm1hdCgK KyAgICAgICAgICAgICAgICAgICAgICAgIGFyY2hpdmVyLnN5c3RlbV9hcmNoaXZlci5uYW1lKSkK KyAgICAgICAgICAgICAgICAgICAgYXJjaGl2ZV91cmwgPSBOb25lCisKKyAgICAgICAgICAgICAg ICBpZiBhcmNoaXZlX3VybCBpcyBub3QgTm9uZToKKyAgICAgICAgICAgICAgICAgICAgcGxhY2Vo b2xkZXIgPSAne31fdXJsJy5mb3JtYXQoYXJjaGl2ZXIuc3lzdGVtX2FyY2hpdmVyLm5hbWUpCisg ICAgICAgICAgICAgICAgICAgIGRbcGxhY2Vob2xkZXJdID0gYXJjaGl2ZV91cmwKKyAgICAgICAg ICAgIGVsc2U6CisgICAgICAgICAgICAgICAgbG9nLmRlYnVnKCclcyBhcmNoaXZlciBkaXNhYmxl ZC4nLCBhcmNoaXZlci5zeXN0ZW1fYXJjaGl2ZXIubmFtZSkKKworICAgICAgICAjIFRoZXNlIHN0 cmluZ3MgYXJlIGRlc2NyaXB0aXZlIGZvciB0aGUgbG9nIGZpbGUgYW5kIHNob3VsZG4ndCBiZSB0 cmFuc2xhdGVkLgorICAgICAgICBkLnVwZGF0ZShtc2dkYXRhLmdldCgnZGVjb3JhdGlvbi1kYXRh Jywge30pKQorCisgICAgICAgIGxvZy5kZWJ1ZygnZGVjb3JhdGUgaGVhZGVyIHRlbXBsYXRlJykK KyAgICAgICAgaGVhZGVyID0gZGVjb3JhdGUoJ2xpc3Q6bWVtYmVyOnJlZ3VsYXI6aGVhZGVyJywg bWxpc3QsIGQpCisKKyAgICAgICAgbG9nLmRlYnVnKCdkZWNvcmF0ZSBoZWFkZXIgdGVtcGxhdGUn KQorICAgICAgICBmb290ZXIgPSBkZWNvcmF0ZSgnbGlzdDptZW1iZXI6cmVndWxhcjpmb290ZXIn LCBtbGlzdCwgZCkKKworICAgICAgICAjIEVzY2FwZSBoYXRjaCBpZiBib3RoIHRoZSBmb290ZXIg YW5kIGhlYWRlciBhcmUgZW1wdHkgb3IgTm9uZS4KKyAgICAgICAgaWYgbGVuKGhlYWRlcikgPT0g MCBhbmQgbGVuKGZvb3RlcikgPT0gMDoKKyAgICAgICAgICAgIGxvZy5kZWJ1ZygnZW1wdHkgdGVt cGxhdGVzLCByZXR1cm4nKQorICAgICAgICAgICAgcmV0dXJuCisKKyAgICAgICAgIyBCZSBNSU1F IHNtYXJ0IGhlcmUuICBXZSBvbmx5IGF0dGFjaCB0aGUgaGVhZGVyIGFuZCBmb290ZXIgYnkKKyAg ICAgICAgIyBjb25jYXRlbmF0aW9uIHdoZW4gdGhlIG1lc3NhZ2UgaXMgYSBub24tbXVsdGlwYXJ0 IG9mIHR5cGUgdGV4dC9wbGFpbi4KKyAgICAgICAgIyBPdGhlcndpc2UsIGlmIGl0IGlzIG5vdCBh IG11bHRpcGFydCwgd2UgbWFrZSBpdCBhIG11bHRpcGFydCwgYW5kIHRoZW4gd2UKKyAgICAgICAg IyBhZGQgdGhlIGhlYWRlciBhbmQgZm9vdGVyIGFzIHRleHQvcGxhaW4gcGFydHMuCisgICAgICAg ICMKKyAgICAgICAgIyBCSkc6IEluIGFkZGl0aW9uLCBvbmx5IGFkZCB0aGUgZm9vdGVyIGlmIHRo ZSBtZXNzYWdlJ3MgY2hhcmFjdGVyIHNldAorICAgICAgICAjIG1hdGNoZXMgdGhlIGNoYXJzZXQg b2YgdGhlIGxpc3QncyBwcmVmZXJyZWQgbGFuZ3VhZ2UuICBUaGlzIGlzIGEKKyAgICAgICAgIyBz dWJvcHRpbWFsIHNvbHV0aW9uLCBhbmQgc2hvdWxkIGJlIHNvbHZlZCBieSBhbGxvd2luZyBhIGxp c3QgdG8gaGF2ZQorICAgICAgICAjIG11bHRpcGxlIGhlYWRlcnMvZm9vdGVycywgZm9yIGVhY2gg bGFuZ3VhZ2UgdGhlIGxpc3Qgc3VwcG9ydHMuCisgICAgICAgICMKKyAgICAgICAgIyBBbHNvLCBp ZiB0aGUgbGlzdCdzIHByZWZlcnJlZCBjaGFyc2V0IGlzIHVzLWFzY2lpLCB3ZSBjYW4gYWx3YXlz CisgICAgICAgICMgc2FmZWx5IGFkZCB0aGUgaGVhZGVyL2Zvb3RlciB0byBhIHBsYWluIHRleHQg bWVzc2FnZSBzaW5jZSBhbGwKKyAgICAgICAgIyBjaGFyc2V0cyBNYWlsbWFuIHN1cHBvcnRzIGFy ZSBzdHJpY3Qgc3VwZXJzZXRzIG9mIHVzLWFzY2lpIC0tCisgICAgICAgICMgbm8sIFVURi0xNiBl bWFpbHMgYXJlIG5vdCBzdXBwb3J0ZWQgeWV0LgorICAgICAgICAjCisgICAgICAgICMgVEs6IE1l c3NhZ2Ugd2l0aCAnY2hhcnNldD0nIGNhdXNlIHRyb3VibGUuIFNvLCBpbnN0ZWFkIG9mCisgICAg ICAgICMgICAgIG1ncy5nZXRfY29udGVudF9jaGFyc2V0KCd1cy1hc2NpaScpIC4uLgorICAgICAg ICBtY3NldCA9IG1zZy5nZXRfY29udGVudF9jaGFyc2V0KCkgb3IgJ3VzLWFzY2lpJworICAgICAg ICBsY3NldCA9IG1saXN0LnByZWZlcnJlZF9sYW5ndWFnZS5jaGFyc2V0CisgICAgICAgIG1zZ3R5 cGUgPSBtc2cuZ2V0X2NvbnRlbnRfdHlwZSgpCisgICAgICAgICMgQkFXOiBJZiB0aGUgY2hhcnNl dHMgZG9uJ3QgbWF0Y2gsIHNob3VsZCB3ZSBhZGQgdGhlIGhlYWRlciBhbmQgZm9vdGVyIGJ5Cisg ICAgICAgICMgTUlNRSBtdWx0aXBhcnQgY2hyb21pbmcgdGhlIG1lc3NhZ2U/CisgICAgICAgIHdy YXAgPSBUcnVlCisgICAgICAgIGlmIG5vdCBtc2cuaXNfbXVsdGlwYXJ0KCkgYW5kIG1zZ3R5cGUg PT0gJ3RleHQvcGxhaW4nOgorICAgICAgICAgICAgIyBTYXZlIHRoZSBSRkMtMzY3NiBmb3JtYXQg cGFyYW1ldGVycy4KKyAgICAgICAgICAgIGZvcm1hdF9wYXJhbSA9IG1zZy5nZXRfcGFyYW0oJ2Zv cm1hdCcpCisgICAgICAgICAgICBkZWxzcCA9IG1zZy5nZXRfcGFyYW0oJ2RlbHNwJykKKyAgICAg ICAgICAgICMgU2F2ZSAnQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZycgaGVhZGVyIGluIGNhc2Ug ZGVjb3JhdGlvbiBmYWlscy4KKyAgICAgICAgICAgIGN0ZSA9IG1zZy5nZXQoJ2NvbnRlbnQtdHJh bnNmZXItZW5jb2RpbmcnKQorICAgICAgICAgICAgIyBoZWFkZXIvZm9vdGVyIGlzIG5vdyBpbiB1 bmljb2RlLgorICAgICAgICAgICAgdHJ5OgorICAgICAgICAgICAgICAgIG9sZHBheWxvYWQgPSBt c2cuZ2V0X3BheWxvYWQoZGVjb2RlPVRydWUpLmRlY29kZShtY3NldCkKICAgICAgICAgICAgICAg ICBkZWwgbXNnWydjb250ZW50LXRyYW5zZmVyLWVuY29kaW5nJ10KLSAgICAgICAgICAgICAgICBt c2dbJ0NvbnRlbnQtVHJhbnNmZXItRW5jb2RpbmcnXSA9IGN0ZQotICAgIGVsaWYgbXNnLmdldF9j b250ZW50X3R5cGUoKSA9PSAnbXVsdGlwYXJ0L21peGVkJzoKLSAgICAgICAgIyBUaGUgbmV4dCBl YXNpZXN0IHRoaW5nIHRvIGRvIGlzIGp1c3QgcHJlcGVuZCB0aGUgaGVhZGVyIGFuZCBhcHBlbmQK LSAgICAgICAgIyB0aGUgZm9vdGVyIGFzIGFkZGl0aW9uYWwgc3VicGFydHMKLSAgICAgICAgcGF5 bG9hZCA9IG1zZy5nZXRfcGF5bG9hZCgpCi0gICAgICAgIGlmIG5vdCBpc2luc3RhbmNlKHBheWxv YWQsIGxpc3QpOgotICAgICAgICAgICAgcGF5bG9hZCA9IFtwYXlsb2FkXQotICAgICAgICBpZiBs ZW4oZm9vdGVyKSA+IDA6Ci0gICAgICAgICAgICBtaW1lZnRyID0gTUlNRVRleHQoCi0gICAgICAg ICAgICAgICAgZm9vdGVyLmVuY29kZShsY3NldCwgZXJyb3JzPSdyZXBsYWNlJyksICdwbGFpbics IGxjc2V0KQotICAgICAgICAgICAgbWltZWZ0clsnQ29udGVudC1EaXNwb3NpdGlvbiddID0gJ2lu bGluZScKLSAgICAgICAgICAgIHBheWxvYWQuYXBwZW5kKG1pbWVmdHIpCisgICAgICAgICAgICAg ICAgZnJvbnRzZXAgPSBlbmRzZXAgPSAnJworICAgICAgICAgICAgICAgIGlmIGxlbihoZWFkZXIp ID4gMCBhbmQgbm90IGhlYWRlci5lbmRzd2l0aCgnXG4nKToKKyAgICAgICAgICAgICAgICAgICAg ZnJvbnRzZXAgPSAnXG4nCisgICAgICAgICAgICAgICAgaWYgbGVuKGZvb3RlcikgPiAwIGFuZCBu b3Qgb2xkcGF5bG9hZC5lbmRzd2l0aCgnXG4nKToKKyAgICAgICAgICAgICAgICAgICAgZW5kc2Vw ID0gJ1xuJworICAgICAgICAgICAgICAgIHBheWxvYWQgPSBoZWFkZXIgKyBmcm9udHNlcCArIG9s ZHBheWxvYWQgKyBlbmRzZXAgKyBmb290ZXIKKworICAgICAgICAgICAgICAgICMgV2hlbiBzZXR0 aW5nIHRoZSBwYXlsb2FkIGZvciB0aGUgbWVzc2FnZSwgdHJ5IHZhcmlvdXMgY2hhcnNldAorICAg ICAgICAgICAgICAgICMgZW5jb2RpbmdzIHVudGlsIG9uZSBkb2VzIG5vdCBwcm9kdWNlIGEgVW5p Y29kZUVycm9yLiAgV2UnbGwgdHJ5CisgICAgICAgICAgICAgICAgIyBjaGFyc2V0cyBpbiB0aGlz IG9yZGVyOiB0aGUgbGlzdCdzIGNoYXJzZXQsIHRoZSBtZXNzYWdlJ3MKKyAgICAgICAgICAgICAg ICAjIGNoYXJzZXQsIHRoZW4gdXRmLTguICBJdCdzIG9rYXkgaWYgc29tZSBvZiB0aGVzZSBhcmUg ZHVwbGljYXRlcy4KKyAgICAgICAgICAgICAgICBmb3IgY3NldCBpbiAobGNzZXQsIG1jc2V0LCAn dXRmLTgnKToKKyAgICAgICAgICAgICAgICAgICAgdHJ5OgorICAgICAgICAgICAgICAgICAgICAg ICAgbXNnLnNldF9wYXlsb2FkKHBheWxvYWQuZW5jb2RlKGNzZXQpLCBjc2V0KQorICAgICAgICAg ICAgICAgICAgICBleGNlcHQgVW5pY29kZUVycm9yOgorICAgICAgICAgICAgICAgICAgICAgICAg cGFzcworICAgICAgICAgICAgICAgICAgICBlbHNlOgorICAgICAgICAgICAgICAgICAgICAgICAg aWYgZm9ybWF0X3BhcmFtOgorICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1zZy5zZXRfcGFy YW0oJ2Zvcm1hdCcsIGZvcm1hdF9wYXJhbSkKKyAgICAgICAgICAgICAgICAgICAgICAgIGlmIGRl bHNwOgorICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1zZy5zZXRfcGFyYW0oJ2RlbHNwJywg ZGVsc3ApCisgICAgICAgICAgICAgICAgICAgICAgICB3cmFwID0gRmFsc2UKKyAgICAgICAgICAg ICAgICAgICAgICAgIGJyZWFrCisgICAgICAgICAgICBleGNlcHQgKExvb2t1cEVycm9yLCBVbmlj b2RlRXJyb3IpOgorICAgICAgICAgICAgICAgIGlmIGN0ZToKKyAgICAgICAgICAgICAgICAgICAg IyBSZXN0b3JlIHRoZSBvcmlnaW5hbCBjLXQtZS4KKyAgICAgICAgICAgICAgICAgICAgZGVsIG1z Z1snY29udGVudC10cmFuc2Zlci1lbmNvZGluZyddCisgICAgICAgICAgICAgICAgICAgIG1zZ1sn Q29udGVudC1UcmFuc2Zlci1FbmNvZGluZyddID0gY3RlCisgICAgICAgIGVsaWYgbXNnLmdldF9j b250ZW50X3R5cGUoKSA9PSAnbXVsdGlwYXJ0L21peGVkJzoKKyAgICAgICAgICAgICMgVGhlIG5l eHQgZWFzaWVzdCB0aGluZyB0byBkbyBpcyBqdXN0IHByZXBlbmQgdGhlIGhlYWRlciBhbmQgYXBw ZW5kCisgICAgICAgICAgICAjIHRoZSBmb290ZXIgYXMgYWRkaXRpb25hbCBzdWJwYXJ0cworICAg ICAgICAgICAgcGF5bG9hZCA9IG1zZy5nZXRfcGF5bG9hZCgpCisgICAgICAgICAgICBpZiBub3Qg aXNpbnN0YW5jZShwYXlsb2FkLCBsaXN0KToKKyAgICAgICAgICAgICAgICBwYXlsb2FkID0gW3Bh eWxvYWRdCisgICAgICAgICAgICBpZiBsZW4oZm9vdGVyKSA+IDA6CisgICAgICAgICAgICAgICAg Zm9vdGJ5dGVzID0gZm9vdGVyLmVuY29kZShsY3NldCwgZXJyb3JzPSdyZXBsYWNlJykKKyAgICAg ICAgICAgICAgICBtaW1lZnRyID0gTUlNRVRleHQoZm9vdGJ5dGVzLCAncGxhaW4nLCBsY3NldCkK KyAgICAgICAgICAgICAgICBtaW1lZnRyWydDb250ZW50LURpc3Bvc2l0aW9uJ10gPSAnaW5saW5l JworICAgICAgICAgICAgICAgIHBheWxvYWQuYXBwZW5kKG1pbWVmdHIpCisgICAgICAgICAgICBp ZiBsZW4oaGVhZGVyKSA+IDA6CisgICAgICAgICAgICAgICAgaGVhZGJ5dGVzID0gaGVhZGVyLmVu Y29kZShsY3NldCwgZXJyb3JzPSdyZXBsYWNlJykKKyAgICAgICAgICAgICAgICBtaW1laGRyID0g TUlNRVRleHQoaGVhZGJ5dGVzLCAncGxhaW4nLCBsY3NldCkKKyAgICAgICAgICAgICAgICBtaW1l aGRyWydDb250ZW50LURpc3Bvc2l0aW9uJ10gPSAnaW5saW5lJworICAgICAgICAgICAgICAgIHBh eWxvYWQuaW5zZXJ0KDAsIG1pbWVoZHIpCisgICAgICAgICAgICBtc2cuc2V0X3BheWxvYWQocGF5 bG9hZCkKKyAgICAgICAgICAgIHdyYXAgPSBGYWxzZQorCisgICAgICAgICMgSWYgd2UgY291bGRu J3QgYWRkIHRoZSBoZWFkZXIgb3IgZm9vdGVyIGluIGEgbGVzcyBpbnRydXNpdmUgd2F5LCB3ZSBj YW4KKyAgICAgICAgIyBhdCBsZWFzdCBkbyBpdCBieSBNSU1FIGVuY2Fwc3VsYXRpb24uICBXZSB3 YW50IHRvIGtlZXAgYXMgbXVjaCBvZiB0aGUKKyAgICAgICAgIyBvdXRlciBjaHJvbWUgYXMgcG9z c2libGUuCisgICAgICAgIGlmIG5vdCB3cmFwOgorICAgICAgICAgICAgcmV0dXJuCisKKyAgICAg ICAgIyBCZWNhdXNlIG9mIHRoZSB3YXkgTWVzc2FnZSBvYmplY3RzIGFyZSBwYXNzZWQgYXJvdW5k IHRvIHByb2Nlc3MoKSwgd2UKKyAgICAgICAgIyBuZWVkIHRvIHBsYXkgdHJpY2tzIHdpdGggdGhl IG91dGVyIG1lc3NhZ2UgLS0gaS5lLiB0aGUgb3V0ZXIgb25lIG11c3QKKyAgICAgICAgIyByZW1h aW4gdGhlIHNhbWUgaW5zdGFuY2UuICBTbyB3ZSdyZSBnb2luZyB0byBjcmVhdGUgYSBjbG9uZSBv ZiB0aGUgb3V0ZXIKKyAgICAgICAgIyBtZXNzYWdlLCB3aXRoIGFsbCB0aGUgaGVhZGVyIGNocm9t ZSBpbnRhY3QsIHRoZW4gY29weSB0aGUgcGF5bG9hZCB0byBpdC4KKyAgICAgICAgIyBUaGlzIHdp bGwgZ2l2ZSB1cyBhIGNsb25lIG9mIHRoZSBvcmlnaW5hbCBtZXNzYWdlLCBhbmQgaXQgd2lsbCBm b3JtIHRoZQorICAgICAgICAjIGJhc2lzIG9mIHRoZSBpbnRlcmlvciwgd3JhcHBlZCBNZXNzYWdl LgorICAgICAgICBpbm5lciA9IE1lc3NhZ2UoKQorICAgICAgICAjIFdoaWNoIGhlYWRlcnMgdG8g Y29weT8gIExldCdzIGp1c3QgZG8gdGhlIENvbnRlbnQtKiBoZWFkZXJzCisgICAgICAgIGZvciBo LCB2IGluIG1zZy5pdGVtcygpOgorICAgICAgICAgICAgaWYgaC5sb3dlcigpLnN0YXJ0c3dpdGgo J2NvbnRlbnQtJyk6CisgICAgICAgICAgICAgICAgaW5uZXJbaF0gPSB2CisgICAgICAgIGlubmVy LnNldF9wYXlsb2FkKG1zZy5nZXRfcGF5bG9hZCgpKQorICAgICAgICAjIEZvciBjb21wbGV0ZW5l c3MKKyAgICAgICAgaW5uZXIuc2V0X3VuaXhmcm9tKG1zZy5nZXRfdW5peGZyb20oKSkKKyAgICAg ICAgaW5uZXIucHJlYW1ibGUgPSBtc2cucHJlYW1ibGUKKyAgICAgICAgaW5uZXIuZXBpbG9ndWUg PSBtc2cuZXBpbG9ndWUKKyAgICAgICAgIyBEb24ndCBjb3B5IGdldF9jaGFyc2V0LCBhcyB0aGlz IG1pZ2h0IGJlIE5vbmUsIGV2ZW4gaWYKKyAgICAgICAgIyBnZXRfY29udGVudF9jaGFyc2V0IGlz bid0LiAgSG93ZXZlciwgZG8gbWFrZSBzdXJlIHRoZXJlIGlzIGEgZGVmYXVsdAorICAgICAgICAj IGNvbnRlbnQtdHlwZSwgZXZlbiBpZiB0aGUgb3JpZ2luYWwgbWVzc2FnZSB3YXMgbm90IE1JTUUu CisgICAgICAgIGlubmVyLnNldF9kZWZhdWx0X3R5cGUobXNnLmdldF9kZWZhdWx0X3R5cGUoKSkK KyAgICAgICAgIyBCQVc6IEhBQ0sgQUxFUlQuCisgICAgICAgIGlmIGhhc2F0dHIobXNnLCAnX192 ZXJzaW9uX18nKToKKyAgICAgICAgICAgIGlubmVyLl9fdmVyc2lvbl9fID0gbXNnLl9fdmVyc2lv bl9fCisgICAgICAgICMgTm93LCBwbGF5IGdhbWVzIHdpdGggdGhlIG91dGVyIG1lc3NhZ2UgdG8g bWFrZSBpdCBjb250YWluIHRocmVlCisgICAgICAgICMgc3VicGFydHM6IHRoZSBoZWFkZXIgKGlm IGFueSksIHRoZSB3cmFwcGVkIG1lc3NhZ2UsIGFuZCB0aGUgZm9vdGVyIChpZgorICAgICAgICAj IGFueSkuCisgICAgICAgIHBheWxvYWQgPSBbaW5uZXJdCiAgICAgICAgIGlmIGxlbihoZWFkZXIp ID4gMDoKLSAgICAgICAgICAgIG1pbWVoZHIgPSBNSU1FVGV4dCgKLSAgICAgICAgICAgICAgICBo ZWFkZXIuZW5jb2RlKGxjc2V0LCBlcnJvcnM9J3JlcGxhY2UnKSwgJ3BsYWluJywgbGNzZXQpCisg ICAgICAgICAgICBoZWFkID0gaGVhZGVyLmVuY29kZShsY3NldCwgZXJyb3JzPSdyZXBsYWNlJykK KyAgICAgICAgICAgIG1pbWVoZHIgPSBNSU1FVGV4dChoZWFkLCAncGxhaW4nLCBsY3NldCkKICAg ICAgICAgICAgIG1pbWVoZHJbJ0NvbnRlbnQtRGlzcG9zaXRpb24nXSA9ICdpbmxpbmUnCiAgICAg ICAgICAgICBwYXlsb2FkLmluc2VydCgwLCBtaW1laGRyKQorICAgICAgICBpZiBsZW4oZm9vdGVy KSA+IDA6CisgICAgICAgICAgICBmb290ID0gZm9vdGVyLmVuY29kZShsY3NldCwgZXJyb3JzPSdy ZXBsYWNlJykKKyAgICAgICAgICAgIG1pbWVmdHIgPSBNSU1FVGV4dChmb290LCAncGxhaW4nLCBs Y3NldCkKKyAgICAgICAgICAgIG1pbWVmdHJbJ0NvbnRlbnQtRGlzcG9zaXRpb24nXSA9ICdpbmxp bmUnCisgICAgICAgICAgICBwYXlsb2FkLmFwcGVuZChtaW1lZnRyKQorCiAgICAgICAgIG1zZy5z ZXRfcGF5bG9hZChwYXlsb2FkKQotICAgICAgICB3cmFwID0gRmFsc2UKLSAgICAjIElmIHdlIGNv dWxkbid0IGFkZCB0aGUgaGVhZGVyIG9yIGZvb3RlciBpbiBhIGxlc3MgaW50cnVzaXZlIHdheSwg d2UgY2FuCi0gICAgIyBhdCBsZWFzdCBkbyBpdCBieSBNSU1FIGVuY2Fwc3VsYXRpb24uICBXZSB3 YW50IHRvIGtlZXAgYXMgbXVjaCBvZiB0aGUKLSAgICAjIG91dGVyIGNocm9tZSBhcyBwb3NzaWJs ZS4KLSAgICBpZiBub3Qgd3JhcDoKLSAgICAgICAgcmV0dXJuCi0gICAgIyBCZWNhdXNlIG9mIHRo ZSB3YXkgTWVzc2FnZSBvYmplY3RzIGFyZSBwYXNzZWQgYXJvdW5kIHRvIHByb2Nlc3MoKSwgd2UK LSAgICAjIG5lZWQgdG8gcGxheSB0cmlja3Mgd2l0aCB0aGUgb3V0ZXIgbWVzc2FnZSAtLSBpLmUu IHRoZSBvdXRlciBvbmUgbXVzdAotICAgICMgcmVtYWluIHRoZSBzYW1lIGluc3RhbmNlLiAgU28g d2UncmUgZ29pbmcgdG8gY3JlYXRlIGEgY2xvbmUgb2YgdGhlIG91dGVyCi0gICAgIyBtZXNzYWdl LCB3aXRoIGFsbCB0aGUgaGVhZGVyIGNocm9tZSBpbnRhY3QsIHRoZW4gY29weSB0aGUgcGF5bG9h ZCB0byBpdC4KLSAgICAjIFRoaXMgd2lsbCBnaXZlIHVzIGEgY2xvbmUgb2YgdGhlIG9yaWdpbmFs IG1lc3NhZ2UsIGFuZCBpdCB3aWxsIGZvcm0gdGhlCi0gICAgIyBiYXNpcyBvZiB0aGUgaW50ZXJp b3IsIHdyYXBwZWQgTWVzc2FnZS4KLSAgICBpbm5lciA9IE1lc3NhZ2UoKQotICAgICMgV2hpY2gg aGVhZGVycyB0byBjb3B5PyAgTGV0J3MganVzdCBkbyB0aGUgQ29udGVudC0qIGhlYWRlcnMKLSAg ICBmb3IgaCwgdiBpbiBtc2cuaXRlbXMoKToKLSAgICAgICAgaWYgaC5sb3dlcigpLnN0YXJ0c3dp dGgoJ2NvbnRlbnQtJyk6Ci0gICAgICAgICAgICBpbm5lcltoXSA9IHYKLSAgICBpbm5lci5zZXRf cGF5bG9hZChtc2cuZ2V0X3BheWxvYWQoKSkKLSAgICAjIEZvciBjb21wbGV0ZW5lc3MKLSAgICBp bm5lci5zZXRfdW5peGZyb20obXNnLmdldF91bml4ZnJvbSgpKQotICAgIGlubmVyLnByZWFtYmxl ID0gbXNnLnByZWFtYmxlCi0gICAgaW5uZXIuZXBpbG9ndWUgPSBtc2cuZXBpbG9ndWUKLSAgICAj IERvbid0IGNvcHkgZ2V0X2NoYXJzZXQsIGFzIHRoaXMgbWlnaHQgYmUgTm9uZSwgZXZlbiBpZgot ICAgICMgZ2V0X2NvbnRlbnRfY2hhcnNldCBpc24ndC4gIEhvd2V2ZXIsIGRvIG1ha2Ugc3VyZSB0 aGVyZSBpcyBhIGRlZmF1bHQKLSAgICAjIGNvbnRlbnQtdHlwZSwgZXZlbiBpZiB0aGUgb3JpZ2lu YWwgbWVzc2FnZSB3YXMgbm90IE1JTUUuCi0gICAgaW5uZXIuc2V0X2RlZmF1bHRfdHlwZShtc2cu Z2V0X2RlZmF1bHRfdHlwZSgpKQotICAgICMgQkFXOiBIQUNLIEFMRVJULgotICAgIGlmIGhhc2F0 dHIobXNnLCAnX192ZXJzaW9uX18nKToKLSAgICAgICAgaW5uZXIuX192ZXJzaW9uX18gPSBtc2cu X192ZXJzaW9uX18KLSAgICAjIE5vdywgcGxheSBnYW1lcyB3aXRoIHRoZSBvdXRlciBtZXNzYWdl IHRvIG1ha2UgaXQgY29udGFpbiB0aHJlZQotICAgICMgc3VicGFydHM6IHRoZSBoZWFkZXIgKGlm IGFueSksIHRoZSB3cmFwcGVkIG1lc3NhZ2UsIGFuZCB0aGUgZm9vdGVyIChpZgotICAgICMgYW55 KS4KLSAgICBwYXlsb2FkID0gW2lubmVyXQotICAgIGlmIGxlbihoZWFkZXIpID4gMDoKLSAgICAg ICAgbWltZWhkciA9IE1JTUVUZXh0KAotICAgICAgICAgICAgaGVhZGVyLmVuY29kZShsY3NldCwg ZXJyb3JzPSdyZXBsYWNlJyksICdwbGFpbicsIGxjc2V0KQotICAgICAgICBtaW1laGRyWydDb250 ZW50LURpc3Bvc2l0aW9uJ10gPSAnaW5saW5lJwotICAgICAgICBwYXlsb2FkLmluc2VydCgwLCBt aW1laGRyKQotICAgIGlmIGxlbihmb290ZXIpID4gMDoKLSAgICAgICAgbWltZWZ0ciA9IE1JTUVU ZXh0KAotICAgICAgICAgICAgZm9vdGVyLmVuY29kZShsY3NldCwgZXJyb3JzPSdyZXBsYWNlJyks ICdwbGFpbicsIGxjc2V0KQotICAgICAgICBtaW1lZnRyWydDb250ZW50LURpc3Bvc2l0aW9uJ10g PSAnaW5saW5lJwotICAgICAgICBwYXlsb2FkLmFwcGVuZChtaW1lZnRyKQotICAgIG1zZy5zZXRf cGF5bG9hZChwYXlsb2FkKQotICAgIGRlbCBtc2dbJ2NvbnRlbnQtdHlwZSddCi0gICAgZGVsIG1z Z1snY29udGVudC10cmFuc2Zlci1lbmNvZGluZyddCi0gICAgZGVsIG1zZ1snY29udGVudC1kaXNw b3NpdGlvbiddCi0gICAgbXNnWydDb250ZW50LVR5cGUnXSA9ICdtdWx0aXBhcnQvbWl4ZWQnCisg ICAgICAgIGRlbCBtc2dbJ2NvbnRlbnQtdHlwZSddCisgICAgICAgIGRlbCBtc2dbJ2NvbnRlbnQt dHJhbnNmZXItZW5jb2RpbmcnXQorICAgICAgICBkZWwgbXNnWydjb250ZW50LWRpc3Bvc2l0aW9u J10KKyAgICAgICAgbXNnWydDb250ZW50LVR5cGUnXSA9ICdtdWx0aXBhcnQvbWl4ZWQnCisKKyAg ICBleGNlcHQgRXhjZXB0aW9uIGFzIGV4OgorICAgICAgICBpbXBvcnQgdHJhY2ViYWNrCisgICAg ICAgIGltcG9ydCBzeXMKKyAgICAgICAgZXhjX3R5cGUsIGV4Y192YWx1ZSwgZXhjX3RyYWNlYmFj ayA9IHN5cy5leGNfaW5mbygpCisgICAgICAgIGVsb2cuZGVidWcoJ0V4Y2VwdGlvbiBpbiBkZWNv cmF0ZSgpOiAlcycsIHJlcHIodHJhY2ViYWNrLmZvcm1hdF90YihleGNfdHJhY2ViYWNrKSkpCiAK IAogQHB1YmxpYwpAQCAtMjQzLDcgKzI3NSw3IEBACiAgICAgICAgICAgICAgICAgICAgICdkZXNj cmlwdGlvbicsCiAgICAgICAgICAgICAgICAgICAgICdpbmZvJywKICAgICAgICAgICAgICAgICAg ICAgKQotICAgICAgICB9CisgICAgfQogICAgIGlmIGV4dHJhZGljdCBpcyBub3QgTm9uZToKICAg ICAgICAgc3Vic3RpdHV0aW9ucy51cGRhdGUoZXh0cmFkaWN0KQogICAgIHRleHQgPSBleHBhbmQo dGVtcGxhdGUsIG1saXN0LCBzdWJzdGl0dXRpb25zKQpkaWZmIC11IC0tbWluaW1hbCAtciAnLS1l eGNsdWRlPSoucHljJyBsaWIvcHl0aG9uMy42L3NpdGUtcGFja2FnZXMvbWFpbG1hbi9tdGEvYmFz ZS5weSBsaWIvcHl0aG9uMy42L3NpdGUtcGFja2FnZXMvbWFpbG1hbi9tdGEvYmFzZS5weQotLS0g bGliL3B5dGhvbjMuNi9zaXRlLXBhY2thZ2VzL21haWxtYW4vbXRhL2Jhc2UucHkJMjAyMC0wOS0y OCAyMTozMzo1My4yMzgwNTg1MjEgKzAxMDAKKysrIGxpYi9weXRob24zLjYvc2l0ZS1wYWNrYWdl cy9tYWlsbWFuL210YS9iYXNlLnB5CTIwMjAtMDktMDIgMTk6MTg6MTEuOTM0Njc5MzM1ICswMTAw CkBAIC00OSw2ICs0OSw4IEBACiAgICAgICAgICAgICBhc19ib29sZWFuKGNvbmZpZy5tdGEuc210 cF92ZXJpZnlfY2VydCksCiAgICAgICAgICAgICBhc19ib29sZWFuKGNvbmZpZy5tdGEuc210cF92 ZXJpZnlfaG9zdG5hbWUpLAogICAgICAgICAgICAgKQorICAgICAgICBsb2cuZGVidWcoJ2Jhc2Uu cHkgQmFzZURlbGl2ZXJ5IF9faW5pdF9fKCVzLCAlZCwgLi4uKSA9PiAlcycsCisgICAgICAgICAg ICAgICAgICBjb25maWcubXRhLnNtdHBfaG9zdCwgaW50KGNvbmZpZy5tdGEuc210cF9wb3J0KSwg c3RyKHNlbGYuX2Nvbm5lY3Rpb24pKQogCiAgICAgZGVmIF9kZWxpdmVyX3RvX3JlY2lwaWVudHMo c2VsZiwgbWxpc3QsIG1zZywgbXNnZGF0YSwgcmVjaXBpZW50cyk6CiAgICAgICAgICIiIkxvdy1s ZXZlbCBkZWxpdmVyeSB0byBhIHNldCBvZiByZWNpcGllbnRzLgpAQCAtNjksMTggKzcxLDIyIEBA CiAgICAgICAgIG1lc3NhZ2VfaWQgPSBtc2dbJ21lc3NhZ2UtaWQnXQogICAgICAgICAjIFNpbmNl IHRoZSByZWNpcGllbnRzIGNhbiBiZSBhIHNldCBvciBhIGxpc3QsIHNvcnQgdGhlIHJlY2lwaWVu dHMgYnkKICAgICAgICAgIyBlbWFpbCBhZGRyZXNzIGZvciBwcmVkaWN0YWJpbGl0eSBhbmQgdGVz dGFiaWxpdHkuCisgICAgICAgIGxvZy5kZWJ1ZygnYmFzZS5weSBfZGVsaXZlcl90b19yZWNpcGll bnRzKCkgY29ubj0lcycsIHN0cihzZWxmLl9jb25uZWN0aW9uKSkKICAgICAgICAgdHJ5OgogICAg ICAgICAgICAgcmVmdXNlZCA9IHNlbGYuX2Nvbm5lY3Rpb24uc2VuZG1haWwoCiAgICAgICAgICAg ICAgICAgc2VuZGVyLCBzb3J0ZWQocmVjaXBpZW50cyksIG1zZy5hc19zdHJpbmcoKSkKKwogICAg ICAgICBleGNlcHQgc210cGxpYi5TTVRQUmVjaXBpZW50c1JlZnVzZWQgYXMgZXJyb3I6CiAgICAg ICAgICAgICBsb2cuZXJyb3IoJyVzIHJlY2lwaWVudHMgcmVmdXNlZDogJXMnLCBtZXNzYWdlX2lk LCBlcnJvcikKICAgICAgICAgICAgIHJlZnVzZWQgPSBlcnJvci5yZWNpcGllbnRzCisKICAgICAg ICAgZXhjZXB0IHNtdHBsaWIuU01UUFJlc3BvbnNlRXhjZXB0aW9uIGFzIGVycm9yOgogICAgICAg ICAgICAgbG9nLmVycm9yKCclcyByZXNwb25zZSBleGNlcHRpb246ICVzJywgbWVzc2FnZV9pZCwg ZXJyb3IpCiAgICAgICAgICAgICByZWZ1c2VkID0gZGljdCgKICAgICAgICAgICAgICAgICAjIHJl Y2lwaWVudCAtPiAoY29kZSwgZXJyb3IpCiAgICAgICAgICAgICAgICAgKHJlY2lwaWVudCwgKGVy cm9yLnNtdHBfY29kZSwgZXJyb3Iuc210cF9lcnJvcikpCiAgICAgICAgICAgICAgICAgZm9yIHJl Y2lwaWVudCBpbiByZWNpcGllbnRzKQorCiAgICAgICAgIGV4Y2VwdCAoc29ja2V0LmVycm9yLCBJ T0Vycm9yLCBzbXRwbGliLlNNVFBFeGNlcHRpb24pIGFzIGVycm9yOgogICAgICAgICAgICAgIyBN VEEgbm90IHJlc3BvbmRpbmcsIG9yIG90aGVyIHNvY2tldCBwcm9ibGVtcywgb3IgYW55IG90aGVy CiAgICAgICAgICAgICAjIGtpbmQgb2YgU01UUEV4Y2VwdGlvbi4gIEluIHRoYXQgY2FzZSwgbm90 aGluZyBnb3QgZGVsaXZlcmVkLApAQCAtOTIsNiArOTgsOCBAQAogICAgICAgICAgICAgICAgICMg cmVjaXBpZW50IC0+IChjb2RlLCBlcnJvcikKICAgICAgICAgICAgICAgICAocmVjaXBpZW50LCAo NDQ0LCBlcnJvcikpCiAgICAgICAgICAgICAgICAgZm9yIHJlY2lwaWVudCBpbiByZWNpcGllbnRz KQorCisgICAgICAgIGxvZy5kZWJ1ZygnYmFzZS5weSBfZGVsaXZlcl90b19yZWNpcGllbnRzKCk6 IHJlZnVzZWQgPSAlZCcsIGxlbihyZWZ1c2VkKSkKICAgICAgICAgcmV0dXJuIHJlZnVzZWQKIAog ICAgIGRlZiBfZ2V0X3NlbmRlcihzZWxmLCBtbGlzdCwgbXNnLCBtc2dkYXRhKToKQEAgLTE0NSwy NyArMTUzLDM3IEBACiAgICAgICAgIGRlbGl2ZXJ5IGFkZHJlc3MgaW4gdGhlIHJldHVybiBlbnZl bG9wZSBzbyB0aGVyZSBjYW4gYmUgbm8gYW1iaWd1aXR5CiAgICAgICAgIGluIGJvdW5jZSBwcm9j ZXNzaW5nLgogICAgICAgICAiIiIKLSAgICAgICAgcmVmdXNlZCA9IHt9Ci0gICAgICAgIHJlY2lw aWVudHMgPSBtc2dkYXRhLmdldCgncmVjaXBpZW50cycsIHNldCgpKQotICAgICAgICBmb3IgcmVj aXBpZW50IGluIHJlY2lwaWVudHM6Ci0gICAgICAgICAgICBsb2cuZGVidWcoJ0luZGl2aWR1YWxE ZWxpdmVyeSB0bzogJXMnLCByZWNpcGllbnQpCi0gICAgICAgICAgICAjIE1ha2UgYSBjb3B5IG9m IHRoZSBvcmlnaW5hbCBtZXNzYWdlcyBhbmQgb3BlcmF0b3Igb24gaXQsIHNpbmNlCi0gICAgICAg ICAgICAjIHdlJ3JlIGdvaW5nIHRvIG11bmdlIGl0IHJlcGVhdGVkbHkgZm9yIGVhY2ggcmVjaXBp ZW50LgotICAgICAgICAgICAgbWVzc2FnZV9jb3B5ID0gY29weS5kZWVwY29weShtc2cpCi0gICAg ICAgICAgICBtc2dkYXRhX2NvcHkgPSBtc2dkYXRhLmNvcHkoKQotICAgICAgICAgICAgIyBTcXVp cnJlbCB0aGUgY3VycmVudCByZWNpcGllbnQgYXdheSBpbiB0aGUgbWVzc2FnZSBtZXRhZGF0YS4K LSAgICAgICAgICAgICMgVGhhdCB3YXkgdGhlIHN1YmNsYXNzJ3MgX2dldF9zZW5kZXIoKSBvdmVy cmlkZSBjYW4gZW5jb2RlIHRoZQotICAgICAgICAgICAgIyByZWNpcGllbnQgYWRkcmVzcyBpbiB0 aGUgc2VuZGVyLCBlLmcuIGZvciBWRVJQLgotICAgICAgICAgICAgbXNnZGF0YV9jb3B5WydyZWNp cGllbnQnXSA9IHJlY2lwaWVudAotICAgICAgICAgICAgIyBTZWUgaWYgdGhlIHJlY2lwaWVudCBp cyBhIG1lbWJlciBvZiB0aGUgbWFpbGluZyBsaXN0LCBhbmQgaWYgc28sCi0gICAgICAgICAgICAj IHNxdWlycmVsIHRoaXMgaW5mb3JtYXRpb24gYXdheSBmb3IgdXNlIGJ5IG90aGVyIG1vZHVsZXMs IHN1Y2ggYXMKLSAgICAgICAgICAgICMgdGhlIGhlYWRlci9mb290ZXIgZGVjb3JhdG9yLiAgWFhY IDIwMTItMDMtMDUgdGhpcyBpcyBwcm9iYWJseQotICAgICAgICAgICAgIyBoaWdobHkgaW5lZmZp Y2llbnQgb24gdGhlIGRhdGFiYXNlLgotICAgICAgICAgICAgbWVtYmVyID0gbWxpc3QubWVtYmVy cy5nZXRfbWVtYmVyKHJlY2lwaWVudCkKLSAgICAgICAgICAgIG1zZ2RhdGFfY29weVsnbWVtYmVy J10gPSBtZW1iZXIKLSAgICAgICAgICAgIGZvciBjYWxsYmFjayBpbiBzZWxmLmNhbGxiYWNrczoK LSAgICAgICAgICAgICAgICBjYWxsYmFjayhtbGlzdCwgbWVzc2FnZV9jb3B5LCBtc2dkYXRhX2Nv cHkpCi0gICAgICAgICAgICBzdGF0dXMgPSBzZWxmLl9kZWxpdmVyX3RvX3JlY2lwaWVudHMoCi0g ICAgICAgICAgICAgICAgbWxpc3QsIG1lc3NhZ2VfY29weSwgbXNnZGF0YV9jb3B5LCBbcmVjaXBp ZW50XSkKLSAgICAgICAgICAgIHJlZnVzZWQudXBkYXRlKHN0YXR1cykKLSAgICAgICAgcmV0dXJu IHJlZnVzZWQKKyAgICAgICAgdHJ5OgorICAgICAgICAgICAgcmVmdXNlZCA9IHt9CisgICAgICAg ICAgICByZWNpcGllbnRzID0gbXNnZGF0YS5nZXQoJ3JlY2lwaWVudHMnLCBzZXQoKSkKKyAgICAg ICAgICAgIGZvciByZWNpcGllbnQgaW4gcmVjaXBpZW50czoKKyAgICAgICAgICAgICAgICBsb2cu ZGVidWcoJ0luZGl2aWR1YWxEZWxpdmVyeSB0bzogJXMnLCByZWNpcGllbnQpCisgICAgICAgICAg ICAgICAgIyBNYWtlIGEgY29weSBvZiB0aGUgb3JpZ2luYWwgbWVzc2FnZXMgYW5kIG9wZXJhdG9y IG9uIGl0LCBzaW5jZQorICAgICAgICAgICAgICAgICMgd2UncmUgZ29pbmcgdG8gbXVuZ2UgaXQg cmVwZWF0ZWRseSBmb3IgZWFjaCByZWNpcGllbnQuCisgICAgICAgICAgICAgICAgbWVzc2FnZV9j b3B5ID0gY29weS5kZWVwY29weShtc2cpCisgICAgICAgICAgICAgICAgbXNnZGF0YV9jb3B5ID0g bXNnZGF0YS5jb3B5KCkKKyAgICAgICAgICAgICAgICAjIFNxdWlycmVsIHRoZSBjdXJyZW50IHJl Y2lwaWVudCBhd2F5IGluIHRoZSBtZXNzYWdlIG1ldGFkYXRhLgorICAgICAgICAgICAgICAgICMg VGhhdCB3YXkgdGhlIHN1YmNsYXNzJ3MgX2dldF9zZW5kZXIoKSBvdmVycmlkZSBjYW4gZW5jb2Rl IHRoZQorICAgICAgICAgICAgICAgICMgcmVjaXBpZW50IGFkZHJlc3MgaW4gdGhlIHNlbmRlciwg ZS5nLiBmb3IgVkVSUC4KKyAgICAgICAgICAgICAgICBtc2dkYXRhX2NvcHlbJ3JlY2lwaWVudCdd ID0gcmVjaXBpZW50CisgICAgICAgICAgICAgICAgIyBTZWUgaWYgdGhlIHJlY2lwaWVudCBpcyBh IG1lbWJlciBvZiB0aGUgbWFpbGluZyBsaXN0LCBhbmQgaWYgc28sCisgICAgICAgICAgICAgICAg IyBzcXVpcnJlbCB0aGlzIGluZm9ybWF0aW9uIGF3YXkgZm9yIHVzZSBieSBvdGhlciBtb2R1bGVz LCBzdWNoIGFzCisgICAgICAgICAgICAgICAgIyB0aGUgaGVhZGVyL2Zvb3RlciBkZWNvcmF0b3Iu ICBYWFggMjAxMi0wMy0wNSB0aGlzIGlzIHByb2JhYmx5CisgICAgICAgICAgICAgICAgIyBoaWdo bHkgaW5lZmZpY2llbnQgb24gdGhlIGRhdGFiYXNlLgorICAgICAgICAgICAgICAgIG1lbWJlciA9 IG1saXN0Lm1lbWJlcnMuZ2V0X21lbWJlcihyZWNpcGllbnQpCisgICAgICAgICAgICAgICAgbXNn ZGF0YV9jb3B5WydtZW1iZXInXSA9IG1lbWJlcgorICAgICAgICAgICAgICAgIGxvZy5kZWJ1Zygn SW5kaXZpZHVhbERlbGl2ZXJ5IGNhbGxiYWNrcycsIHN0cihyZWNpcGllbnQpKQorICAgICAgICAg ICAgICAgIGZvciBjYWxsYmFjayBpbiBzZWxmLmNhbGxiYWNrczoKKyAgICAgICAgICAgICAgICAg ICAgY2FsbGJhY2sobWxpc3QsIG1lc3NhZ2VfY29weSwgbXNnZGF0YV9jb3B5KQorCisgICAgICAg ICAgICAgICAgbG9nLmRlYnVnKCdJbmRpdmlkdWFsRGVsaXZlcnkgY2FsbGluZyBkZWxpdmVyOiAl cycsIHN0cihyZWNpcGllbnQpKQorICAgICAgICAgICAgICAgIHN0YXR1cyA9IHNlbGYuX2RlbGl2 ZXJfdG9fcmVjaXBpZW50cygKKyAgICAgICAgICAgICAgICAgICAgbWxpc3QsIG1lc3NhZ2VfY29w eSwgbXNnZGF0YV9jb3B5LCBbcmVjaXBpZW50XSkKKyAgICAgICAgICAgICAgICByZWZ1c2VkLnVw ZGF0ZShzdGF0dXMpCisKKyAgICAgICAgICAgIGxvZy5kZWJ1ZygnSW5kaXZpZHVhbERlbGl2ZXJ5 IERPTkU6ICVzJywgc3RyKHJlZnVzZWQpKQorICAgICAgICAgICAgcmV0dXJuIHJlZnVzZWQKKwor ICAgICAgICBleGNlcHQgRXhjZXB0aW9uIGFzIGVycm9yOgorICAgICAgICAgICAgbG9nLmRlYnVn KCdJbmRpdmlkdWFsRGVsaXZlcnkgRXhjZXB0aW9uOiAlcycsIHN0cihlcnJvcikpCisgICAgICAg ICAgICByYWlzZSBlcnJvcgpkaWZmIC11IC0tbWluaW1hbCAtciAnLS1leGNsdWRlPSoucHljJyBs aWIvcHl0aG9uMy42L3NpdGUtcGFja2FnZXMvbWFpbG1hbi9tdGEvY29ubmVjdGlvbi5weSBsaWIv cHl0aG9uMy42L3NpdGUtcGFja2FnZXMvbWFpbG1hbi9tdGEvY29ubmVjdGlvbi5weQotLS0gbGli L3B5dGhvbjMuNi9zaXRlLXBhY2thZ2VzL21haWxtYW4vbXRhL2Nvbm5lY3Rpb24ucHkJMjAyMC0w OS0yOCAyMTozMzo1My4yMzgwNTg1MjEgKzAxMDAKKysrIGxpYi9weXRob24zLjYvc2l0ZS1wYWNr YWdlcy9tYWlsbWFuL210YS9jb25uZWN0aW9uLnB5CTIwMjAtMDktMDIgMTc6NTc6MTYuNTU3ODY4 OTIyICswMTAwCkBAIC00Nyw3ICs0Nyw2IEBACiAgICAgQ29udmVydCBhIHN0cmluZyB0byBhbiBl bnVtIHZhbHVlLiAgQWNjZXB0cyBhbnkgY2FzZS4KICAgICAiIiIKICAgICBzID0gcy5sb3dlcigp Ci0KICAgICB0cnk6CiAgICAgICAgIHJldHVybiBTZWN1cmVNb2RlKHMpCiAgICAgZXhjZXB0IFZh bHVlRXJyb3I6CkBAIC0xMDgsNiArMTA3LDcgQEAKICAgICAgICAgc2VsZi52ZXJpZnlfaG9zdG5h bWUgPSB2ZXJpZnlfaG9zdG5hbWUgYW5kIHZlcmlmeV9jZXJ0CiAgICAgICAgIHNlbGYuX3VzZXJu YW1lID0gc210cF91c2VyCiAgICAgICAgIHNlbGYuX3Bhc3N3b3JkID0gc210cF9wYXNzCisgICAg ICAgIGxvZy5kZWJ1ZygnQ29ubmVjdGlvbi5weSBfX2luaXRfXyglcy4uLiknLCBzZWxmLl9ob3N0 KQogCiAgICAgICAgIHNlbGYuX3Nlc3Npb25fY291bnQgPSBOb25lCiAgICAgICAgIHNlbGYuX2Nv bm5lY3Rpb24gPSBOb25lCkBAIC0xMTksMTMgKzExOSwxNyBAQAogCiAgICAgZGVmIHNlbmRtYWls KHNlbGYsIGVudnNlbmRlciwgcmVjaXBpZW50cywgbXNndGV4dCk6CiAgICAgICAgICIiIk1pbWlj IGBzbXRwbGliLlNNVFAuc2VuZG1haWxgLiIiIgorCisgICAgICAgIGxvZy5kZWJ1ZygnQ29ubmVj dGlvbi5weSBzZW5kbWFpbCglcy4uLiknLCBzZWxmLl9ob3N0KQogICAgICAgICBpZiBhc19ib29s ZWFuKGNvbmZpZy5kZXZtb2RlLmVuYWJsZWQpOgogICAgICAgICAgICAgIyBGb3JjZSB0aGUgcmVj aXBpZW50cyB0byB0aGUgc3BlY2lmaWVkIGFkZHJlc3MsIGJ1dCBzdGlsbCBkZWxpdmVyCiAgICAg ICAgICAgICAjIHRvIHRoZSBzYW1lIG51bWJlciBvZiByZWNpcGllbnRzLgogICAgICAgICAgICAg cmVjaXBpZW50cyA9IFtjb25maWcuZGV2bW9kZS5yZWNpcGllbnRdICogbGVuKHJlY2lwaWVudHMp CisKICAgICAgICAgaWYgc2VsZi5fY29ubmVjdGlvbiBpcyBOb25lOgogICAgICAgICAgICAgc2Vs Zi5fY29ubmVjdCgpCiAgICAgICAgICAgICBzZWxmLl9sb2dpbigpCisKICAgICAgICAgIyBzbXRw bGliLlNNVFAuc2VuZG1haWwgcmVxdWlyZXMgdGhlIG1lc3NhZ2Ugc3RyaW5nIHRvIGJlIHB1cmUg YXNjaWkuCiAgICAgICAgICMgV2UgaGF2ZSBzZWVuIG1hbGZvcm1lZCBtZXNzYWdlcyB3aXRoIG5v bi1hc2NpaSB1bmljb2Rlcywgc28gZW5zdXJlCiAgICAgICAgICMgd2UgaGF2ZSBwdXJlIGFzY2lp LgpAQCAtMTcwLDcgKzE3NCwxNiBAQAogICAgICAgICAgICAgICAgIGlmIHNlbGYuc2VjdXJlX21v ZGUgPT0gU2VjdXJlTW9kZS5TVEFSVFRMUzoKICAgICAgICAgICAgICAgICAgICAgbG9nLmRlYnVn KCdTdGFydHRscycpCiAgICAgICAgICAgICAgICAgICAgIHNlbGYuX2Nvbm5lY3Rpb24uc3RhcnR0 bHMoY29udGV4dD1zZWxmLl90bHNfY29udGV4dCkKLSAgICAgICAgZXhjZXB0IChzb2NrZXQuZXJy b3IsIElPRXJyb3IsIHNtdHBsaWIuU01UUEV4Y2VwdGlvbik6CisgICAgICAgIGV4Y2VwdCBzb2Nr ZXQuZXJyb3IgYXMgZXJyb3I6CisgICAgICAgICAgICBsb2cuZGVidWcoJ3NvY2tldC5lcnJvciB3 aGlsZSBjb25uZWN0aW5nIHRvIFNNVFA6ICcgKyBzdHIoZXJyb3IpKQorICAgICAgICAgICAgc2Vs Zi5xdWl0KCkKKyAgICAgICAgICAgIHJhaXNlCisgICAgICAgIGV4Y2VwdCBzbXRwbGliLlNNVFBF eGNlcHRpb24gYXMgZXJyb3I6CisgICAgICAgICAgICBsb2cuZGVidWcoJ3NtdHBsaWIgZXJyb3Ig d2hpbGUgY29ubmVjdGluZyB0byBTTVRQOiAnICsgc3RyKGVycm9yKSkKKyAgICAgICAgICAgIHNl bGYucXVpdCgpCisgICAgICAgICAgICByYWlzZQorICAgICAgICBleGNlcHQgSU9FcnJvciBhcyBl cnJvcjoKKyAgICAgICAgICAgIGxvZy5kZWJ1ZygnSU9FcnJvciB3aGlsZSBjb25uZWN0aW5nIHRv IFNNVFA6ICcgKyBzdHIoZXJyb3IpKQogICAgICAgICAgICAgc2VsZi5xdWl0KCkKICAgICAgICAg ICAgIHJhaXNlCiAgICAgICAgIGV4Y2VwdCBFeGNlcHRpb24gYXMgZXJyb3I6CkBAIC0xODcsNiAr MjAwLDkgQEAKIAogICAgIGRlZiBfbG9naW4oc2VsZik6CiAgICAgICAgICIiIlNlbmQgbG9naW4g aWYgYm90aCB1c2VybmFtZSBhbmQgcGFzc3dvcmQgYXJlIHNwZWNpZmllZC4iIiIKKyAgICAgICAg aWYgc2VsZi5fY29ubmVjdGlvbiBpcyBOb25lOgorICAgICAgICAgICAgbG9nLmRlYnVnKCdsb2dp bjogbm8gY29ubmVjdGlvbicpCisgICAgICAgICAgICByZXR1cm4KICAgICAgICAgaWYgc2VsZi5f dXNlcm5hbWUgaXMgbm90IE5vbmUgYW5kIHNlbGYuX3Bhc3N3b3JkIGlzIG5vdCBOb25lOgogICAg ICAgICAgICAgbG9nLmRlYnVnKCdsb2dnaW5nIGluJykKICAgICAgICAgICAgIHRyeToK --===============3664788302143954096== Content-Type: text/x-patch Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="mailman-listen.patch" MIME-Version: 1.0 ZGlmZiAtdSAtLW1pbmltYWwgLXIgJy0tZXhjbHVkZT0qLnB5YycgbWFpbG1hbi9jb25maWcvc2No ZW1hLmNmZyBtYWlsbWFuL2NvbmZpZy9zY2hlbWEuY2ZnCi0tLSBtYWlsbWFuL2NvbmZpZy9zY2hl bWEuY2ZnCTIwMjAtMDktMjggMjE6MzM6NTMuMTk0MDU5MDkwICswMTAwCisrKyBtYWlsbWFuL2Nv bmZpZy9zY2hlbWEuY2ZnCTIwMjAtMDktMDQgMDA6NDA6NDUuMzk4ODEwMTc1ICswMTAwCkBAIC0z NzgsOSArMzc4LDEyIEBACiAKIAogW3dlYnNlcnZpY2VdCi0jIFRoZSBob3N0bmFtZSBhdCB3aGlj aCBhZG1pbiB3ZWIgc2VydmljZSByZXNvdXJjZXMgYXJlIGV4cG9zZWQuCisjIFRoZSBhZHZlcnRp c2VkIGhvc3RuYW1lIGF0IHdoaWNoIGFkbWluIHdlYiBzZXJ2aWNlIHJlc291cmNlcyBhcmUgZXhw b3NlZC4KIGhvc3RuYW1lOiBsb2NhbGhvc3QKIAorIyBUaGUgSVAgYWRkcmVzcyBhdCB3aGljaCBw b3N0b3JpdXMgc2VydmVyIGxpc3RlbnMgZm9yIHJlcXVlc3RzLgorbGlzdGVuOiAwLjAuMC4wCisK ICMgVGhlIHBvcnQgYXQgd2hpY2ggdGhlIGFkbWluIHdlYiBzZXJ2aWNlIHJlc291cmNlcyBhcmUg ZXhwb3NlZC4KIHBvcnQ6IDgwMDEKIApkaWZmIC11IC0tbWluaW1hbCAtciAnLS1leGNsdWRlPSou cHljJyBtYWlsbWFuL3Jlc3QvZ3VuaWNvcm4ucHkgbWFpbG1hbi9yZXN0L2d1bmljb3JuLnB5Ci0t LSBtYWlsbWFuL3Jlc3QvZ3VuaWNvcm4ucHkJMjAyMC0wOS0yOCAyMTozMzo1My4yNDYwNTg0MTcg KzAxMDAKKysrIG1haWxtYW4vcmVzdC9ndW5pY29ybi5weQkyMDIwLTA5LTI0IDE4OjMyOjA4LjA2 NjY1MTYwMCArMDEwMApAQCAtNTMsMTEgKzUzLDExIEBACiAgICAgIyBXZSBsb2FkIHNvbWUgb3B0 aW9ucyBmcm9tIGNvbmZpZy53ZWJzZXJ2aWNlLCB3aGlsZSBvdGhlciBleHRlbmRlZCBvcHRpb25z CiAgICAgIyBmb3IgZ3VuaWNvcm4gY2FuIGJlIGRlZmluZWQgaW4gY29uZmlndXJhdGlvbjogc2Vj dGlvbi4gV2UgYWxzbyBsb2FkIHVwCiAgICAgIyBzb21lIGxvZ2dpbmcgb3B0aW9ucyBzaW5jZSBn dW5pY29ybiBzZXRzIHVwIGl0J3Mgb3duIGxvZ2dlcnMuCi0gICAgaG9zdCA9IGNvbmZpZy53ZWJz ZXJ2aWNlLmhvc3RuYW1lCisgICAgbGlzdGVuID0gY29uZmlnLndlYnNlcnZpY2UubGlzdGVuCiAg ICAgcG9ydCA9IGludChjb25maWcud2Vic2VydmljZS5wb3J0KQogICAgIGxvZ19wYXRoID0gb3Mu cGF0aC5qb2luKGNvbmZpZy5MT0dfRElSLCBjb25maWcubG9nZ2luZy5odHRwWydwYXRoJ10pCiAg ICAgb3B0aW9ucyA9IHsKLSAgICAgICAgJ2JpbmQnOiAne306e30nLmZvcm1hdChob3N0LCBwb3J0 KSwKKyAgICAgICAgJ2JpbmQnOiAne306e30nLmZvcm1hdChsaXN0ZW4sIHBvcnQpLAogICAgICAg ICAnYWNjZXNzbG9nJzogbG9nX3BhdGgsCiAgICAgICAgICdlcnJvcmxvZyc6IGxvZ19wYXRoLAog ICAgICAgICAnbG9nbGV2ZWwnOiBjb25maWcubG9nZ2luZy5odHRwWydsZXZlbCddLApAQCAtNzIs MyArNzIsNCBAQAogICAgICAgICBvcHRpb25zW2tleV0gPSBndW5pY29ybl9jb25maWdbJ2d1bmlj b3JuJ11ba2V5XQogCiAgICAgcmV0dXJuIEd1bmljb3JuQXBwbGljYXRpb24obWFrZV9hcHBsaWNh dGlvbigpLCBvcHRpb25zKQorCg== --===============3664788302143954096== Content-Type: text/x-patch Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="mailman-runner.patch" MIME-Version: 1.0 ZGlmZiAtdSAtLW1pbmltYWwgLXIgJy0tZXhjbHVkZT0qLnB5YycgbGliL3B5dGhvbjMuNi9zaXRl LXBhY2thZ2VzL21haWxtYW4vY29yZS9ydW5uZXIucHkgbGliL3B5dGhvbjMuNi9zaXRlLXBhY2th Z2VzL21haWxtYW4vY29yZS9ydW5uZXIucHkKLS0tIGxpYi9weXRob24zLjYvc2l0ZS1wYWNrYWdl cy9tYWlsbWFuL2NvcmUvcnVubmVyLnB5CTIwMjAtMDktMjggMjE6MzM6NTMuMTk0MDU5MDkwICsw MTAwCisrKyBsaWIvcHl0aG9uMy42L3NpdGUtcGFja2FnZXMvbWFpbG1hbi9jb3JlL3J1bm5lci5w eQkyMDIwLTA5LTAyIDE4OjM5OjIyLjAzMDc5ODUxMiArMDEwMApAQCAtMTQ1LDcgKzE0NSw3IEBA CiAKICAgICBkZWYgX29uZV9pdGVyYXRpb24oc2VsZik6CiAgICAgICAgICIiIlNlZSBgSVJ1bm5l cmAuIiIiCi0gICAgICAgIG1lID0gc2VsZi5fX2NsYXNzX18uX19uYW1lX18KKyAgICAgICAgbWUg PSBzZWxmLm5hbWUKICAgICAgICAgZGxvZy5kZWJ1ZygnWyVzXSBzdGFydGluZyBvbmVsb29wJywg bWUpCiAgICAgICAgICMgTGlzdCBhbGwgdGhlIGZpbGVzIGluIG91ciBxdWV1ZSBkaXJlY3Rvcnku ICBUaGUgc3dpdGNoYm9hcmQgaXMKICAgICAgICAgIyBndWFyYW50ZWVkIHRvIGhhbmQgdXMgdGhl IGZpbGVzIGluIEZJRk8gb3JkZXIuCkBAIC0yOTEsNyArMjkxLDEwIEBACiAKICAgICBkZWYgX3Nu b296ZShzZWxmLCBmaWxlY250KToKICAgICAgICAgIiIiU2VlIGBJUnVubmVyYC4iIiIKLSAgICAg ICAgaWYgZmlsZWNudCBvciBzZWxmLnNsZWVwX2Zsb2F0IDw9IDA6CisgICAgICAgIG1lID0gc2Vs Zi5uYW1lCisgICAgICAgIGRsb2cuZGVidWcoJ1slc10gc25vb3plIChjb3VudD0lZCkgJXMnLCBt ZSwgZmlsZWNudCwgc2VsZi5zbGVlcF9mbG9hdCkKKyAgICAgICAgIyBJZiB0aGVyZSB3YXMgc3Rp bGwgd29yaywgb3Igc2xlZXB0aW1lIGlzIGRpc2FibGVkLgorICAgICAgICBpZiBmaWxlY250ID4g MCBvciBzZWxmLnNsZWVwX2Zsb2F0IDw9IDA6CiAgICAgICAgICAgICByZXR1cm4KICAgICAgICAg dGltZS5zbGVlcChzZWxmLnNsZWVwX2Zsb2F0KQogCg== --===============3664788302143954096==-- From mark@msapiro.net Wed Sep 30 14:49:19 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Various patches for mm3 stable 3.3.1 Date: Wed, 30 Sep 2020 07:49:09 -0700 Message-ID: <7405f1a9-c2ab-17fc-c810-c6b98218b003@msapiro.net> In-Reply-To: <49ea79e4-72c3-a6bc-805e-31d40b91c9a2@ivimey.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3581538903706353041==" --===============3581538903706353041== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/30/20 6:57 AM, Ruth Ivimey-Cook wrote: > Folks, > > For my own purposes I have created some patches which apply to the > current mailman pip package. I'm only responding here to the issue of how to suggest patches. If you hope that they could be incorporated into the distribution, posting them to this list is not very effective. It is too easy for list posts to get lost, buried or forgotten. The best thing to do is to create an issue or issues at (each issue should address only one thing). Then you can either attach the suggested patch to the issue or better, create a merge request with the patch to fix the issue. Ideally, the merge request would also contain relevant additions to unit tests. The first advantage is open issues do not get lost. They are always there in the tracker. The second is that all of the above is work that needs to be done in order to actually get the patch into the master branch, and the more of it you do, the less of it needs to be done by someone else who may not be as motivated as you are. Granted, this may seem like a burden for you. After all, you fixed your problem so why bother, but open source works best if we all contribute what we can to improve the product. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============3581538903706353041==-- From gordon@dickens.com Wed Sep 30 15:21:27 2020 From: gordon@dickens.com To: mailman-users@mailman3.org Subject: [MM3-users] Re: Cron error Date: Wed, 30 Sep 2020 15:21:23 +0000 Message-ID: <160147928325.4681.5007494210726426321@mailman.iad1.psf.io> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0521536503634656950==" --===============0521536503634656950== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The error has not re-occurred in the last 24 hours since I removed the lock f= ile as Mark recommended. So, I am hopefull that the problem is solved. FYI, Gordon --===============0521536503634656950==-- From mark@msapiro.net Wed Sep 30 15:52:48 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Various patches for mm3 stable 3.3.1 Date: Wed, 30 Sep 2020 08:52:38 -0700 Message-ID: <237ed32d-35a8-0270-cdbe-3459cc6931c8@msapiro.net> In-Reply-To: <49ea79e4-72c3-a6bc-805e-31d40b91c9a2@ivimey.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5978569760687523106==" --===============5978569760687523106== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/30/20 6:57 AM, Ruth Ivimey-Cook wrote: > > 1. If I do not set up 'individual' or 'full' personalisation, all is > well and the default footer templates are expanded as expected. However > setting up personalisation causes template expansion to fail > completely, and messages get sent with a blank expansion. Your mailman-debug.patch makes changes to mailman/handlers/decorate.py including wrapping all the relevant code in a try:/except: that logs any exception. If you set the level for logging.error to debug, what do you get in the log? This issue doesn't occur in the unpatched code on my servers - personalized footers work fine - so I think the problem must be something in your patch. > 2. I would find the postorius UI much easier to cope with if the > default template was 'populated' into the UI wherever it was non-blank, > rather than the UI starting off as if no templates were defined at all. This is . -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============5978569760687523106==-- From mark@msapiro.net Wed Sep 30 16:00:43 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: Cron error Date: Wed, 30 Sep 2020 09:00:30 -0700 Message-ID: <3de12e5e-8c36-ebc4-b78f-968b25ea94f0@msapiro.net> In-Reply-To: <160147928325.4681.5007494210726426321@mailman.iad1.psf.io> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1060898027874436638==" --===============1060898027874436638== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/30/20 8:21 AM, gordon(a)dickens.com wrote: > The error has not re-occurred in the last 24 hours since I removed the lock= file as Mark recommended. So, I am hopefull that the problem is solved. As I suggested at , I don't think the underlying issue was with the `mailman digests --send` run by cron. I think any `mailman` command would have failed as long as the dbcreate.db lock residue was in /var/lib/mailman3/locks/. The question is what left that there in the first place. Did you manually run any `mailman` commands that terminated abnormally or were killed in the day before this started? --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============1060898027874436638==-- From mmasoudf@aaas.org Wed Sep 30 16:25:45 2020 From: Mohsen Masoudfar To: mailman-users@mailman3.org Subject: [MM3-users] How to prevent members to attach executables (exe, bat, ...) to the emails sent to list Date: Wed, 30 Sep 2020 16:25:40 +0000 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4188738293500748016==" --===============4188738293500748016== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi, I am trying to find a way for content filtering to remove executables from th= e emails sent to the list. I found some hints under: https://mailman.readthedocs.io/en/stable/build/lib/mailman/handlers/docs/filt= ering.html?highlight=3Dcontent%20filter but still not clear how to do it for all type of excecutables. Best regards Mohsen --===============4188738293500748016==-- From brian_carpenter@emwd.com Wed Sep 30 16:28:45 2020 From: Brian Carpenter To: mailman-users@mailman3.org Subject: [MM3-users] Re: Various patches for mm3 stable 3.3.1 Date: Wed, 30 Sep 2020 12:14:13 -0400 Message-ID: In-Reply-To: <237ed32d-35a8-0270-cdbe-3459cc6931c8@msapiro.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3836642801148054994==" --===============3836642801148054994== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 9/30/20 11:52 AM, Mark Sapiro wrote: >> 2. I would find the postorius UI much easier to cope with if the >> default template was 'populated' into the UI wherever it was non-blank, >> rather than the UI starting off as if no templates were defined at all. Our Affinity admin interface does that: Affinity Template -- Brian Carpenter Harmonylists.com Emwd.com --===============3836642801148054994==-- From mark@msapiro.net Wed Sep 30 17:07:04 2020 From: Mark Sapiro To: mailman-users@mailman3.org Subject: [MM3-users] Re: How to prevent members to attach executables (exe, bat, ...) to the emails sent to list Date: Wed, 30 Sep 2020 10:06:53 -0700 Message-ID: <494b798c-6351-cd89-610a-6c8babae7123@msapiro.net> In-Reply-To: =?utf-8?q?=3CMN2PR20MB3245E5A68A259869048C63A0B6330=40MN2PR20MB?= =?utf-8?q?3245=2Enamprd20=2Eprod=2Eoutlook=2Ecom=3E?= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1398447250237252535==" --===============1398447250237252535== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On 9/30/20 9:25 AM, Mohsen Masoudfar wrote: > Hi, >=20 > I am trying to find a way for content filtering to remove executables from = the emails sent to the list. I found some hints under: > https://mailman.readthedocs.io/en/stable/build/lib/mailman/handlers/docs/fi= ltering.html?highlight=3Dcontent%20filter >=20 > but still not clear how to do it for all type of excecutables. There are four list settings that affect this. They are pass_types, pass_extensions, filter_types and filter_extensions. the pass_* settings specify message parts the list will accept while the filter_* settings specify message parts the list won't accept. The page at only discusses filter_types and needs updating for the others. Filtering by extension only works on message parts that have an associated file name. Thus, while you can add things like 'exe', 'bat', 'cmd', 'com', 'pif', 'scr', 'vbs' and 'cpl' to filter_extensions, it won't be completely effective. A better approach is to use MIME types[1] and only allow those you want. For a discussion list, a reasonable set is what this list uses: multipart message/rfc822 text application/pgp-signature This will accept any multipart and message/rfc822 parts for further analysis and ultimately accept only text/* and application/pgp-signature elemental parts. If you want to accept images and pdfs, you can add image application/pdf See for a list of all registered MIME types. Note that these are all of the form main_type/sub_type and putting just main_type in *_types passes or filters all subtypes for that maintype. [1] You can use both, e.g. filter_extensions to remove those you don't want and pass_types to accept the MIME types you do want. --=20 Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan --===============1398447250237252535==--