- Mailman-users - lists.mailman3.org

Mailman 3 with AWS auto scaling and potentially load balancing
by Chris McMacken 30 Mar '18

30 Mar '18

After doing a lot of googling I can't seem to find solid answers to a couple of basic mailman hosting questions so I thought I would ask here. I'm looking to build an auto scaling group so that my mailman services can self heal if the EC2 instance goes away. What state if any is stored locally on the filesystem that would need to be backed up and restored when an instance is replaced? I'm also interested in a high availability deployment so I would like to do some basic load balancing to ensure that this requirement is met but I haven't been able to find much on this topic for mailman 3. Can someone point me in the direction of a write up on this problem or comment on it?

2 5

Re: Templates and docker-mailman
by Hathaway, John W. 22 Mar '18

22 Mar '18

Hah, adding the extension did the trick. Should have inferred that. Thanks! John W. Hathaway | IT Director LOCKRIDGE GRINDAL NAUEN P.L.L.P. 100 Washington Avenue S | Suite 2200 | Minneapolis MN 55401 V: 612-339-6900 | F: 612-339-0981 | www.locklaw.com<http://www.locklaw.com/> This e-mail may contain information that is privileged, confidential or otherwise protected from disclosure. If you are not the intended recipient or otherwise have received this message in error, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you are not the intended recipient or otherwise have received this message in error, please notify us immediately by e-mail, discard any paper copies and delete all electronic files of the message.

1 0

TypeError: authenticate() takes exactly 0 arguments (3 given)
by Bjoern Franke 22 Mar '18

22 Mar '18

Hi, after migrating from a PIP install to Debian Backports packages, I get the following error: https://pastebin.com/tCnTuKYz The whole config of mailman3-suite: https://pastebin.com/nzJufYBN Regards Bjoern

2 4

Templates and docker-mailman
by jwhathaway＠locklaw.com 20 Mar '18

20 Mar '18

I have mailman installed through docker-mailman and I'm trying to get rid of the footer decoration in our messages. Based on earlier threads, I have placed an empty file on the host running Docker at: /opt/mailman/core/var/templates/site/en/list:member:generic:footer But this is not being honored by the mailman container. I have restarted all of the containers. Is there something else I need to do?

3 3

Re: Getting rid of held messages from the command line
by Mark Sapiro 19 Mar '18

19 Mar '18

On 03/19/2018 02:22 PM, Aymeric Mansoux wrote: > > Ah, so if I understand you correctly, as long as I'm on 3.1.1, I could > run into this issue again, but then: Yes, this issue will recur with Mailman 3.1.1 and a MySQL database. > - Would patching 3.1.1 with 5477a201 be enough to prevent further issues > until the next stable is released? I think so. The only potential issue is if mailman/database/alembic/versions/4bd95c99b2e7_fix_template_password.py is not in 3.1.1. I think it is, so I think it's OK, but if not, you'd need to modify mailman/database/alembic/versions/3f31035ed0d7_further_increase_pendedkeyvalue_value_.py to point to the 3.1.1 head of the alembic versions. > - How can I clean the current pendedkeyvalue table? With phpMyAdmin or the mysql command. I think you can just delete those entries from the pendedkeyvalue table. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

Getting rid of held messages from the command line
by Aymeric Mansoux 17 Mar '18

17 Mar '18

Hello, We're currently running some of Mailman software, as available via pip, namely: - mailman 3.1.1 (pip3) - postorius 1.1.2 (pip2) - hyperkitty 1.1.4 (pip2) - mailmanclient 3.1.1 (pip3) So far so good, however, for some reasons, a couple of our lists have started to accumulate held messages that cannot be manipulated via the REST API. Regardless if we use the web interface or mailmanclient, it ends up with: Mar 14 10:58:32 2018 (24287) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3.5/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/wsgiapp.py", line 214, in __call__ return super().__call__(environ, start_response) File "/usr/local/lib/python3.5/dist-packages/falcon/api.py", line 244, in __call__ responder(req, resp, **params) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 167, in on_get resource = self._make_collection(request) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/local/lib/python3.5/dist-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 157, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 70, in _make_resource resource = super()._make_resource(request_id) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 39, in _make_resource results = requests.get_request(request_id) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/model/requests.py", line 120, in get_request result.data_hash, expunge=False) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/model/pending.py", line 138, in confirm value = json.loads(keyvalue.value) File "/usr/lib/python3.5/json/__init__.py", line 319, in loads return _default_decoder.decode(s) File "/usr/lib/python3.5/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.5/json/decoder.py", line 355, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 1 (char 0) Mar 14 10:58:32 2018 (24287) 127.0.0.1 - - "GET /3.1/lists/xx(a)xx.xx/held HTTP/1.1" 500 59 Is there a way to manipulate/see/remove these mails differently? Right now this prevent moderation of any held messages (note: some lists do not suffer from this and the held panel or mailmanclient works perfectly fine). Thanks!

3 4

disable sign-ups
by Peter Münster 14 Mar '18

14 Mar '18

Hi, I would like to add new accounts only manually, the mailman3 site should not be open for sign-ups. So I've added this code to settings.py: --8<---------------cut here---------------start------------->8--- from allauth.account.adapter import DefaultAccountAdapter class MyAccountAdapter(DefaultAccountAdapter): def is_open_for_signup(self, request): return False ACCOUNT_ADAPTER = 'MyAccountAdapter' --8<---------------cut here---------------end--------------->8--- It seems to work, but the user experience is not optimal: the user can click on "Sign Up", and then gets to "Server Error (500)". This is the log: --8<---------------cut here---------------start------------->8--- ERROR 2018-01-31 21:53:47,116 6769 django.request Internal Server Error: /accounts/signup/ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 34, in _wrapper return bound_func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper return view(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 30, in bound_func return func.__get__(self, type(self))(*args2, **kwargs2) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 210, in dispatch return super(SignupView, self).dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 79, in dispatch **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 182, in dispatch if not self.is_open(): File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 191, in is_open return get_adapter(self.request).is_open_for_signup(self.request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/adapter.py", line 501, in get_adapter return import_attribute(app_settings.ADAPTER)(request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/utils.py", line 150, in import_attribute pkg, attr = path.rsplit('.', 1) ValueError: need more than 1 value to unpack ERROR 2018-01-31 21:53:47,116 6769 django.request Internal Server Error: /accounts/signup/ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 34, in _wrapper return bound_func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper return view(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 30, in bound_func return func.__get__(self, type(self))(*args2, **kwargs2) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 210, in dispatch return super(SignupView, self).dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 79, in dispatch **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 182, in dispatch if not self.is_open(): File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 191, in is_open return get_adapter(self.request).is_open_for_signup(self.request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/adapter.py", line 501, in get_adapter return import_attribute(app_settings.ADAPTER)(request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/utils.py", line 150, in import_attribute pkg, attr = path.rsplit('.', 1) ValueError: need more than 1 value to unpack --8<---------------cut here---------------end--------------->8--- Do you have any hints, how to avoid such errors please? TIA for any help, -- Peter

3 2

Internal Server Error: /hyperkitty/api/mailman/archive: ValueError: A string literal cannot contain NUL (0x00) characters.
by Henrik Rasmussen 14 Mar '18

14 Mar '18

This could potentially be associated with [1] and [2], but it does not regard import21. When mails are released or otherwise archived I get an Internal Server Error regarding "ValueError: A string literal cannot contain NUL (0x00) characters." in the web/mailmanweb.log, but unlike importing corrupted archives, this is mails coming from the users in the daily life. Would it be possible for Mailman to check for NUL characters and skip those before archiving? ERROR 2018-03-13 10:38:12,602 39 django.request Internal Server Error: /hyperkitty/api/mailman/archive Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python2.7/site-packages/hyperkitty/views/mailman.py", line 68, in _decorator return func(request, *args, **kwargs) File "/usr/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view return view_func(*args, **kwargs) File "/usr/local/lib/python2.7/site-packages/hyperkitty/views/mailman.py", line 114, in archive add_to_list(mlist_fqdn, msg) File "/usr/local/lib/python2.7/site-packages/hyperkitty/lib/incoming.py", line 149, in add_to_list email.save() File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 796, in save force_update=force_update, update_fields=update_fields) File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 824, in save_base updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 908, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 947, in _do_insert using=using, raw=raw) File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 1043, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1054, in execute_sql cursor.execute(sql, params) File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) ValueError: A string literal cannot contain NUL (0x00) characters. [1] https://gitlab.com/mailman/hyperkitty/issues/155 [2] https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/….

1 1

A short link to this page -- TinyURL
by John Griessen 14 Mar '18

14 Mar '18

The wiki page has a convenient TinyURL. Is it feasible to make TinyURLs for archived list threads such as: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…

3 2

Database running VERY hot
by Darren Smith 12 Mar '18

12 Mar '18

Hello all, We had an old installation of Mailman 2.1 across three servers with over 32,000 mailing lists on them. Due to circumstances beyond my control, we had to shut them down and quickly set up Mailman 3 on a new server. We have Mailman3 set up on a single server this time, as we were having problems with a 3-server setup and Hyperkitty and archives. Long story short, it is up and running, and our users are starting to come back. However, we do not have very many users hitting Postorius and Hyperkitty yet, and we are seeing a distressing number of 504 gateway timeouts. We noticed that there is a significant number of queries to the API whenever hitting Postorius's front page to load the mailing lists, and then on every load. It looks as if it makes a single call to the API to get information about a single list, and does this 10 times - once for each list - to render the page. This gets much worse when users set the view to 200 per page, as they are inclined to do with 32,000 mailing lists... What we've done, actually, is to override the front page and point the users to a different set of static pages where we have all of our lists categorized and searchable, and when they find the list they are looking for, we have a link back to postorius. It has taken a lot of the stress off of the server. However, I still see where the database at times can run VERY hot. We have set up this server to run with PostgreSQL. When I do a ps aux | grep postgres, I find a lot of processes that have been spawned to take care of mailman - a dozen or so. However, one of the threads is taking up about 85% of the load, another thread about 14%, and little dribbles here and there from the rest of the threads. Here are the questions: 1) Has anyone run into problems like these? I would think that maybe this is an issue having so many mailing lists, but honestly there aren't that many people using the system yet. It is going to ramp up as the users realize that we are live again, but we aren't getting that much traffic. 2) Is there any configuration that I can set, especially with regards to the database, so that it can be used more efficiently and not throttle through one or two threads? Please let me know what further information I can provide. Thanks in advance for any help or advice! Silas

5 11