- Mailman-users - lists.mailman3.org

update_index
by Darren Smith 03 Apr '18

03 Apr '18

Hello, If I'm reading the docs correctly, when we run manage.py update_index, it should be an incremental index of new emails that have been added since the last time update_index was run - can anyone confirm that this is the expected behavior? Or is there another command line option that I am missing? Right now we are indexing EVERYTHING every time that this is run. Probably this wouldn't be much of a problem except that we are importing all of our old archives (they go back to 1998 and comprise over 40.5 million emails) into the new system. As of now we have around 700,000 emails imported into hyperkitty, and since it is indexing everything it is taking around 1 hour and 10 minutes. Obviously we can't run that once a minute. Right now we're doing once a day, meaning that new emails won't be indexed until the next day. Fast forward to when we have all 40 million imported...the full index process will take (we estimate) around 4 days or so. So...can anyone help with how to get the update_index to do only emails added since the last time the update_index was run? Thanks! Darren

2 4

Mailman 3 with AWS auto scaling and potentially load balancing
by Chris McMacken 30 Mar '18

30 Mar '18

After doing a lot of googling I can't seem to find solid answers to a couple of basic mailman hosting questions so I thought I would ask here. I'm looking to build an auto scaling group so that my mailman services can self heal if the EC2 instance goes away. What state if any is stored locally on the filesystem that would need to be backed up and restored when an instance is replaced? I'm also interested in a high availability deployment so I would like to do some basic load balancing to ensure that this requirement is met but I haven't been able to find much on this topic for mailman 3. Can someone point me in the direction of a write up on this problem or comment on it?

2 5

Re: Templates and docker-mailman
by Hathaway, John W. 22 Mar '18

22 Mar '18

Hah, adding the extension did the trick. Should have inferred that. Thanks! John W. Hathaway | IT Director LOCKRIDGE GRINDAL NAUEN P.L.L.P. 100 Washington Avenue S | Suite 2200 | Minneapolis MN 55401 V: 612-339-6900 | F: 612-339-0981 | www.locklaw.com<http://www.locklaw.com/> This e-mail may contain information that is privileged, confidential or otherwise protected from disclosure. If you are not the intended recipient or otherwise have received this message in error, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you are not the intended recipient or otherwise have received this message in error, please notify us immediately by e-mail, discard any paper copies and delete all electronic files of the message.

1 0

TypeError: authenticate() takes exactly 0 arguments (3 given)
by Bjoern Franke 22 Mar '18

22 Mar '18

Hi, after migrating from a PIP install to Debian Backports packages, I get the following error: https://pastebin.com/tCnTuKYz The whole config of mailman3-suite: https://pastebin.com/nzJufYBN Regards Bjoern

2 4

Templates and docker-mailman
by jwhathaway＠locklaw.com 20 Mar '18

20 Mar '18

I have mailman installed through docker-mailman and I'm trying to get rid of the footer decoration in our messages. Based on earlier threads, I have placed an empty file on the host running Docker at: /opt/mailman/core/var/templates/site/en/list:member:generic:footer But this is not being honored by the mailman container. I have restarted all of the containers. Is there something else I need to do?

3 3

Re: Getting rid of held messages from the command line
by Mark Sapiro 19 Mar '18

19 Mar '18

On 03/19/2018 02:22 PM, Aymeric Mansoux wrote: > > Ah, so if I understand you correctly, as long as I'm on 3.1.1, I could > run into this issue again, but then: Yes, this issue will recur with Mailman 3.1.1 and a MySQL database. > - Would patching 3.1.1 with 5477a201 be enough to prevent further issues > until the next stable is released? I think so. The only potential issue is if mailman/database/alembic/versions/4bd95c99b2e7_fix_template_password.py is not in 3.1.1. I think it is, so I think it's OK, but if not, you'd need to modify mailman/database/alembic/versions/3f31035ed0d7_further_increase_pendedkeyvalue_value_.py to point to the 3.1.1 head of the alembic versions. > - How can I clean the current pendedkeyvalue table? With phpMyAdmin or the mysql command. I think you can just delete those entries from the pendedkeyvalue table. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

Getting rid of held messages from the command line
by Aymeric Mansoux 17 Mar '18

17 Mar '18

Hello, We're currently running some of Mailman software, as available via pip, namely: - mailman 3.1.1 (pip3) - postorius 1.1.2 (pip2) - hyperkitty 1.1.4 (pip2) - mailmanclient 3.1.1 (pip3) So far so good, however, for some reasons, a couple of our lists have started to accumulate held messages that cannot be manipulated via the REST API. Regardless if we use the web interface or mailmanclient, it ends up with: Mar 14 10:58:32 2018 (24287) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3.5/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/wsgiapp.py", line 214, in __call__ return super().__call__(environ, start_response) File "/usr/local/lib/python3.5/dist-packages/falcon/api.py", line 244, in __call__ responder(req, resp, **params) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 167, in on_get resource = self._make_collection(request) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/local/lib/python3.5/dist-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 157, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 70, in _make_resource resource = super()._make_resource(request_id) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 39, in _make_resource results = requests.get_request(request_id) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/model/requests.py", line 120, in get_request result.data_hash, expunge=False) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/model/pending.py", line 138, in confirm value = json.loads(keyvalue.value) File "/usr/lib/python3.5/json/__init__.py", line 319, in loads return _default_decoder.decode(s) File "/usr/lib/python3.5/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.5/json/decoder.py", line 355, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 1 (char 0) Mar 14 10:58:32 2018 (24287) 127.0.0.1 - - "GET /3.1/lists/xx(a)xx.xx/held HTTP/1.1" 500 59 Is there a way to manipulate/see/remove these mails differently? Right now this prevent moderation of any held messages (note: some lists do not suffer from this and the held panel or mailmanclient works perfectly fine). Thanks!

3 4

disable sign-ups
by Peter Münster 14 Mar '18

14 Mar '18

Hi, I would like to add new accounts only manually, the mailman3 site should not be open for sign-ups. So I've added this code to settings.py: --8<---------------cut here---------------start------------->8--- from allauth.account.adapter import DefaultAccountAdapter class MyAccountAdapter(DefaultAccountAdapter): def is_open_for_signup(self, request): return False ACCOUNT_ADAPTER = 'MyAccountAdapter' --8<---------------cut here---------------end--------------->8--- It seems to work, but the user experience is not optimal: the user can click on "Sign Up", and then gets to "Server Error (500)". This is the log: --8<---------------cut here---------------start------------->8--- ERROR 2018-01-31 21:53:47,116 6769 django.request Internal Server Error: /accounts/signup/ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 34, in _wrapper return bound_func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper return view(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 30, in bound_func return func.__get__(self, type(self))(*args2, **kwargs2) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 210, in dispatch return super(SignupView, self).dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 79, in dispatch **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 182, in dispatch if not self.is_open(): File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 191, in is_open return get_adapter(self.request).is_open_for_signup(self.request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/adapter.py", line 501, in get_adapter return import_attribute(app_settings.ADAPTER)(request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/utils.py", line 150, in import_attribute pkg, attr = path.rsplit('.', 1) ValueError: need more than 1 value to unpack ERROR 2018-01-31 21:53:47,116 6769 django.request Internal Server Error: /accounts/signup/ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 34, in _wrapper return bound_func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/decorators/debug.py", line 76, in sensitive_post_parameters_wrapper return view(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/utils/decorators.py", line 30, in bound_func return func.__get__(self, type(self))(*args2, **kwargs2) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 210, in dispatch return super(SignupView, self).dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 79, in dispatch **kwargs) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 182, in dispatch if not self.is_open(): File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/views.py", line 191, in is_open return get_adapter(self.request).is_open_for_signup(self.request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/account/adapter.py", line 501, in get_adapter return import_attribute(app_settings.ADAPTER)(request) File "/usr/lib/python2.7/site-packages/django_allauth-0.34.0-py2.7.egg/allauth/utils.py", line 150, in import_attribute pkg, attr = path.rsplit('.', 1) ValueError: need more than 1 value to unpack --8<---------------cut here---------------end--------------->8--- Do you have any hints, how to avoid such errors please? TIA for any help, -- Peter

3 2

Internal Server Error: /hyperkitty/api/mailman/archive: ValueError: A string literal cannot contain NUL (0x00) characters.
by Henrik Rasmussen 14 Mar '18

14 Mar '18

This could potentially be associated with [1] and [2], but it does not regard import21. When mails are released or otherwise archived I get an Internal Server Error regarding "ValueError: A string literal cannot contain NUL (0x00) characters." in the web/mailmanweb.log, but unlike importing corrupted archives, this is mails coming from the users in the daily life. Would it be possible for Mailman to check for NUL characters and skip those before archiving? ERROR 2018-03-13 10:38:12,602 39 django.request Internal Server Error: /hyperkitty/api/mailman/archive Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python2.7/site-packages/hyperkitty/views/mailman.py", line 68, in _decorator return func(request, *args, **kwargs) File "/usr/local/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view return view_func(*args, **kwargs) File "/usr/local/lib/python2.7/site-packages/hyperkitty/views/mailman.py", line 114, in archive add_to_list(mlist_fqdn, msg) File "/usr/local/lib/python2.7/site-packages/hyperkitty/lib/incoming.py", line 149, in add_to_list email.save() File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 796, in save force_update=force_update, update_fields=update_fields) File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 824, in save_base updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 908, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 947, in _do_insert using=using, raw=raw) File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 1043, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1054, in execute_sql cursor.execute(sql, params) File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) ValueError: A string literal cannot contain NUL (0x00) characters. [1] https://gitlab.com/mailman/hyperkitty/issues/155 [2] https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/….

1 1

A short link to this page -- TinyURL
by John Griessen 14 Mar '18

14 Mar '18

The wiki page has a convenient TinyURL. Is it feasible to make TinyURLs for archived list threads such as: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…

3 2