November 2023 - Mailman-users - lists.mailman3.org

hyperkitty.lib.utils Failed to update the fulltext index
by Ben Harris 13 Apr '24

13 Apr '24

hello there, i migrated to a new host last night and i've been getting this error from a cronjob mail every minute. anyone have any ideas what would cause this? ERROR 2019-04-17 14:26:33,056 27205 hyperkitty.lib.utils Failed to update the fulltext index: ord() expected a character, but string of length 0 found Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/hyperkitty/lib/utils.py", line 186, in run_with_lock fn(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/hyperkitty/search_indexes.py", line 87, in update_index update_cmd.update_backend("hyperkitty", "default") File "/usr/lib/python2.7/dist-packages/haystack/management/commands/update_index.py", line 276, in update_backend last_max_pk=max_pk) File "/usr/lib/python2.7/dist-packages/haystack/management/commands/update_index.py", line 97, in do_update backend.update(index, current_qs, commit=commit) File "/usr/lib/python2.7/dist-packages/haystack/backends/whoosh_backend.py", line 178, in update self.setup() File "/usr/lib/python2.7/dist-packages/haystack/backends/whoosh_backend.py", line 126, in setup self.index = self.storage.open_index(schema=self.schema) File "/usr/lib/python2.7/dist-packages/whoosh/filedb/filestore.py", line 176, in open_index return indexclass(self, schema=schema, indexname=indexname) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 421, in __init__ TOC.read(self.storage, self.indexname, schema=self._schema) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 632, in read check_size("int", _INT_SIZE) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 626, in check_size sz = stream.read_varint() File "/usr/lib/python2.7/dist-packages/whoosh/filedb/structfile.py", line 191, in read_varint return read_varint(self.read) File "/usr/lib/python2.7/dist-packages/whoosh/util/varints.py", line 102, in read_varint b = ord(readfn(1)) thanks, ben

5 8

export/import list?
by Dave McGuire 01 Mar '24

01 Mar '24

I'm migrating to a new Mailman3 instance, finally escaping a fairly old (3.1.1) and problematic distribution repo install on a Linux VM and moving to a venv install in a native SmartOS Zone. I've gotten everything running well and integrated with our (fairly complex) mail server. I'd like to begin migration of the lists on the old mm3 instance. Is there a way to export a list from one Mailman 3 instance and import it into another, on a different machine? Ideally some way to capture its settings, members, members' settings, etc? Thanks, -Dave -- Dave McGuire, AK4HZ New Kensington, PA

6 9

Aborted connection message in log in every 5 minutes
by zoltan＠circle-interactive.co.uk 31 Jan '24

31 Jan '24

Hi Everyone, we have mailman 3.3.8 venv test installation which in general works fine however our mysql error log is filled up with messages below because they are so frequent. 2023-02-24 0:31:30 199567 [Warning] Aborted connection 199567 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:36:30 201063 [Warning] Aborted connection 201063 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:41:31 202559 [Warning] Aborted connection 202559 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:46:31 204056 [Warning] Aborted connection 204056 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) As suggested here https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… I tried to correlate with error logs and /opt/mailman/web/logs/mailmanweb.log shows some hyperkitty.lib.utils Failed to update the fulltext index: (2002, "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (111)" /opt/mailman/web/logs/uwsgi-error.log Message: OperationalError(2013, "Lost connection to MySQL server at 'handshake: reading initial communication packet', sy stem error: 104") but they have not such frequency so unsure if they are related. Thanks for the suggestions

5 14

Single Click Unsubscribe
by J D 26 Jan '24

26 Jan '24

I know this has been discussed in the past: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… However, Google has recently announced that they will be enforcing one-click unsubscribe for bulk senders: https://techcrunch.com/2023/10/03/gmail-to-enforce-harsher-rules-in-2024-to… Thoughts?

5 5

mailman-web compress - Invalid template socialaccount/login_cancelled.html
by Odhiambo Washington 24 Jan '24

24 Jan '24

Running an install on Debian12 VM. Getting a new error message that I haven't seen previously: (venv) mailman@debian12:~$ mailman-web compress Compressing... Invalid template socialaccount/login_cancelled.html: Invalid block tag on line 7: 'element', expected 'endblock'. Did you forget to register or load this tag? Invalid template account/verified_email_required.html: Invalid block tag on line 7: 'element', expected 'endblock'. Did you forget to register or load this tag? Error parsing template socialaccount/signup.html: socialaccount/base.html done Compressed 2 block(s) from 78 template(s) for 1 context(s). -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

7 13

Splitting hosts: mta&core vs postorius
by Ruth Ivimey-Cook 18 Jan '24

18 Jan '24

I'm trying to set up mailman3 on two hosts, one of which runs core + exim, the other runs postorius (&possibly hyperkitty). I have I think got core set up correctly, and postorius is sort of usable, but anything in the web ui that talks to core ends up with a white screen. I think what is happening is a postorius URL ends up with the core (or perhaps no) hostname, and as there's nothing there to answer it fails. I've checked the settings and I don't think I have misconfigured postorius. Is it possible that the bad url is being passed from core to postorius in some way and reused without changing the host? In the config at present, both hosts are visible to the browser, and each host can communicate with the other. Any ideas on how to pin down the problem better? Ruth

6 31

how to migrate mailing lists to a new domain using mailman 3
by C. Souza 18 Jan '24

18 Jan '24

In a few days my current domain will be disabled (it is out of my control) and I would like to know how to migrate the mailing lists to the new domain name. As a solution I added a new domain in mailman 3 and created a script that replicates the lists in this new domain. To forward the emails received by the old lists, the script removes the members and adds the email of the new list. Is there a fashionable way to do that? Also, one problem that I don't have a solution is the archive of the old lists. How to bring them to the new ones? Thanks for your help. My mailing list server is running on a CenOS-7x with postgreSQL and the following mailman packages: Package Version ------------------- ---------- 1 aiosmtpd 1.2.4 2 alembic 1.5.3 3 arrow 0.17.0 4 asgiref 3.3.1 5 atpublic 2.1.2 6 authheaders 0.13.0 7 authres 1.2.0 8 bleach 3.3.0 9 blessed 1.17.12 10 certifi 2020.12.5 11 cffi 1.14.4 12 chardet 4.0.0 13 click 7.1.2 14 cmarkgfm 0.5.2 15 cryptography 3.3.1 16 decorator 4.4.2 17 defusedxml 0.6.0 18 Django 3.0.12 19 django-allauth 0.44.0 20 django-appconf 1.0.4 21 django-compressor 2.4 22 django-extensions 3.1.0 23 django-gravatar2 1.4.4 24 django-haystack 3.0 25 django-mailman3 1.3.5 26 django-picklefield 3.0.1 27 django-q 1.3.4 28 djangorestframework 3.12.2 29 dkimpy 1.0.5 30 dnspython 2.1.0 31 docutils 0.16 32 falcon 2.0.0 33 flufl.bounce 3.0.1 34 flufl.i18n 2.0.2 35 flufl.lock 3.2 36 gunicorn 20.0.4 37 HyperKitty 1.3.3 38 idna 2.10 39 importlib-resources 5.1.0 40 lazr.config 2.2.3 41 lazr.delegates 2.0.4 42 mailman 3.3.2 43 mailman-hyperkitty 1.1.0 44 mailman-web 0.0.3 45 mailmanclient 3.3.2 46 Mako 1.1.4 47 MarkupSafe 1.1.1 48 networkx 2.5 49 nose 1.3.7 50 oauthlib 3.1.0 51 packaging 20.9 52 passlib 1.7.4 53 pip 21.0.1 54 postorius 1.3.3 55 psycopg2-binary 2.8.6 56 publicsuffix2 2.20191221 57 pycparser 2.20 58 Pygments 2.7.4 59 PyJWT 2.0.1 60 pyparsing 2.4.7 61 python-dateutil 2.8.1 62 python-editor 1.0.4 63 python3-openid 3.2.0 64 pytz 2021.1 65 rcssmin 1.0.6 66 readme-renderer 28.0 67 requests 2.25.1 68 requests-oauthlib 1.3.0 69 rjsmin 1.1.0 70 robot-detection 0.4 71 setuptools 39.2.0 72 six 1.15.0 73 SQLAlchemy 1.3.23 74 sqlparse 0.4.1 75 typing-extensions 3.7.4.3 76 urllib3 1.26.3 77 uWSGI 2.0.19.1 78 wcwidth 0.2.5 79 webencodings 0.5.1 80 wheel 0.36.2 81 Whoosh 2.7.4 82 zipp 3.4.0 83 zope.component 4.6.2 84 zope.configuration 4.4.0 85 zope.deferredimport 4.3.1 86 zope.deprecation 4.4.0 87 zope.event 4.5.0 88 zope.hookable 5.0.1 89 zope.i18nmessageid 5.0.1 90 zope.interface 5.2.0 91 zope.proxy 4.3.5 92 zope.schema 6.0.1 -- Carlos Souza

2 1

Loops back to myself
by David Krantz 12 Jan '24

12 Jan '24

Hi, I have a problem with my Mailman 3-setup. The symptom is that postfix rejects non-existing addresses for my list domains with <adr(a)example.com>: mail for example.com loops back to myself What I guess happens happens is that postfix for some reason accepts the mail as it is for a relay domain and finds that it is not deliverable locally as the address does not exist locally and then tries to relay it on to the appropriate MX server, which is itself and then fails and generates a bounce message. This is a bit annoying as I have set up postfix to only accept valid addresses. I have set the transport_maps, local_recipient_maps and relay_domains to the files generated by MM3. I run an older version of MM3 (3.0) but prefer to fix this without upgrading as upgrading is a relatively big undertaking. This is mostly a backscatter problem for spam I guess. Any ideas of very simple and easy to forget things that I've probably missed? Cheers // David

5 6

The doubled entries issue
by Lars Bjørndal 07 Jan '24

07 Jan '24

I've got the problem Multiple rows were found when one or none was required. I found the issue and a script here: https://gitlab.com/mailman/mailman/-/issues/657 I did su - mailman3 and ran: python scriptname.py and got: Traceback (most recent call last): File "/opt/mailman3/fix-doubled-entries.py", line 21, in <module> listmgr = get_list_manager() ^^^^^^^^^^^^^^^^^^ File "/opt/mailman3/fix-doubled-entries.py", line 6, in get_list_manager return getUtility(IListManager) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman3/venv/lib64/python3.11/site-packages/zope/component/_api.py ", line 180, in getUtility raise ComponentLookupError(interface, name) zope.interface.interfaces.ComponentLookupError: (<InterfaceClass mailman.interfa ces.listmanager.IListManager>, '') (venv) 1[mailman3@dalen ~]$ Do I miss something? How can I fix the problem? Thanks, Lars

4 10

Archive import fails: "IndexError: list index out of range"
by Johannes Rohr 06 Jan '24

06 Jan '24

I am now importing the archives of my old mailman2 installation. Unfortunately, with some of the larger and older lists, import fails halfway through, with an exception, the last message of which is IndexError: list index out of range Below is an example. This means that the last few years are missing from the archive in several lists. Are there any suggestions what I could do to still get them importet? Thanks! Johannes File "/usr/lib/python3.8/email/_header_value_parser.py", line 2069, in get_msg_id token, value = get_dot_atom_text(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 1334, in get_dot_atom_text raise errors.HeaderParseError("expected atom at a start of " email.errors.HeaderParseError: expected atom at a start of dot-atom-text but found '@localhost>' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/opt/mailman/venv/lib/python3.8/site-packages/mailman_web/manage.py", line 37, in main execute_from_command_line(sys.argv) File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line utility.execute() File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/base.py", line 402, in run_from_argv self.execute(*args, **cmd_options) File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/base.py", line 448, in execute output = self.handle(*args, **options) File "/opt/mailman/venv/lib/python3.8/site-packages/hyperkitty/management/commands/hyperkitty_import.py", line 387, in handle importer.from_mbox(mbfile, report_name) File "/opt/mailman/venv/lib/python3.8/site-packages/hyperkitty/management/commands/hyperkitty_import.py", line 212, in from_mbox progress_marker.tick(unquote(message.get("message-id", 'n/a'))) File "/usr/lib/python3.8/email/message.py", line 471, in get return self.policy.header_fetch_parse(k, v) File "/usr/lib/python3.8/email/policy.py", line 163, in header_fetch_parse return self.header_factory(name, value) File "/usr/lib/python3.8/email/headerregistry.py", line 607, in __call__ return self[name](name, value) File "/usr/lib/python3.8/email/headerregistry.py", line 202, in __new__ cls.parse(value, kwds) File "/usr/lib/python3.8/email/headerregistry.py", line 535, in parse kwds['parse_tree'] = parse_tree = cls.value_parser(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 2126, in parse_message_id token, value = get_msg_id(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 2073, in get_msg_id token, value = get_obs_local_part(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 1516, in get_obs_local_part if (obs_local_part[0].token_type == 'dot' or IndexError: list index out of range

6 10