June 2022 - Mailman-users - lists.mailman3.org

invalid parameter response of the rest-api
by Jacob Sievert Sept. 22, 2022

Sept. 22, 2022

Hello, Since a few days we get an error response from our automated membership script. We use the following code to control the membership for some lists: ``` def mailinglist_manage(list_address, members): """ make sure all 'members' are present on the list and no remove everyone else list_address : mail address of mailing list to manage addresses : mail address that are supposed to be on the list """ print("[*] update subscriptions for {}".format(list_address)) ml = mmc.get_list(list_address) # make sure we dont spam users ml.settings['send_welcome_message'] = False ml.settings.save() addresses = set(members.keys()) ml_members = ml.members ml_addresses = set((x.address.__str__() for x in ml_members)) for addr in ml_addresses - addresses: print("unsubscribe {} from {}".format(addr, list_address)) ml.unsubscribe(addr) for addr in addresses - ml_addresses: print("subscribe {} to {}".format(addr, list_address)) ml.subscribe(addr, members[addr], pre_verified=True, pre_confirmed=True, pre_approved=True) ``` Since Tuesday without any update performed by us we get the following error from the script when it's running: ``` Traceback (most recent call last): File "./mail_user_control.py", line 423, in <module> mailinglist_manage("XXX(a)EXAMPLE.COM", members) File "./mail_user_control.py", line 412, in mailinglist_manage pre_verified=True, pre_confirmed=True, pre_approved=True) File "/root/.local/lib/python3.6/site-packages/mailmanclient/restobjects/mailinglist.py", line 339, in subscribe response, content = self._connection.call('members', data) File "/root/.local/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 112, in call error_msg, response, None) urllib.error.HTTPError: HTTP Error 400: Invalid Parameter "subscriber": . ``` If you have an idea why that code now returns this. I would be thankful. regards Jacob Sievert

4 7

Message-Footer after migrating to Mailman3
by christopher.claus＠tgcamberg1848.de Sept. 13, 2022

Sept. 13, 2022

Hello together, I migrated a number of lists from mailman2 to mailman3 using the import functionality. Everthing works fine and the new lists are in use. Currently I´m using the maxking-docker-images with postorius: Mailman Core Version GNU Mailman 3.3.3 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.8.5 (default, Feb 19 2021, 09:27:16) [GCC 9.3.0] After this migration I started a refactoring with the list-styles specially with message-footers. Already the old lists in mailman2 were confused in some configuration details an I was clear to import this "misconfiguration" and clean it up afterwards. But this was much more difficult than expected: some lists are using a footer (from the mailman2-config) and there is no record in the template-section visible. Further I found no configuration item which points to this "old" footer. I found the file [code] /usr/lib/python3.8/site-packages/mailman/templates/de/list:member:generic:footer.txt [/code] which includes the old footer: [code] _______________________________________________ $display_name Mailingliste -- $listname Zur Abmeldung von dieser Mailingliste senden Sie eine Nachricht an ${short_listname}-leave@${domain} [/code] but no configuration item to disable this behaviour. I created a new template list:member:regular:footer for the list to override the footer. But the old footer is still used - in detail: the resulting message sent to all subcribers of the list contains only the footer and the content sent to the list is an attachement of that "footer-message". This is the basic reason why I want to change it. In the logfiles I found an error and a warning: [code] ERROR 2021-05-24 16:22:22,895 1648 django.security.DisallowedHost Invalid HTTP_HOST header: 'tgc_mailman_web:8000'. The domain name provided is not valid according to RFC 1034/1035. WARNING 2021-05-24 16:22:22,896 1648 django.request Bad Request: /postorius/api/templates/list/homepage.gruppe.tgcamberg1848.de/list:member:regular:footer [/code] Obviously I have a problem with the hostname. I set [code] POSTORIUS_TEMPLATE_BASE_URL=http://tgc_mailman_web:8000 [/code] where tgc_mailman_web is my container name. This URL ist stored in the DB-table "template" for the name=list:member:regular:footer. A call to my fqdn or to localhost [code] wget http://localhost:8000/postorius/api/templates/list/homepage.gruppe.tgcamber… [/code] will return the footer itself - this works fine. Therefore, should i set POSTORIUS_TEMPLATE_BASE_URL to http://localhost:8000? I think, resolving this issue will use the newly configured footer. But I started with the intention to remove alle footers due to the attachement situation described above. Does anybody have an idea how to change the configuration of my list? Are these problems probably related to the import? Currently we are using more then 25 lists with a lot of users. Manually recreating all lists is not possible an could only be a test for a single list. Best regards, chrclaus

6 18

Using REST with CLI using curl .... help needed
by Krešimir Mihalj Aug. 27, 2022

Aug. 27, 2022

Hi ! I need simple add/delete member to list using rest ..... but all documentation is for python and not helpfull for me, .... and I need these in CLI using curl command .... To add member using CLI i tried this command: curl -d "{'list_id' : 'list.domain.com', 'subscriber' : 'user(a)email.com'}" -H 'Content-Type: application/json' --user restadmin:restpass http://localhost:8001/3.0/members and I got this response: Missing parameters: list_id, subscriber What is wrong in above command here ? ..... I need to add user: user(a)email.com to list: list(a)domain.com Also, if someone is willing to give example: delete user: user(a)email.com from list: list(a)domain.com I actually create PHP application .... but CLI example is easy to reproduce in PHP function AddUserToList($user,$lista) { $url = 'http://llocalhost:8001/3.0'; $request_url = $url . '/members'; $data = array("list_id" => $lista,"subscriber" => $user ); $postdata = json_encode($data); $ch = curl_init($request_url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_USERPWD, "restadmin:restpass"); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json')); $result = curl_exec($ch); curl_close($ch); print_r ($result); } AddUserToList('user(a)email.com','list.domain.com');

6 8

User display name entry field input ignored
by Christian July 27, 2022

July 27, 2022

Whether I try and add users using mass subscribe, or individually, it seems the display name field entry is ignored. The default is then "none" for the user. Please advise. Thank you!

2 3

Problem replying via Hyperkitty
by Jeff July 27, 2022

July 27, 2022

Hello Everyone, A few of my users read our mailing list through Hyperkitty. Whenever they reply through the web, their replies simply disappear. One example from today was logged as follows: [pid: 2287|app: 0|req: 116007/153551] xxx.xx.xx.xx () {72 vars in 1992 bytes} [Thu Jun 30 11:26:16 2022] POST /archives/list/listname(a)domainname.org/message/46WUOPZITVMKJN4TXR6CP3WMFUEAVJKD/reply => generated 3213 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 188 bytes (3 switches on core 0) A couple of basic questions: 1) when a users replies in this manner, does their response go to the list or to the subscriber to whom they are replying? 2) what log files could I check to see what — if anything — happens to these emails? 3) Are there any settings that need to be set to make this work? Thanks much. Any help will be appreciated. Jeff

2 3

Apache Conf
by Guiseppin Thomas July 26, 2022

July 26, 2022

Hi, Currently, I use a nginx configuration for mailman. It's work perfectly. I tried to do the same with Apache, but the statics files are not delivered (404). Basically, it's the same configuration: /server { listen 80; server_name mailing.ddd.fr; access_log /var/log/nginx/mailing.access.log; error_log /var/log/nginx/mailing.error.log; location /static/ { alias /opt/mailman/static/; expires 30d; } proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; location / { proxy_pass http://localhost:8000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } } <VirtualHost *:80> ServerName mailing.ddd.fr ErrorLog ${APACHE_LOG_DIR}/mailing.error.log CustomLog ${APACHE_LOG_DIR}/mailing.access.log combined <Location /static/> Alias "/opt/mailman/static/" </Location> <Location /> ProxyPass http://localhost:8000/ ProxyPassReverse http://localhost:8000/ ProxyPreserveHost On </Location> </VirtualHost>/ I missed something ? Thanks.

5 10

Adding user - Error 409 Subscription request already pending
by kevin＠coastalaska.org July 14, 2022

July 14, 2022

We have a couple of people trying to subscribe to our list but receive an error similar to: {"title": "409 Conflict", "description": "Subscription request already pending"} If I try to Mass Subscribe them in Postorius I get this error: HTTP Error 409: b'{"title": "409 Conflict", "description": "Subscription request already pending"}' In Postorius I don't see them hung up in Subscription requests. What to do? Thanks, Kevin

5 17

Walk-thru for Ubuntu Bionic Mailman 3 Full setup
by kelly.close＠lrewater.com July 9, 2022

July 9, 2022

I hope this is helpful to the Ubuntu folks out there. Once some critical small things were figured out, turns out it's pretty easy to get Mailman 3 running successfully under Ubuntu 18.04. The linked google doc includes my own walkthru, which includes using a cloud server image for Ubuntu 18.04 and a LAMP setup with Apache 2 and PHP, as well as Python and PostgreSQL, and the mailman3-full package that comes with that Ubuntu distribution. The walk-thru uses Postfix for mail and PostgreSQL for the back end database. https://docs.google.com/document/d/1xIcSsoNFp2nHi7r4eQys00s9a0k2sHhu1V5Plan… If you use this and see needed corrections or additions, please let me know! ~Kelly

6 21

Problem with database change sqlite -> mysql
by moritz both July 7, 2022

July 7, 2022

Greetings, I am totally new to this list and relatively new to mailman3 and came here because I have a problem I would like to solve soon. I hope somebody has a clue: I am running a really small list site with 10 or so lists, all of them have low traffic and no more than ~70 members. The lists are running on a shared server at uberspace. I followed their guide at https://lab.uberspace.de/guide_mailman-3/ Mailman Core Version GNU Mailman 3.3.5 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.8.13 (default, May 19 2022, 15:06:19) [GCC 9.3.1 20200408 (Red Hat 9.3.1-2)] Since my migration from mm2 to mm3 I see what I consider high loads and excessive i/o and memory usage. My guess is that the main problem is the database backend for mailman, posterious and hyperkitty, which is sqlite. The database is >200MB at the moment (I have not found out why it is so huge, I tried to delete the hyperkitty archives from there and no longer archive messages, but maybe I have not managed to delete all). I take any recommandation for this situation. My current idea is changing the database backend to something else, i.e. mysql. I tried to follow this guide https://www.rockandnull.com/django-dumpdata/ but the "loaddata" command fails. The stack trace is below. Unfortunatelly, I have close to no knowledge of django and don't know why the Email matching query not exists. Will postgres make things easier? Thanks in advance for any help. Moritz Here is the stacktrace: Traceback (most recent call last): File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 173, in __get__ rel_obj = self.field.get_cached_value(instance) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/mixins.py", line 15, in get_cached_value return instance._state.fields_cache[cache_name] KeyError: 'starting_email' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv self.execute(*args, **cmd_options) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute output = self.handle(*args, **options) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 78, in handle self.loaddata(fixture_labels) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 123, in loaddata self.load_label(fixture_label) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 190, in load_label obj.save(using=self.using) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/serializers/base.py", line 223, in save models.Model.save_base(self.object, using=using, raw=True, **kwargs) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/base.py", line 763, in save_base pre_save.send( File "/home/daneben/.local/lib/python3.8/site-packages/django/dispatch/dispatcher.py", line 180, in send return [ File "/home/daneben/.local/lib/python3.8/site-packages/django/dispatch/dispatcher.py", line 181, in <listcomp> (receiver, receiver(signal=self, sender=sender, **named)) File "/home/daneben/.local/lib/python3.8/site-packages/hyperkitty/signals.py", line 96, in Thread_on_pre_save kwargs["instance"].on_pre_save() File "/home/daneben/.local/lib/python3.8/site-packages/hyperkitty/models/thread.py", line 157, in on_pre_save self.find_starting_email() File "/home/daneben/.local/lib/python3.8/site-packages/hyperkitty/models/thread.py", line 149, in find_starting_email if self.starting_email is not None: File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 187, in __get__ rel_obj = self.get_object(instance) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 307, in get_object return super().get_object(instance) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 154, in get_object return qs.get(self.field.get_reverse_related_filter(instance)) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get raise self.model.DoesNotExist( hyperkitty.models.email.DoesNotExist: Problem installing fixture '/home/daneben/mailman-suite/datadump.json': Email matching query does not exist.

3 5

Django errors after deleting a thread
by peter＠chubb.wattle.id.au July 6, 2022

July 6, 2022

After deleting a thread via the web interface (because it's spam), I get lots of messages from hyperkitty E.g. Internal Server Error: /hyperkitty/list/list@domain/top-threads DoesNotExist at /hyperkitty/list/list@domain/top-threads Thread matching query does not exist. Request Method: GET Request URL: https://example.com/hyperkitty/list/list@domain/top-threads Django Version: 2.2.17 Python Executable: /usr/bin/uwsgi-core Python Version: 3.8.6 Python Path: ['.', '', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Wed, 25 Nov 2020 07:12:00 +1100 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Traceback: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 115. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 113. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/lib/view_helpers.py" in inner 135. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mlist.py" in overview_top_threads 220. 'threads': mlist.top_threads, File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py" in top_threads 168. return self.cached_values["top_threads"]() File "/usr/lib/python3/dist-packages/hyperkitty/models/common.py" in __call__ 59. return self.get_or_set(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py" in get_or_set 376. return [Thread.objects.get(pk=pk) for pk in thread_ids] File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py" in <listcomp> 376. return [Thread.objects.get(pk=pk) for pk in thread_ids] File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_method 82. return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py" in get 406. raise self.model.DoesNotExist( Exception Type: DoesNotExist at /hyperkitty/list/devel(a)sel4.systems/top-threads Exception Value: Thread matching query does not exist. Request information: USER: AnonymousUser GET: No GET data POST: No POST data FILES: No FILES data COOKIES: No cookie data

3 4