June 2022 - Mailman-users - lists.mailman3.org

invalid parameter response of the rest-api
by Jacob Sievert 22 Sep '22

22 Sep '22

Hello, Since a few days we get an error response from our automated membership script. We use the following code to control the membership for some lists: ``` def mailinglist_manage(list_address, members): """ make sure all 'members' are present on the list and no remove everyone else list_address : mail address of mailing list to manage addresses : mail address that are supposed to be on the list """ print("[*] update subscriptions for {}".format(list_address)) ml = mmc.get_list(list_address) # make sure we dont spam users ml.settings['send_welcome_message'] = False ml.settings.save() addresses = set(members.keys()) ml_members = ml.members ml_addresses = set((x.address.__str__() for x in ml_members)) for addr in ml_addresses - addresses: print("unsubscribe {} from {}".format(addr, list_address)) ml.unsubscribe(addr) for addr in addresses - ml_addresses: print("subscribe {} to {}".format(addr, list_address)) ml.subscribe(addr, members[addr], pre_verified=True, pre_confirmed=True, pre_approved=True) ``` Since Tuesday without any update performed by us we get the following error from the script when it's running: ``` Traceback (most recent call last): File "./mail_user_control.py", line 423, in <module> mailinglist_manage("XXX(a)EXAMPLE.COM", members) File "./mail_user_control.py", line 412, in mailinglist_manage pre_verified=True, pre_confirmed=True, pre_approved=True) File "/root/.local/lib/python3.6/site-packages/mailmanclient/restobjects/mailinglist.py", line 339, in subscribe response, content = self._connection.call('members', data) File "/root/.local/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 112, in call error_msg, response, None) urllib.error.HTTPError: HTTP Error 400: Invalid Parameter "subscriber": . ``` If you have an idea why that code now returns this. I would be thankful. regards Jacob Sievert

4 7

Re: mail stuck in "out" queue
by Dmitry Makovey 16 Sep '22

16 Sep '22

So our mailman got stuck again, but I did follow your advise. Here's what I've found: On 3/29/19 2:58 PM, Mark Sapiro wrote: > On 3/29/19 2:28 PM, Dmitry Makovey wrote: >> I've had encountered this situation now twice: mail is being stuck in >> "out" queue and nothing in the logs indicates the issue. Is there a way >> to ask mailman-core politely to attempt delivery (perhaps with increased >> debugging in the logs) *without* restarting the core (which does solve >> *this* problem but creates other problems, like availability of the >> mailman as a whole) > > > Is the "out" runner running? yes: # docker top mailman-core | grep -F out: 100 21681 13215 0 Mar28 ? 01:14:16 /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg > ps -fwwA |grep runner=out > > If not, you should be able to start it as the Mailman user with a > command like > > /path/to/python3 /path/to/mailman/bin/runner -C /path/to/mailman.cfg > --runner=out:0:1 > > See 'ps -fwwA|grep runner' for examples > > If the out runner is running, is there a file named *.bak along with all > the *.pck files in queue/out/ there is one, yes: # ls -l /srv/mail/mailman/core/var/queue/out/*.bak -rw-rw---- 1 100 65533 21512 Apr 14 19:45 /srv/mail/mailman/core/var/queue/out/1555296331.8386078+2c905825b82708e5a682b872b0aff320c9dd0b2e.bak what does it indicate and what should my steps be to resolve this (other than restarting queues)? > And if it's not, is there anything in Mailman's logs to indicate why it > died? > > Also, 'mailman restart' should only take a matter of seconds, but if the > out runner is running but not processing its queue, I think sending it a > SIGINT should restart it. SIGINT killed the process allright, but it didn't auto-respawn. I ended up spawning it with: /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg but soon realized it doesn't go to background nicely so I've nohup'ed it in the end. but it did solve my problem. -- Sr System and DevOps Engineer SoM IRT

5 12

Message-Footer after migrating to Mailman3
by christopher.claus＠tgcamberg1848.de 13 Sep '22

13 Sep '22

Hello together, I migrated a number of lists from mailman2 to mailman3 using the import functionality. Everthing works fine and the new lists are in use. Currently I´m using the maxking-docker-images with postorius: Mailman Core Version GNU Mailman 3.3.3 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.8.5 (default, Feb 19 2021, 09:27:16) [GCC 9.3.0] After this migration I started a refactoring with the list-styles specially with message-footers. Already the old lists in mailman2 were confused in some configuration details an I was clear to import this "misconfiguration" and clean it up afterwards. But this was much more difficult than expected: some lists are using a footer (from the mailman2-config) and there is no record in the template-section visible. Further I found no configuration item which points to this "old" footer. I found the file [code] /usr/lib/python3.8/site-packages/mailman/templates/de/list:member:generic:footer.txt [/code] which includes the old footer: [code] _______________________________________________ $display_name Mailingliste -- $listname Zur Abmeldung von dieser Mailingliste senden Sie eine Nachricht an ${short_listname}-leave@${domain} [/code] but no configuration item to disable this behaviour. I created a new template list:member:regular:footer for the list to override the footer. But the old footer is still used - in detail: the resulting message sent to all subcribers of the list contains only the footer and the content sent to the list is an attachement of that "footer-message". This is the basic reason why I want to change it. In the logfiles I found an error and a warning: [code] ERROR 2021-05-24 16:22:22,895 1648 django.security.DisallowedHost Invalid HTTP_HOST header: 'tgc_mailman_web:8000'. The domain name provided is not valid according to RFC 1034/1035. WARNING 2021-05-24 16:22:22,896 1648 django.request Bad Request: /postorius/api/templates/list/homepage.gruppe.tgcamberg1848.de/list:member:regular:footer [/code] Obviously I have a problem with the hostname. I set [code] POSTORIUS_TEMPLATE_BASE_URL=http://tgc_mailman_web:8000 [/code] where tgc_mailman_web is my container name. This URL ist stored in the DB-table "template" for the name=list:member:regular:footer. A call to my fqdn or to localhost [code] wget http://localhost:8000/postorius/api/templates/list/homepage.gruppe.tgcamber… [/code] will return the footer itself - this works fine. Therefore, should i set POSTORIUS_TEMPLATE_BASE_URL to http://localhost:8000? I think, resolving this issue will use the newly configured footer. But I started with the intention to remove alle footers due to the attachement situation described above. Does anybody have an idea how to change the configuration of my list? Are these problems probably related to the import? Currently we are using more then 25 lists with a lot of users. Manually recreating all lists is not possible an could only be a test for a single list. Best regards, chrclaus

6 18

Using REST with CLI using curl .... help needed
by Krešimir Mihalj 27 Aug '22

27 Aug '22

Hi ! I need simple add/delete member to list using rest ..... but all documentation is for python and not helpfull for me, .... and I need these in CLI using curl command .... To add member using CLI i tried this command: curl -d "{'list_id' : 'list.domain.com', 'subscriber' : 'user(a)email.com'}" -H 'Content-Type: application/json' --user restadmin:restpass http://localhost:8001/3.0/members and I got this response: Missing parameters: list_id, subscriber What is wrong in above command here ? ..... I need to add user: user(a)email.com to list: list(a)domain.com Also, if someone is willing to give example: delete user: user(a)email.com from list: list(a)domain.com I actually create PHP application .... but CLI example is easy to reproduce in PHP function AddUserToList($user,$lista) { $url = 'http://llocalhost:8001/3.0'; $request_url = $url . '/members'; $data = array("list_id" => $lista,"subscriber" => $user ); $postdata = json_encode($data); $ch = curl_init($request_url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_USERPWD, "restadmin:restpass"); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json')); $result = curl_exec($ch); curl_close($ch); print_r ($result); } AddUserToList('user(a)email.com','list.domain.com');

6 8

User display name entry field input ignored
by Christian 27 Jul '22

27 Jul '22

Whether I try and add users using mass subscribe, or individually, it seems the display name field entry is ignored. The default is then "none" for the user. Please advise. Thank you!

2 3

Problem replying via Hyperkitty
by Jeff 27 Jul '22

27 Jul '22

Hello Everyone, A few of my users read our mailing list through Hyperkitty. Whenever they reply through the web, their replies simply disappear. One example from today was logged as follows: [pid: 2287|app: 0|req: 116007/153551] xxx.xx.xx.xx () {72 vars in 1992 bytes} [Thu Jun 30 11:26:16 2022] POST /archives/list/listname(a)domainname.org/message/46WUOPZITVMKJN4TXR6CP3WMFUEAVJKD/reply => generated 3213 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 188 bytes (3 switches on core 0) A couple of basic questions: 1) when a users replies in this manner, does their response go to the list or to the subscriber to whom they are replying? 2) what log files could I check to see what — if anything — happens to these emails? 3) Are there any settings that need to be set to make this work? Thanks much. Any help will be appreciated. Jeff

2 3

Apache Conf
by Guiseppin Thomas 26 Jul '22

26 Jul '22

Hi, Currently, I use a nginx configuration for mailman. It's work perfectly. I tried to do the same with Apache, but the statics files are not delivered (404). Basically, it's the same configuration: /server { listen 80; server_name mailing.ddd.fr; access_log /var/log/nginx/mailing.access.log; error_log /var/log/nginx/mailing.error.log; location /static/ { alias /opt/mailman/static/; expires 30d; } proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; location / { proxy_pass http://localhost:8000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } } <VirtualHost *:80> ServerName mailing.ddd.fr ErrorLog ${APACHE_LOG_DIR}/mailing.error.log CustomLog ${APACHE_LOG_DIR}/mailing.access.log combined <Location /static/> Alias "/opt/mailman/static/" </Location> <Location /> ProxyPass http://localhost:8000/ ProxyPassReverse http://localhost:8000/ ProxyPreserveHost On </Location> </VirtualHost>/ I missed something ? Thanks.

5 10

Adding user - Error 409 Subscription request already pending
by kevin＠coastalaska.org 14 Jul '22

14 Jul '22

We have a couple of people trying to subscribe to our list but receive an error similar to: {"title": "409 Conflict", "description": "Subscription request already pending"} If I try to Mass Subscribe them in Postorius I get this error: HTTP Error 409: b'{"title": "409 Conflict", "description": "Subscription request already pending"}' In Postorius I don't see them hung up in Subscription requests. What to do? Thanks, Kevin

5 17

Walk-thru for Ubuntu Bionic Mailman 3 Full setup
by kelly.close＠lrewater.com 09 Jul '22

09 Jul '22

I hope this is helpful to the Ubuntu folks out there. Once some critical small things were figured out, turns out it's pretty easy to get Mailman 3 running successfully under Ubuntu 18.04. The linked google doc includes my own walkthru, which includes using a cloud server image for Ubuntu 18.04 and a LAMP setup with Apache 2 and PHP, as well as Python and PostgreSQL, and the mailman3-full package that comes with that Ubuntu distribution. The walk-thru uses Postfix for mail and PostgreSQL for the back end database. https://docs.google.com/document/d/1xIcSsoNFp2nHi7r4eQys00s9a0k2sHhu1V5Plan… If you use this and see needed corrections or additions, please let me know! ~Kelly

6 21

Problem with database change sqlite -> mysql
by moritz both 07 Jul '22

07 Jul '22

Greetings, I am totally new to this list and relatively new to mailman3 and came here because I have a problem I would like to solve soon. I hope somebody has a clue: I am running a really small list site with 10 or so lists, all of them have low traffic and no more than ~70 members. The lists are running on a shared server at uberspace. I followed their guide at https://lab.uberspace.de/guide_mailman-3/ Mailman Core Version GNU Mailman 3.3.5 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.8.13 (default, May 19 2022, 15:06:19) [GCC 9.3.1 20200408 (Red Hat 9.3.1-2)] Since my migration from mm2 to mm3 I see what I consider high loads and excessive i/o and memory usage. My guess is that the main problem is the database backend for mailman, posterious and hyperkitty, which is sqlite. The database is >200MB at the moment (I have not found out why it is so huge, I tried to delete the hyperkitty archives from there and no longer archive messages, but maybe I have not managed to delete all). I take any recommandation for this situation. My current idea is changing the database backend to something else, i.e. mysql. I tried to follow this guide https://www.rockandnull.com/django-dumpdata/ but the "loaddata" command fails. The stack trace is below. Unfortunatelly, I have close to no knowledge of django and don't know why the Email matching query not exists. Will postgres make things easier? Thanks in advance for any help. Moritz Here is the stacktrace: Traceback (most recent call last): File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 173, in __get__ rel_obj = self.field.get_cached_value(instance) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/mixins.py", line 15, in get_cached_value return instance._state.fields_cache[cache_name] KeyError: 'starting_email' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv self.execute(*args, **cmd_options) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute output = self.handle(*args, **options) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 78, in handle self.loaddata(fixture_labels) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 123, in loaddata self.load_label(fixture_label) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 190, in load_label obj.save(using=self.using) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/serializers/base.py", line 223, in save models.Model.save_base(self.object, using=using, raw=True, **kwargs) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/base.py", line 763, in save_base pre_save.send( File "/home/daneben/.local/lib/python3.8/site-packages/django/dispatch/dispatcher.py", line 180, in send return [ File "/home/daneben/.local/lib/python3.8/site-packages/django/dispatch/dispatcher.py", line 181, in <listcomp> (receiver, receiver(signal=self, sender=sender, **named)) File "/home/daneben/.local/lib/python3.8/site-packages/hyperkitty/signals.py", line 96, in Thread_on_pre_save kwargs["instance"].on_pre_save() File "/home/daneben/.local/lib/python3.8/site-packages/hyperkitty/models/thread.py", line 157, in on_pre_save self.find_starting_email() File "/home/daneben/.local/lib/python3.8/site-packages/hyperkitty/models/thread.py", line 149, in find_starting_email if self.starting_email is not None: File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 187, in __get__ rel_obj = self.get_object(instance) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 307, in get_object return super().get_object(instance) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 154, in get_object return qs.get(self.field.get_reverse_related_filter(instance)) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get raise self.model.DoesNotExist( hyperkitty.models.email.DoesNotExist: Problem installing fixture '/home/daneben/mailman-suite/datadump.json': Email matching query does not exist.

3 5