- Mailman-users - lists.mailman3.org

Hyperkitty fulltext search performances
by Loïc Dachary 04 Oct '20

04 Oct '20

Hi, I'm in the process of importing a large number of mbox (~30,000) for a few hundred mailing lists. So far around 300,000 mails (~12GB) were imported from two mailing lists. The hyperkitty_import process took ~8 hours and created a 10GB MySQL database. The update_index_one_list for the two lists took a total of ~72 hours and created a ~2GB worth of index in /var/lib/mailman3/web/fulltext_index, which is consistent with the fact that there are lot of attachments (probably 10GB out of 12GB). When I search for one word via the full text search web interface, it takes around 30 seconds to complete (even when repeated twice) and I can see the process grow to use up to 6GB of resident memory. It is running on a recent physical machine with decent IO and a Intel(R) Xeon(R) CPU E5-2609 v2 @ 2.50GHz. Is this consistent with other people experience? Cheers -- Loïc Dachary, Artisan Logiciel Libre

4 4

Mailman3 not delivering list mail
by Gordon Dickens 02 Oct '20

02 Oct '20

Hello, I have a fresh mailman3 installation that I recently upgraded from mailman2 on a debian system.Â In any event, I imported some test lists and tested my installation thoroughly and everything seemed to be working fine.Â Then, today, I imported a important live working mailing list from mailman2 and everything appeared to go well both with the import of the list addresses and the archives.Â However, when I sent out the first post on Mailman3 the post was not echoed to the list members.Â Then, I subsequently discovered that the queue had lots of mail undelivered in /var/lib/mailman3/queue/bad. It appears that my mailman3 installation has stopped delivering list mail and that all of the list mail is stuck in /var/lib/mailman3/queue/bad. How do I resolve this situation?Â Can I just remove the files with "rm *" in the /var/lib/mailman3/queue/bad directory?Â Please let me know what I should do to get things working again. Thanks, Gordon

3 7

Various patches for mm3 stable 3.3.1
by Ruth Ivimey-Cook 30 Sep '20

30 Sep '20

Folks, For my own purposes I have created some patches which apply to the current mailman pip package. The first of them is for my purposes necessary, and has been discussed here before: being able to specify the address used to listen for tcp connections independently of the address the server considers it to be accessible at. A possible change to this patch would be to check whether this address was undefined but the old hostname was defined, and use that if so. For me this is not needed, and in any case it is a trivial change to make. The second patch adds a bunch of additional logging calls to various network-related parts of the package. I wrote these into the package because I was seeking the cause of template expansion causing messages to be silently dropped. I eventually found the cause was that I had an incorrect setting of the template root URL, but I regard the lack of anything in the logs as just as much of a bug. One important part of those changes was that I added a try/catch around two parts of the 'decorate' code, which is where the silent dropping occurred. If you only take one part of this patch, I encourage you to take this part. For the third patch, I discovered that the task runner was at one point busy-waiting to the tune of 100% cpu, and the cause was the way that the runner checked to see if there was work left in the queue. I am not quite sure what was the exact problem, but I think it was that there was something to do, but it was not achievable at that time, and the result was continual retry. The third patch changes the logic slightly so that this no longer occurs - retry will happen, but somewhat more gracefully. I hope you find some use in these three. Finally, I am currently experiencing problems with templates and would appreciate any thoughts: 1. If I do not set up 'individual' or 'full' personalisation, all is well and the default footer templates are expanded as expected. However setting up personalisation causes template expansion to fail completely, and messages get sent with a blank expansion. 2. I would find the postorius UI much easier to cope with if the default template was 'populated' into the UI wherever it was non-blank, rather than the UI starting off as if no templates were defined at all. Regards, Ruth

3 3

Unable to remove non-member
by Brian Carpenter 29 Sep '20

29 Sep '20

Hi Everyone, I ran across a strange issue today. A client reported the inability to remove a non-member in the following format: z_email/domain.org.za(a)domain.org.za The reported error from Postorius was: "The user could not be removed" I used https://verifalia.com/validate-email to validate the email address and it is syntactically correct. -- Brian Carpenter Harmonylists.com Emwd.com

2 4

Discard all mail with the subject "The resutls of your email commands"
by w.schoener＠alp.dillingen.de 29 Sep '20

29 Sep '20

Hello, I am using mailman 3.2.1 for several mailing lists. My colleagues want me to stop all mails with the subject "The results of your email commands". I found some contributions here, but nowhere a solution for this question. Does anybody has any suggestion? Maybe I could try to handle this outside of mailman, but then I would have to pass this task on to someone I dont know .... :-( Thanks for all hints. :-) Wolfgang

3 5

New releases
by Abhilash Raj 25 Sep '20

25 Sep '20

Hi Everyone, I am pleased to announce new stable releases for: - Postorius: 1.3.3 Changelog: https://docs.mailman3.org/projects/postorius/en/latest/news.html#id1 Download: https://pypi.org/project/postorius/1.3.3/ - Hyperkitty: 1.3.3 Changelog: https://hyperkitty.readthedocs.io/en/latest/news.html#id1 Download: https://pypi.org/project/HyperKitty/1.3.3rc1/ - Mailmanclient: 3.3.1 Changelog: https://docs.mailman3.org/projects/mailmanclient/en/latest/src/mailmanclien… Download: https://pypi.org/project/mailmanclient/3.3.1/ - Django-mailman3: 1.3.3 Changelog: https://gitlab.com/mailman/django-mailman3/-/blob/master/README.rst Download: https://pypi.org/project/django-mailman3/1.3.3 Python 3.6+ and Django 2.0+ is supported for all of them. Django 3.0 support for Hyperkitty requires manually upgrading a dependency (django-haystack>=3.0b2, once a stable version of this has been released, it shouldn't require manually upgrading). There are tons of bug-fixes across the board and some new features. Biggest visible change is switch to Bootstrap 4, which has been long pending for us. Bootstrap 4 completely changes the CSS grid model using the new Flexbox. There might be some small changes or breakages when using on mobile. Please report such issues to us via Gitlab! Some other notable changes are in Postorius, which includes many more list settings exposed include content filtering settings, bounce processing (which was added in the previous release of Mailman Core) settings and some other ones. Settings page is also slightly different with all the sections on the vertical menu on left instead of horizontal tabs. You can now also specify a reason when rejecting held messages. There was also a gnarly bug, which caused the name of some members to be the string "None". For the longest time, I couldn't figure out the reason for it, but it ended up being a simple fix in mailmanclient's json serialization of display_name, which would result in Python's None value being passed as string "None" to Mailman's API for subscription. There is also better support for filtering visible lists based on the current vhost, which I see a few people are already waiting for from mailman-users list. A full changelog has been added to each project in the top. -- thanks, Abhilash Raj (maxking)

5 23

My Server is unable to connect to Mailman REST API few times
by Shashikanth Komandoor 24 Sep '20

24 Sep '20

Hi Team, I am currently running Mailman 3 with version 3.3.1 and in built postfix version 2.10.1-6 on RHEL 7.5 with PostgreSQL 11.7 version with default values in production environment from the past 4 months almost. As of now, we are having around 1327 lists created on this. When I am trying to do some operations like creating the lists, deleting the lists or accessing the lists through the GUI, I am getting the below error. If I am trying to do the operations through API or through mailman shell, still I am not able to connect to the API. Something went wrong Mailman REST API not available. Please start Mailman core. But my mailman core was working fine in the background. At the same time I was able to do other operations on a few other lists. By the time I am getting the above error, the mailman.log recorded the below messages. *[2020-09-16 12:49:08 +0530] [2600] [CRITICAL] WORKER TIMEOUT (pid:20234)* *[2020-09-16 12:49:08 +0530] [20234] [INFO] Worker exiting (pid: 20234)* *[2020-09-16 12:49:09 +0530] [21042] [INFO] Booting worker with pid: 21042* My mailman.cfg configuration is as below. The timeout value is the default value. I did not set any customized value. *(venv3) [root@lsmgr mailman]# grep -v ^# /var/lib/mailman/mailman/var/etc/mailman.cfg* *[archiver.hyperkitty]* *class: mailman_hyperkitty.Archiver* *enable: yes* *configuration: /var/lib/mailman/mailman-hyperkitty/mailman-hyperkitty.cfg* *[mta]* *incoming: mailman.mta.postfix.LMTP* *outgoing: mailman.mta.deliver.deliver* *lmtp_host: 0.0.0.0* *lmtp_port: 8024* *smtp_host: localhost* *smtp_port: 25* *[webservice]* *workers: 10* *[database]* *class: mailman.database.postgresql.PostgreSQLDatabase* *url: postgres://<postgresdbuser>:<postgresdbpassword>@127.0.0.1/mailman <http://127.0.0.1/mailman>* Then as per the suggestion I got earlier I configured the [webservice] section in the above file as below. *[webservice]* *configuration: /var/lib/mailman/mailman/var/etc/gunicorn.cfg* * [root@lsmgr logs]# cat /var/lib/mailman/mailman/var/etc/gunicorn.cfg* *[gunicorn]* *workers = 10* *timeout = 90 * After the above configuration, I found many errors like below in mailman.log file: *sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'pended.id <http://pended.id>'"* *sqlalchemy.exc.ResourceClosedError: This result object does not return rows. It has been closed automatically.* *sqlalchemy.exc.DatabaseError: (psycopg2.DatabaseError) error with status PGRES_TUPLES_OK and no message from the libpq* The above might be few of the errors. FYI, for the creation of each list, it is taking almost around 15 to 20 minutes. After multiple times of the above error messages, at some time, I am fortunately getting the list created or some other operation getting done. I found the same scenario when the workers are from default 2 to 10. I don't understand where exactly I need to focus to solve the issue. Please let me know if I have to share any other details for you to suggest to me. -- Thanks & Regards, Shashi Kanth.K 9052671936 <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

3 9

including a template failed
by Wolfgang Bock 24 Sep '20

24 Sep '20

Hi, I created a new mailinglist and got during the test-process the following errormessage in my mailman.log after: ... Sep 19 22:55:08 2020 (1433) ACCEPT: <000001d68ec7$2553e6e0$6ffbb4a0$(a)domain-01.de> Errors: Sep 19 22:55:09 2020 (1436) Cannot connect to SMTP server localhost on port 25 HTTPSConnectionPool(host='webmail.domain-02.de', port=443): Max retries exceeded with url: /mailman3/postorius/api/templates/list/mylist.lists.domain-03.de/list:member :regular:footer (Caused by NewConnectionError('<urllib3.connec tion.VerifiedHTTPSConnection object at 0x7fc8380f1748>: Failed to establish a new connection: [Errno -2] Name or service not known')) ... domain02.de isn't involved in mailman3 either as a user domain nor as a list-domain. After deleting the template mailman3/postorius/api/templates/list/mylist.lists.domain-03.de/list:member: regular:footer the mail to the list is processed without problems: Sep 19 22:56:22 2020 (1436) <000001d68ec7$2553e6e0$6ffbb4a0$(a)domain-01.de> smtp to mylist(a)list.domain-03.de for 1 recips, completed in 0.21389174461364746 seconds Sep 19 22:56:22 2020 (1436) <000001d68ec7$2553e6e0$6ffbb4a0$(a)wbock.de> post to mylist(a)list.domain-03.de from user(a)domain-01.de, 1331 bytes Please help to solve this problem? Thanks and regards Wolfgang

2 3

Missing whitespace in postfix_vmap after mailman genaliases
by Wolfgang Bock 23 Sep '20

23 Sep '20

Hi, after creating a new list I got the following errormessage: mailman aliases -> postmap: warning: /var/lib/mailman3/data/postfix_vmap, line 17: expected format: key whitespace value /var/lib/mailman3/data/postfix_vmap # AUTOMATICALLY GENERATED BY MAILMAN ON 2020-09-19 07:44:34 .... xxxxxxxxxxx(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-bounces(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-bounces(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-confirm(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-confirm(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-join(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-join(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-leave(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-leave(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-owner(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-owner(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-request(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-request(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-subscribe(a)lists.xxxxxxxxxxxxxxxx.de xxxxxxxxxxx-subscribe(a)xxxxxxxxxxxxx.de xxxxxxxxxxx-unsubscribe@lists.xxxxxxxxxxxxxxxx.dexxxxxxxxxxx-unsubscribe@xxx xxxxxxxxxx.de The last line in the left part contains 48 chars. I solved the problem by adding a whitespace in the last line and I made a new postmap for postfix_vmap. But it would be better to solve the problem by changing the genaliases process in the next update. Regards Wolfgang

2 3

Re: cc'd e-mail addresses rewritten with @invalid.domain
by Williams, Peter B 23 Sep '20

23 Sep '20

Hi Mark, Thanks for the help with this! Other list members will also see the e-mail as coming from user(a)invalid.domain<mailto:user@invalid.domain>. I will continue testing the other scenarios and will reply back as soon as possible. Thanks again for the help with this and have a nice day! Peter On Mon, Sep 21, 2020 at 9:00 PM Mark Sapiro <mark(a)msapiro.net> wrote: On 9/21/20 10:49 AM, peterbw(a)gmail.com<mailto:peterbw@gmail.com> wrote: > Hi, > > We've recently noticed an issue where users who are cc'd in a message where a mailing list is also within the cc list are getting rewritten a user(a)invalid.domain. > > This issue looks like it may be related to Outlook and how user names are displayed specifically. Here is an example: > > To: test_user(a)gmail.com<mailto:test_user@gmail.com> > cc: listserv(a)lists.test.org<mailto:listserv@lists.test.org>; Williams, Peter; > > "Williams, Peter" maps to a user e-mail in the Outlook Global Address book. When the message is sent the list will send the message with the first name and last name separated, for example the message will appear: > > to: test_user(a)gmail.com<mailto:test_user@gmail.com> > cc: listserv(a)lists.test.org<mailto:listserv@lists.test.org>, Williams(a)invalid.domain, Peter(a)invalid.domain Because apparently Outlook is sending the message with the literal > Cc: listserv(a)lists.test.org<mailto:listserv@lists.test.org>; Williams, Peter; instead of something like > Cc: listserv(a)lists.test.org<mailto:listserv@lists.test.org>, "Williams, Peter" <peters_real_address(a)example.com<mailto:peters_real_address@example.com>> Then some MTA in the chain is seeing the unqualified addresses Williams and Peter and appending its own local domain. > This only seems to happen with users who are cc'd who are also members of the list that is being cc'd. For example in the scenario above my e-mail that is cc'd is also a member of listserv(a)lists.test.org<mailto:listserv@lists.test.org>. In this case, where do you see this? i.e., do other list members see it in the message they receive from the list? I don't know why that would be. I'm certain that Mailman is not involved in this and the Williams(a)invalid.domain and Peter(a)invalid.domain addresses are in the Cc: as received by Mailman. > This issue does not occur if the user e-mail address is listed before the mailing list in the cc field. So for example, if the cc in the scenario above contained Williams, Peter; listserv(a)lists.test.org<mailto:listserv@lists.test.org>; it would route properly. Again, I think this is Outlook. What's in the Cc: of the mail that test_user(a)gmail.com<mailto:test_user@gmail.com> receives directly? What happens if you send mail in this way with To: test_user(a)gmail.com<mailto:test_user@gmail.com> cc: other_user(a)gmail.com<mailto:other_user@gmail.com>; Williams, Peter; Again, what's in the Cc: of the mail that test_user(a)gmail.com<mailto:test_user@gmail.com> and other_user(a)gmail.com<mailto:other_user@gmail.com> receive? Also, please see <https://wiki.list.org/DOC/Mailman%20is%20not%20Listserv>. -- Mark Sapiro <mark(a)msapiro.net<mailto:mark@msapiro.net>> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org<mailto:mailman-users@mailman3.org> To unsubscribe send an email to mailman-users-leave(a)mailman3.org<mailto:mailman-users-leave@mailman3.org> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

3 4