Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: lmtp runner not starting

by Guillermo Hernandez (Oldno7)

El 1/4/23 a las 21:00, Guillermo Hernandez (Oldno7) via Mailman-users escribió: > > El 1/4/23 a las 20:44, Mark Sapiro escribió: >> ... >> lmtp runner did not start because at some point it had died and >> restarting Mailman won't restart a runner that died and was marked to >> not restart. >> >> You need to stop Mailman and make sure when mailman is stopped that >> there are no files in mailman's var/locks/ directory (remove any that >> are there) and then start Mailman and lmtp runner should start. >> > I stopped mailman and removed all that was in there (there were many > locked registers). Restarted mailman, and it shows the same error that > in my previous mail. > > Now there is two new locked files, one (master.lck) which its content > is the name of the other file. And the second one has the same content: > > master.lck|www.proteccion-civil.eu|9389|7022761192045613065 > > will try deleting the content of var/locks and restarting the server same results lmtp runner is diying. Now I know thata part of the lock file is the pid of the master runner. Apr 01 21:03:03 2023 (11548) Master started Apr 01 21:03:11 2023 (11563) virgin runner started. Apr 01 21:03:13 2023 (11553) bounces runner started. Apr 01 21:03:14 2023 (11561) retry runner started. Apr 01 21:03:15 2023 (11560) rest runner started. [2023-04-01 21:03:15 +0200] [11560] [INFO] Starting gunicorn 20.1.0 [2023-04-01 21:03:15 +0200] [11560] [INFO] Listening at: http://10.0.1.42:8001 (11560) [2023-04-01 21:03:15 +0200] [11560] [INFO] Using worker: sync [2023-04-01 21:03:15 +0200] [11567] [INFO] Booting worker with pid: 11567 [2023-04-01 21:03:15 +0200] [11568] [INFO] Booting worker with pid: 11568 Apr 01 21:03:16 2023 (11564) digest runner started. Apr 01 21:03:17 2023 (11562) task runner started. Apr 01 21:03:17 2023 (11562) Task runner evicted 0 expired pendings Apr 01 21:03:17 2023 (11562) Task runner deleted 0 orphaned workflows Apr 01 21:03:17 2023 (11562) Task runner deleted 0 orphaned requests Apr 01 21:03:17 2023 (11562) Task runner deleted 0 orphaned messages Apr 01 21:03:17 2023 (11562) Task runner evicted expired cache entries Apr 01 21:03:18 2023 (11559) pipeline runner started. Apr 01 21:03:19 2023 (11555) in runner started. Apr 01 21:03:19 2023 (11554) command runner started. Apr 01 21:03:20 2023 (11558) out runner started. Apr 01 21:03:22 2023 (11557) nntp runner started. Apr 01 21:03:23 2023 (11552) archive runner started. a ps ax output: 11548 - IsJ 0:01.92 /usr/local/bin/python3.9 /usr/local/bin/master --force -C /usr/local/mailman3/var/etc/mailman.cfg 11552 - SJ 0:01.97 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=archive:0:1 11553 - IJ 0:02.10 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=bounces:0:1 11554 - SJ 0:01.94 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=command:0:1 11555 - SJ 0:01.94 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=in:0:1 11556 - ZJ 0:02.21 <defunct> 11557 - SJ 0:01.95 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=nntp:0:1 11558 - SJ 0:01.93 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=out:0:1 11559 - SJ 0:01.94 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=pipeline:0:1 11560 - SJ 0:02.02 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=rest:0:1 11561 - IJ 0:01.94 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=retry:0:1 11562 - IJ 0:02.02 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=task:0:1 11563 - SJ 0:01.95 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=virgin:0:1 11564 - SJ 0:01.93 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=digest:0:1 11567 - IJ 0:00.01 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=rest:0:1 11568 - IJ 0:00.01 /usr/local/bin/python3.9 /usr/local/bin/runner -C /usr/local/mailman3/var/etc/mailman.cfg --runner=rest:0:1 > > > > > >> As to why it died in the first place, there should be something in >> mailman.log from the time it died. >> -- ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: firma-GHP-emails.png Replaced multipart/alternative part with first alternative.

2 years, 9 months

[MM3-users] Re: Prototype import [was: Re: Setup Question: Disk-Layout (Migrating from Mailman 2.1)]

by Mihai Moldovan

* On 4/18/25 04:23, Mark Sapiro wrote: > On 4/17/25 11:35 AM, Mihai Moldovan wrote: >> >> New incoming messages will have the correct data, of course, but >> imported ones wouldn't, so I'll have to use a more sophisticated >> approach to handle them, probably by going through mailman's email >> wrapper, figuring out how to generate a msgdata object for a message, >> using RFC2369.process and maybe more. > > Here's an example script. You need to run this with > /opt/mailman/mm/venv/bin/python to get access to the mailman imports. > ``` > from mailbox import Maildir, mbox > from mailman.email.message import Message > from mailman.handlers.rfc2369 import process > from mailman.interfaces.listmanager import IListManager > from mailman.utilities.email import add_message_hash > from zope.component import getUtility > mb = mbox('path/to/mbox', factory=Message, create=False) > md = Maildir('path/to/maildir', create=False) > mlist = getUtility(IListManager).get_by_list_id('your.list.id') > for msg in mb: > add_message_hash(msg) > process(mlist, msg, {}) > md.add(msg) > mb.close() > md.close() > ``` Thank you. Unfortunately, I'm having a hard time getting this to work correctly. My current approach is (which is modified from yours, for instance to use Prototype.archive_message() instead of writing directly to a Maildir): ```py import copy import sys from mailbox import mbox from mailman.archiving.prototype import Prototype from mailman.core import initialize from mailman.email.message import Message from mailman.handlers.rfc_2369 import process from mailman.interfaces.listmanager import IListManager from mailman.interfaces.mailinglist import IMailingList from mailman.utilities.email import add_message_hash from zope.component import getUtility initialize.initialize() if (len(sys.argv) < 3): print('Usage: {0} <list-id> <mbox-file>'.format(sys.argv[0]), file=sys.stderr) exit(1) mb = mbox(sys.argv[2], factory=Message, create=False) mlist = getUtility(IListManager).get_by_list_id(sys.argv[1]) for msg in mb: try: add_message_hash(msg) process(mlist, msg {}) Prototype.archive_message(mlist, msg) except Exception as e: print("Error when adding {0}: {1}".format(msg['message-id'], str(e)), file=sys.stderr) mb.close() ``` This returns "Error when adding None: '_PartialFile' object has no attribute 'header_max_count'" for each message. This, including getting None for the Message-ID, stumped me and I got on to debugging this. Indeed, even something as simple as ```py for msg in mb: print(type(msg)) print(msg['message-id']) print(msg) exit(0) ``` results in getting a None for the Message-ID and a stack trace: <class 'mailman.email.message.Message'> None Traceback (most recent call last): File "/root/mailman3/prototype-import.py", line 29, in <module> print(msg) File "/usr/lib/python3.12/email/message.py", line 165, in __str__ return self.as_string() ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/email/message.py", line 55, in as_string value = email.message.Message.as_string(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/email/message.py", line 188, in as_string g.flatten(self, unixfrom=unixfrom) File "/usr/lib/python3.12/email/generator.py", line 98, in flatten policy = policy.clone(max_line_length=self.maxheaderlen) ^^^^^^^^^^^^ AttributeError: '_PartialFile' object has no attribute 'clone'. Did you mean: 'close'? This eventually led me to realize that passing mailman.email.message.Message as the factory to mbox() internally calls factory(msg), and the base class of Message (email.message.Message) has an __init__ function that takes the policy parameter, so it looks as if it's registering the mailbox message incorrectly as the policy handler, which is totally wrong of course. If I change the call to mb = mbox(sys.argv[2], factory=None, create=False), the output looks more promising, so converting the data to mailman.email.message.Message first seems to have been the wrong idea on my part: ``` <class 'mailbox.mboxMessage'> <mailman.1.1399151409.10314.x2go-i18n(a)lists.x2go.org> Return-Path: <mailman-bounces(a)lists.x2go.org> [...] ``` and indeed, if I let the import actually happen, it works. The imported messages, do have a Message-ID-Hash, but the Archived-At and List-Archive headers are empty (literally <>). Do you have an example message that was archived through Prototype? What are the proper header values for this module? I believe that Archived-At should be the Message-ID-Hash and the List-Archive header contain... well, given that the Prototype archiver is not meant to be publicly available, probably a file:// URL? Mihai

8 months, 4 weeks

[MM3-users] Re: error changed after restart (was: KeyError: 'get_requests_count' after an upgrade)

by Abhilash Raj

On Sat, Feb 6, 2021, at 3:04 AM, Guillermo Hernandez (Oldno7) via Mailman-users wrote: > I restarted de server and the error changed. Now the log shows > "KeyError: 'subscription_mode'": Did you also restart Mailman Core after the upgrade? > > ERROR 2021-02-06 11:47:49,015 26798 django.request Internal Server > Error: /mailman3/mailman3/lists/name_and_domain.of.the.list > Traceback (most recent call last): > File > "/usr/local/lib/python3.7/site-packages/mailmanclient/restbase/base.py", > line 119, in __getattr__ > return self._get(name) > File > "/usr/local/lib/python3.7/site-packages/mailmanclient/restbase/base.py", > line 86, in _get > raise KeyError(key) > KeyError: 'subscription_mode' > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File > "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", > line 34, in inner > response = get_response(request) > File > "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", > line 115, in _get_response > response = self.process_exception_by_middleware(e, request) > File > "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", > line 113, in _get_response > response = wrapped_callback(request, *callback_args, **callback_kwargs) > File > "/usr/local/lib/python3.7/site-packages/django/views/generic/base.py", > line 71, in view > return self.dispatch(request, *args, **kwargs) > File > "/usr/local/lib/python3.7/site-packages/postorius/views/generic.py", > line 74, in dispatch > return super(MailingListView, self).dispatch(request, *args, **kwargs) > File > "/usr/local/lib/python3.7/site-packages/django/views/generic/base.py", > line 97, in dispatch > return handler(request, *args, **kwargs) > File > "/usr/local/lib/python3.7/site-packages/postorius/views/list.py", line > 295, in get > member.subscription_mode == > File > "/usr/local/lib/python3.7/site-packages/mailmanclient/restbase/base.py", > line 124, in __getattr__ > self.__class__.__name__, name)) > AttributeError: 'Member' object has no attribute 'subscription_mode' > *********************** > some info about the installed versions via pip list: > pip list | grep django > django-allauth 0.44.0 > django-appconf 1.0.4 > django-compressor 2.4 > django-extensions 3.1.0 > django-gravatar2 1.4.4 > django-haystack 3.0 > django-mailman3 1.3.5 > django-picklefield 3.0.1 > django-q 1.3.4 > djangorestframework 3.12.2 > > pip list | grep mailman > django-mailman3 1.3.5 > mailman 3.3.3 > mailman-hyperkitty 1.1.0 > mailmanclient 3.3.2 > > pip list | grep postorius > postorius 1.3.4 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 6/2/21 11:12, Guillermo Hernandez (Oldno7) via Mailman-users wrote: > > I've just upgrade mailman 3 installation following Mr. Sapiro advice: > > did a > > > > pip install --upgrade django-mailman3 hyperkitty mailman mailmanclient > > mailman-hyperkitty postorius > > > > I did a "python3 manage.py migrate" after, too. > > > > And all seemed to run well. > > > > All the lists showed in postorius via web, but when I try to accesss > > into one of them the browser shows an error. > > > > In the log you can see: > > > > *-*-*-*-*-*-* > > > > Traceback (most recent call last): > > File > > "/usr/local/lib/python3.7/site-packages/mailmanclient/restbase/base.py", > > line 119, in __getattr__ > > return self._get(name) > > File > > "/usr/local/lib/python3.7/site-packages/mailmanclient/restbase/base.py", > > line 86, in _get > > raise KeyError(key) > > KeyError: 'get_requests_count' > > > > ... (And after all the traceback lines) > > > > AttributeError: 'MailingList' object has no attribute > > 'get_requests_count' > > > > *-*-*-*-*-*-*-* > > > > The lists seem to be distributing messeages well.. but I cannot acces > > via web administration (django/postorius) > > > > Can anyone point me in the right direction to solve this, please? > > > > _______________________________________________ > > Mailman-users mailing list -- mailman-users(a)mailman3.org > > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

4 years, 11 months

[MM3-users] Re: Held messages: Long time waiating for response of the Mailman API

by Stephan Krinetzki

Mark Sapiro wrote: > On 1/29/22 05:48, Jacob Sievert via Mailman-users wrote: > > Hello, > > we seem to have the same problem right now. > > We are still on python3.6 and postgresql 12.9 > > I looked at our Table pendedkeyvalue and we have roughly 35k rows in > > that table, > > is that normal or shouldn't those get cleaned up after a while? > > Yes they should. This is https://gitlab.com/mailman/mailman/-/issues/257 > fixed in Mailman core 3.3.5. Also in Mailman 3.3.5 is a new Task runner > that runs periodic tasks, one of which is to remove orphaned pendings. > This may also be the issue for the OP in this thread if that > pendedkeyvalue table is also large. I can confirm this. We have a total of ~580000 Rows in the pendedkeyvalue table. Since last wednsday even the hourly job quits (Out of memory exception) > Here is a mailman shell script that will clean that up. > # Prior to Mailman 3.3.5, some tokens for user confirmations were pended > with > # too long a lifetime. This script removes those pendings based on when > they > # were pended and the configured pending_request_life rather than their > # expiration. > > # Also prior to Mailman 3.3.5, pended held_message tokens for email handling > # of the message were not removed when the message was handled via REST. > This > # script removes those pendings too. > > # This is run with > # mailman shell -r delete_orphans_expireds > # after saving it as > # /opt/mailman/mm/venv/bin/delete_orphans_expireds.py > > from datetime import datetime > from lazr.config import as_timedelta > from mailman.config import config > from mailman.database.transaction import transactional > from mailman.interfaces.pending import IPendings > from zope.component import getUtility > > pendings = getUtility(IPendings) > > def is_request(id): > if config.db.store.execute( > 'SELECT * FROM _request WHERE id = {};'.format(id)).rowcount > 0: > return True > return False > > then = datetime.now() - as_timedelta(config.mailman.pending_request_life) > thenm = datetime.now() - as_timedelta(config.mailman.moderator_request_life) > > def fromisoformat(x): > if hasattr(datetime, 'fromisoformat'): > return datetime.fromisoformat(x) > try: > return datetime.strptime(x, '%Y-%m-%dT%H:%M:%S.%f') > except ValueError: > return datetime.strptime(x, '%Y-%m-%dT%H:%M:%S') > > @transactional > def delete_orphans_expireds(): > count = 0 > for token, data in pendings.find(pend_type='held message'): > if data and not is_request(data['id']): > result = pendings.confirm(token, expunge=True) > count += 1 > print(f'expunged {count} orphaned pended held messages') > > count = 0 > for token, data in pendings.find(pend_type='data'): > if data and data['_mod_hold_date']: > when = data['_mod_hold_date'] > if isinstance(when, str): > when = fromisoformat(when) > if when < thenm: > result = pendings.confirm(token, expunge=True) > count += 1 > print(f'expunged {count} expired held messages') > > pends = list(pendings.find(pend_type='subscription')) > pends += list(pendings.find(pend_type='unsubscription')) > count = 0 > for token, values in pends: > if values and values['token_owner'] == 'subscriber': > when = values['when'] > if isinstance(when, str): > when = fromisoformat(when) > if when < then: > result = pendings.confirm(token, expunge=True) > count += 1 > print(f'expunged {count} expired (un)subscription confirmations') > > The above says to save the script at > /opt/mailman/mm/venv/bin/delete_orphans_expireds.py but that path may > need to be adjusted based in where Mailman's bin/ directory is in your > installation. Thanks for the Script Mark! WIth the Script, the result is: mailman shell -r delete_orphans_expireds Jan 31 09:15:46 2022 (7980) Database url: postgres://mailman:XXXXXXXX@127.0.0.1/YYYYYYY expunged 95918 orphaned pended held messages expunged 7958 expired held messages expunged 22372 expired (un)subscription confirmations And the pendedkeyvalue table has significant lower entries (~ 92000) and now the process to accept or decline a held message ist a lot of faster. Maybe we will take the database to a seperate server to get more speed - but for the moment it is fast enough. Thanks again and thanks to Jacob Sievert for the pointer to the size of the pendedkeyvalue table.

3 years, 11 months

[MM3-users] Re: integrating mm3 with postfix / lmtp

by Abhilash Raj

On Nov 13 2017, at 1:34 am, Thor Atle Rustad <thor.rustad(a)gmail.com> wrote: > There is a way around it! > > I have had two issues with the the maxking docker image. One is that the regexp is not working properly. I reported that, and it has been fixed in newer code. My other problem is that the docker image creates a user, mailman, that receives uid 103. Well, uid 103 on my system is already taken by systemd-bus-proxy (grep 103 /etc/passwd returns "systemd-bus-proxy:x:103:105:systemd Bus Proxy,,,:/run/systemd:/bin/false"). Containers usually run with uid namespace, so it doesn't really matter what uid is used outside of container. Unless, you mount /etc/shadow from host in to the container, which isn't really needed for the images. > > My solution includes downloading the corrected postfix.py, and replacing the Dockerfile. I put the postfix.py in <docker-mailman>/core/assets/. > > My Dockerfile: > FROM maxking/mailman-core > > RUN grep mailman /etc/passwd && grep mailman /etc/group \ > && deluser mailman \ > && addgroup -S -g900 mailman \ > && adduser -S -u900 mailman mailman \ > && grep mailman /etc/passwd && grep mailman /etc/group > COPY assets/postfix.py /usr/local/lib/python3.6/site-packages/mailman/mta/postfix.py > > > I then run docker build (with -t parameter, you must look up that yourself). I use a different name for my images, so I end up with (note, there are two tags per image): > root@mailer:/home/mailman/docker/docker-mailman_mods/core# docker images > REPOSITORY TAG IMAGE ID CREATED SIZE > local/mailman_core_900 20171110_2 9649e84767e1 2 days ago 176MB > local/mailman_core_900 latest 9649e84767e1 2 days ago 176MB > local/mailman_web_900 20171110_2 07a9b3d7ddd6 2 days ago 247MB > local/mailman_web_900 latest 07a9b3d7ddd6 2 days ago 247MB > > > I do the same with the web image, as I need to change the user there, too. > > Then, in docker-compose.yaml, I change the line(s) referring to the image name(s): > > services: > mailman-core: > image: local/mailman_core_900 > container_name: mailman-core > hostname: mailman-core > > > > mailman-web: > image: local/mailman_web_900 > container_name: mailman-web > hostname: mailman-web > > > I don't know if this is a good solution, but at least it fixes some serious issues with the 3.1's postfix integration that wouldn't otherwise be fixed until the 3.2 release. The bottom line is that it works for me, but it adds an additional layer of complication. > > 2017-11-03 19:40 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in (mailto:maxking@asynchronous.in)>: > > On Fri, Nov 3, 2017, at 08:29 AM, Fabian A. Santiago wrote: > > > October 26, 2017 11:07 PM, "Mark Sapiro" <mark(a)msapiro.net (mailto:mark@msapiro.net)> wrote: > > > > > > > On October 26, 2017 7:30:35 PM PDT, "Fabian A. Santiago" <fsantiago(a)garbage-juice.com (mailto:fsantiago@garbage-juice.com)> wrote: > > > > > > > >> That was it. Perfect. I manually modified my regexp map and it works > > > >> now. Excellent and Thank you. You're the man. Does mm3 ever refresh > > > >> those maps or only as I add new domains / lists? > > > > > > > > Only when you make changes to domains or lists. > > > > > > > > -- > > > > Mark Sapiro <mark(a)msapiro.net (mailto:mark@msapiro.net)> > > > > Sent from my Not_an_iThing with standards compliant, open source software. > > > > > > Mark, > > > > > > I've noticed that even simply restarting the mm3 components those alias > > > maps get rewritten and the problem returns until I can manually edit it. > > > > Yeah, that is true. Transport maps are re-generated everytime the > > container restarts. > > > > I don't think think there is any way around this right now :( > > > > > > -- > > Abhilash Raj > > maxking(a)asynchronous.in (mailto:maxking@asynchronous.in) > > _______________________________________________ > > Mailman-users mailing list > > mailman-users(a)mailman3.org (mailto:mailman-users@mailman3.org) > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

8 years, 2 months

[MM3-users] Re: mail loops back to myself

by Florian Sukup

On 6/9/25 05:05, Mark Sapiro wrote: > On 6/8/25 16:04, Florian Sukup wrote: >> >> I set up mailman, created a site/domain (lists.yyy.com) My hostname >> is host.xxx.com . My IP has a RDNS entry: reverse.rrr.com . > > Why is the RDNS to reverse.rrr.com and not host.xxx.com. It is important > for mail delivery to have full circle DNS. I.e. the sending server > should have an A record for its IP and revers DNS for that IP should > point back to the sending server's name. I'm guessing that rrr.com is a > hosting provider and you don't control the rDNS for that IP, but you > should try to get them to change it for you. Without that change, > delivery of your outbound mail, at least to large ISPs, will be > problematic at best. > The setup has historic reasons. However I can eliminate reverse.rrr.com completely and replace it by host.xxx.com. Right now reverse.rrr.com has an A-record pointing to the host's ip address. > >> When I send an email to my mailing list mylist(a)lists.yyy.com I receive >> an error email. The logfile says the following: >> >> ... >> Jun 9 00:23:38 arvak postfix/relay/smtp[]: 094895FADB: >> to=<mylist(a)lists.yyy.com>, relay=reverse.rrr.com[m.y.i.p]:25, >> delay=0.06, delays=0.03/0.01/0.02/0, dsn=5.4.6, status=bounced (mail >> for lists.yyy.com loops back to myself) > > What is the output from `postconf -n`? > alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases append_dot_mydomain = no biff = no compatibility_level = 2 inet_interfaces = all inet_protocols = all local_recipient_maps = proxy:unix:passwd.byname $alias_maps hash:/var/lib/mailman3/data/postfix_lmtp mailbox_size_limit = 0 message_size_limit = 0 mydestination = localhost, localhost.localdomain, arvak myhostname = host.xxx.com mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128, 192.168.77.0/24 myorigin = /etc/mailname owner_request_special = no readme_directory = no recipient_delimiter = + relay_domains = ${{$compatibility_level} < {2} ? {$mydestination} : {}} hash:/var/lib/mailman3/data/postfix_domains relayhost = smtp_tls_CApath = /etc/ssl/certs smtp_tls_cert_file = /etc/letsencrypt/live/host.xxx.com/fullchain.pem smtp_tls_key_file = /etc/letsencrypt/live/host.xxx.com/privkey.pem smtp_tls_loglevel = 3 smtp_tls_security_level = may smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtp_use_tls = yes smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination smtpd_tls_cert_file = /etc/letsencrypt/live/host.xxx.com/fullchain.pem smtpd_tls_key_file = /etc/letsencrypt/live/host.xxx.com/privkey.pem smtpd_tls_loglevel = 3 smtpd_tls_security_level = may smtpd_use_tls = yes transport_maps = hash:/var/lib/mailman3/data/postfix_lmtp virtual_alias_domains = 9 different domains, however not host.xxx.com, lists.yyy.com or reverse.rrr.com virtual_alias_maps = hash:/etc/postfix/virtual >> Jun 9 00:23:38 arvak postfix/smtpd[114119]: disconnect from >> host.xxx.com[m.y.i.p] ehlo=1 quit=1 commands=2 >> ... >> >> The MX record of lists.yyy.com points to reverse.rrr.com. Not sure if >> this is the best idea? > > reverse.rrr.com has no A or AAAA record. An MX MUST point to a domain > that has an A or AAAA record. The MX should point to host.xxx.com. > Will be resolved (s. above). >> Can anyone give me a hint where to search for this error? > > The output from `postconf -n` would help. Also, have you set up postfix > per > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h… ? > Basically yes, but I see a few differences which all worked on my test installation. Here comes my mailman.cfg: [mailman] site_owner: mailman@... noreply_address: noreply default_language: en sender_headers: from from_ reply-to sender email_commands_max_lines: 10 pending_request_life: 3d cache_life: 7d pre_hook: post_hook: layout: debian filtered_messages_are_preservable: no html_to_plain_text_command: /usr/bin/lynx -dump $filename listname_chars: [-_.0-9a-z] [shell] prompt: >>> banner: Welcome to the GNU Mailman shell use_ipython: no history_file: [paths.debian] var_dir: /var/lib/mailman3 queue_dir: $var_dir/queue bin_dir: /usr/lib/mailman3/bin list_data_dir: $var_dir/lists log_dir: /var/log/mailman3 lock_dir: $var_dir/locks data_dir: $var_dir/data cache_dir: $var_dir/cache etc_dir: /etc/mailman3 ext_dir: $var_dir/ext messages_dir: $var_dir/messages archive_dir: $var_dir/archives template_dir: $var_dir/templates pid_file: /run/mailman3/master.pid lock_file: $lock_dir/master.lck [database] class: mailman.database.sqlite.SQLiteDatabase url: sqlite:///$DATA_DIR/mailman.db debug: no [logging.debian] format: %(asctime)s (%(process)d) %(message)s datefmt: %b %d %H:%M:%S %Y propagate: no level: info path: mailman.log [webservice] hostname: localhost port: 8001 use_https: no show_tracebacks: yes api_version: 3.1 admin_user: ... admin_pass: ... [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver smtp_host: localhost smtp_port: 25 smtp_user: smtp_pass: lmtp_host: 127.0.0.1 lmtp_port: 8024 configuration: python:mailman.config.postfix Thanks for your help, Florian.

7 months, 1 week

[MM3-users] Re: Member Issue Discovered

by Brian Carpenter

On 10/20/20 11:19 PM, Mark Sapiro wrote: > On 10/20/20 6:54 AM, Brian Carpenter wrote: >> Respectively, I think you are asking the wrong question here. The real >> question is why isn't a display_name being removed when a list >> subscriber is unsubscribed. > > I'd like to understand the real requirement. It seems to me that this > issue has come up because a list admin wanted to change the display name > shown in the membership roster for a user. Since there is currently no > UI to do this, the list admin tried to do it by unsubscribing and > resubscribing the user. That didn't work which led to this > "unsubscribing a user should remove the user's information" thread, but > the real issue is the lack of a UI for changing display names. It seems > if that UI existed and was available, the "unsubscribing a user should > remove the user's information" issue would never have been raised. There are two real requirements. One is to be able to do something as easy as changing a name for a list member. I did a lot of testing with the relationship between a name used for a subscription versus a name used for registering via the U.I. (Postorius/Django) and it is very confusing. I still am having a very difficult time understanding the logic presented here for the way Mailman 3 handles user information. The second requirement is ALL data should be removed if someone unsubscribes from a list that is just a list member of a single list. I feel very strongly about that. I don't really care for the reasoning behind why the data is retained. I just think it should be removed for a list member who has no need for an account that manages multiple email address and is subscribed to multiple lists. I host many single lists. So it is very important to me, and as an advocate for my clients, I will state very clearly how important it is to me (and my clients) regards of other user scenarios out there (looking at you Mr. Turnbull). I care about my own. > So perhaps what we should be talking about is UIs for changing user > information, what they would look like and who should be able to change > what. That is a start and I thought I brought that up. We also need a separate conversation on the retention of data apparently. > Note that I personally am a member of many lists, an admin of multiple > lists and a site admin for multiple mailman installations. I am well > aware of the frustrations of list admins who wind up just doing it > because it's way easier than instruction some users as to how to do it > themselves. However, I don't think that is necessarily sufficient reason > to hand over control of global, non-list specific user information to > the admin of one particular list that the user happens to be a member of. I never asked for global control for list owners. You have made that almost a necessity with the multiple email address per user account feature that you brought in. I don't think List owners should have global control but server owners certainly do. But the rightful avoidance of such control for List owners, I think has resulted in a wrongful limiting of what they can do currently. I so disagree with S. Turnbull's disparaging comments that I think we ought to be designing for List owners primarily and not list members/users when it comes to user interfaces. From what I see, it is mostly server and list owners that are interacting with this (mailman-user) list and not list members/users. In my experience, I never hear from list members. Just list owners. Whatever issue list owners have with their own list members are easily handled by them when it comes to Mailman 2. Not so much with Mailman 3. > > Even in mailman 2.1, while a list admin could go to a user's options > page for the list and change things, the "change globally" check boxes > only worked for the user, not for the list admin. > -- Brian Carpenter Harmonylists.com Emwd.com

5 years, 2 months

[MM3-users] Re: signup / registration error - permissions and cert chains

by David Newman

On Dec 31, 2021, at 03:43, Victoriano Giralt <victoriano(a)uma.es> wrote: > > El viernes, 31 de diciembre de 2021 1:48:38 (CET) David Newman escribió: >> I'd like for regular (non-admin) list subscribers to be able to manage >> their subscription preferences and view list archives. > > That's a good way to go :-) > > My response is more of a (very) old sysadmin and Django user (since 2008) > hunch that a proper one based on code and documentation review, but I've been > trying to contribute several times and always (super) Mark Sapiro beats me :-) > >> If I'm reading the error correctly, this is related to an inability to >> verify the cert chain. The /etc/mailman3/settings.py file points to the >> same cert and key files used by Nginx, Postfix, and Dovecot. > > You are right in your diagnose but not in your interpretation (see my comment > below inside the traceback). It is certificate related, but not for server > TLS, but for CLIENT authentication. > > >> EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' >> EMAIL_HOST = 'localhost' >> EMAIL_PORT = 25 >> EMAIL_HOST_USER = 'dnewman(a)networktest.com' >> EMAIL_HOST_PASSWORD = 'wouldnt-you-like-to-know' >> EMAIL_USE_TLS = 'True' >> EMAIL_SSL_CERTFILE = '/etc/ssl/certs/myhost.crt' >> EMAIL_SSL_KEYFILE = '/etc/ssl/private/myhost.key' > > All these settings above are used for SENDING messages and, if I'm not > mistaken, the SSL key and cert are used for authenticating the user sending > the email. Actually, using TLS and SMTP Auth for localhost is a bit too much. > I've been configuring SMTP servers since 1990 and my mail servers just accept > mail form localhost, if they are broken into, the user and password have > already been exposed :-) > >> But this might only be for email, not Postorius/Django. > > You are right (if I also am) > >> What additional configuration is needed to allow regular users to create >> and manage their own accounts? > > I'd say that is more what is not needed (the SMTP TLS authentication) > > I'll remove the "noise". These are the tell tale lines: > >> "/opt/mailman/venv/lib/python3.9/site-packages/django/core/mail/backends/smt >> p.py", line 67, in open >> self.connection.starttls(keyfile=self.ssl_keyfile, >> certfile=self.ssl_certfile) > > The SMTP Django backend is trying to connect to the mail server to send the > Mailman account confirmation message and failing, probably because the user > Django runs as cannot open the private key (which is a very sensible thing if > that private key is the one used for the web facing TLS certificate, I can > tell you how bad in private or search for my name, wasd, apache and VMS ;-)) > > That certificate is not needed for sending email from Django, and, as I said, > not even SMTP Auth for sending via localhost. Actually, doing SMTP Auth on > port 25 is not even recommended practice. Hi Victoriano, Thanks for this. I could use some clarification on what specific changes you are suggesting. I *think* you are saying to remove the EMAIL_USE_TLS stuff and also move to another port (maybe 587), but I am not sure. Also, the reason I added the TLS in the first place was that I was getting errors without it. And I am unclear why the cert / private key pair do not work for Django when they do work OK for Postfix, Nginx, and Dovecot. Thanks for clarifying — and happy and safe 2022 to you as well! dn > > Happy, healthy, safe and well ventilated New Year to all. > > -- > Victoriano Giralt Innovation Director > Digital Transformation Vicerectorate University of Malaga > +34952131415 SPAIN > ================================================================== > Note: signature.asc is the electronic signature of present message > A: Yes. >> Q: Are you sure ? >>> A: Because it reverses the logical flow of conversation. >>>> Q: Why is top posting annoying in email ? > > > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

4 years

[MM3-users] Re: Customise Postorius templates?

by Abhilash Raj

> On Feb 7, 2022, at 4:35 AM, Duane Raymond <duane(a)fairsay.com> wrote: > > On Mon, 7 Feb 2022 at 00:56, Abhilash Raj <maxking(a)asynchronous.in> wrote: > >>> On Feb 1, 2022, at 11:01 AM, Mark Sapiro <mark(a)msapiro.net> wrote: >>> >>> On 2/1/22 09:47, Duane Raymond wrote: >>>> Hi, >>>> I'm looking to do some radical customisation of the Postorius (and >> Hyperkitty) templates over the next 6 ish months and was wondering if there >> is a 'best practice' way to do this that will survive updates to MM3. I've >> searched around and not found much and also tested copying >>>> >> venv/lib/python3.7/site-packages/postorius/templates/postorius/lists/summary.html >>>> to >>>> var/templates/lists/summary.html >>>> As suggested in some posts - but it didn't seem to pick up the >> customisation - only customisations in the venv path worked - and they get >> overwritten on update. > > >> The best way to do this would be utilize Django’s template loader. See >> here[1] for the documentation on how you can do this. You want to put the >> path where you are putting the templates under `DIRS` option of the >> `TEMPLATES` section. >> >> Do make sure that you are keeping the directory structure correct so that >> Django can discover them. Like, Hyperkitty’s base.html should be in >> `/custom/path/hyperkitty/base.html`, where `/custom/path` is what you’ve >> setup in DIRS setting above. >> > I hadn’t tried it when I sent email, but when I tried it locally on Postorius, it seems to work for me. > This looks promising! I've not got it working yet, but this is what I've > tried: > > 1. Created folder 'custom' in the mm root (in my case: /opt/mailman/mm/) > 2. Copied the files and folders from postorius/templates/postorius/* and > hyperkitty/templates/hyperkitty/* to the custom/ folder (not the > full postorius/templates/postorius/ match, just evening in the > second postorius/ dirrectory) Note that you want to copy from “src/postorius/templates” directory to your "/opt/mailman/mm/custom” directory. It is important that you retain the “postorius” and “hyperkitty” directories. Here is what I have in my settings.py: - 'DIRS': [], + 'DIRS': ['/Users/maxking/Documents/mm3/postorius/example_project/templates’], And here is the structure of the templates directory you are seeing above: $ tree /Users/maxking/Documents/mm3/postorius/example_project/templates /Users/maxking/Documents/mm3/postorius/example_project/templates └── postorius └── lists └── summary.html Note that I am only overriding the Summary page here. > 3. Recursive update of permissions of the custom/ folder and files to be > mailman > 4. Updated /opt/mailman/mm/settings.py by changing "'DIRS': []" > to "'DIRS': [BASE_DIR, '/custom/']" in the "TEMPLATES =" section Note that you don’t want [“BASE_DIR”, “/custom/“] here, the comma is the wrong value here. The docs I mentioned have a “/“. Although, it is possible that even “/“ since that needs BASE_DIR to be a Path datatype and not string. Easiest thing with least surprise would be to use the full path, atleast to ensure that template discovery works as expected. > 5. Edited the original AND copied postorius/lists/summary.html to have a > small change in the footer of each so I know what version it is serving Since I was running a dev server, it doesn’t need restart, but simply restarting gunicorn should should do it. You don’t need to restart anything else. > 6. Restarted: gunicorn qcluster mailman nginx and cleared browser cache > 7. Reloaded the 'info' page of a few lists to see which version ti was > serving > > Variants: I've also changed the DIRS value to the hard-coded path: > /opt/mailman/mm/custom > > Result: it is still serving the default/original Postorius template vs the > custom one. > > Any suggestions for things I should do to make it use the /custom/ folder > instead? Any recompiling or other actions beyond just restarting? (I > probably don't need to restart qcluster and maybe not even mailman core for > this to take effect but I do it just in case!) Hope that helps. -- thanks, Abhilash Raj (maxking)

3 years, 11 months

[MM3-users] Re: Does hyperkitty show JPEGs in line and what about attachments?

by tlhackque

On 12-Jul-17 12:39, Mark Sapiro wrote: > > Also, with respect to Mailman 2.1, if you want the scrubber to preserve > file names and extensions, set the following in mm_cfg.py > > SCRUBBER_DONT_USE_ATTACHMENT_FILENAME = False > SCRUBBER_USE_ATTACHMENT_FILENAME_EXTENSION = True > > and if you want scrubbed HTML to not be HTML escaped so it renders > rather than looking like raw html, set > > ARCHIVE_HTML_SANITIZER = 3 > > but note this comment from Defaults.py > >> # 3 - Remove text/html as attachments but don't HTML-escape them. Note: this >> # is very dangerous because it essentially means anybody can send an HTML >> # email to your site containing evil JavaScript or web bugs, or other >> # nasty things, and folks viewing your archives will be susceptible. You >> # should only consider this option if you do heavy moderation of your list >> # postings. > > This is an issue with HyperKitty as it appears this is what HyperKitty > does and there's no way to turn it off. > This warning seems a bit dated, though it's not completely wrong. It comes from the days when HTML was new, browsers were fragile, and javascript treated with suspicion. And virus/spam scanners for email were non-existent. Today, it's a very rare website that doesn't rely on javascript (Postorious and Hyperkitty use JS). Browsers, while they still have bugs, are much more defensive. And there are plenty of truly evil sites that they have to defend against. It is certainly true that archived e-mail can turn your site into an unknowing distributor of malware: FLASH bugs, documents with embedded buffer overflows, cross-site scripting and the other many ills of the day. Wikis deal with this frequently. However, in these cases, your mailing list has distributed the same bits to your subscribers - a community that you probably care more about than a random visitor to your (open) archive. I wouldn't run a list - public or private - where the traffic doesn't go through SPAM and virus filtering before Mailman sees it. (SpamAssassin and ClamAV are good open-source solutions.) And once you've done that (and Mailman 3's optional DMARC), most of these attacks are defanged/mitigated. This is essentially automated moderation - to a point. Note that all the Djano authentication schemes packaged with Mailman (facebook, google, etc) rely on javascript and are sites littered with what the comment refers to as "webbugs" - Google Analytics, tracking cookies, browser fingerprinting, 0 size images (the original webbug). They make money (and have become mainstream) using technologies that were considered anti-social when that warning was written. (Personally, I still think of them as anti-social, but the public has chosen to pay for services with privacy...) While some may elect to stick with the highly restrictive policies of "plain text only", this limits the information content and applicability of the the platform. Whether this is acceptable depends on the community that you serve. Mailman can be an effective mechanism to deliver rich media on a "push" basis. And that's "rich" by 1980 standards (bold, well-formatted tables, an attached agenda or document package); not even "rich" by today's (sleeping cat videos...). I think that Mailman has to be able to handle today's rich media with a reasonable degree of safety and convenience. Including in the archives. I thought that was one of the goals for Mailman Version 3... I also think that the advice quoted above should be modified to better reflect these realities. Mailman isn't the only tool available to protect users from evil content, and aggressively filtering to plaintext is a very blunt instrument. Including anti-spam, anti-virus, DNS blacklisting, DKIM/DMARC tests in the delivery pipeline (most of which can be/is done before Mailman touches a post) should be strongly recommended. Checks for headers indicating checked-by local (anti-spam/anti-virus) agents should be available in the Mailman rulesets (and require some cooperation from the MTA to ensure that they can't be passed through from outside.) There is nothing wrong with running a plain text only site, if it serves your community. But if Mailman wants to be relevant in today's environment, it has to adapt to rich content as more than an unwelcome guest. (As I have :0)

8 years, 6 months

Search results for query "sapiro"