Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: Newbie question 2: Rewriting issue for bounced emails

by William Oliver

On Sun, 2021-12-26 at 18:23 -0800, Mark Sapiro wrote: > On 12/26/21 6:19 PM, Mark Sapiro wrote: > > > > Otherwise, you can do this in mailman shell, e.g. > > ``` > > $ mailman shell > > > > Welcome to the GNU Mailman shell > > Use commit() to commit changes. > > Use abort() to discard changes since the last commit. > > Exit with ctrl+D does an implicit commit() but exit() does not. > > > > >>> dm = getUtility(IDomainManager) > > >>> d = dm.get('example.com') > > >>> d.alias_domain = x.example.com > > >>> commit() > > ``` > > > Ooops. That should be > > d.alias_domain = 'x.example.com' > > I.e., quoted > > Thanks. I am *almost* there. Adding the vmap in mailman fixed *almost* everything. Now I can add users, get receive mails to the list, archive messages correctly, and send emails. Except... The list sends emails to local recipients fine. I have three users on my test list: fp145(a)libertyfp.org (local) billo(a)billoblog.com (not local) oliver(a)billoblog.com (not local) When I send a message to the list from *any* of these three, the mailman3 accepts it and attempts to broadcast it out. It is delivered sucessfully to the local email address, but it looks like there's a relaying problem with external addresses. In the syslog snippet below, note that the fp145(a)libertyfp.org mail is delivered, but the mail to the recipients at billoblog.com get an "Access denied" Dec 27 15:33:25 libertyfp postfix/smtpd[419593]: connect from mail.libertyfp.org[2.56.57.28] Dec 27 15:33:25 libertyfp postfix/smtpd[419593]: DC203421BE: client=mail.libertyfp.org[2.56.57.28] Dec 27 15:33:25 libertyfp postfix/cleanup[420001]: DC203421BE: message- id=<9785b5a56581104fe079221bb947eb3a03813d97.camel(a)billoblog.com> Here it is getting delivered locally: Dec 27 15:33:25 libertyfp postfix/cleanup[420001]: DC203421BE: warning: header Subject: [Testlist] bbbbb from mail.libertyfp.org[2.56.57.28]; from=<testlist-bounces+fp145=libertyfp.org(a)libertyfp.org> to=<fp145(a)libertyfp.org> proto=ESMTP helo=<mail.libertyfp.org> Dec 27 15:33:25 libertyfp postfix/qmgr[418758]: DC203421BE: from=<testlist-bounces+fp145=libertyfp.org(a)libertyfp.org>, size=2651, nrcpt=1 (queue active) Dec 27 15:33:25 libertyfp postfix/virtual[420016]: DC203421BE: to=<fp145(a)libertyfp.org>, relay=virtual, delay=0.03, delays=0.01/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to maildir) Dec 27 15:33:25 libertyfp postfix/qmgr[418758]: DC203421BE: removed Here it is getting bounced for billo(a)billoblog.com and oliver(a)billoblog.com: Dec 27 15:33:26 libertyfp postfix/smtpd[419593]: NOQUEUE: reject: RCPT from mail.libertyfp.org[2.56.57.28]: 554 5.7.1 <billo(a)billoblog.com>: Recipient address rejected: Access denied; from=<testlist-bounces+billo=billoblog.com(a)libertyfp.org> to=<billo(a)billoblog.com> proto=ESMTP helo=<mail.libertyfp.org> Dec 27 15:33:26 libertyfp postfix/smtpd[419593]: disconnect from mail.libertyfp.org[2.56.57.28] ehlo=1 mail=2 rcpt=1/2 data=1 rset=1 quit=1 commands=7/8 Dec 27 15:33:26 libertyfp postfix/smtpd[419593]: connect from mail.libertyfp.org[2.56.57.28] Dec 27 15:33:26 libertyfp postfix/smtpd[419593]: NOQUEUE: reject: RCPT from mail.libertyfp.org[2.56.57.28]: 554 5.7.1 <oliver(a)billoblog.com>: Recipient address rejected: Access denied; from=<testlist-bounces+oliver=billoblog.com(a)libertyfp.org> to=<oliver(a)billoblog.com> proto=ESMTP helo=<mail.libertyfp.org> This error occurs whether the account posting to the list is in the libertyfp.org domain or billoblog.com domain. Note that "regular" mail from libertyfp.org does get delivered to outside addresses just fine, originating from inside the domain or through an email client outside the domain. I tried changing the smtp port in mailman.cfg to 587, but that didn't change anything. In my reading, there seems to be differing discussions about what should be in the mydestinations value, but I don't know what it should be. Here's the current last bit of my main.cf, along with the mydestinations setting: mydestination = localhost.org, localhost # add to the end (add virtual users) # if specify multiple domains, specify comma or space separated virtual_mailbox_domains = libertyfp.org virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/virtual-mailbox virtual_uid_maps = static:20000 virtual_gid_maps = static:20000 # mailman3 changes owner_request_special = no always_add_missing_headers = yes transport_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp local_recipient_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp relay_domains = hash:/opt/mailman/mm/var/data/postfix_domains default_destination_recipient_limit = 30 default_destination_concurrency_limit = 15 virtual_alias_maps = hash:/opt/mailman/mm/var/data/postfix_vmap header_checks = regexp:/etc/postfix/header_checks Here's the current values in /opt/mailman/mm/var/data/postfix_domains: # AUTOMATICALLY GENERATED BY MAILMAN ON 2021-12-27 21:10:32 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. x.libertyfp.org libertyfp.org here's postfix_lmtp: # AUTOMATICALLY GENERATED BY MAILMAN ON 2021-12-27 21:10:32 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. # Aliases which are visible only in the @x.libertyfp.org domain. testlist(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-bounces(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-confirm(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-join(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-leave(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-owner(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-request(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-subscribe(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-unsubscribe(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 Heres the postfix_vmap that was (finally) successfully created: # AUTOMATICALLY GENERATED BY MAILMAN ON 2021-12-27 21:10:32 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. # Virtual mappings for the @libertyfp.org domain. testlist(a)libertyfp.org testlist(a)x.libertyfp.org testlist-bounces(a)libertyfp.org testlist-bounces(a)x.libertyfp.org testlist-confirm(a)libertyfp.org testlist-confirm(a)x.libertyfp.org testlist-join(a)libertyfp.org testlist-join(a)x.libertyfp.org testlist-leave(a)libertyfp.org testlist-leave(a)x.libertyfp.org testlist-owner(a)libertyfp.org testlist-owner(a)x.libertyfp.org testlist-request(a)libertyfp.org testlist-request(a)x.libertyfp.org testlist-subscribe(a)libertyfp.org testlist-subscribe(a)x.libertyfp.org testlist-unsubscribe(a)libertyfp.org testlist-unsubscribe(a)x.libertyfp.org For what it's worth, here's the extended header info for the mail that *was* delivered: Return-Path: <testlist-bounces+fp145=libertyfp.org(a)libertyfp.org> X-Original-To: fp145(a)libertyfp.org Delivered-To: fp145(a)libertyfp.org Received: from mail.libertyfp.org (mail.libertyfp.org [2.56.57.28]) by mail.libertyfp.org (Postfix) with ESMTP id 226F6421BA for <fp145(a)libertyfp.org>; Mon, 27 Dec 2021 16:27:17 -0500 (EST) Received: from [10.112.157.251] (unknown [92.60.40.252]) by mail.libertyfp.org (Postfix) with ESMTPSA id F3608421BA for <testlist(a)libertyfp.org>; Mon, 27 Dec 2021 16:27:11 -0500 (EST) Message-Id: <562c2f8aaeb3c9e2ccf198aff197dbd299fc002a.camel(a)libertyfp.org> From: fp145(a)libertyfp.org <fp145(a)libertyfp.org> To: testlist(a)libertyfp.org Date: Mon, 27 Dec 2021 16:27:07 -0500 In-Reply-To: <eb3380eecc734e240f65ddf974d81ee7dac6ca20.camel(a)libertyfp.org> References: <eb3380eecc734e240f65ddf974d81ee7dac6ca20.camel(a)libertyfp.org> User-Agent: Evolution 3.40.0-1 Mime-Version: 1.0 Message-Id-Hash: FKEDFEFMXBD6YUDWKL235PQRIMUYYPEQ X-Message-Id-Hash: FKEDFEFMXBD6YUDWKL235PQRIMUYYPEQ X-Mailfrom: fp145(a)libertyfp.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember- moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.5 Precedence: list Subject: [Testlist] Re: test from fp145 List-Id: Test list <testlist.libertyfp.org> Archived-At: <https://libertyfp.org/archives/list/testlist@libertyfp.org/message/FKEDFEFM… > List-Archive: <https://libertyfp.org/archives/list/testlist@libertyfp.org/> List-Help: <mailto:testlist-request@libertyfp.org?subject=help> List-Owner: <mailto:testlist-owner@libertyfp.org> List-Post: <mailto:testlist@libertyfp.org> List-Subscribe: <mailto:testlist-join@libertyfp.org> List-Unsubscribe: <mailto:testlist-leave@libertyfp.org> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Evolution-Source: 14004b3d9af2d67c898fd6d6c16b487f796088f5 I'm beginning to hate postfix. billo

3 years, 4 months

[MM3-users] Re: Confusing mailman user model

by Gilles Filippini

Gilles Filippini a écrit le 19/07/2020 à 18:00 : > Mark Sapiro a écrit le 19/07/2020 à 17:30 : >> On 7/19/20 8:10 AM, Gilles Filippini wrote: >>> Hi, >>> >>> I'm trying to understand the Mailman user model, and I'm confused about >>> the respective roles of tables 'auth_user' and 'user'. >> >> >> auth_user is a Django auth table. These are the users listed in the >> Django admin web UI. >> >> user is Mailman core's users. >> >> >>> Both have an 'id' column referred to as 'user_id' when used as a foreign >>> key. But there seems to be no relation at all between them as they use >>> different ids for the same user. >> >> >> Right, they are unrelated. >> >> >>> Our mailman server was migrated from mailman2 to mailman3 about 1.5 year >>> ago, and there is an inconsistency I'd like to fix: >>> 2 of our users have a registered Mailman account with their respective >>> email addresses in table 'auth_user', but these very same addresses in >>> the 'address' table are linked to another old admin account in the >>> 'user' table. >> >> Their Mailman account is the 'old' one. the auth_user account is a >> Django account >>> >>> Then if add one of these addresses as a domain owner, it it the old >>> admin address which is selected instead. How could I fix that? >> >> >> That's because domain owner in a Mailman thing and involvs the Mailman >> user. You could delete the Django users via the Django admin UI and then >> re-register them via Postorius. That *might* work. > > When login into postorius, it is the Django account that is used, right? > Then how is the mailman account retrieved from there? It seems UUIDs are > used in the process, but I fail understanding how, so far. Wouldn't it > be safe to retrieve the new Mailman account associated to these Django > users, and link there emails from 'address' table' to them? Reading the mailman-web source code I understand now that the mailman account is retrieved from the Django account email address. Then both users are tied to the same old mailman admin account, and deleting then registering them again won't change anything on this aspect. Would this work? 1- For both addresses, update their record in table 'address' to set 'user_id' to null 2- On their next login to Mailman-web, a new mailman account would be created and associated with their own email address. Is my understanding correct? _g.

4 years, 9 months

[MM3-users] Re: Member Issue Discovered

by Brian Carpenter

On 10/19/20 11:32 PM, Mark Sapiro wrote: > On 10/19/20 7:42 PM, Brian Carpenter wrote: >> On 10/19/20 10:23 PM, Mark Sapiro wrote: >>> And the import21 created the user and address records for this user. >> Does the same thing for a new subscriber as well. So there is no pathway >> to change a real name that is associated with an email address. None. >> Zilch. Mailman 2 made this so easy and Mailman 3 made it impossible. I >> will let someone else file the bug report. You don't really think that >> this is an issue which means it will be years before it is addressed. So >> I will save me some time. I will learn to live with it. > > I didn't say I didn't think it was a real issue. I mostly questioned > whether a new subscription should change an existing display_name. I > agree that there should be a way for a user to change the display_name > associated with her address. I'm not so sure about a list admin. Respectively, I think you are asking the wrong question here. The real question is why isn't a display_name being removed when a list subscriber is unsubscribed. Also list members being forced into a powerless role by Mailman 3's architecture have no way of changing a name except through a list owner unless they register an account but then again, *even that doesn't allow them to change their display name*. There are also other use cases in which a list owner will want to identify portions of their membership roster through the use of the display_name. Most list members interact with a Mailing list program via email. I know you understand that. However it is List owners that are the "real" power users of Mailman and so the admin interface should be designed in such a way that empowers List owners. Taking the ability away from List owners (and List members to make), what should be a simple name change, weakens them. > And in any case, I'm only one person. I'm not the only one deciding > what's important and what's not. I only decide what I want to work on, > not what anyone else thinks is important or wants to work on, so even if > I think something is not worth doing, that doesn't mean it won't get > done, and once again for emphasis, I do thing a user should be able to > change the display name associated with her address(es). Again I think we are missing something here from this conversation: data is not being removed. A list subscriber being removed (unsubscribed) should have their information totally removed. It is not. I now have one list owner who realizes this and it is not good. How does this not violate GDPR? This is why I think all the mailman developers ought to make this issue a high priority. It is creepy to see a name returned out of nowhere when someone resubscribes to a list without associating a name with a second resubscribing. I think this puts List owners at a disadvantage. > > Mailman 3 is totally different from Mailman 2.1 in this respect. Mailman > 2.1 had no concept of user. All it knew was addresses subscribed to > lists and an address subscribed to one list had no connection to the > same address subscribed to another list or being an owner or moderator > of a list. I think this should still be the case for non-registered list members. List owners/moderators are different. They HAVE to be registered users of Postorius/Affinity in order to manage/moderate a list. I wonder what the majority of list members' scenarios are. Is the majority scenario in play today the one where most list members are only subscribed to a single list on a single mailman instance? Or is it the scenario that is in majority use the one where a single list member is subscribed to multiple lists? I think that it is important to find out because it should govern the development process to a certain degree. This single approach of having registered user account with multiple associated email accounts elevates one scenario at the expense of the other. It also assumes/requires that list members have a registered user account and I can tell you from my experience that is not going to happen. The majority of mailman 2 users will not registered with a Mailman 3 admin interface. Some will of course, the majority will not. Maybe the behavior should and will change but it will take years for that to happen. So what I am challenging and questioning here is the approach where such assumptions are being made already that results in the weakening of list owners. > > Mailman 3 does have a concept of user and addresses belonging to a user. > This complicates things in some ways. In Mailman 2.1 we could have "The > Boss <user(a)example.com" as a member of one list and "Just a Peon > <user(a)example.com>" as a member of another list. In Mailman 3 that is > not possible unless the addresses are tweaked in some way to make them > different. Perhaps there is a better approach here to handle single member w/multiple subscriptions that doesn't hurt the ability of List owners to serve the needs of list members who do not have a registered user account and prevents the removal of data when a SINGLE user unsubscribes from a list. > > You don't seem to be concerned about the case where I subscribe to a > second list with a different display name and am surprised to find my > display name changed on the first list, but it's something that I have > to consider. I am not concerned because I think such cases are in the extreme minority. Are you saying that people are now going to be using different real names associated with their other subscriptions as well and that is going to be so needed it warrants this complex system that you have come up with? If someone whats to have a different name associated with a different subscription/email address then I am going to tell them create a new user account and use a good password management program to keep track of their logins. Right now I think this minority use case scenario is negatively impacting other administrative tasks of Mailman, particularly in the removal of user data when someone unsubscribes from a list. Its the use of a square peg being forced into a round hole. Except in this case most of the holes are round but by golly, we are going to use that square peg regardless! > > As far as filing or not filing an issue, issues in the GitLab trackers > are how we track these things. Threads on mailing lists are appropriate > for discussion of issues, but if something is going to get changed or > fixed, an issue in the tracker is the way to ensure it doesn't get put > aside and forgotten. If this is as important to you as it seems to be, > please file the issue. I will but what is disturbing is that you don't think that the non-removal of user data in cases of unsubscribing is not important or even an issue. -- Brian Carpenter Harmonylists.com Emwd.com

4 years, 6 months

[MM3-users] Re: Turn off social logins?

by Torge Riedel

Am 16.02.19 um 13:50 schrieb Stephen J. Turnbull: > Mark Sapiro writes: > > On 2/14/19 11:10 PM, Stephen J. Turnbull wrote: > > > Torge Riedel writes: > > > > > > > - there is a way to do this now via settings_local.py, but feels > > > > a little bit risky if default configuration changes on update of > > > > mailman > > > > > > This should not be true. If it is, we need to fix that. Will check. > > > > > > It is more or less true [...] the only way settings_local.py can > > change something like INSTALLED_APPS is to redefine INSTALLED_APPS > > in it's entirety which leads to the possibility of INSTALLED_APPS > > changing in settings.py and those changes not being picked up in > > settings_local.py. > > That's a different issue from the one I believe Torge is worried > about, which is an update overwriting settings_local.py. But it's not > good. Well, Mark got it what I was meaning. Maybe my explanation was not precise enough. Hopefully an update is not overwriting settings_local.py in any case. Is it part of delivery and then changed afterwards? Haven't checked that yet. I personally like the way other services are importing custom settings by including all config files from a given directory. Maybe it is not worth for mailman cause custom settings will not grow very big - so no need to split it in tiny files. > > > The way around this is to remove the import of settings_local from > > settings.py and then put > > > > from settings import * > > > > at the beginning of settings_local.py and then point Django at > > settings_local rather than settings. > > I don't understand the ramifications of making this change (presumably > Barry or somebody had a reason for the change from Mailman 2's > method), but at first glance it looks like a big step in the right > direction. > > > Then you can do things like > > > > try: > > INSTALLED_APPS.remove('allauth.socialaccount.providers.google') > > except ValueError: > > pass > > > > in settings_local.py except there are still issues because, e.g., > > INSTALLED_APPS is defined as a tuple (immutable) and not a list in > > settings.py. > > Given the concerns of privacy advocates, I would say that it would be > better to have a separate > > AUTH_PROVIDERS = ['allauth.socialaccount.providers.google', ...] > > in settings_local.py, and merge that in to INSTALLED_APPS. Of course > removing specific providers would be OK too, but I think that this is > a place where "deny all / accept what you want" is appropriate. > > Steve >

6 years, 3 months

[MM3-users] Re: Mailman web interface / uwsgi

by Jan Eden

On 2022-11-26 12:02, Mark Sapiro wrote: > On 11/25/22 23:00, Jan Eden via Mailman-users wrote: > > > > proxy_pass http://127.0.0.1:8000; â†’ proxy_pass http://unix:/opt/mailman/mm/var/mailman.sock; > > > > and received an error message (as expected): > > > Because that won't work. You can't http to a unix socket in that way. > > Use > ``` > uwsgi_pass unix:/opt/mailman/mm/var/mailman.sock; > ``` > > > There are obviously a couple of places where the http-socket on port > > 8000 is referenced (e.g. mailman_web/settings/mailman.py, > > /opt/mailman/mm/hyperkitty.cfg). Is there an overview of the > > required changes when using a different (file) socket? > > > If you configure uwsgi with only a unix `socket` and no `http-socket`, > nothing will be listening on port 8000. Thus, you need to configure those > references to public facing URLs and let ngnix determine where to go. > > e.g., for mailman-hyperkitty.cfg > ``` > [general] > base_url: http://example.com/archives/ > ``` > and in settings.py > ``` > POSTORIUS_TEMPLATE_BASE_URL = 'https://example.com' > ``` > to override the `localhost:8000` reference in > mailman_web/settings/mailman.py. Thank you! This all makes sense, but uwsgi logged the following error after applying the changes above: > Traceback (most recent call last): > File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/wsgi.py", line 130, in __call__ > request = self.request_class(environ) > File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/wsgi.py", line 78, in __init__ > self.method = environ["REQUEST_METHOD"].upper() > KeyError: 'REQUEST_METHOD' > Traceback (most recent call last): > File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/wsgi.py", line 130, in __call__ > request = self.request_class(environ) > File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/wsgi.py", line 78, in __init__ > self.method = environ["REQUEST_METHOD"].upper() > KeyError: 'REQUEST_METHOD' So I checked my local Django configuration and found an include directive in mysite_nginx.conf (include /etc/nginx/uwsgi_params;) where the referenced file uwsgi_params contained the following: > uwsgi_param QUERY_STRING $query_string; > uwsgi_param REQUEST_METHOD $request_method; > uwsgi_param CONTENT_TYPE $content_type; > uwsgi_param CONTENT_LENGTH $content_length; > > uwsgi_param REQUEST_URI $request_uri; > uwsgi_param PATH_INFO $document_uri; > uwsgi_param DOCUMENT_ROOT $document_root; > uwsgi_param SERVER_PROTOCOL $server_protocol; > uwsgi_param REQUEST_SCHEME $scheme; > uwsgi_param HTTPS $https if_not_empty; > > uwsgi_param REMOTE_ADDR $remote_addr; > uwsgi_param REMOTE_PORT $remote_port; > uwsgi_param SERVER_PORT $server_port; > uwsgi_param SERVER_NAME $server_name; Replicating this setup on the server solved the KeyError problem, everything works now! I just wonder why these uwsgi parameters need to be passed explicitly when using a file socket, but not with a http socket. Another (unrelated) question: Why is mailman_hyperkitty.cfg placed in /opt/mailman/mm and not in /etc/mailman3 with the other configuration files when using the virtualenv installation instructions (https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-inst… Thanks again for all your help (and patience)! - Jan

2 years, 5 months

[MM3-users] Re: Digests not working correctly

by Joel Lord

The May 4th digest that went out was _also_ size-triggered, so this may have nothing to do with periodic digests at all, and possibly my periodic digests aren't working. I'm not on any of my own lists in digest mode, I'm slowly extracting diagnostic information out of people who are. Also, since this is a ~2 month cycle, it's really difficult to get data points to work with. I'll need to remember to go in and look when this settles down again (new cycle of activity started last night) to see if there's anything left pending. (venv) root@host2:/home/lists/mailman/venv/bin# pip freeze | grep -i hyper HyperKitty==1.3.7 On 6/4/2023 10:05 PM, Mark Sapiro wrote: > On 6/4/23 18:35, Joel Lord wrote: >> >> The periodic digests do seem to be coming out. I also now have >> confirmation that the one message in this morning's digest that was >> from May 4th was also included in the last digest back on May 4th, so >> it seems that the one message was left behind in the digest queue when >> the periodic digest was sent. > > I don't see how that can happen. The process that sends a digest renames > the var/lists/<list-id>/digest.mmdf mailbox file in which the messages > are accumulated to var/lists/<list-id>/digest.<volume>.<issue>.mmdf, > where <volume> and <issue> are the volume and issue numbers of that > digest, and then queues a message in the `digest` queue to tell the > digest runner to create the digest from the messages in that mbox and > send it. Thus, it leaves no var/lists/<list-id>/digest.mmdf mailbox file > behind and that is created anew when the next post arrives. Further, if > there is a non-empty digest.mmdf file, its messages should be sent no > later than the next 11 PM `cron digests`. > > >> There was one earlier message to the list back on May 4th, before the >> one that got duplicated, but I can't tell if that triggered a >> size-based digest to be sent: the logs aren't clear enough on that >> detail for me to tell > > > OK > > >> Just to inform things: >> >> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep mailman >> django-mailman3==1.3.9 >> mailman==3.3.8 >> mailman-hyperkitty==1.2.1 >> mailman-web==0.0.6 >> mailmanclient==3.3.5 >> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep hyper >> mailman-hyperkitty==1.2.1 > > Actually, it's HyperKitty, not hyperkitty, but I assume HyperKitty is up > to date as are the others. > >> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep post >> postorius==1.3.8 >> >> > -- Joel Lord

1 year, 11 months

[MM3-users] Re: migrate MM3 list from an old server to a new one

by Odhiambo Washington

On Wed, Aug 7, 2024 at 11:52 AM IEM Network Operation Center (IOhannes m zmölnig) <noc(a)iem.at> wrote: > thanks both Mark and Odhiambo (who replied off-list) for your answers. > > On 8/6/24 18:40, Mark Sapiro wrote: > > On 8/6/24 01:43, noc(a)iem.at wrote: > >> > >> but how can i export an entire MM3 list (including 20 years of > >> archives) to some serialization format, and import it again in another > >> instance? > > > > > > With the dump and other utilities from your database manager. Almost > > everything is in the database. There are some things in Mailman's var/ > > directory, but those can simply be moved. > > > > urgh. > while this will (hopefully) work with a fresh installation, i'm somewhat > afraid whether this will also work for: > - migrating a single list from one instance to another (at least, if you > (like me) do not have have an intimate knowledge about the db layout) > Hmm.. I think I did something like this in the recent past. I need to remember how I went about it though, but it could be something like: (Assuming a virtualenv install) I believe you could: 1. Make a full backup of the old MM2 install - I mean the DB and the MM3 files. 2. update the install to the latest version corresponding to the one on the new server. 3. Delete all lists with their archives EXCEPT that one list you want to migrate. 4. Export the DB(s) 6. !! Backup the DBs on the new server (just in case something bad happens!! 7. Import the DB dumps from the old server into the new one. Hopefully, there are no conflicts with the sites - importing lists into an instance that is already populated with other > lists (things like changed db schemas come to my mind) > Here again, you should have both sites running at the same software level. right now my task is to move both some MM3 lists (from an old MM3 > instance) and some MM2 lists onto a single server with a fresh > installation, so i guess i will be fine. > Okay. > my worries are mostly about future migrations. > Sit down and write down your plans. Always come back here for 'sanity checks' on those plans. > is a feature-request warranted, or am i worrying too much? > A feature to automate your migrations? Have you consulted ChatGPT yet? :-) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

9 months, 1 week

[MM3-users] Re: MM2.1 -> MM3.3 migration. To hold or not hold

by mark＠suburbia.org.au

Mark Sapiro wrote: > To diagnose this further, can you create a test list, set Default action > to take when a member posts to the list to Hold, mass subscribe an > address and send a post from that address and see if it's held. It > should be, and if it is, I think things are working as they should be > and a test post to the original list from one of these mass subscribed > members will be held. I've done some testing as follows: - Created a test list - Migrated the existing list in question from its mm21 config.pck to the test list - Changed the display name to avoid confusion :-) - Checked moderator settings for the list: - Default action to take when a member posts to the list: Hold for moderation - Default action to take when a non-member posts to the list: Discard (no notification) - Emergency moderation: No - Accept/Hold/Reject/Discard non-members: all blank - CSV exported the 247 existing list members - Pasted all exported member addresses from the CSV into the mass removal web page, *except* for the list-owner. After doing so hoped that they didn't all get an unsubscribe mail. Given that "Send goodbye message" was set to yes, I suspect I did. :-\ - Confirmed List members only had the list owner - Added a first new test user (me) via mass subscribe, ticked Pre confirm, Pre approved, Pre verified. Received welcome email - Confirmed List members; list owner "Hold for moderation", test user "None" - Added a second new test user (me with a second email address) via mass subscribe with same settings as above. Received welcome email - Modified second test user to "Hold for moderation" in Members list. Noted the Moderation was set to "List default" before changing it to "Hold for moderation" - Sent an email from test user 1 to the list - Sent an email from test user 2 to the list - Sent an email from a third email address that isn't on the list - Looked at Held messages: - test user 1 email was held for moderation in Web UI. User 1 received an email indicating as such. - test user 2 email was held for moderation in Web UI. User 2 received an email indicating as such. - non-member email was neither held for moderation, nor sent to the list. > If that is true, I can only think that at the time of that post, > settings weren't as they are now. I think for now I will have to assume that's the case, until the next email can be readied against the proper list. If you hear nothing further, assume that this was all based on some random quirk (by humans or machines). I'll update if it happens again. Cheers, Mark

5 months, 3 weeks

[MM3-users] Re: SPF check fails for lists subdomain

by Jan Eden

On 2023-01-04 12:40, Mark Sapiro wrote: > On 1/4/23 11:39, Jan Eden via Mailman-users wrote: > > Hi, > > > > my question is not related to Mailman directly, apologies for using this > > list. I configured the DNS records for my base domain and my lists > > subdomain identically (the DMARC policy records are also identical, but > > not listed here): > > > > MX @ mail.eden.one > > TXT @ "v=spf1 mx ~all" > > MX lists mail.eden.one > > TXT lists "v=spf1 mx ~all" > > > > A mail 123.123.123.123 > > > > But both Yahoo and Google report different SPF results for the two domains: > > What could possibly cause this difference? The SPF test also fails for a > > [...] > > different base domain with the same MX and SPF records. > > Your spf for lists.mail.eden.one specifies its MX which is also > lists.mail.eden.one, however mail from that domain arrives from IP > 123.123.123.123 and presumably an rDNS lookup returns mail.eden.one which is > not lists.mail.eden.one, thus the failure. > > Add the IP 123.123.123.123 to the spf and drop the MX since it doesn't work > > TXT lists "v=spf1 123.123.123.123 ~all" This would explain a lot, but it also invalidates everything I thought to have understood about DNS records. Maybe the abbreviated records above (quoted from my provider's web interface) were misleading, so here's the output of dig: eden.one. 60 IN MX 10 mail.eden.one. eden.one. 60 IN TXT "v=spf1 mx ~all" lists.eden.one. 60 IN MX 10 mail.eden.one. lists.eden.one. 60 IN TXT "v=spf1 mx ~all" mail.eden.one. 60 IN A 123.123.123.123 So an MX lookup for both eden.one and lists.eden.one returns the hostname mail.eden.one, which points to the address 123.123.123.123. The SPF records for eden.one and lists.eden.one refer to the respective MX records (with the same target hostname). According to RFC 7208[1], the mx mechanism "matches if <ip> is one of the MX hosts for a domain. [...] check_host() first performs an MX lookup on the <target-name>. Then it performs an address lookup on each MX name returned. The <ip> is compared to each returned IP address. [...] If any address matches, the mechanism matches." So in both cases, the MX mechanism should first retrieve mail.eden.one, and then 123.123.123.123 via DNS queries, and should match accordingly when the message was sent from mail.eden.one/123.123.123.123. Although I could specify the IP address in my SPF records directly (as you suggested), I do hope that my understanding of DNS records laid out above is not entirely misguided. My current setup does work as expected for eden.one, after all. - Jan [1] https://www.rfc-editor.org/rfc/rfc7208#section-5.4

2 years, 4 months

[MM3-users] Timeouts

by tlhackque

On 25-Apr-21 18:08, Mark Sapiro wrote: > On 4/25/21 2:20 PM, Andrew Hodgson wrote: >> Hi, >> >> Hyperkitty 1.3.4. >> >> I am trying to download a complete list mbox by going to all threads view >> and using the download option. I have tried a couple of tools (Gunzip and >> Winrar) and both are giving me an unexpected end of file when trying to decompress the gz file. >> >> Here is the list URL I am using: https://lists.hodgsonfamily.org/hyperkitty/list/plextalk@lists.hodgsonfamil… > Depending in the web server configuration, timeouts can occur when > downloading large archive mboxes. instead of downloading the entire mbox > with > <https://lists.hodgsonfamily.org/hyperkitty/list/plextalk@lists.hodgsonfamil…>, > do it in pieces by adjusting start and end. e.g. > > <https://lists.hodgsonfamily.org/hyperkitty/list/plextalk@lists.hodgsonfamil…> > > <https://lists.hodgsonfamily.org/hyperkitty/list/plextalk@lists.hodgsonfamil…> > > <https://lists.hodgsonfamily.org/hyperkitty/list/plextalk@lists.hodgsonfamil…> > > > Although, I don't think timing out is the issue, and I'm not sure what > is, but I think it has something to do with messages in the archive. If > I try to get the 3 pieces as above, the first piece with > start=2008-02-19&end=2012-12-31 works but the others don't and even the > smaller > > <https://lists.hodgsonfamily.org/hyperkitty/list/plextalk@lists.hodgsonfamil…> > > fails the same way. > > If I try to get 2013 month by month, all work except December which > gives me an internal server error. What's the traceback from that error. > The described timeouts are something that hyperkitty ought to be able to avoid. For apache, the timeout is idle time between blocks of output. Hyperkitty can avoid this by generating the archive in segments (based on size, or elapsed time), flushing its output buffer, generating a multi-file archive, and/or using Transfer-Encoding: chunked (chunked doesn't work for http/2). It ought to be able to break the work into blocks of "n" messages & do something to generate output. Besides avoiding timeouts, working in segments allows the GUI to display meaningful progress (e.g. if you're loading with XMLHttpRequest, "onprogress") It really oughtn't be up to the user to break up the request. Until then: the apache directive is "TimeOut" (or "ProxyTimeout"), with a default value of 60 (seconds). It's a server config/virtual host parameter, so if you're running in an environment where you only have .htaccess, you need admin help or you're out of luck. Other webservers (especially those with accelerators) may have more granular timeouts.

4 years

Search results for query "sapiro"