Search results for "Postorius" - Mailman-users

Re: mailman-web migrate error

by Mark Sapiro

On 11/12/25 11:48, pashdown(a)xmission.com wrote: > I'm migrating from Ubuntu 20.04, mailman 3.2.2, postgresql 12 to Debian 13, mailman 3.3.10, postgresql 17. > > After migrating the postgresql databases and assuring the perms and passwords work, this is what I get with a "mailman-web migrate": > > (user(a)domain.com) redacted from the original > > # mailman-web migrate > Operations to perform: > Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, \ > sessions, sites, socialaccount > Running migrations: > Applying account.0003_alter_emailaddress_create_unique_verified_email... OK > Applying account.0004_alter_emailaddress_drop_unique_email... OK > Applying account.0005_emailaddress_idx_upper_email... OK > Applying account.0006_emailaddress_lower...Traceback (most recent call last): > File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 89, in _execute > return self.cursor.execute(sql, params) > ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ > psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "account_emailaddress_user_id_\ > email_987c8728_uniq" > DETAIL: Key (user_id, email)=(24, user(a)domain.com) already exists. This is actually a django-allauth issue. This issue is due to changes in Django allauth to prevent creating multiple addresses differing only in case. The problem is if you have such multiple addresses in your account_emailaddress table, this migration fails. There is a script at https://www.msapiro.net/scripts/UC_fix that will fix this. You may need to adjust the shebang and replace psycopg2 with sqlite3. See https://github.com/pennersr/django-allauth/issues/3019 perhaps starting at This issue is due to changes in Django allauth to prevent creating multiple addresses differing only in case. The problem is if you have such multiple addresses in your account_emailaddress table, this migration fails. There is a script at https://www.msapiro.net/scripts/UC_fix that will fix this. You may need to adjust the shebang. See https://github.com/pennersr/django-allauth/issues/3019 perhaps starting at https://github.com/pennersr/django-allauth/issues/3019#issuecomment-1748028… -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

11 months, 1 week

Re: Can disabled users enable themself?

by Stephen J. Turnbull

Flo via Mailman-users writes: > How can I find these settings in Postorius? There is only a user's > language and a list's language. For the user: I visit https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ At the bottom of the page it tells me: Primary Address (steve(a)turnbull.jp) ... Manage Subscription ... Click on the "Manage Subscription" button, I get a page with links Subscriptions Global Mailman preferences Address-based preferences List-based preferences across the top under the "Mailman Settings stephenjturnbull" title. For "Global Mailman preferences" there is a page with all of the settings documented, and "Preferred language" is the last setting. For "Address-based preferences" and "List-based preferences", there is a table, with "Preferred language" the rightmost column. Do you not see that? For the list owner: I go to https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/member… to see the members, click on a member (I accessed your subscription since I logged in as myself, hope you don't mind) and see a full page with your subscription options. There is no way for you as *list* owner to see the users' Global or Address settings. As *site* owner (superuser) you probably can, but can't test, I don't have that privilege on a running Mailman 3 at the moment. I don't know enough about the relevant views to be sure you can access that, but usually "superuser" allows you to Manage All The Things. Hope that helps. Due to GSoC evaluations, I don't have time to get up to speed to help with the other issues you're seeing. Mark probably will figure it out much more quickly, anyway. :-) Steve -- GNU Mailman consultant (installation, migration, customization) Sirius Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan

3 months, 2 weeks

Getting original message headers

by Stephen J. Turnbull

Peter Chubb via Mailman-users writes: > To do that I need to find the original message headers. By the time > the message gets to the archive, or to the mailing list recipients, > all the Received: headers seem to have been deleted, leaving only > the ones from my list server outwards. That's not something Mailman does by default (maybe it does for anonymous lists?) For example, the Received chain in your post as I received it goes back through several subdomains of "chubb" before getting to mailman3.org. > Is there a way to access the original incoming email, before DMARC > mitigation etc., has changed the headers? What are you looking for in the received chain? If you just need the malicious source addresses and message-ids, those should be in the MTA log already. The only ways I can think of in a stock Mailman to preserve the original message are (1) emergency moderation, which should catch the mail before anything is done to it, and (2) reverting the anonymous configuration to default. In case (1) you can see the headers of held messages in Postorius, or with the "mailman qfile" subcommand. The held message remains in the "in" queue until its disposition is decided, I think. If you don't find it there, look around the queues (not bad or shunt, those have a different purpose) for messages that have been there for a while. In case (2), end recipients should get the whole received chain with the original from heder. The less disruptive way to do it involves configuring the MTA to both log the whole message to a file and send it to Mailman. To say more we'd need to know about your MTA. -- GNU Mailman consultant (installation, migration, customization) Sirius Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan

9 months, 2 weeks

Re: Add old email address

by Quanah Gibson-Mount

--On Friday, March 13, 2020 4:18 PM +0900 "Stephen J. Turnbull" <turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote: > You can certainly add extra addresses to a Mailman 3 user (I don't > know offhand whether/how an admin can do it through Postorius, but the > core database certainly has that facility and the REST API allows it). I was able to resolve this via the Django admin interface. > However, this doesn't have the same semantics as in Mailman 2 because > Mailman 2 doesn't have users at all. So as an admin how do you know > which addresses correspond to particular users? More generally, what > do you mean by "associate with an account", and why do you want to do > it? > > Normally I would not ask the latter question, it's your business why > you want to do it. But here I'm trying to get at what semantics you > want. If I were going to do this for the kind of purposes I can > imagine, it would be a matter of informing the users that this feature > is available, and I'd ask the users to do it if they care. It seems > you have some project goal in mind though. Because I've been a member of the project for close to 20 years, and the email addresses I've had over that time have changed in relation to the jobs I've had. Same deal for other long time members of the project. By adding addresses I no longer have to my account and marking them verified, I can easily associate nearly 20 years of postings that I've made with myself when I look at the archives. --Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

6 years, 7 months

Re: using SSH/TLS with external MTA

by Mark Sapiro

On 7/28/24 06:26, Roland Giesler via Mailman-users wrote: > On 2024/07/28 00:39, Mark Sapiro wrote: >> >> What do you see in Mailman's smtp.log and in the mail.log on the mail >> server? > > The smtp log is in syslog... The smtp.log I refer to is Mailman's smtp.log which is in Mailman's var/logs/ directory. > I think I see what's going on here. The mail is being sent from > postorius(a)giesler.za.net and there is no log entry on > box2.gtahardware.co.za that shows this logon attempt. So it seems it's > not going to that server, but rather to the local postfix. > > A mail was sent to root on the localhost with the following content: > > Return-Path: <root(a)giesler.za.net> > X-Original-To: root@localhost > Delivered-To: root@localhost > Received: from mailman.fast.za.net (localhost [IPv6:::1]) > by mailman.localdomain (Postfix) with ESMTP id 2174A120A5B > for <root@localhost>; Sun, 28 Jul 2024 13:11:03 +0000 (UTC) > Content-Type: text/plain; charset="utf-8" > MIME-Version: 1.0 > Content-Transfer-Encoding: quoted-printable > Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: > /accounts/login/ > From: root(a)giesler.za.net > To: root@localhost > Date: Sun, 28 Jul 2024 13:11:03 -0000 > Message-ID: <172217226313.2150.10895652368679013629(a)mailman.fast.za.net> > > Internal Server Error: /accounts/login/ > > SMTPRecipientsRefused at /accounts/login/ > {'roland(a)giesler.za.net': (454, b'4.7.1 <roland(a)giesler.za.net>: Relay > acce= > ss denied')} This is mail from django-allauth, not from Mailman. Your settings in mailman.cfg do not affect how this mail is sent. See https://docs.djangoproject.com/en/5.0/topics/email/#smtp-backend for the Django settings for this. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 2 months

Re: Problem updating mailman

by Brian Carpenter

Mark Sapiro wrote: > On 1/24/21 1:03 AM, Odhiambo Washington wrote: > > I uninstalled rubygem-sass from the OS. I don't > > remember consciously > > installing that because it was never in the guide I followed (which is your > > own writeup). > > But I still have a problem: > > (venv) [mailman3@gw ~/mm]$ /opt/mailman/mm/bin/mailman-post-update > > > > '[' False == False ']' > > mkdir -p /opt/mailman/mm/static > > /opt/mailman/mm/bin/django-admin collectstatic --clear --noinput > > --verbosity 0 > > /opt/mailman/mm/bin/django-admin compress > > CommandError: An error occurred during rendering > > /opt/mailman/mm/venv/lib/python3.7/site-packages/django_mailman3-1.3.5-py3.7.egg/django_mailman3/templates/socialaccount/signup.html: > > /bin/sh: sass: not found > > > > You can't just uninstall rubygem-sass. You have to replace it. Here's > one recipe for Ubuntu > cd /usr/local/lib > # See https://github.com/sass/dart-sass/releases/tag/1.32.5 for the > # appropriate version. > wget > https://github.com/sass/dart-sass/releases/download/1.32.5/dart-sass-1.32.5-... > sudo tar -xf dart-sass-1.32.5-linux-x64.tar.gz > sudo chmod -R o-w dart-sass > sudo ln -s dart-sass/sass /usr/local/bin/sass > rm dart-sass-1.32.5-linux-x64.tar.gz > Then you should see the command > sass --version > prints > 1.32.5 > Then, because dart-sass doesn't recognize the short form -t option for > --style, you have to edit your settings(_local).py and change > COMPRESS_PRECOMPILERS = ( > ('text/x-scss', 'sass -t compressed {infile} {outfile}'), > ('text/x-sass', 'sass -t compressed {infile} {outfile}'), > ) > to > COMPRESS_PRECOMPILERS = ( > ('text/x-scss', 'sass --style compressed {infile} {outfile}'), > ('text/x-sass', 'sass --style compressed {infile} {outfile}'), > ) I had to do a new installation of Mailman 3 with Postorius/Hyperkitty today and these directions were very helpful. Thank you, Mark.

5 years, 8 months

Re: Template for Password Reset Email

by Mark

On 2024-11-18 01:38, Mark Sapiro wrote: > On 11/17/24 16:57, Mark wrote: >> >> I can see the French text for the "Password Reset Email" in the >> django.po file. And running the compilemessages command reports the >> "django.po" for French is okay and up to date. Should I be recompiling >> the ".mo" file? > > No, It's fine. The issue is the language context for this message has > nothing to do with any Mailman list's preferred language. > > Again, see > https://docs.djangoproject.com/en/5.1/topics/i18n/translation/#how-django-d… > > If all the lists on your server are French, you could set > > LANGUAGE_CODE = 'fr' > > in your Django settings. Otherwise, the user's browser has to include > an Accept-Language: header specifying French as the most preferred > language in it's request. > > Also note the ordering of SessionMiddleware, LocaleMiddleware and > CommonMiddleware in the above doc. I have recently updated the various > places where we suggest or supply MIDDLEWARE settings, but you may > find that your settings have LocaleMiddleware after CommonMiddleware > and you should fix that. It's just the one list that is using French. Others are in English I've edited the order of the middleware settings in this file ... /opt/mailman/venv/lib/python3.11/site-packages/mailman_web/settings/base.py I reordered as below, restarted everything but no joy :-( ... at Line 76 ... MIDDLEWARE = ( 'allauth.account.middleware.AccountMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.common.CommonMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware', ) ... I'm thinking this is not the file I'm looking for. Apologies for my cluelessness here.

1 year, 11 months

Mailman 3 deployment: ok to use same database instance for core, web, and hyperkitty tables?

by Stephen J. Turnbull

Hi Serban, s.dobrau(a)ucl.ac.uk writes: > We are currently working on a migration plan for Mailman -> Mailman > 3. We have around over 3000 mailing lists with a total of 50K > members Medium-size. I guess a 2-CPU/4GB Linode or similar should handle the Mailman load (including Postorius and HyperKitty), you might want more memory and maybe more CPUs if you're running the RDBMS on the same host. In the 23k lists/50k users/100k inbound messages/day migration described below, we ended up reconfiguring the VMs 3 times before declaring the system stable. IMO the 2 x 8CPU/16GB final setup was way overpowered for the need, but we didn't try to calibrate finer than that. > and 500 worth of archived data. 500 whats? 500MB, 500GB, 500TB? Assuming 500GB, > Our plan is to have core+web (instance 1) and Hyperkitty (instance > 2). With the archives on the FS local to the Hyperkitty. The archives are in the database as BLOBs. There's no reason to have a separate instance for HyperKitty. It's not obvious to me that you need more than that 2x4 Linode for both Mailman and RDBMS, but if you're going to split it's Mailman vs RDMBS, not Mailman vs HyperKitty. > And all 3 tables (core, web, Hyperkitty) pointing to the same > database on a remote SQL (mysql-enterprise) instance. Using class > mailman.database.mysql.MySQLDatabase (and whatever necessary on the > Django/Hyperkitty instance). Make sure your MySQL database(s) for Mailman are configured with the utf8mb4 option for 4 byte UTF-8 support, or it will choke on emojis and the like. I don't think there are any other gotchas for MySQL. Note there are a lot more than 3 tables in Mailman's databases. The "traditional" configuration is a "mailman" database for core and a "mailmanweb" database for Django and the archives, but Mark says that as the tables are disjoint across Mailman, Django, and the archives there's no reason not to use a single "mailman" database for all of them. I believe that's how this list's host is configured. > Is this feasible, Yes. > if so what would be the recommended compute/storage > requirements. I would start with the spec above (2CPU/4GB). If that doesn't perform, bump it to taste. Mailman itself uses about a GB of RAM for all its runner processes. (I'm not a cloud expert, I just assume you can do that the way we did, very simply.) The Python code (including compiled .pycs) is about 300MB. The database storage requirements are nominal for Mailman and Django (maybe 100MB total in the database for a 23k lists, 50k users installation I worked on recently), and I think the database usage for the archives was approximately 1:1 vs. the Mailman2 mboxes. Mark may have a better estimate on that. There is also the full-text search index for the archives. I recommend Xapian, in my experience Whoosh is really slow.[1] I don't recall offhand how big those were but I would assume they're no smaller than 1/10 the size of the mboxes, and maybe quite a bit bigger. Importing list configurations, users, and subscriptions into Mailman 3 is pretty fast, as long as you're not using the stock Postfix support.[2] The problem is that generating the Postfix alias files for the lists seems to be noticably linear in number of lists, which means it's quadratic for the mass import. IIRC with 5000 lists you'd be looking at >1m/list just to keep regenerating Postfix's alias database, ie 5000 minutes. I found two solutions, both of which required patching Mailman. I hope to get both into the next release. I think we got it down to <5s/list, for 5000 lists that's < 1hr. Importing archives is ... yeeech. The client decided they didn't want a ton of archives anyway (you know corporations, if it's not required by law, shred it after 6 months). I don't recall the size estimate accurately but we kept 6 months X 4500 lists, maybe 100GB out of 2.3TB of mboxes. That took 24 hours to import into HyperKitty, without doing the full-text indexing. I do know the original conservative estimate was 20 days to import the whole 2,3 TB. The full-text indexing took more than a week if I remember correctly. The easiest thing to do is to just do no migration, keep the old archive online, and start populating the new archive when you switch over. It's possible to do both the HyperKitty import and full-text indexing incrementally with a bit of planning. It's also quite possible to migrate incrementally, a few lists at a time. Mailman 2 and Mailman 3 can coexist happily on the same host. Mark can advise on that, I think. The only other advice I remember offhand is that Mailman's outgoing smart host should be in the same datacenter as Mailman. We never did debug it, but the system described above with 23k lists had problems with the out queue stalling when smtp.client.com resolved to a different datacenter 500km away. :-) The problem went away as soon as they pinned the MX to the MTAs in the same datacenter. Footnotes: [1] If you're familiar with another full-text search engine, check the django-haystack docs. Haystack supports several other engines, your preference may be among them. [2] Exim4 does not have this problem. I don't know about other supported MTAs.

2 years, 11 months

Can creating too many Django accounts crash Mailman3 server?

by Stephen J. Turnbull

Mohsen Masoudfar writes: > It seems anybody can go to the site: > https://LISTSERVNAME/mailman3/postorius/lists/ > and click on the [Sign up] on the top-right corner and create an > account. Yes. > This can easily be automated, even though the next step, confirming > the email address, being ignored. I believe it can be used as a > target for a DOS attack by creating so many accounts, that > eventually causes an 'out of space' error. > > Is this a justified concern? "Justified" depends on who you think the attacker is. Theoretically it's a real possibility, but the space used by an unverified account is not large. I suspect there are always much more painful DoS attacks to worry about. > Is there a way to manage this feature in a secure way? In some sense, I don't think there is. The point of the feature is to allow anybody to sign up without authentication beyond having a usable email address. Either you allow the DoS attack or anything you do is likely to deny service to legitimate users. Brian's approach (cron job purging unverified users) is generally good hygiene, but I don't think it would do much against an deliberate attack. And it shouldn't be hard to automate the confirmation process. I guess you could handle that by purging accounts with no subscriptions or other roles on any lists, but that might annoy a few legitimate users. Probably the best you could do for an open server would be to throttle user creation to say 1 per minute (adjustable to the scale of your site). I don't know if there is currently a way to require admin approval for account creation (as opposed to list subscription). Perhaps we should add that if we don't have it. Steve

6 years, 1 month

Ansible role to install Mailman3

by samuel.d.darwin＠gmail.com

Hi, Before installing the Mailman Suite, I searched on https://galaxy.ansible.com/ for Ansible roles that might automate the whole process. https://github.com/galaxyproject/ansible-mailman3 turned out to be quite good. However, it was slightly out-of-date. I cloned it, made a few bug fixes, and corrected linting errors. Then, going further, added optional basic installations of Nginx, Postfix, PostgreSQL into the role. If anyone is installing Mailman they are welcome to try it out. https://github.com/cppalliance/ansible-mailman3 Now, I see there was a discussion on this very list about https://github.com/rivimey/ansible-mailman3 which was also a fork of natefoo's repo. What are the differences? As Stephen Turnbull wrote: "her (rivimey's) setup may be more complicated than most folks need (IIRC, she has a multi-homed setup with Mailman core, Postorius, and HyperKitty on three separate IPs on a separate subnet in the DMZ, while Mailman was really designed for three Mailman applications running on the same host, along with the database server and MTA)." My version of the role didn't follow the tactic of "three separate IPs on a separate subnet in the DMZ". Mainly, it adds postfix and postgres. Fixes small errors. Also, in a git branch of the repository named mailman3-core, I removed all the web code, so there is a version of the role that only installs core. We will use that while bundling the web components into another Django app, already deployed with docker. There is an open question about maintenance. Among the three roles mentioned here, which is being actively kept up-to-date? Perhaps there are no guarantees from any of them. At this instant in time, https://github.com/cppalliance/ansible-mailman3 is the most recent. Would it make sense to mention these roles in the official documentation? Not sure. Installing mailman3 is complicated and because of that, is a good fit for Ansible.

3 years, 2 months

Mailman-users search results for query "Postorius"