Search results for "dO" - lists.mailman3.org

[MM3-users] Re: List mails blocked from GMX - web.de > Spamfolder

by Stephen J. Turnbull

As Mark writes, MX has fixed IP A record, reverse DNS round-trips with A lookup, stay off RBLs, DKIM sign your mail are absolutely required. Using at least conditional DMARC mitigation is required, too. However, conditional DMARC mitigation won't help with gmx.net, it has the same DMARC policy as web.de: % host -t TXT _dmarc.gmx.net _dmarc.gmx.net descriptive text "v=DMARC1; p=none; sp=quarantine; rua=mailto:dmarcreport@gmx.net" % host -t TXT _dmarc.web.de _dmarc.web.de descriptive text "v=DMARC1; p=none; sp=quarantine; rua=mailto:dmarcreport@web.de" The important part is "p=none", which says that GMX does not reject or quarantine *solely* because of a DMARC fail.[1] The conditional DMARC mitigation will *not* be applied for that domain for this reason. However, some sites consider From alignment (domain of DKIM signature = domain of author address in From:) in scoring spam. If GMX does that, unconditional DMARC mitigation may help. Two additional configurations that may help: 1. Authenticated Received Chain (ARC) is a relatively new standard which is used by some of the large American providers, specifically GMail and outlook.com. The basic idea is that your host will validate the digital signature on receipt of the post, add a header field reporting the results, and digitally sign that report. Many receiving hosts will then apply the same trust to your validation report that they do to your signatures. Maybe GMX will do it, too. 2. SPF is an older standard than DKIM, which uses the IP address of the source of the incoming connection instead of a digital signature. This may help in some cases, but it's much less important than DKIM signatures nowadays. As a last resort, you may disable all Mailman features (adding list tags or serial numbers to Subject:, adding header or footer to the body) that invalidate signatures. Unfortunately this may not be legal (some jurisdictions may require that you add instructions on how to leave the list). Steve Footnotes: [1] IIRC, the "sp=quarantine" means that some fraction of DMARC failing mail should be quarantined for testing purposes, but the default fraction is 0 so that is ignored.

1 year, 9 months

[MM3-users] Re: Notifying owners of membership changes but not moderators

by Perry Kollmorgen

Thanks Mark. So, I have python scripts which do the adding of nonmembers and moderators. For the nonmember script I needed to add something so that the nonmembers can send, i.e the moderaction_action = 'defer' This is my code for nonmembers: mailing_list.add_role('nonmember', nonmember.email, display_name=nonmember.display_name) list_member = mailing_list.get_nonmember(nonmember.email) list_member.moderation_action = 'defer' list_member.save() For the code for moderators it currently only adds them: mailing_list.add_moderator(moderator.email, display_name=moderator.display_name) Would I need to add something similar which sets a delivery_status for the moderator added ? Cheers, Perry ________________________________ From: Mark Sapiro <mark(a)msapiro.net> Sent: Wednesday, 27 November 2024 2:16 PM To: mailman-users(a)mailman3.org <mailman-users(a)mailman3.org> Subject: [MM3-users] Re: Notifying owners of membership changes but not moderators On 11/26/24 4:54 PM, Perry Kollmorgen wrote: > > Is there any way to have these only for Owners ? No. Essentially all admin notices are sent to LISTNAME-owner@DOMAIN and that in turn resends to owners and moderators. There are no controls to restrict this to only owners and not moderators, however it doesn't send to those with delivery disabled, so you could do something like this in `mailman shell` ``` $ mailman shell -l LIST_ID Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. The variable 'm' is the LIST_ID mailing list >>> for moderator in m.moderators.members: ... moderator.preferences.delivery_status = DeliveryStatus.by_moderator ... >>> commit() ``` This will not affect the delivery status of the user's role as list member. This is not available in Postorius and also, the moderator can't set the delivery_status for their moderator role either, only for their member role. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… This message sent to pcjkollmorgen(a)hotmail.com

4 months, 4 weeks

[MM3-users] Re: HTML header / footer template

by Mark Sapiro

On 06/18/2018 02:33 AM, Asaf Ohayon wrote: > I understand Mailman 3 doesn't want to support HTML Emails, > This is perfectly legitimate, sadly however, This is a requirement of my client, > Which is in this case the Ministry of Education of Israel. Mailman does support HTML email. Just adjust content filtering to not remove HTML and not collapse alternatives. What Mailman does not support (and which I think is what you want) is the addition of headers and footers either as text/html parts or inserted into text/html message bodies. The former would be fairly easy. Look at mailman/handlers/decorate.py. If the messages you are decorating are text/html or multipart/alternative or something more complex, the header and footer are going to be separate MIME parts anyway, so where we have > if len(footer) > 0: > mimeftr = MIMEText( > footer.encode(lcset, errors='replace'), 'plain', lcset) > mimeftr['Content-Disposition'] = 'inline' > payload.append(mimeftr) > if len(header) > 0: > mimehdr = MIMEText( > header.encode(lcset, errors='replace'), 'plain', lcset) > mimehdr['Content-Disposition'] = 'inline' > payload.insert(0, mimehdr) in two different places, just change the 'plain' arguments to 'html' and then the headers and footers will be added as text/html and can include html tags to be rendered. On the other hand, if what you want is to insert headers and footers at the beginning and end of an existing HTML body part, We will not make Mailman do that because it is very difficult to do it right. This is because just prepending or appending some additional HTML to an existing HTML part gives unpredictable results. You need to actually parse or render the existing HTML to determine how to add your additions and in complex cases even that may not render as you wish in all various MUA renderings. If you want to try for HTML what decorate does for simple plain text, you can try it and you may find it works for you, but it is way to fragile for a Mailman feature. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years, 10 months

[MM3-users] Re: Digests not working correctly

by Joel Lord

On 6/4/2023 9:25 PM, Mark Sapiro wrote: > On 6/4/23 11:45, Joel Lord wrote: Thanks for the explanation on size-triggered digests. >> Periodic is much more complicated, even to explain. I have the cron >> job running nightly at 11:00 PM and it certainly seems to be working, >> except for one really odd thing. The list also definitely has >> periodic digests enabled in the web UI. This is a very bursty list, >> so it will go mad with traffic for a few weeks and then will go mostly >> idle for a month or so. The first digest of a burst is definitely >> size-triggered, but pretty reliably it has the last few messages from >> the previous burst in there. So the one I got today had three >> messages from today and one message from May 4th. But that May 4th >> message should have gone out at 11:00 PM on May 4th with the daily >> digest and then gotten flushed from the pending digest queue, right? > > That's correct. Do you ever get periodic digests sent at 11:00 PM? If > not, there is some issue either with configuration or something. If you > get them sometimes, I don't know what could be wrong. The periodic digests do seem to be coming out. I also now have confirmation that the one message in this morning's digest that was from May 4th was also included in the last digest back on May 4th, so it seems that the one message was left behind in the digest queue when the periodic digest was sent. There was one earlier message to the list back on May 4th, before the one that got duplicated, but I can't tell if that triggered a size-based digest to be sent: the logs aren't clear enough on that detail for me to tell. Just to inform things: (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep mailman django-mailman3==1.3.9 mailman==3.3.8 mailman-hyperkitty==1.2.1 mailman-web==0.0.6 mailmanclient==3.3.5 (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep hyper mailman-hyperkitty==1.2.1 (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep post postorius==1.3.8 -- Joel Lord

1 year, 10 months

[MM3-users] Mailman3 Issue with Sending Mail to Yahoo/AOL

by Stephen J. Turnbull

chris--- via Mailman-users writes: > I've been getting complaints from the Yahoo people that mail is > delayed at times, or messages that come out of order. That could be due to greylisting. > (host mta6.am0.yahoodns.net[67.195.204.74] said: 452 Too many > recipients tnmpmscs (in reply to RCPT TO command)) You could try enabling personalization in Mailman. Then each user's message is different, so it will send one per transaction. Depending on negotiation between your MTA and the receiving MTA, this may not be too inefficient because the connection will be reused (aka pipelining). The other possibility would be to tune your MTA to send smaller batches. However ... > (host mta6.am0.yahoodns.net[67.195.204.74] said: 421 [IPTS04] > Messages from 74.112.73.65 temporarily deferred due to unexpected > volume or user complaints - 4.16.55.1; see > https://postmaster.verizonmedia.com/error-codes (in reply to end of > DATA command)) Personalization could exacerbate the problem with unexpected volume depending on how they measure volume (by the number of messages or by the number of recipients). Not much you can do about that except hope they'll learn that sometimes there's list traffic and sometimes there's not. Have you looked up this error code (IPTS04)? > So it seems that Yahoo/AOL keeps detecting my mailman3 server as > either spam, sending too much email, or too many recipients. I don't think it's detecting spam, or it would say so (or perhaps just blackhole your mail). > Anyone anyone shed some light on how to appropriately solve this > issue? The only real solution is to boot subscribers with AOL/Yahoo/Verizon addresses. That's likely not something you want to do, unfortunately, and subscribers are unlikely to take suggestions that their email provider sucks, and they should change to a competent one, kindly. Ironically enough, the Japanese government did exactly that in 2014, prohibiting use of Yahoo addresses by government employees or entities -- but yahoo.co.jp is a completely different company from yahoo.com (they licensed the name and the search engine etc), and their email is apparently more competently run, in any case they don't bounce list traffic or cause DMARC problems.

3 years, 11 months

[MM3-users] Re: Turn off social logins?

by Stephen J. Turnbull

Mark Sapiro writes: > On 2/14/19 11:10 PM, Stephen J. Turnbull wrote: > > Torge Riedel writes: > > > > > - there is a way to do this now via settings_local.py, but feels > > > a little bit risky if default configuration changes on update of > > > mailman > > > > This should not be true. If it is, we need to fix that. Will check. > > > It is more or less true [...] the only way settings_local.py can > change something like INSTALLED_APPS is to redefine INSTALLED_APPS > in it's entirety which leads to the possibility of INSTALLED_APPS > changing in settings.py and those changes not being picked up in > settings_local.py. That's a different issue from the one I believe Torge is worried about, which is an update overwriting settings_local.py. But it's not good. > The way around this is to remove the import of settings_local from > settings.py and then put > > from settings import * > > at the beginning of settings_local.py and then point Django at > settings_local rather than settings. I don't understand the ramifications of making this change (presumably Barry or somebody had a reason for the change from Mailman 2's method), but at first glance it looks like a big step in the right direction. > Then you can do things like > > try: > INSTALLED_APPS.remove('allauth.socialaccount.providers.google') > except ValueError: > pass > > in settings_local.py except there are still issues because, e.g., > INSTALLED_APPS is defined as a tuple (immutable) and not a list in > settings.py. Given the concerns of privacy advocates, I would say that it would be better to have a separate AUTH_PROVIDERS = ['allauth.socialaccount.providers.google', ...] in settings_local.py, and merge that in to INSTALLED_APPS. Of course removing specific providers would be OK too, but I think that this is a place where "deny all / accept what you want" is appropriate. Steve -- Associate Professor Division of Policy and Planning Science http://turnbull.sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull(a)sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN

6 years, 2 months

[MM3-users] Re: Two trivial questons:

by Bill Oliver

I take that back. I went back and made another superuser using mailman-web create superuser. But... I still get bounced with the error: Forbidden (403) CSRF verification failed. Request aborted. More information is available with DEBUG=True. my uwsgi.log generates: [pid: 1385|app: 0|req: 14/22] 127.0.0.1 () {52 vars in 1078 bytes} [Wed Oct 9 23:15:17 2024] GET /accounts/login/?next=/mailman3/lists/ => generated 8204 bytes in 33 msecs (HTTP/1.0 200) 11 headers in 509 bytes (1 switches on core 1) [pid: 1385|app: 0|req: 15/23] 127.0.0.1 () {62 vars in 1230 bytes} [Wed Oct 9 23:15:27 2024] POST /accounts/login/ => generated 1019 bytes in 2 msecs (HTTP/1.0 403) 8 headers in 268 bytes (1 switches on core 0) So it wasn't that I forgot the superuser name. It's getting denied. That's all I got. Any help would be appreciated... billo On Wed, 2024-10-09 at 19:04 -0400, Bill Oliver wrote: > First: > > Last night I got everything installed and, I think, working. > Unfortunately, I forgot to write down the name and password of the > superuser I created. I know what I put in as password, but I cant' > remamber the name of the superuser. > > I know I could just create another superuser with "mailmanweb > createsuperuser" but I was wondering if there was a "mailmanweb" > command for *listing* superusers. > > If not, I'll just make another superuser... > > Second: > > I have the "example.com" error in the login screen. The instructions > state that this is a problem with making SITE_ID=1. Okay, but I > don't > know what to change it to. I tried SITE_ID=2 on a lark and everything > broke, and the instruction say not to do SITE_ID=0. Apparently it > has > to be an integer (I also tried putting in site name and got that > error). So... where do I change it? > > Thanks! > > billo > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to billo(a)billoblog.com

6 months, 2 weeks

[MM3-users] Re: Turn off social logins?

by Torge Riedel

I'll do a top post to sum things a bit up (from my point of view and requirements): - no need to disable social logins by default, there are and will be setups where this is required and wanted, but: - allow disable of social logins either by config file or admin web UI in a stable way, means: after update of mailman nothing gets broken if default configuration has changed - there is a way to do this now via settings_local.py, but feels a little bit risky if default configuration changes on update of mailman Maybe there is a possibility for a simple approach via config file: EXTRA_LOGINS = no or ENABLED_EXTRA_LOGINS = [] (maybe not valid py code, but just to show how it might look like) Best regards Torge Am 14.02.19 um 03:42 schrieb Stephen J. Turnbull: > Jonas Meurer writes: > > > But still, adding so-called "social" authenticators to the mailman3 > > django application will expose users to the risk of clicking on > > those ID provider buttons. > > I can see this point for Debian users. I suspect Ubuntu users are > somewhat less paranoid. :-) > > Since that risk holds even for sites that enable explicitly (assuming > we adopt the same policy) I will take a look at making that risk hard > to realize (more distance from anything else clickable, smaller > buttons with visible and accurate boundaries. > > > Personally, I consider it a major privacy issue if one central > > instance (e.g. Facebook) is able to track on which platforms and > > services you authenticate. > > Sure, but that ship has pretty well sailed AFAICT. Most users use > unconfigured versions of IE (or Edge) and Safari, which means they're > subject to all manner of webbugs. My employer just asked me to stop > using Firefox because it's too pedantic for their website development > vendors. :-( GDPR enforcement seems to primarily be an arm of the EU > trade offensive against large American services (that's the > Economist's recent opinion, not mine), while Europe-based globals are > undoubtedly doing the same crap. > > > And to be honstes, I'm a bit irritated that those tracking features > > from big corporates like Facebook and Google, whose main business > > model is to aggregate as as much private data points as possible > > about as much users as possible are enabled by default in Mailman3 > > upstream. > > Your irritation is not our problem, though, since you can use Debian's > version, and as I mentioned earlier, as far as I know most of our > sites are happy to have social auth. I will be paying attention to > the list to see what others think. > > > Besides the privacy concerns raised above, on problem with central > > authentication services is that they also create new single points > > of entry/failure. If your Facebook account gets hacked, you now > > loose control over all other services/platforms, that you used the > > Facebook authenticator service for. > > Same thing if you use a weak password. Effective and secure > authentication is not something we're good at, and we can't really > afford the effort to be good at it. These things are tradeoffs, and > the default should be something that most of our users (ie, site and > list admins, NOT subscribers) want. Hopefully their preferences in > this respect reflect those of *their* users (== subscribers). > > > I agree with Torge that those social auth providers should be > > disabled per default. IMHO, A sane default would be to list them in > > the settings.py but have them commented out. > > "Those" social auth providers? Are there social auth providers who > provide what you consider acceptable privacy guarantees (a la > Duck-Duck-Go in search engines)? If so, we could make those higher in > the list/easier to use. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

6 years, 2 months

[MM3-users] Re: Mailman issues after re-boot

by paulrubin37＠gmail.com

PROGRESS, Thank you. mail:/etc # zypper info python311-Django Loading repository data... Reading installed packages... Information for package python311-Django: ----------------------------------------- Repository : Main Repository (OSS) Name : python311-Django Version : 4.2.5-1.1 Arch : noarch Vendor : openSUSE Installed Size : 29.3 MiB Installed : Yes (automatically) Status : up-to-date Source package : python-Django-4.2.5-1.1.src Upstream URL : https://www.djangoproject.com Summary : A high-level Python Web framework Description : Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design. (venv) mailman@mail:~> pip install --force-reinstall psycopg2-binary Collecting psycopg2-binary Obtaining dependency information for psycopg2-binary from https://files.pythonhosted.org/packages/ce/85/62825cabc6aad53104b7b6d12eb2a… Downloading psycopg2_binary-2.9.9-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.4 kB) Downloading psycopg2_binary-2.9.9-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 6.9 MB/s eta 0:00:00 Installing collected packages: psycopg2-binary Attempting uninstall: psycopg2-binary Found existing installation: psycopg2-binary 2.8.6 Uninstalling psycopg2-binary-2.8.6: Successfully uninstalled psycopg2-binary-2.8.6 Successfully installed psycopg2-binary-2.9.9 (venv) mailman@mail:~> pip install --force-reinstall psycopg2 Collecting psycopg2 Downloading psycopg2-2.9.9.tar.gz (384 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 384.9/384.9 kB 2.2 MB/s eta 0:00:00 Preparing metadata (setup.py) ... done Building wheels for collected packages: psycopg2 Building wheel for psycopg2 (setup.py) ... done Created wheel for psycopg2: filename=psycopg2-2.9.9-cp311-cp311-linux_x86_64.whl size=505693 sha256=f0288149e9e27a5b88cc041b9385bc1f1c3a8e95958d0edf5de6d47ef3d6aadb Stored in directory: /opt/mailman/.cache/pip/wheels/ab/34/b9/78ebef1b3220b4840ee482461e738566c3c9165d2b5c914f51 Successfully built psycopg2 Installing collected packages: psycopg2 Attempting uninstall: psycopg2 Found existing installation: psycopg2 2.8.6 Uninstalling psycopg2-2.8.6: Successfully uninstalled psycopg2-2.8.6 Successfully installed psycopg2-2.9.9 (venv) mailman@mail:~> python -c 'import psycopg2' (venv) mailman@mail:~> mailman info GNU Mailman 3.3.8 (Tom Sawyer) Python 3.11.5 (main, Aug 31 2023, 07:57:41) [GCC] config file: /etc/mailman3/mailman.cfg db url: postgresql://mailman:XXXXX@localhost/mailman devmode: DISABLED REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:restpass (venv) mailman@mail:~> I went the through the rest of the install got the login screen and after logging in I get this: Forbidden (403) CSRF verification failed. Request aborted. Looking at the PostgreSQL db, the user table has certainly been emptied the only user is mailman. The UID, address and mailing list tables seem to contain many records. I am not sure how much to restore of the original DB? Short of doing a diff on two backup file do you have any insight? Thanks Paul.

1 year, 6 months

[MM3-users] My Server is unable to connect to Mailman REST API few times

by Stephen J. Turnbull

Hi Shashikanth, It appears I may have sent you on a wild goose chase. You have my apologies. There's a very similar report today, so this appears to be a Mailman 3 issue. For now, pleasee see Mark's post for the correct way to increase timeout: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… I am working with Mark to diagnose the underlying problem. Steve Stephen J. Turnbull writes: > Shashikanth Komandoor writes: > > > I am currently running Mailman 3 with version 3.3.1 and in > > built postfix version 2.10.1-6 on RHEL 7.5 with PostgreSQL 11.7 version > > with default values in production environment from the past 4 months > > almost. As of now, we are having around 1327 lists created on this. > > Aren't you the crew that modified Mailman to interface with SMS or > something like that? If so, don't worry, it doesn't invalidate your > warranty, but it seems possible that it has something to do with this: > > > After the above configuration, I found many errors like below in > > mailman.log file: > > > > *sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for > > column 'pended.id <http://pended.id>'"* > > > > *sqlalchemy.exc.ResourceClosedError: This result object does not return > > rows. It has been closed automatically.* > > > > *sqlalchemy.exc.DatabaseError: (psycopg2.DatabaseError) error with status > > PGRES_TUPLES_OK and no message from the libpq* > > Whether you have that modified Mailman or not, the messages above > suggest that there may have been a change in the schema of the > database that one side (presumably SQLAlchemy) knows about but have > not been propagated to the other (PostgreSQL). Have you perhaps done > an upgrade but not run the database migration script? > > About the REST API not available error: > > > Something went wrong > > > > Mailman REST API not available. Please start Mailman core. > > > > But my mailman core was working fine in the background. At > > the same time I was able to do other operations on a few other lists. > > > > By the time I am getting the above error, the mailman.log > > recorded the below messages. > > > > *[2020-09-16 12:49:08 +0530] [2600] [CRITICAL] WORKER TIMEOUT (pid:20234)* > > > > *[2020-09-16 12:49:08 +0530] [20234] [INFO] Worker exiting (pid: 20234)* > > > > *[2020-09-16 12:49:09 +0530] [21042] [INFO] Booting worker with pid: 21042* > > Something is taking too long, the worker is timing out. That's > probably why you get the REST API not available message. Do the > sqlalchemy errors correlate with the timeouts? > > I suspect the exit and reboot of the worker is normal but I haven't > looked at this part of the code. > > > My mailman.cfg configuration is as below. The timeout > > value is the default value. I did not set any customized value. > > Thanks for sending it; sometimes it speeds up debugging. But in this > case you say that other lists are working, so site configuration > doesn't seem to be the problem. Is this particular list special in > some way? List configuration is different? > > > The above might be few of the errors. FYI, for the creation of each list, > > it is taking almost around 15 to 20 minutes. After multiple times of the > > above error messages, at some time, I am fortunately getting the list > > created or some other operation getting done. > > This is quite strange. I don't see why this would suddenly start > working after many minutes. > > Does PostgreSQL have any logs with "interesting" (or scary) messages > in them? > > Maybe somebody else has some ideas, but this is as far as I can go. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > >

4 years, 7 months

[MM3-users] Re: Verification Failed Error

by Odhiambo Washington

On Fri, 21 Dec 2018 at 20:19, Abhilash Raj <maxking(a)asynchronous.in> wrote: > On Fri, Dec 21, 2018, at 4:00 AM, Stephen J. Turnbull wrote: > > Abhilash Raj writes: > > > > > Exim experts would be able to help you better with this. > > > > I'm the one who set up the original sample Exim .conf, because (a) I > > didn't want to migrate all my bots to Postfix, and (b) the one-real- > > host .conf turned out to be trivial. I'm an Exim expert only for my > > own configuration, which is a (pre-global-warming) glacially-moving > > target. I've never seen anything like the problems described here, > > sorry. Can't help. > > > > Since it's looking like we don't have any real Exim experts hanging > > around here, perhaps we should move the sample containers and other > > default configs to assume Postfix and document "Exim should work but > > you're kinda on your own, we'll do what we can but it's limited." At > > least Mark is maintaining such systems, and they're probably better > > documented than Exim for the single-real-host configuration, though > > they would probably run into similar problems with the containerized > > setup. WDOT? > > > Originally, the reason I went with Exim was because the postmap + hash > files were hard to generate for Mailman from within the container when > the postfix was running on the host. Exim checks existence of certain > paths to verify that the Mailing List exists, which was easier to do. > > However, now Mailman Core supports regex maps for Postfix, which > can be used with containers. > > Images already support using Postfix, it is just not the default in setup > scripts and documentation. I have opened an issue[1] to maybe change > that if some initial testing doesn't goes fine. > > [1]: https://github.com/maxking/docker-mailman/issues/282 > Hello Maxking, It is importantest to maintain Exim support :-) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

6 years, 4 months

[MM3-users] Re: [Mailman-Developers] Better documentation for Mailman Core's config options

by Barry Warsaw

Wow, this is amazing Abhilash! So much better than the old wall of text, and so cool that you learned how to write a Sphinx plugin while you were at it. Cheers, -Barry > On Aug 7, 2020, at 12:31, Abhilash Raj <maxking(a)asynchronous.in> wrote: > > Hey All, > > For the past few days, I've been trying to improve the documentation > style for Mailman Core's config options. Internally, Mailman has > all configurations defined in an ini formatted `schema.cfg` file, which > serves as a schema for our `zope.configuration` library. We had that file > simply embedded in the sphinx documentation previously as a wall of text. > > I ended up writing a Sphinx (our documentation generator) plugin which > parses the config file and presents the same documentation with more > bells and whistles so it is easier to lookup and link sections and config > options.[1] > > The process of doing that was fun and I learnt how to write a simple > sphinx plugin, so yay! It treats the comments above each config option > in the ini file as it's doc and basically generates ReST text that it passes > to Sphinx to parse and convert to the markup format that you see in > browser. You can look at it in the source repo[2] . > > Do note that not all sections are documented yet, I am still figuring out what > would be a good way to document template sections like `[plugins.master]` > and other similar sections of that kind. > > [1]: https://gnu-mailman--667.org.readthedocs.build/projects/mailman/en/667/src/… ((URL is for the PR, > the main doc seems like is serving the cached version from an older > build). > [2]: https://gitlab.com/mailman/mailman/-/blob/master/_ext/configplugin.py > > -- > thanks, > Abhilash Raj (maxking) > _______________________________________________ > Mailman-Developers mailing list -- mailman-developers(a)python.org > To unsubscribe send an email to mailman-developers-leave(a)python.org > https://mail.python.org/mailman3/lists/mailman-developers.python.org/ > Mailman FAQ: https://wiki.list.org/x/AgA3 > > Security Policy: https://wiki.list.org/x/QIA9

4 years, 8 months

[MM3-users] Re: Uncaught runner exception: 'str' object has no attribute 'decode'

by Abhilash Raj

On Tue, Oct 3, 2017, at 12:28 AM, Henrik Rasmussen wrote: > > Fra: Mark Sapiro [mailto:mark@msapiro.net] > > Sendt: 2. oktober 2017 17:26 > > Emne: [MM3-users] Re: Uncaught runner exception: 'str' object has no attribute 'decode' > > > >> Is there some kind of workaround, I can give my users or something that the list admins can do as a workaround? > > > > There is no user or list admin workaround, but the defense is a fairly > > simple patch. See the MR at > > <https://gitlab.com/mailman/mailman/merge_requests/322> and the commit > > at > > <https://gitlab.com/mailman/mailman/commit/37b3f62afdf225a8e29b66fcefcdb56b9…>. > > The commit changes 7 files, but only two are needed. The rests are tests > > or reverting <https://gitlab.com/mailman/mailman/issues/256> which isn't > >necessary. > > > > The only changes you need are those to mailman/email/message.py and > > mailman/mta/connection.py. > > I am using Docker-mailman (not yet very familiar Docker and version > control). I would expect I have to wait for a Docker-mailman release > including this change, right? > > I have at least five+ lists which almost haven't been working since my > switch from Mailman 2.1 to Mailman 3.1 (Docker-mailman 0.1.1), and I am > desperate trying to figure out whether it is best to fall back to the > Mailman 2 server, for the time being, where the lists came from, or it is > better to wait for an update to the Docker-image. > > Will https://github.com/maxking/docker-mailman/milestone/2 fix my > problems regarding failed mail handling and will it become available soon > (when?) (https://github.com/maxking/docker-mailman/issues/147)? The container images have stable releases of Mailman components, so those changes will make in to the container when a new Core release happens. The next image release is pending a version upgrade of Hyperkitty, to 1.1.3, but other changes are just meant to make the process of setting up Mailman better. They do not include changes from git repo after the release. -- Abhilash Raj maxking(a)asynchronous.in

7 years, 6 months

[MM3-users] Re: Verification Failed Error

by Abhilash Raj

On Fri, Dec 21, 2018, at 4:00 AM, Stephen J. Turnbull wrote: > Abhilash Raj writes: > > > Exim experts would be able to help you better with this. > > I'm the one who set up the original sample Exim .conf, because (a) I > didn't want to migrate all my bots to Postfix, and (b) the one-real- > host .conf turned out to be trivial. I'm an Exim expert only for my > own configuration, which is a (pre-global-warming) glacially-moving > target. I've never seen anything like the problems described here, > sorry. Can't help. > > Since it's looking like we don't have any real Exim experts hanging > around here, perhaps we should move the sample containers and other > default configs to assume Postfix and document "Exim should work but > you're kinda on your own, we'll do what we can but it's limited." At > least Mark is maintaining such systems, and they're probably better > documented than Exim for the single-real-host configuration, though > they would probably run into similar problems with the containerized > setup. WDOT? Originally, the reason I went with Exim was because the postmap + hash files were hard to generate for Mailman from within the container when the postfix was running on the host. Exim checks existence of certain paths to verify that the Mailing List exists, which was easier to do. However, now Mailman Core supports regex maps for Postfix, which can be used with containers. Images already support using Postfix, it is just not the default in setup scripts and documentation. I have opened an issue[1] to maybe change that if some initial testing doesn't goes fine. [1]: https://github.com/maxking/docker-mailman/issues/282 > > Steve > > -- > Associate Professor Division of Policy and Planning Science > http://turnbull.sk.tsukuba.ac.jp/ Faculty of Systems and Information > Email: turnbull(a)sk.tsukuba.ac.jp University of Tsukuba > Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

6 years, 4 months

[MM3-users] Re: Postorius: Show only own lists

by Terri Oda

On January 23, 2018 6:26:51 AM PST, Thor Atle Rustad <thor.rustad(a)gmail.com> wrote: >2018-01-23 14:37 GMT+01:00 Simon Hanna <simon(a)hannaweb.eu>: > >> On 01/23/2018 12:10 PM, Thor Atle Rustad wrote: >> >>> Imagine an installation with 500+ mailing lists and almost as many >local >>> admins that are supposed to maintain their own local lists with a >few >>> local >>> members. As site admin, I see all 500+ lists, but I was expecting >*local* >>> admins to see only the lists they maintain. I have set all lists to >>> "advertised = False", so they are hidden from ordinary view. Is >there a >>> way >>> to show only relevant lists to local admins? >I believe I wrote Postorius in the subject line, but it may not be so >obvious. So yes, in Posterius I would like to hide the lists from email >harvesters, while relatively "untrusted" admins should have access to >basic >operations on the one list he/she "owns" (and without having to browse >through 50 pages of mailing lists). Somehow I don't think 500 >superusers in >our system is a good idea! What do you mean by local admins? Is this 1. Giantsite.example.com hosts lists for a bunch of domains, and you want the logged in admins of mydomain.com to see their list admin pages but not otherdomain.org's lists 2. You want each logged in admin to have their own personal dashboard of list admin pages for lists they administrate, even though they're all under one big domain. (Neither of those scenarios talks about archives, since you say you're only talking about postorious) I think #2 is something we should be doing, and the old "todo" section of the interface kind of was made with that in mind. Probably worth mocking up some screenshots and opening a feature request to keep the idea in mind, but it should be more a matter of prioritizing how things are displayed than any huge change. I'm not sure if we have a "local admin" concept in the system that lines up with #1, though.

7 years, 3 months

[MM3-users] Re: Set Postorius interface locale

by Abhilash Raj

On Tue, Mar 2, 2021, at 7:55 AM, Duane Raymond wrote: > Hi everyone, > > Thanks to Brian I'm up and running with my migrated mailman3 lists and > am now starting to get into customising the interface (styling and > content). I've been trying to figure out how to set/change the > Postorius locale - I specifically want to change some text strings on > the info page without breaking the i18n mechanisms. I've worked with po > and mo files before, so changing and compiling them isn't a problem. > The problems I've encountered so far are: > 1) Where do I set the local of the Postorius interface? The installation has a default Language setting which you can set in the settings.py LANGUAGE_CODE = "en" > 2) Individual lists have a language setting, but it doesn't seem to > reflect what the Postorius locales are as the only english option is > "English (USA)" and none of the other English speaking locales and on > the file system there is no en-us PO files....so I suspect I'm looking > in the wrong place. List level settings affect the Mailman level emails that go out, not the interface. The users can choose which language the interface shows up in using their browser settings. We primarily depend on the Accept-Language header in the request to determine the language of the response. While users do have a preffered-language setting, it currently isn't used to determine the language of the UI. Like list setting, it is only used to determine the language of the emails sent out (user's preferences override the list's default language). > > Am I looking at this all wrong? The documentation doesn't seem to have > any info about this and yet the mechanisms seem to be there. > > Once I learn more I'm quite keen to start working on improving the > documentation based on what I learn. > > Cheers, > > Duane > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

4 years, 1 month

[MM3-users] Re: how to integrate subscription into static jekyll website

by Hagen Bauer

Since I am running mailman in a docker container I thought it may be an approach to put https://github.com/dthree/mailit along. This seems to be used by a good amount of people and would allow me to have a rest api just for sending the mails to mailman. I am not a javascript specialist but currently I do not see any security risks. Do I miss something? Regards Hagen On 10/19/18 8:05 PM, Stephen J. Turnbull wrote: > Hagen Bauer writes: > > > Has anybody tried to hide the direct api access behind a nginx reverse > > proxy url? > > Sounds straightforward. > > I doubt anybody's done it, though, because you still face the problem > that anybody with that access is a site manager, with unclear limits > on their power outside of Mailman. (Shouldn't be much they can do > outside of Mailman, but I wouldn't bet my host on it.) In many cases, > the real site manager is also the host admin, so they have sufficient > access to a shell account. There's no need for her to have a proxy; > why make it available to people who are supposed to have less power? > > Note I'm *not* saying this *never* makes sense. I'm just saying that > in a lot of cases it won't buy you very much because Mailman isn't set > up to provide granular authorization of REST endpoints. > > > on the Webseite I would have a url like this > > > > mailmanserver.tld/subcsribe/user(a)mail.com > > > > And within the nginx proxy configuration on the server this would > > translate to > > > > localhost:80xx/"mailman-restapi-with-credentials"user(a)mail.com > > First of all, the address to subscribe is a variable, so at a bare > minimum your nice static site has to provide a form, which isn't very > static. I guess you could put the URL to the nginx proxy as the > action URL in the form, but then the proxy needs to deal with a > form-encoded query string to translate to the appropriate REST > endpoint. > > I suppose you could also have the user type the URL including the > email address into the browser address bar, but Abhilash's idea of > providing a mailto:subscribe@mailmanserver.tld URL seems more > convenient. The mailto URL also solves the problem of confirming > ownership of the mailbox, since that logic is built in to the > subscribe-by-mail code. > > Second, if your proxy is configured very carefully, you might be able > to restrict this to only the exact operations you want to expose, here > subscription. However, although the email interface implements > verification by providing a one-time token that must be read from the > mailbox of the address (and so proves ownership of the mailbox), I > don't think the REST API implements operations like that. IIRC that > is implemented in Postorius. (Abhilash?) If my memory is correct, > you would also need to implement that, and that's definitely not static. > > > So the public does not see the credentials. And if my nginx host > > config is hacked I have a problem anyway. > > I can't speak for Abhilash, but what would worry me is not that nginx > is already hacked, but rather that if not properly configured, it > might be possible for someone to access REST endpoints you don't want > exposed through this interface, and maybe even to other parts of the > webserver. To give a simple example, consider a variant on the > ancient Apache traversal bug: > > mailmanserver.tld/subscribe/../unsubscribe/user(a)mail.com > > Probably nginx and/or Mailman and/or your proposed proxy config are > already capable of stopping this particular well-known attack, but > undoubtedly there are more sophisticated attacks that I don't know > about. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

6 years, 6 months

[MM3-users] Re: Splitting hosts: mta&core vs postorius

by Ruth Ivimey-Cook

Stephen J. Turnbull wrote: > I don't understand. I'm not in a position to test with Mailman itself > right now, but the standalone gunicorn[1] does the right thing (takes > an IP address and binds to the corresponding interface, or takes a > domain name, resolves it to the IP address, and binds that to the > corresponding interface). It does not substitute localhost. The localhost address is coming in via the hosts file, which is a relatively standard practice IMO. I could change it, but then perhaps other things would not work. I would prefer to be able to specify what I need. There are also things like systemd-resolvd which will do the localhost-substitution for you even if /etc/hosts doesn't. I did try with the gunicorn patch Mark mentioned (host = "0.0.0.0") and changing the cfg file hostname back to the full name of the host, and that now works!! -- in that the problem I showed earlier is gone. However I then get another issue. From the mailman.log file I now see in the last few lines. For clarity I've numbered them like this "{1}": {1} [22/Jun/2020:00:58:24 +0000] "GET /3.1/domains HTTP/1.1" 200 320 "-" "GNU Mailman REST client v3.3.1" {2} [22/Jun/2020:00:58:24 +0000] "GET /3.1/domains/ch-bc.org.uk HTTP/1.1" 200 215 "-" "GNU Mailman REST client v3.3.1" {3} [22/Jun/2020:00:58:24 +0000] "GET /3.1/domains/greyarea-web1.cam.ivimey.org/lists?advertised=true&count=0&page=1 HTTP/1.1" 404 26 "-" "GNU Mailman REST client v3.3.1" At this point the browser screen reports "Something went wrong" and the message "HTTP Error 404: {"title": "404 Not Found"}" Previously, the process stopped at line {1}. Now that and line {2} is executed (AFAIK) correctly, but then line {3} is isssued. I have no idea why something is looking for "domains/greyarea-web1": - Only the "ch-bc.org.uk" domain is listed in the reply to line {1}, as expected; - "greyarea-web1" doesn't appear in any lists I can show with the "mailman" cli command (and I think I tried them all); - "greyarea-web1" doesn't appear anywhere in the source or config other than as the name of the host its running on or as an ALLOWED_HOST; - I tried using "curl -X DELETE" (from the api docs) to delete a domain called that and it says no such list exists. My only possible clue is that I think, at some point in the journey of configuring mailman, I might have created a list called that on the host. However since then I have wiped the install files and reinstalled. I cannot make sense of this being the answer given the checks just listed. I went looking through the python source and can see that the get() function does indeed throw a 404 exception if the domain isn't found, but I cannot see which code is parsing the reply from the first line So I have two questions: 1) How can I fix this problem? 2) why does getting an 404 reply here cause the postorius web page to produce a "something went wrong" /404 and nothing else, when it could have simply reported that something had gone wrong with one specific domain; > I'd rather not enable the complexity of a multihomed application > unless there's an important use case. (I don't make the final > decision, but I am listened to by Those Who Do.) You wonder about multihome hosts, but should a host have more than one interface (as several of my VMs and physical hosts do), being able to specify which of the possible addresses to listen on could be very useful in the case that "all of them" is the wrong answer. There is no complexity for the developers beyond what is already supported and working. The only point I can see wrong here is the mailman3-core is using one hostname value for two distinct purposes, and IMO it shouldn't. Ideally, and this is icing on the cake, a listen address specification should allow multiple IPs to be passed in, in which case the server listens on all of them. I expect the python libraries already support this. Thanks to all for the help so far, Ruth

4 years, 10 months

[MM3-users] Re: SPF check fails for lists subdomain

by Jan Eden

On 2023-01-05 11:13, Odhiambo Washington wrote: > On Thu, Jan 5, 2023 at 10:56 AM Jan Eden via Mailman-users < > mailman-users(a)mailman3.org> wrote: > > > On 2023-01-04 15:07, Mark Sapiro wrote: > > > On 1/4/23 14:02, Jan Eden via Mailman-users wrote: > > > > Although I could specify the IP address in my SPF records directly (as > > > > you suggested), I do hope that my understanding of DNS records laid out > > > > above is not entirely misguided. My current setup does work as expected > > > > for eden.one, after all. > > > > > > As I said at < > > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > > > > > I was misreading your DNS and now I am as puzzled as you about the > > failure. > > > > It gets even more mysterious. I tried sending messages from both > > lists.eden.one and janeden.net (my other domain) to a gmail account and > > to another mail provider. Both messages passed the SPF checks on both > > services (s. the relevant headers quoted below). Now because I send > > those messages via my SMTP user (smtpuser(a)eden.one) and use SRS, the SPF > > check operates (and succeeds) on e.g. > > > > srs0=jdm+=5c=lists.eden.one=testlist-bounces+gmailuser=googlemail.com(a)eden.one > > (and not on somethingsomething(a)lists.eden.one or > > somethingsomething(a)janeden.net) Could this be the reason for the failed > > SPF checks in Yahoo's and Google's DMARC reports? > > > - Jan > > Might you be willing to use proven tools instead of trying so hard to > understand the myth? > > I use https://easydmarc.com/tools#spf-tools > It allows me to generate, lookup, test SPF, DKIM, DMARC, etc. I did use adequate tools to generate and test my DNS records, and this is why the DMARC reports quoted in my initial message are mysterious to me. Trying hard to solve a mystery might not be productive in every case, but I do not want to stop being curious. - Jan

2 years, 3 months

[MM3-users] Re: Installation Help on Ubuntu 18

by Odhiambo Washington

On Sat, 25 Jul 2020 at 20:35, <joseverde(a)gmail.com> wrote: > Hi folks, I spent many hours last night trying to get Mailman 3 up and > running. I am using Ubuntu on a Digital Ocean LAMP droplet (so PHP, Apache, > Sql, and Postfix are already installed). The LAMP stack works as I have a > successful installation of YOURLS going on. > > So, I tried to use this guide ( > https://docs.google.com/document/d/1xIcSsoNFp2nHi7r4eQys00s9a0k2sHhu1V5Plan…) > to install the Mailman3 package and got to the configuration screen (the > one with all the DOS-esque GUI). I ran it but it kept saying the DB wasn't > installed. Was I supposed to create SQL databases and users before running > that and use those usernames/databases in the credentials it asks for? It > doesn't seem like that in the way it asks the questions (since it asks to > create a password) but I wasn't sure. > > Anywho, I tried to remove the package completely and restart but I keep > spinning my wheels. So my questions: > > 1. Does that guide actually work? Did I just foul up the database part? > 2. Is there a better way to install mailman3 with some nice step-by-step > directions to follow? > 3. Is there a way to completely wipe my previous installs without wiping > the server? ( I didn't back up...silly I know ). I used apt-get --purge > remove to uninstall mailman3 and mailman to start over, did that do the > trick? > > Sorry for all the questions. > DISCLAIMER: I have no experience at all with MM3. I haven't even succeeded in installing it as yet because I am trying to do it manually on FreeBSD. I have an Ubuntu box that runs mailman-2.1.34 manually installed as well, but I am wondering if you tried the following: root@lists:/home/wash# apt search mailman3 Sorting... Done Full Text Search... Done mailman3/bionic 3.2.0-1~ubuntu18.04.1 all [residual-config] Mailing list management system mailman3-doc/bionic 3.2.0-1~ubuntu18.04.1 all Mailing list management system documentation mailman3-full/bionic 3.2.0-1~ubuntu18.04.1 all Full Mailman3 mailing list management suite (metapackage) mailman3-web/bionic 0+20180916-2~ubuntu18.04.1 all [residual-config] Django project integrating Mailman3 Postorius and HyperKitty python-django-hyperkitty/bionic 1.1.4-4 all Web user interface to access GNU Mailman3 archives python-django-mailman3/bionic 1.1.0-4 all Django library to help interaction with Mailman3 (Python 2 version) python-django-postorius/bionic 1.1.2-3 all Web user interface to access GNU Mailman3 python-mailmanclient/bionic 3.1.1-5 all Python bindings for Mailman3 REST API (Python 2 version) python-mailmanclient-doc/bionic 3.2.0-2~ubuntu18.04.1 all Python bindings for Mailman3 REST API (Documentation) python3-django-hyperkitty/bionic 1.2.1-3~ubuntu18.04.1 all Web user interface to access GNU Mailman3 archives python3-django-mailman3/bionic 1.2.0-2~ubuntu18.04.1 all Django library to help interaction with Mailman3 (Python 3 version) python3-django-postorius/bionic 1.2.2-2~ubuntu18.04.1 all Web user interface to access GNU Mailman3 python3-mailman-hyperkitty/bionic 1.1.0-9~ubuntu18.04.1 all [residual-config] Mailman3 plugin to archive emails with HyperKitty python3-mailmanclient/bionic 3.2.0-2~ubuntu18.04.1 all Python bindings for Mailman3 REST API (Python 3 version) root@lists:/home/wash# apt install mailman3 Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: python3-aiosmtpd python3-alembic python3-dateutil python3-falcon python3-flufl.bounce python3-flufl.i18n python3-flufl.lock python3-lazr.config python3-lazr.delegates python3-mako python3-mimeparse python3-nose python3-passlib python3-psycopg2 python3-public python3-sqlalchemy python3-sqlalchemy-ext python3-zope.configuration python3-zope.i18nmessageid python3-zope.schema Suggested packages: mailman3-doc python3-editor python-flufl.bounce-doc python-flufl.i18n-doc python-flufl.lock-doc python-lazr.config-doc python-lazr.delegates-doc python3-beaker python-mako-doc python-nose-doc python-psycopg2-doc python-public-doc python-sqlalchemy-doc python3-mysqldb python3-fdb python3-zope.testing The following NEW packages will be installed: mailman3 python3-aiosmtpd python3-alembic python3-dateutil python3-falcon python3-flufl.bounce python3-flufl.i18n python3-flufl.lock python3-lazr.config python3-lazr.delegates python3-mako python3-mimeparse python3-nose python3-passlib python3-psycopg2 python3-public python3-sqlalchemy python3-sqlalchemy-ext python3-zope.configuration python3-zope.i18nmessageid python3-zope.schema 0 upgraded, 21 newly installed, 0 to remove and 2 not upgraded. Need to get 2,862 kB of archives. After this operation, 16.7 MB of additional disk space will be used. Do you want to continue? [Y/n] Where would that end you??? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

4 years, 8 months

[MM3-users] Re: clean up spambot generated accounts

by Johannes Rohr

Sorry, I should have looked into the script more closely. I simply had to set the variable CFG_PATH to the correct value. Johannes Am 23.02.25 um 15:29 schrieb Johannes Rohr: > Dear all, > > I'm trying to find a way to clean up hundreds of spambot generated > accounts on my system. I was pointed to this script that should be > capable of doing the task: > > https://www.msapiro.net/scripts/clean_users.py > > However, it seems outdated. First, the path to the interpreter is not > what I find on my mailman3 installation. It says: > > #! /opt/mailman/mm/venv/bin/python > > However, the python command is located in /opt/mailman/venv/bin/python > instead, no "mm" > > After fixing this, the script still throws an error: > > (venv) mailman@ida:~$ clean_users.py > Traceback (most recent call last): > File "/usr/local/bin/clean_users.py", line 55, in <module> > django.setup() > File > "/opt/mailman/venv/lib/python3.10/site-packages/django/__init__.py", > line 19, in setup > configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) > File > "/opt/mailman/venv/lib/python3.10/site-packages/django/conf/__init__.py", > line 102, in __getattr__ > self._setup(name) > File > "/opt/mailman/venv/lib/python3.10/site-packages/django/conf/__init__.py", > line 89, in _setup > self._wrapped = Settings(settings_module) > File > "/opt/mailman/venv/lib/python3.10/site-packages/django/conf/__init__.py", > line 217, in __init__ > mod = importlib.import_module(self.SETTINGS_MODULE) > File "/usr/lib/python3.10/importlib/__init__.py", line 126, in > import_module > return _bootstrap._gcd_import(name[level:], package, level) > File "<frozen importlib._bootstrap>", line 1050, in _gcd_import > File "<frozen importlib._bootstrap>", line 1027, in _find_and_load > File "<frozen importlib._bootstrap>", line 1004, in > _find_and_load_unlocked > ModuleNotFoundError: No module named 'settings' > > Can somebody explain to me, why this module is not found and what I > can do to fix it, if anything? > > Thanks a lot in advance! > > Johannes > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to johannes(a)rohr.org

2 months

[MM3-users] Re: Treatment of hostname in /etc/hosts [was: Splitting hosts ...]

by Abhilash Raj

On Thu, Jun 25, 2020, at 12:33 AM, Stephen J. Turnbull wrote: > Ruth Ivimey-Cook writes: > > > The localhost address is coming in via the hosts file, which is a > > relatively standard practice IMO. > > OK, so I did a little research on this. You don't need to read this > to continue discussion, but since other users have expressed interest > in splitting hostname from listener IP address, I am posting what I've > learned about aliasing hostname to localhost. This doesn't apply > directly to multihomed hosts as also mentioned in this thread, and > additions and corrections are, of course, welcome. > > It had been my understanding that conventional wisdom is that > "127.0.0.1 localhost" is *all* that you should put in /etc/hosts in > most cases, so I did a couple of web searches. As far as my search-fu > got me, there is no standard beyond putting localhost itself in there, > which is used by a lot of boot scripts. But as a matter of common > practice, the Debian family supplies an /etc/hosts that defaults to > aliasing the host's FQDN to 127.0.1.1 (not 127.0.0.1), apparently as a > kludge to keep GNOME happy: > http://www.leonardoborda.com/blog/127-0-1-1-ubuntu-debian/ > > systemd-resolvd does something different; in the case that there is no > interface configured *and* no /etc/hosts entry, it resolves the > "local, configured hostname" to 127.0.0.2 (again, not 127.0.0.1). But > if there is an interface configured, it resolves to that interface's > IP. Apparently this is intended to speed up systemd-based boot > processes by allowing them to run in parallel. Fedora issue tracker > reports it leads to a problem similar to the one being discussed, eg, > https://bugzilla.redhat.com/show_bug.cgi?id=1116538#c25 > > Bottom line: neither the Debian family nor the Red Hat family aliases > hostname to localhost; by default it's a separate "interface". And > Red Hat does it conditionally on no real interface found. > > As far as I can tell, these workarounds are about placating software > that breaks in one way or another if the host's own FQDN does not > resolve at all when they start up, presumably because the resolver > can't reach the network -- but they don't care if it's consistent with > DNS as long as it connects to the local host, which leads to a fairly > arbitrary practice of aliasing the hostname to localhost. All of > which suggests to me that such software should be special-casing the > host's various identifiers (hostname, canonical FQDN, /etc/hosts > aliases, all of which it can easily discover) and translate them to > "localhost", rather than asking the resolver to do it and creating an > inconsistency with DNS. > > Of course, getting other software to Do The Right Thing is pretty > hopeless, but the solution is simple: wait for the network to come up > and use the configured interface's address by looking up the canonical > hostname. This works for GNOME and Postfix (mentioned in the > references above), and for Mailman. > > AFAICS, the exceptions are (1) running a high-availability "cloud" > system that needs to scale very quickly on-demand (in which case you > want to start things in parallel and need to deal with the real race > conditions involved), and (2) a need to do work while disconnected. > The former is over my pay grade but apparently Fedora has not managed > to get it 100% reliable even with systemd. The latter case can be > dealt with by putting your static IP address in /etc/hosts when you > have one, which I presume Sorry to chime in pretty late on this topic, but I didn't know about the research you did above. Thanks Steve! I have had some of these issues in the past, but they were related to Mail routing from Postfix to the Mailman container. The problem was that docker would assign these DNS names to each container based on their names and magically update every container's `/etc/hsots` so they can talk to other containers through these docker given DNS names. However, the postfix running on the host won't be because docker doesn't put this DNS name anywhere else. So, if I put 0.0.0.0 for LMTP server to bind to, it would use that address to generate Postfix's transport maps and of course it won't be able to reach Core running at an IP assigned at runtime. But the way I got around the problem was by adding regex based routing for Postfix. Exim worked because all I had to do was make the directories visible on the host. About HTTP requests, I would say Core doesn't really need to know the external address that it is reachable at, only the interface it needs to bind to. In a containerized environment, the IP/FQDN could actually change, so if we are optimizing this for situations like containers/multi-host, it would require an Admin to update the configs. Instead, what I was thinking was to just ignore the host, port in the self_link in Mailmanclient. The purpose of self_link is mostly to reflect the path to the resource itself in the API and I am not sure if it requires the complete URL with scheme, host and port. That should allow Web components to reach the resources without Core knowing or caring about what address it is externally reachable at. Thoughts? > > > I could change it, but then perhaps other things would not work. > > I would prefer to be able to specify what I need. > > Notwithstanding the discussion above about why it "shouldn't" break > anything, I wouldn't be surprised if something breaks -- such > configuration choices tend to ramify. I sympathize with you not > wanting to change a configuration that works for you. > > Steve > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

4 years, 9 months

[MM3-users] Re: Mailman, etc. upgrade woes and persistent bugs

by Ruth Ivimey-Cook

Steve, What I mean by per-user permissions is that some transitions, for example bounce-disabled to enabled, should be only possible by someone with appropriate permission to do so -- perhaps the list admin, perhaps someone else, but not just anyone. Another example _could_ be if a user disables themselves then only that user can re-enable them again (though I can see that might not be good). Of course, you can to some extent group users by role, so 'admin' role is always ok, 'list-owner' role is ok for any action on that list, etc. If you had a 'role' which was 'account-owner' then you can probably code the rule for re-enabling an account disabled by the account owner as "role = admin or role = list-owner or role = account-owner". Whether this needs to be extended to "role=admin or user = guy" (where 'guy' is a named account holder) I don't know... I suspect it would be better if that wasn't done unless it was shown to be necessary, as "here be dragons". Do you see where I'm at? In writing this up it occurs to me that being able to see/edit rules such as these, stored in per-list config, would probably be a good thing. Don't know how practical that is though. Regards, Ruth On 16/02/2021 15:29, Stephen J. Turnbull wrote: > Hi, Ruth, > > Thanks for the comments. I'll have to think about most of it, but I > have one immediate question. > > Ruth Ivimey-Cook writes: > > > > On 13/02/2021 16:34, Stephen J. Turnbull wrote: > > > > I don't think separate booleans is a good idea, since they're > > > basically mutually exclusive states[.] > > > > While using one variable makes some sense as you say, it does > > complicate the allowed state transitions especially when per-user > > permissions as well as logic are applied to the situation. > > I don't understand how "per-user permissions" apply here. > > Regards, > Steve -- Software Manager & Engineer Tel: 01223 414180 Blog: http://www.ivimey.org/blog LinkedIn: http://uk.linkedin.com/in/ruthivimeycook/

4 years, 2 months

[MM3-users] Re: Broken web UI after upgrade

by Odhiambo Washington

On Wed, Dec 14, 2022 at 8:02 PM Stephen J. Turnbull < stephenjturnbull(a)gmail.com> wrote: > Odhiambo Washington writes: > > > The site is public, so I can share the link: > https://list.rc.org/archives/ > > The CSS is not working. Two possibilities to try first: > 1. (more likely) You didn't run the command that compiles the CSS. > (probably) mailman-web compress > 2. (much less likely) The CSS got replaced in the upgrade and the > ownership changed because you were working as root, and the server > process can't access the files. > 3. other possibilities: the SASS compiler (usually sassc, possibly sass) > > 1. The post-update script runs everything required. For the avoidance of doubt, here it is: <CUT> #!/bin/bash set -e BINDIR="/opt/mailman/mm/bin" CONFDIR="/opt/mailman/mm" SETTINGS="settings" DJANGO_CONTROL_SCRIPT="$BINDIR/django-admin" STATIC_DIR="$CONFDIR/static" DEBUG="False" set -x if [ "$DEBUG" == "False" ]; then mkdir -p "$STATIC_DIR" || : "$DJANGO_CONTROL_SCRIPT" collectstatic --clear --noinput --verbosity 0 "$DJANGO_CONTROL_SCRIPT" compress "$DJANGO_CONTROL_SCRIPT" compilemessages fi syncdb_cmd="$DJANGO_CONTROL_SCRIPT migrate" if [[ ! -t 1 ]]; then # stdout is not a terminal syncdb_cmd="$syncdb_cmd --noinput" fi $syncdb_cmd echo 'Do you want to rebuild the search index? This takes a long time.' echo 'Type "Yes" to continue; anything else to quit.' read -p "> " response if [ "x$response" != "xYes" ] ; then exit; fi "$DJANGO_CONTROL_SCRIPT" update_index </CUT> The script is run within the venv. 2. That isn't possible. Why? Because when I do 'su -mailman' it automatically activates the venv via a .bashrc. I have run this: find /opt/mailman -type f -name "*.css" -exec ls -al {} \; And all files are owned by mailman.mailman. 3. I have the following in settings_local.py: COMPRESS_PRECOMPILERS = ( ('text/x-scss', 'sass -t compressed {infile} {outfile}'), ('text/x-sass', 'sass -t compressed {infile} {outfile}'), ) COMPRESS_OFFLINE = True COMPRESS_ENABLED = True And: root@list:/home/wash# su - mailman (venv) mailman@list:~$ which sass /usr/bin/sass (venv) mailman@list:~$ `which sass` --version Ruby Sass 3.7.4 (venv) mailman@list:~$ Anything else that might be the issue? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)

2 years, 4 months

[MM3-users] Re: Single-Sign-out for List member

by Abhilash Raj

On Thu, Aug 13, 2020, at 10:04 AM, Mark Sapiro wrote: > On 8/13/20 6:21 AM, w.schoener(a)alp.dillingen.de wrote: > > > > So I would like to ad a link in the footer of every message, with which every receiver/member can sign out of the newsletter with one click. > > It does not seam to be possible, to put this into effect with mailman 3. > > Does anybody have any hints, how to create a link, and when this link is called, the user is signed out of the newsletter. > > > You can set the list's unsubscription_policy to open and set the link as > a mailto: to the list-leave address, but that still requires a second > click to send the mail. > > The solution is to create a web based CGI script which is called with a > URL like > <https://example.com/unsub_script/list.example.com/user@example.net> and > have that script invoke the appropriate `mailman` command to remove the > member from the list. Beginning with Mailman 3.3.2, this would be If this is one-click, can't this URL be hit by anyone trying to remove the user from any MailingList without any auth or anything? > > mailman delmembers -l list.example.com -m user(a)example.net -g > > to remove the member and send a goodbye message. In older Mailman, it > would be something like > > echo user(a)example.net | mailman members -x - list.example.com > > I understand you may have a legal requirement do do this, but these > "one-click - no user confirmation" links are a bad idea because usera > will reply to a list post or forward it and not remove the link and > userb will receive usera's reply and click usera's unsubscribe link > either maliciously or thinking it will unsubscribe userb. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

4 years, 8 months

[MM3-users] Re: MM3 occasionally chokes (?)

by Dmitry Makovey

thanks Abhilash for all those pointers. I'll bee following those threads to see whether I can find something there. thanks! On 3/18/19 10:14 AM, Abhilash Raj wrote: > On Mon, Mar 18, 2019, at 10:13 AM, Abhilash Raj wrote: >> On Mon, Mar 18, 2019, at 9:46 AM, Dmitry Makovey wrote: >>> Thanks for fast response Abhilash! >>> >>> See inline for responses: >>> >>> On 3/18/19 9:24 AM, Abhilash Raj wrote: >>>> On Mon, Mar 18, 2019, at 9:14 AM, Dmitry Makovey wrote: >>>>> In the last couple of month we've observed some odd behavior from MM3 >>>>> (namely the core) where it seems like email that is about to be sent to >>>>> a large-ish mailing list (10K users) has been processed yet never gets >>>>> sent out until we restart the core. >>>> >>>> This is indeed weird, I don't think 10k is too much, but it also depends >>>> on your MTA. How fast do they allow sending emails and if there is any >>>> rate limiting. >>> >>> I'm using Mailu.io for MTA. We're controlling it so there's not much >>> throttling as I've whitelisted mailman to send anything over LMTP. >> >> Mailman sends out email using SMTP and receives it from MTA using LMTP. >> >>> >>>> I wouldn't necessarily consider this a performance problem in Core, >>>> since it is still just processing only one email. All it needs to do is >>>> send them all out, which is where the problem should be. >>> >>> inferred assumption is that mailman-core loses LMTP connection with >>> Mailu stack. but then upon restart of service it regains it? >> >> Mailman acts as am SMTP client sending out emails via MTA. AFAIK, >> Mailman 3 recycles SMTP connections after certain number of emails are >> sent out. >> >> You can configure how many emails are sent-out per-connection and >> when should each connection be recycled. You can also disable the >> recycling altogether and always keep a connection open to the MTA. >> See [1] here for detailed docs. >> >> [1]: >> https://mailman.readthedocs.io/en/latest/src/mailman/mta/docs/connection.ht… >> >> Again, copying from the docs, these are some extra MTA related settings >> that you may want to change: >> >> --- >> # Ceiling on the number of recipients that can be specified in a single SMTP >> # transaction. Set to 0 to submit the entire recipient list in one >> # transaction. >> max_recipients: 500 >> >> # Ceiling on the number of SMTP sessions to perform on a single socket >> # connection. Some MTAs have limits. Set this to 0 to do as many as we like >> # (i.e. your MTA has no limits). Set this to some number great than 0 and >> # Mailman will close the SMTP connection and re-open it after this number of >> # consecutive sessions. >> max_sessions_per_connection: 0 >> >> # Maximum number of simultaneous subthreads that will be used for SMTP >> # delivery. After the recipients list is chunked according to max_recipients, >> # each chunk is handed off to the SMTP server by a separate such thread. If >> # your Python interpreter was not built for threads, this feature is disabled. >> # You can explicitly disable it in all cases by setting max_delivery_threads >> # to 0. >> max_delivery_threads: 0 >> --- >> >> max_sessions_per_connection is by default set to 0, so that shouldn't be >> causing any problems. >> >> For a list of 10k members with `max_delivery_threads` set to to >> default(500), > > I meant, max_recipients set to default (500). > >> Mailman could be sequentially sending out emails in 20 (10k/500) >> connections, which >> could be causing the delay that you see. >> >> You could increase the threads or if your MTA can handle it, set the >> max_delivery_threads >> to 20 and see if that helps? >> >>> >>>> I'd imagine debug logging on outgoing runner should give some more >>>> pointers on what is going on. >>>> >>>> You can configure the smtp runner to log extra information (copied from >>>> docs)[1] >>>> >>>> [1]: >>>> https://mailman.readthedocs.io/en/latest/src/mailman/config/docs/config.html >>> >>> awesome - I'll give it a go. thanks for the pointer. >>> >>>> I'd also recommend looking into MTA logs for anything un-usual. >>> >>> we briefly did , but nothing stood out. >>> >>>>> troubleshooting/digging before I just restart the core (which by the way >>>>> takes about 45min to fully start... which is another puzzle to solve at >>>> >>>> How do you measure that it takes 45 mins to start? >>> >>> normally it's "docker ps" command output along with "docker top" so once >>> I see all the sub-processes kicked off I'm assuming core is started and >>> from that point it's anywhere between 35 and 45min before I can get any >>> *response* from core (i.e. mailman-web complains, REST commands fail, etc.) >> >> Maybe we should discuss this in another thread? REST Runner (and other runners) >> shouldn't be taking this long to boot up. >> >>> >>>> What database backend are you using? >>> >>> Postgres. The whole setup is docker-setup (0.1.7 version of containers >>> at this point). >>> >>> >>> -- >>> Sr System and DevOps Engineer SoM IRT >>> >>> >>> Attachments: >>> * signature.asc >> >> -- >> thanks, >> Abhilash Raj (maxking) > -- Sr System and DevOps Engineer SoM IRT

6 years, 1 month

[MM3-users] Re: Two Questions: One General, the Other Specific

by Mark Sapiro

On 2/26/21 9:10 AM, David Wilson wrote: > > *Is there another site that might best communicate the future goals, > plans, objectives, and time frames for mailman3 or is the status of the > project at the state of being primarily reacting to problems, as > evidenced in this problem solving forum, at least as it seems to me, > with no offense meant or intended?* We are a small project with few volunteers and no paid staff. While we would like to be able to devote significant time to future goals, plans and objectives, often we only have time and resources to devote to fixing immediate issues. The way to get issues/objectives/etc. on our radar it to post them to the various sub-project issue trackers at <https://gitlab.com/groups/mailman>. > Something I find surprising is a problem described in an image grab > below relating to an inability to amend a list entry to include a name > to be associated with an email address, as if the data needs to be > immutable. When a workaround was unsuccessfully explored to delete the > entry and then resubmit an amended entry but with the same already > deleted email address, that was still refused, presumably because of > pointers to archived data don't permit, but this seems like a > particularly onerous restriction, or perhaps even a bug, or at least a > severe design constraint. *Is this item on an existing to do list?* There has been much discussion of this on this list. See for example the recent thread at <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…> This item is not on a ToDo list because even though <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…> asked for an issue to be filed, the most interested person refused to do so because of his assessment of our priorities <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…>. And if you don't read the whole thread, at least see the response at <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…>. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 1 month

[MM3-users] Re: Edit archived email? Is it possible?

by Prasanth Nair

Hi Mark, Thank you for your help. Let me try this. Kind regards, Prasanth On Tue, 28 Sept 2021 at 18:03, Mark Sapiro <mark(a)msapiro.net> wrote: > On 9/28/21 6:02 AM, Prasanth Nair wrote: > > Hello, > > > > Does anyone know how to edit an archived email in Mailman3? It is > possible > > in Mailman2 ( > > > https://wiki.list.org/DOC/How%20do%20I%20edit%20the%20archives%20of%20a%20M… > ). > > Is there any option similar to this in Mailman3? > > > In some sense the process is the same but possibly even simpler in > Mailman 3. You need to modify the underlying data which requires > appropriate access to the server. > > In Mailman 3, the data are in the hyperkitty_email table in Mailman's > database. You can get the Message-ID hash from the URL to the archived > message, e.g. for the message to which I'm replying, the URL is > > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > and the hash is 7PVJL23DLSWMXXL44ALQUO6VSYBRZ4OA. Then if you have > access, you can do database queries like > ``` > SELECT subject, content FROM hyperkitty_email WHERE message_id_hash = > '7PVJL23DLSWMXXL44ALQUO6VSYBRZ4OA'; > ``` > and > ``` > UPDATE hyperkitty_email SET content = '<new content>' WHERE > message_id_hash = '7PVJL23DLSWMXXL44ALQUO6VSYBRZ4OA'; > UPDATE hyperkitty_email SET subject = '<new subject>' WHERE > message_id_hash = '7PVJL23DLSWMXXL44ALQUO6VSYBRZ4OA'; > ``` > If the message has attachments, they are in the hyperkitty_attachment > table and can be referenced by `email_id` which is the corresponding > `id` from the hyperkitty_email entry. > > You can also do similar things via `django-admin shell`. E.g. > ``` > $ django-admin shell > Python ... > (InteractiveConsole) > >>> from hyperkitty.models.email import Email > >>> x = > Email.objects.filter(message_id_hash='7PVJL23DLSWMXXL44ALQUO6VSYBRZ4OA') > >>> print(x[0].content) > ``` > but this requires some knowledge of HyperKitty's models and Django's ORM. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

3 years, 6 months

[MM3-users] Re: Upgrade Mailmam Core from

by Odhiambo Washington

On Thu, Sep 29, 2022 at 9:18 AM <sascha.rissel(a)gmail.com> wrote: > Hi all, > > as Mark suggested, I delete the venv and started anew, making sure that I > do > > sudo su mailman > before creating the venv again. > > Executing > > pip install wheel mailman psycopg2-binary\<2.9 > led to the same error message as in my post > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > saying > > error: invalid command 'bdist_wheel' > six times, namely for: > > ERROR: Failed building wheel for mailman > > ERROR: Failed building wheel for authheaders > > ERROR: Failed building wheel for authres > > ERROR: Failed building wheel for lazr.config > > ERROR: Failed building wheel for dkimpy > > ERROR: Failed building wheel for lazr.delegates > > After this, I deleted the venv again and startet anew, again. > But this time, I explicitly did (as Stephen suggested above) > > pip install wheel > prior to > > pip install wheel mailman psycopg2-binary\<2.9 > > This time, it ran through without errors, although "wheel" is also > contained in the latter command. > Can you explain that? > > Second question: > There is still no directory "/etc/mailman3" created, where I can create > mailman.cfg in. > Does this maybe result from the fact that in first instance, I tried to > install mailman3-full via "apt-get install" and removed it again via > "apt-get remove"; with manually deleting /etc/mailman3 afterwards. > > Shall I create /etc/mailman3 manually and adding "mailman" system user > permissions? > > In advance, thanks again for your help Can you possibly blow away everything and start afresh again - using the following howto - https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10 ? And IF you get stuck, feel free to ping me even offlist. PS: I don't use Postfix and Nginx so I will not be able to assist you with those two. However, I will do my best to hand-hold you on the rest, as has been done to me on this list too. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)

2 years, 6 months

[MM3-users] Re: New Mailman Releases

by Odhiambo Washington

On Wed, Oct 26, 2022 at 8:44 AM Abhilash Raj <maxking(a)asynchronous.in> wrote: > Hi, > > I am pleased to announce several new releases for GNU Mailman project. > > 1. Mailman Core 3.3.6 [1] > 2. Postorius 1.3.7 [2] > 3. Hyperkitty 1.3.6 [3] > 4. Mailmanclient 3.3.4 [4] > 5. Django-mailman3 1.3.8 [5] > > All the releases are published to PyPI and you can find links to all the > PyPI projects: > > https://docs.mailman3.org/en/latest/ > > A full list of changelogs are available below: > > [1]: > > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.… > [2]: > https://docs.mailman3.org/projects/postorius/en/latest/news.html#section-2 > -- Something weird is going on with the page missing the bullets. > [3]: > > https://docs.mailman3.org/projects/hyperkitty/en/latest/news.html#news-1-3-6 > [4]: > > https://docs.mailman3.org/projects/mailmanclient/en/latest/src/mailmanclien… > [5]: https://gitlab.com/mailman/django-mailman3/-/blob/master/README.rst > > > Finally, the minimum Python version required is 3.7 although, 3.9 is > recommended if you are doing a new install for longest support. 3.10 is > supported for Django apps, but Mailman Core is missing that due to one > of the core dependencies not support 3.10 yet (aiosmtpd). > Hello @Abhilash Raj <maxking(a)asynchronous.in> Thank you and the whole MM3 team for all the free work you do to make us have mailing lists :) I have a question regarding the Python version. I have a very recent installation of MM3, via the Virtual Environment method, running smoothly - so far. The Python version that got installed is 3.10.6 wash@lists:~$ sudo su - mailman (venv) mailman@lists:~$ which python /opt/mailman/mm/venv/bin/python (venv) mailman@lists:~$ /opt/mailman/mm/venv/bin/python -V Python 3.10.6 (venv) mailman@lists:~$ mailma mailmail3 mailman (venv) mailman@lists:~$ mailman --version GNU Mailman 3.3.5 (Tom Sawyer) (venv) mailman@lists:~$ 1. What does the incompatibility with Python-3.10 portend for my setup and anyone else with Python-3.10? 2. Is it possible to downgrade the Python in a Venv down to 3.9 without breaking the installation? And finally, is there documentation/procedure to upgrade a VENV install in this case? Thank you in advance. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)

2 years, 5 months

[MM3-users] Re: signup / registration error - permissions and cert chains

by David Newman

On Dec 31, 2021, at 03:43, Victoriano Giralt <victoriano(a)uma.es> wrote: > > El viernes, 31 de diciembre de 2021 1:48:38 (CET) David Newman escribió: >> I'd like for regular (non-admin) list subscribers to be able to manage >> their subscription preferences and view list archives. > > That's a good way to go :-) > > My response is more of a (very) old sysadmin and Django user (since 2008) > hunch that a proper one based on code and documentation review, but I've been > trying to contribute several times and always (super) Mark Sapiro beats me :-) > >> If I'm reading the error correctly, this is related to an inability to >> verify the cert chain. The /etc/mailman3/settings.py file points to the >> same cert and key files used by Nginx, Postfix, and Dovecot. > > You are right in your diagnose but not in your interpretation (see my comment > below inside the traceback). It is certificate related, but not for server > TLS, but for CLIENT authentication. > > >> EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' >> EMAIL_HOST = 'localhost' >> EMAIL_PORT = 25 >> EMAIL_HOST_USER = 'dnewman(a)networktest.com' >> EMAIL_HOST_PASSWORD = 'wouldnt-you-like-to-know' >> EMAIL_USE_TLS = 'True' >> EMAIL_SSL_CERTFILE = '/etc/ssl/certs/myhost.crt' >> EMAIL_SSL_KEYFILE = '/etc/ssl/private/myhost.key' > > All these settings above are used for SENDING messages and, if I'm not > mistaken, the SSL key and cert are used for authenticating the user sending > the email. Actually, using TLS and SMTP Auth for localhost is a bit too much. > I've been configuring SMTP servers since 1990 and my mail servers just accept > mail form localhost, if they are broken into, the user and password have > already been exposed :-) > >> But this might only be for email, not Postorius/Django. > > You are right (if I also am) > >> What additional configuration is needed to allow regular users to create >> and manage their own accounts? > > I'd say that is more what is not needed (the SMTP TLS authentication) > > I'll remove the "noise". These are the tell tale lines: > >> "/opt/mailman/venv/lib/python3.9/site-packages/django/core/mail/backends/smt >> p.py", line 67, in open >> self.connection.starttls(keyfile=self.ssl_keyfile, >> certfile=self.ssl_certfile) > > The SMTP Django backend is trying to connect to the mail server to send the > Mailman account confirmation message and failing, probably because the user > Django runs as cannot open the private key (which is a very sensible thing if > that private key is the one used for the web facing TLS certificate, I can > tell you how bad in private or search for my name, wasd, apache and VMS ;-)) > > That certificate is not needed for sending email from Django, and, as I said, > not even SMTP Auth for sending via localhost. Actually, doing SMTP Auth on > port 25 is not even recommended practice. Hi Victoriano, Thanks for this. I could use some clarification on what specific changes you are suggesting. I *think* you are saying to remove the EMAIL_USE_TLS stuff and also move to another port (maybe 587), but I am not sure. Also, the reason I added the TLS in the first place was that I was getting errors without it. And I am unclear why the cert / private key pair do not work for Django when they do work OK for Postfix, Nginx, and Dovecot. Thanks for clarifying — and happy and safe 2022 to you as well! dn > > Happy, healthy, safe and well ventilated New Year to all. > > -- > Victoriano Giralt Innovation Director > Digital Transformation Vicerectorate University of Malaga > +34952131415 SPAIN > ================================================================== > Note: signature.asc is the electronic signature of present message > A: Yes. >> Q: Are you sure ? >>> A: Because it reverses the logical flow of conversation. >>>> Q: Why is top posting annoying in email ? > > > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

3 years, 3 months

[MM3-users] Re: Configure MM3 with separate IMAP Mailserver

by Mark Sapiro

On 1/2/23 11:18, Clemens Leider wrote: > Hi Mark, much appreciated! This is for sending messages, right? How about the recieving part…I read different things around that i need multiple mailboxes created, postfix installed, relays, ASMTPDirect etc…can you guide me a bit here, please? I havent found that one instruction that would solve this for me until now. Would just like to avoid to have to administer my own mailserver locally, if i have a perfectly configured one with my provider. CL Yes, what I posted is for outgoing mail from Mailman. Incoming mail is much more difficult. There are threads on this in the archives of this list. E.g., <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…> The incoming MTA needs to relay the mail to Mailman's LMTP server for every address - that's 9 addresses per list. An example postfix transport_maps for one list is: ``` list(a)example.com lmtp:[127.0.0.1]:8024 list-bounces(a)example.com lmtp:[127.0.0.1]:8024 list-confirm(a)example.com lmtp:[127.0.0.1]:8024 list-join(a)example.com lmtp:[127.0.0.1]:8024 list-leave(a)example.com lmtp:[127.0.0.1]:8024 list-owner(a)example.com lmtp:[127.0.0.1]:8024 list-request(a)example.com lmtp:[127.0.0.1]:8024 list-subscribe(a)example.com lmtp:[127.0.0.1]:8024 list-unsubscribe(a)example.com lmtp:[127.0.0.1]:8024 ``` This relays mail to those addresses to Mailman's LMTP server listening on the localhost, port 8024. If you could arrange for the external IMAP server to do that, that would work, but I suspect you don't have access to even try. For MM 2.1 which is different in that it doesn't use LMTP in this way, some people have used fetchmail <https://www.fetchmail.info/> to do things like this. This requires setting up mailboxes on the remote server to receive the mail and using fetchmail to retrieve that mail and relay it to Mailman. The issues with this are keeping the remote mailboxes and fetchmail in sync with Mailman as lists are created and removed. Another possibility is to have all your lists in a dedicated domain and configure the remote MTA to forward all mail to that domain to the Mailman host and then configure an MTA on the Mailman host to simply relay all mail to Mailman via LMTP. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 3 months

[MM3-users] Re: moving lists to another server

by Mark Sapiro

On 11/21/23 04:32, Odhiambo Washington wrote: > > I tested the above out of curiosity. It failed. There are 9 lists in the > setup, but I am only interested in 1. > > ``` > (venv) mailman@debian12:~$ mailman-web shell > Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] > Type 'copyright', 'credits' or 'license' for more information > IPython 8.17.2 -- An enhanced Interactive Python. Type '?' for help. > > In [1]: from hyperkitty.lib.compat import get_list_by_name > > In [2]: mlist = get_list_by_name('list_name_only', 'list_domain') You can't use literally ('list_name_only', 'list_domain'). If the list of interest is for example alist(a)example.com, the above should be mlist = get_list_by_name('alist', 'example.com') ... > Was I supposed to substitute something for the real listname of the list I > wanted? Yes. > Another thing - totally unrelated - that surprised me: > While using the commented-out block below, I could not click and get > anything beyond http://http://list.name/mailman3 as anything to do with > /postorius/xxxxx gave "Page not found" error. > However, with the second config block that uses the UNIX socket, I am able > to access everything > > ``` > #<IfModule mod_proxy.c> > # ProxyPreserveHost On > # ProxyPass "/mailman3" "http://127.0.0.1:8010/mailman3" > # ProxyPass "/archives" "http://127.0.0.1:8010/archives" > # ProxyPass "/accounts" "http://127.0.0.1:8010/accounts" > # ProxyPass "/admin" "http://127.0.0.1:8010/admin" > # ProxyPass "/user-profile" "http://127.0.0.1:8010/user-profile" > # </IfModule> > > ProxyPassMatch ^/static/ ! > ProxyPassMatch ^/favicon.ico ! > ProxyPass / unix:/opt/mailman/mm/var/uwsgi.sock|uwsgi://localhost/ > ProxyPassReverse / unix:/opt/mailman/mm/var/uwsgi.sock|uwsgi://localhost/ > ``` > > Why is it that the UNIX socket option worked flawlessly while the other > option failed??? The first option does not have ProxyPass for urls matching /postorius or /hyperkitty so such urls don't work. The second option passes everything (ProxyPass / ) to uwsgi via the socket so all urls that uwsgi can handle will work. You could do something similar with the first option with simply ProxyPass "/" "http://127.0.0.1:8010/" assuming uwsgi is listening on port 8010. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 5 months

[MM3-users] Re: HyperKitty: discussions not updating, misreporting stats

by Abhilash Raj

On Sun, Nov 12, 2017, at 04:24 PM, Cameron Smith wrote: > These are a couple of HyperKitty issues that I have talked about in > August [1]. I’ve been doing a bit more experimentation in the last few > days and am reporting my findings. > > The two issues are: > > — Missing messages: HyperKitty doesn’t show all of the messages in a discussion > - Misreporting of discussion statistics: HyperKitty doesn’t update the number of participants and comments in a discussion > > > I’m using the latest Docker setup. > > Neither of these issues are solved by updating or rebuilding indexes: > > sudo docker exec mailman-web ./manage.py update_index > sudo docker exec mailman-web ./manage.py rebuild_index > > > However, if I stop and then restart the containers (sudo docker-compose > down; sudo docker-compose up -d), the discussion statistics are updated > and properly display the number of participants and comments in all > discussions. This does not, however, solve the problem of missing > messages. This sounds like there is something wrong with how django_q tasks are run inside container images. Either sum bug in uwsgi, or some configuration error. I do remember Mark having an issue on lists.mailman3.org with django_q tasks not running properly, I wonder if it is the same issue here? I have created an issue[1] to track. [1]: https://github.com/maxking/docker-mailman/issues/196 > > > I have discovered two methods of displaying the missing discussion > messages: > > add a new message to the thread in HyperKitty; I’m not convinced that this works *every* time, but it at least works *most* times > > run the yearly cron job; this seems to work *every* time: > > sudo docker exec mailman-web ./manage.py runjobs yearly I am not sure I have seen this before, but it is possible other MM3 admins might have. Hyperkitty itself doesn't have any yearly jobs though, maybe some dependency does. > > > [1]: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… > > > Cameron Smith > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ -- Abhilash Raj maxking(a)asynchronous.in

7 years, 5 months

[MM3-users] Re: Splitting hosts: mta&core vs postorius

by Mark Sapiro

On 6/20/20 1:37 PM, Ruth Ivimey-Cook wrote: > Thanks Mark, Ignore bracketed numbers below ([1], [2], etc.) They come from conversion of the HTML only post to plain text. > On 20/06/2020 03:53, Mark Sapiro wrote: > > What are your Django settings for MAILMAN_REST_API_URL, > MAILMAN_REST_API_USER and MAILMAN_REST_API_PASS and do these agree with > the [webservice] settings for hostname, port, use_https, admin_user and > admin_pass in mailman.cfg in core and is the port open to the outside on > the core machine? > > greyarea-post is the mailman-core and exim server. > > greyarea-web1 is the postorius and hyperkitty server. > > On greyarea-web1, in django-settings.py, > > MAILMAN_REST_API_URL = '[1]http://greyarea-post:8870' > MAILMAN_REST_API_USER = 'restadmin' > MAILMAN_REST_API_PASS = XXXX > > and both the above-named hosts are also included in ALLOWED_HOSTS along > with localhost. > > On greyarea-post, in mailman.cfg: > > [webservice] > admin_pass: XXXXX > admin_user: restadmin > api_version: 3.1 > hostname: 0.0.0.0 > port: 8870 > use_https: no > > The password and username are as on web1. That all looks good. > Do you see REST requests from the Postorius machine in core's > mailman.log and access.log? > > Yes, there are lots of requests of the form: > > [07/Jun/2020:15:00:01 +0000] "GET /3.1/lists?count=10&page=1 > HTTP/1.1" 200 470 "-" "GNU Mailman REST client v3.3.1" > > I made a new list using the core CLI, and as a result of navigating > directly to the list archive page I get to see: > > [06/Jun/2020:23:00:02 +0000] "GET /3.1/lists/chbcdaily(a)ch-bc.org.uk > HTTP/1.1" 200 365 "-" "GNU Mailman REST client v3.3.1" > > ... but in neither case do I get a complete screen on the browser or > 'follow-up' requests. > > I don't have a file access.log on the greyarea-post. Yes, that's actually a gunicorn log so would only be on greyarea-web1, but the mailman.log tells us the requests are getting to the core REST API, so we don't need that info. > On the greyarea-web1 I have mailmansuite.log with the last few lines: > > File > "/opt/mailman3/lib/python3.6/site-packages/mailmanclient/restbase/ba > se.py", line 74, in rest_data > response, content = self._connection.call(self._url) > File > "/opt/mailman3/lib/python3.6/site-packages/mailmanclient/restbase/co > nnection.py", line 124, in call > 'Could not connect to Mailman API: ', repr(e)) > mailmanclient.restbase.connection.MailmanConnectionError: ('Could > not connect to Mailman API: ', > 'ConnectionError(MaxRetryError("HTTPConnectionPool(host=\'0.0.0.0\', > port=8870): Max retries exceeded with url: /3.1/domains/ch-bc.org.uk > (Caused by NewConnectionError(\'<urllib3.connection.HTTPConnection > object at 0x7f5416db8518>: Failed to establish a new connection: > [Errno 111] Connection refused\',))",),)') > ERROR 2020-06-03 01:02:06,115 29002 django.request Service > Unavailable: /postorius/lists/ OK. This is telling us that mailmanclient (the REST API Python bindings that Postorius uses to access core) is having trouble establishing a connestion to the core REST server, but it clearly gets there some of the time based on the mailman.log entries. > the uwsgi-error_*.log contains many lines of the form: > > api: url is > [2]http://greyarea-post.cam.ivimey.org:8870/3.1/lists?count=10&page= > 1 > api: url is > [3]http://greyarea-post.cam.ivimey.org:8870/3.1/lists?count=10&page= > 1 > > My reason for thinking, in my original post, that the wrong host > address is being used is that the ConnectionError above contains > 0.0.0.0 in its url for /3.1/domains/... but 0.0.0.0 is only present as > the listen address on the core web service and does not appear in the > config for postorius. The ConnectionError above comes from requests.request (docs at <https://requests.readthedocs.io/en/master/>). Client only provides a URL in the call to request and the URL is the baseurl + the path and the baseurl is MAILMAN_REST_API_URL which you have as http://greyarea-post:8870. Entries above have http://greyarea-post.cam.ivimey.org:8870/. Is that the correct FQDN? I suspect a DNS lookup issue. How does greyarea-web1 resolve greyarea-post? You could try a FQDN that greyarea-web1 can lookup in DNS rather than relying on /etc/resolv.conf and/or /etc/hosts. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 10 months

[MM3-users] Re: reset bounce-scores?

by Mark Sapiro

On 10/3/24 01:15, IEM Network Operation Center (IOhannes m zmölnig) wrote: > i'm running a mailman-3.3.8 instance, and would like to reset the > bounce-score for a (number of) user(s). > i searched the internet and this list's archives, but haven't found > anything useful so far. > > due to a misconfiguration of our universities reverse DNS lookup, the > bounce score of some users has rocketed in the last days - some users > have already been unsubscribed because of that. > > could anybody please share how i can reset the bounce score? First of all, there are 3 categories of affected list members - those who are already unsubscribed, those who are still members but have delivery disabled by bounce, and those with delivery enabled but with bounce score > 0. For those that have been unsubscribed, you just have to re-subscribe them. For those that have delivery enabled but a non-zero bounce score, you may not need to do anything because they are presumably not currently bouncing so eventually the old bounce info will become stale and be ignored, but to guard against spurious bounces that may result in disabling, e.g.from an over aggressive user's spam filter, you can add `verp_probes: yes` to the `[mta]` section of mailman.cfg. The effect of this is when a members bounce score reaches threshold the score is reset and a special probe message is sent and delivery is disabled only if the probe bounces. See https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/model/docs… That leaves those with delivery disabled by bounce. Assuming you want to enable delivery for all members of all lists with delivery disabled by bounce, the following `mailman shell` interaction will do it. ``` $ bin/mailman shell Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. >>> for mlist in getUtility(IListManager): ... for member in mlist.members.members: ... if member.delivery_status == DeliveryStatus.by_bounces: ... member.preferences.delivery_status = DeliveryStatus.enabled ... >>> commit() >>> ``` Watch out for a possible wrapped line above. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 months, 2 weeks

[MM3-users] Re: Restore/export existing Mailman lists + hyperkitty archives on a new server

by Stephen J. Turnbull

Kimmo L writes: > Here is the list of processes. That's an unfamiliar format to me. Is this a Linux system running systemd? > Not sure yet, which stores the most memory in the cache. For > example at the moment i have 6GB ram - 250mb free and 3gb cached. "Cache" in the context of Linux kernel operation normally means "data file contents we keep around for possible future use, but if you need memory we will evict some stuff and give you some memory". It's not memory committed to running processes. So a couple hundred MB of memory with lots of cache is to be expected after a Linux system has been running for a while. So if you have 6GB RAM and 3GB cached, that's consistent with a machine that's running Mailman, using about 2.5 GB of RAM and not much else is running except the kernel. This report does not indicate a machine on the edge of OOM-killing processes, and it's quite consistent with Mark's experience that the whole Mailman suite runs comfortably in 4 GB. How much swap do you have, and how much is in use? What other daemons are running on the host besides Mailman? I guess you must have an MTA (Postfix, Exim) running and filtering software, as well as a front-end webserver (you shouldn't be running gnuicorn directly on the web). Systemd, dbus, I guess. How much memory are they using? Have you modified any web pages associated with Mailman, including any of the Django templates? Are there any pages with unusually long load times? I checked, and neither of the errors that you report for the uwsgi configuration ("OSError: write error" and "django.request Internal Server Error") seems to be related to a memory problem, but rather to difficulties serving certain resources, typically non-existent ones, or a user leaning on the "refresh" function in their browser. (Information from checking a few StackOverflow articles.) By the way, Mark is going to be mostly unavailable for about 4 weeks. I'll do what I can to help, but Mark is probably a lot more expert on this than I am, so bear with me. Steve

2 years, 8 months

[MM3-users] Re: Archives visible to all

by worik

tl;dr location ~ ^/hyperkitty { # Turn off archiving for Mailman3 allow 1.2.3.4; deny all; } There should be a better way (https://gitlab.com/mailman/hyperkitty/issues/219) Problem fixed, but at he cost of no archives and a broken "archives" link. On 22/02/19 9:03 PM, Johannes Winter wrote: > Am Thu, 21 Feb 2019 12:23:41 +1300 > schrieb worik <root(a)worik.org>: > >> When I load my new mailing list system, before I log in, I can see the >> archives of all the lists. Without logging in. Even lists with >> archives set to private archives. >> >> What can I do about that? > Are you sure that you really can see the list contents? In my case only > the listnames are shown, which is not nice, but bearable. Yes, that is true. Since I made my first post on this thread (which was a bit premature) I have researched this further. All messages posted before the archives are made private are displayed for all to ee, and I can find no ay of changing that except by making hyperkitty a dead link, as above. Displaying all lists on the server is unacceptable for my users. My users are involved in political activism and displaying the names of all the lists is a political risk. (As distinct from privacy or security risks). > > There is an open issue about that: > https://gitlab.com/mailman/hyperkitty/issues/197 > > Would be happy for workaround ideas, too. Yes. It has been open for half a year, https://gitlab.com/mailman/hyperkitty/issues/129 for a year. The problem is that Hyperkitty does not "...use the list's |advertised| setting to show or hide it...". IMO this is also a bug in Mailman core. Clearly Archivers need to respect changes in the statius of a list and change the status of archives already collected (not a trivial matter) but Mailman should not hand over the names of lists where archives are turned off. As I write I realise I have not tested creating a list with archives off from the get go... Worik > > > Johannes Winter > -- If not me then who? If not now then when? If not here then where? So, here I stand, I can do no other root(a)worik.org 021-1680650, (03) 4821804 Aotearoa (New Zealand)

6 years, 1 month

[MM3-users] Re: Error: ConnectionRefusedError at /accounts/login/

by Stephen J. Turnbull

Kim Sunggun via Mailman-users writes: > Have you set any of those to other than default in your Django > settings? > everything is default. What we're doing is not relevant because we're not trying to connect to your MTA. Some of us do because our MTAs expect authentication or are running on a different port for some reason. Many of us don't because our MTAs use the defaults too, at least for local connections. The question *you* need to figure out is why won't your MTA talk to Django *at all*? Any number of reasons, in approximate order of when they arise in the connection process: 0. You can't get there from here because of a firewall (very unlikely on localhost). 1. The MTA isn't listening on that port at that address. The port might work at a different address, another port might work at that address, or maybe both are different. As Mark recommended, check the Mailman configuration at [mta]. If it says anything about smtp_host or smtp_port, use those in your Django configuration too. If it doesn't, they default to 127.0.0.1:25 or localhost:25, and the Django defaults should be OK. 2. The MTA is listening for IPv6 (eg, [::1]). If you specify "localhost", maybe it automatically tries IPv6, but maybe it doesn't. If Django is assuming 127.0.0.1, it won't try [::1]. 3. The MTA expects mandatory TLS (usually port = 465). 4. The MTA expects negotiated TLS (STARTTLS), mostly on port 589 but some sites require it on port 25. 5. The MTA expects authentication in TLS initialization (don't ask me, I've never set this up). I don't recall whether generic SASL applies here, although I know you can use X.509 certificates. (SASL is the authentication protocol that most MTAs use *after* the connection is established, but you aren't getting that far.) I wouldn't be surprised if there are other possible issues, but that's enough for now. ;-) Your MTA logs should show that there was a connection attempt at the right time from the right host (probably localhost), and give the point at which the connection failed and maybe a reason why. If there's nothing at all in the MTA log, the problem is #1 above.

1 month, 3 weeks

[MM3-users] MM3 Docker

by Odhiambo Washington

[Requesting everyone to be patient with me because I am not so good with Linux, Docker is a completely new thing to me, and MM3, well....] So, I have a VM running Ubuntu-18.04. It is in a Data Centre somewhere. It has an IP n.n.n.n (a public IP). Also, it does not have high resources. However, I hope I can at least get to install MM3 and test it using https://asynchronous.in/docker-mailman/ If I could run MM3 using the apt-get install method, perhaps that would have made my life a bit easier, hoping that MM3 can actually be run in parallel with MM2.1 on the same host. Is anyone doing that already?? Anyway, I need some hand-holding using the docker method for now. I am reading the steps as documented on the https://asynchronous.in/docker-mailman/ and somethings aren't clear to me: I have created two files: (a) /opt/mailman/web/settings_local.py: SERVE_FROM_DOMAIN: 'mm3.my.co.ke' ALLOWED_HOSTS: 'my.co.ke' HYPERKITTY_API_KEY: 'Whatever' MAILMAN_ADMIN_USER: 'admin' MAILMAN_ADMIN_EMAIL: 'odhiambo(a)gmail.com' SECRET_KEY: 'Whatever' EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' EMAIL_HOST = '172.19.199.1' EMAIL_PORT = 25 (b) /opt/mailman/core/mailman-extra.cfg HYPERKITTY_API_KEY: 'Whatever' DATABASE_URL: 'mysql+pymysql://mailman:mailmanpass@database/mailmandb' DATABASE_TYPE: 'mysql' DATABASE_CLASS: 'mailman.database.mysql.MySQLDatabase' Questions: Are the formats correct? Do I use quotes or not? My docker-compuse.yaml: version: '2' services: mailman-core: image: maxking/mailman-core:latest container_name: mailman-core hostname: mailman-core volumes: - /opt/mailman/core:/opt/mailman/ links: - database:database depends_on: - database environment: - DATABASE_URL=mysql+pymysql://mailman:mailmanpass@database/mailmandb - DATABASE_TYPE=mysql - DATABASE_CLASS=mailman.database.mysql.MySQLDatabase - HYPERKITTY_API_KEY=someapikey networks: mailman: ipv4_address: 172.19.199.2 mailman-web: image: maxking/mailman-web:latest container_name: mailman-web hostname: mailman-web depends_on: - database links: - mailman-core:mailman-core - database:database volumes: - /opt/mailman/web:/opt/mailman-web-data environment: - DATABASE_URL=mysql://mailman:mailmanpass@database/mailmandb - DATABASE_TYPE=mysql - HYPERKITTY_API_KEY=someapikey - SECRET_KEY=thisisaverysecretkey - DYLD_LIBRARY_PATH=/usr/local/mysql/lib/ networks: mailman: ipv4_address: 172.19.199.3 database: environment: MYSQL_DATABASE: mailmandb MYSQL_USER: mailman MYSQL_PASSWORD: mailmanpass MYSQL_RANDOM_ROOT_PASSWORD: "yes" restart: always image: mariadb:10.3 volumes: - /opt/mailman/database:/var/lib/mysql After all that, I have run the following: wash@lists:~/docker-mailman$ docker-compose up -d Creating network "docker-mailman_mailman" with driver "bridge" Creating docker-mailman_database_1 ... done Creating mailman-core ... done Recreating 4b7bc1ecef65_mailman-web ... done Question - Should I assume everything is okay and proceed to the next steps? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

6 years, 3 months

[MM3-users] Re: Archive search failing on one list

by Mark

On Sat, 2023-09-23 at 20:28 -0700, Mark Sapiro wrote: > > What if you do something like the following where django-admin is > whatever command you use for the Django admin command. > ``` > $ django-admin shell > Python 3.9.16 (main, Dec 11 2022, 12:49:23) > [GCC 9.4.0] on linux > Type "help", "copyright", "credits" or "license" for more > information. > (InteractiveConsole) > >>> from hyperkitty.models.mailinglist import MailingList > >>> mlist_name = 'listname(a)domain.com' > >>> mlist = MailingList.objects.get(name=mlist_name) > >>> mlist > <MailingList listname(a)domain.com (listname.domain.com)> > >>> > ``` Hi Mark, I'm still on this problem of one list's archive-search not returning any results. I created a new list today and it also has the same problem. In short, the older lists all seem fine -- their search results are up- to-date so that tells me the cron is okay. It's just these 2 newer lists that have the problem. It's taken me while to get my head around the Django admin command you have suggested, as I kept getting an error when starting: #django-admin shell >>> from hyperkitty.models.mailinglist import MailingList ... django.core.exceptions.ImproperlyConfigured ... That sent me down a rabbit hole which I haven't emerged from yet. But instead, starting with # python3 manage.py shell ... # python3 manage.py shell Python 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110] on linux Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>> from hyperkitty.models.mailinglist import MailingList >>> mlist_name = 'thepost(a)example.com' >>> mlist = MailingList.objects.get(name=mlist_name) >>> mlist <MailingList: <MailingList thepost(a)example.com>> >>> The format of my final line is different to yours ... Mine: <MailingList: <MailingList thepost(a)example.com>> Yours: <MailingList listname(a)domain.com (listname.domain.com)> Does this (missing list_id) give any clue as to why the archive-search isn't working, or is just that starting with "# python3 manage.py shell" is the wrong thing to do? By the way, the list_id shows correctly in the SQL query result. select * from hyperkitty_mailinglist where name like 'thepost(a)example.com'; ... name | thepost(a)example.com | list_id | thepost.example.com ... Any ideas appreciated. Mark

1 year, 6 months

[MM3-users] Re: MM3 Postfix FROM header issue

by eboltz＠lhtservices.com

Mark, Thanks for the assistance thus far. When I create a new Domain within Postorius. If I set the Mail Host to 'example.com' and the Alias Domain to 'lists.example.com', create a new list such as 'staff_test', it'll default to the 'example.com' domain (which is the real domain). Then when I subscribe, I'll get an email from 'staff_test(a)example.com', when I want it to be from 'staff_test(a)lists.example.com' instead, also if I reply back to the 'staff_test(a)example.com' then I"ll get a bounce-back that it doesn't exist, but if I send to 'staff_test(a)lists.example.com' it'll go to the newly created list. I'd like all defaults to be '@lists.example.com', if that makes sense. Here are the settings of my postfix_lmtp and postfix_vmap # AUTOMATICALLY GENERATED BY MAILMAN ON 2023-12-17 20:05:34 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. # Aliases which are visible only in the @lists.example.com domain. staff_test(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-bounces(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-confirm(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-join(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-leave(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-owner(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-request(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-subscribe(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-unsubscribe(a)lists.example.com lmtp:[127.0.0.1]:8024 # AUTOMATICALLY GENERATED BY MAILMAN ON 2023-12-17 20:05:34 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. # Virtual mappings for the @example.com domain. staff_test(a)example.com staff_test(a)lists.example.com staff_test-bounces(a)example.com staff_test-bounces(a)lists.example.com staff_test-confirm(a)example.com staff_test-confirm(a)lists.example.com staff_test-join(a)example.com staff_test-join(a)lists.example.com staff_test-leave(a)example.com staff_test-leave(a)lists.example.com staff_test-owner(a)example.com staff_test-owner(a)lists.example.com staff_test-request(a)example.com staff_test-request(a)lists.example.com staff_test-subscribe(a)example.com staff_test-subscribe(a)lists.example.com staff_test-unsubscribe(a)example.com staff_test-unsubscribe(a)lists.example.com

1 year, 4 months

[MM3-users] Re: Single-Sign-out for List member

by Abhilash Raj

On Thu, Aug 13, 2020, at 10:04 AM, Mark Sapiro wrote: > On 8/13/20 6:21 AM, w.schoener(a)alp.dillingen.de wrote: > > > > So I would like to ad a link in the footer of every message, with which every receiver/member can sign out of the newsletter with one click. > > It does not seam to be possible, to put this into effect with mailman 3. > > Does anybody have any hints, how to create a link, and when this link is called, the user is signed out of the newsletter. > > > You can set the list's unsubscription_policy to open and set the link as > a mailto: to the list-leave address, but that still requires a second > click to send the mail. > > The solution is to create a web based CGI script which is called with a > URL like > <https://example.com/unsub_script/list.example.com/user@example.net> and > have that script invoke the appropriate `mailman` command to remove the > member from the list. Beginning with Mailman 3.3.2, this would be > > mailman delmembers -l list.example.com -m user(a)example.net -g > > to remove the member and send a goodbye message. In older Mailman, it > would be something like > > echo user(a)example.net | mailman members -x - list.example.com > > I understand you may have a legal requirement do do this, but these > "one-click - no user confirmation" links are a bad idea because usera > will reply to a list post or forward it and not remove the link and > userb will receive usera's reply and click usera's unsubscribe link > either maliciously or thinking it will unsubscribe userb. There is also the issue that these days some Mail clients will try to create a preview of all the URLs if they can and it would inadvertently GET the URL resulting in the un-subscription. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

4 years, 8 months

[MM3-users] Re: Database running VERY hot

by Darren Smith

Abhilash, Thanks for the response. I'm going over to lists.fedoraproject.org to see if they have done any fine tuning. Our installation is at https://lists.rootsweb.ancestry.com, if you're interested in seeing an installation with over 32,000 mailing lists. As I mentioned, it was big enough that we have disabled the browse experience in Postorius and are hosting our own pages for browsing to a new list. If we get time to work on this we will certainly create a pull request. :) On Thu, Mar 8, 2018 at 11:49 AM, Abhilash Raj <maxking(a)asynchronous.in> wrote: > Hi Darren, > > > On 3/8/18 9:43 AM, Darren Smith wrote: > >> Hello all, >> >> We had an old installation of Mailman 2.1 across three servers with over >> 32,000 mailing lists on them. Due to circumstances beyond my control, we >> had to shut them down and quickly set up Mailman 3 on a new server. >> >> We have Mailman3 set up on a single server this time, as we were having >> problems with a 3-server setup and Hyperkitty and archives. Long story >> short, it is up and running, and our users are starting to come back. >> >> However, we do not have very many users hitting Postorius and Hyperkitty >> yet, and we are seeing a distressing number of 504 gateway timeouts. >> >> We noticed that there is a significant number of queries to the API >> whenever hitting Postorius's front page to load the mailing lists, and >> then >> on every load. It looks as if it makes a single call to the API to get >> information about a single list, and does this 10 times - once for each >> list - to render the page. This gets much worse when users set the view to >> 200 per page, as they are inclined to do with 32,000 mailing lists... >> >> What we've done, actually, is to override the front page and point the >> users to a different set of static pages where we have all of our lists >> categorized and searchable, and when they find the list they are looking >> for, we have a link back to postorius. It has taken a lot of the stress >> off >> of the server. >> >> However, I still see where the database at times can run VERY hot. >> >> We have set up this server to run with PostgreSQL. When I do a ps aux | >> grep postgres, I find a lot of processes that have been spawned to take >> care of mailman - a dozen or so. However, one of the threads is taking up >> about 85% of the load, another thread about 14%, and little dribbles here >> and there from the rest of the threads. >> >> Here are the questions: >> >> 1) Has anyone run into problems like these? I would think that maybe this >> is an issue having so many mailing lists, but honestly there aren't that >> many people using the system yet. It is going to ramp up as the users >> realize that we are live again, but we aren't getting that much traffic. >> > > I can say that in general, Postorius doesn't really optimize REST API > queries and so loading a single page can sometimes make things bad. > > Currently, the largest installation of Mailman 3 that we know of runs at > lists.fedoraproejct.org and the administrators of the list would know how > well does our Web UI scale. > > However, I don't think we have anyone actively working on trying to fix > this particular issue. Pull Requests and fixes are welcome :) > > >> 2) Is there any configuration that I can set, especially with regards to >> the database, so that it can be used more efficiently and not throttle >> through one or two threads? >> > > There isn't any configuration that would change how databases are used in > Mailman Core. Django *may* have some, but I don't know about any of them of > the top of my head. > > thanks, > Abhilash >

7 years, 1 month

[MM3-users] Re: Hyperkitty 1.1.1 to HyperKitty 1.2.1

by Abhilash Raj

Sorry for the top post! You don't have to do the export import thing, you just need to delete the index for fulltext search, which is typically in a directory alongside your settings.py called "fulltext_index" ( not 100% on the name of directory). After that you need to build your index (python manag.py rebuild_index) for the search to work. thanks, Abhilash On Sun, Sep 30, 2018, at 11:42 PM, Thomas G wrote: > Hi, > > I would like to migrate my current Mailman3.1/Hyperkitty data to a fresh > install in Mailman3.2. > > I didn't found how migrate the archives. > From : HyperKitty - Version: 1.1.1 - Python2.7 > To : HyperKitty - Version: 1.2.1 - Python3.5 > > I exported a mbox from hyperkitty (web interface) and tried to re-import > it, without result. > """""""""""""""""""""""""""""""""""""""" > xxxx @xxxxxx :/opt/mailman# python3.5 manage.py hyperkitty_import -l > listedetests(a)hatlab.fr /home/admin/listedetests(a)hatlab.fr.mbox.gz > > Importing from mbox file /home/admin/listedetests(a)hatlab.fr.mbox.gz to > listedetests(a)hatlab.fr > Computing thread structure > Synchronizing properties with Mailman > Warming up cache > The full-text search index is not updated for this list. It will not be > updated by the 'minutely' incremental update job. To update the index > for this list, run the 'manage.py update_index_one_list > listedetests(a)hatlab.fr' command. > """""""""""""""""""""""""""""""""""""" > The database stay empty (and web interface): > """""""""""""""""""""""""""""""""""""" > mailmandbnameprod=# select * from hyperkitty_email; > id | message_id | message_id_hash | subject | content | date | timezone > | in_reply_to | archived_date | thread_depth | thread_order | > mailinglist_id | parent_id | sender_id | thread_id | sender_name > ----+------------+-----------------+---------+---------+------ > +----------+-------------+---------------+--------------+-------------- > +----------------+-----------+-----------+-----------+------------- > (0 rows) > """""""""""""""""""""""""""""""""""""" > > Any idea to do this importation ? > > Thanks ! > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ -- thanks, Abhilash Raj (maxking)

6 years, 6 months

[MM3-users] Re: Prototype import [was: Re: Setup Question: Disk-Layout (Migrating from Mailman 2.1)]

by Mark Sapiro

On 4/19/25 1:55 PM, Mihai Moldovan wrote: > > This eventually led me to realize that passing > mailman.email.message.Message as the factory to mbox() internally calls > factory(msg), and the base class of Message (email.message.Message) has > an __init__ function that takes the policy parameter, so it looks as if > it's registering the mailbox message incorrectly as the policy handler, > which is totally wrong of course. This appears to be a bug in mailbox.mbox. To work around it, try this: ``` from email import message_from_bytes mb = mbox(sys.argv[2], create=False) for key in mb.iterkeys(): msg = message_from_bytes(mb.get_bytes(key), Message) ``` > and indeed, if I let the import actually happen, it works. Still, it's probably better if the message is a mailman.email.message.Message object rather than an email.message.Message object. > The imported messages, do have a Message-ID-Hash, but the Archived-At > and List-Archive headers are empty (literally <>). Those headers are added by the rfc_2369 handler based on the archivers configured for the list. They will never point to the prototype archiver or the message in the prototype archive. Normally they will point to HyperKitty and to the archived message in HyperKitty. This is the case even for messages in the prototype archiver as those messages are just the message as delivered to the list members absent any personalization. > Do you have an example message that was archived through Prototype? What > are the proper header values for this module? I believe that Archived-At > should be the Message-ID-Hash and the List-Archive header contain... > well, given that the Prototype archiver is not meant to be publicly > available, probably a file:// URL? Typically they will be something like List-Archive: <https://example.com/archives/list/listname@example.com/> Archived-At: <https://example.com/archives/list/listname@example.com/message/xxxxxx/> where xxxxxx is the message-id hash value. Note that the generation of these headers by the rfc_2369 handler depends on this script being run in an installation that has hyperkitty and mailman-hyperkitty installed and the hyperkitty archiver enabled for the list. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 days, 9 hours

[MM3-users] hosted signup is producing unexpected results

by tokyoprogressive＠mac.com

In my test of the two forms of signup, I am getting these unexpected results. Is it my settings or is something weird within the program itself? 1) https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres… 2) Choose "You can also subscribe without creating an account. If you wish to do so, please use the form below" 3) Then I have to approve it. They get "Welcome to the "Discuss" mailing list! To post to this list, send your email to: discuss(a)list.tokyoprogressive.org 4) But I have to approve it.I assume this is because they do not yet have a real account. And they should be listed as a NON MEMBER? 5) But even when they sign up here: https://list.tokyoprogressive.org/accounts/signup/?next=%2Fpostorius%2Flist… they get an email from my provider (NOT MY LIST????) as follows: "Hello from mailman3.emwd.com! You're receiving this e-mail because user iCloud has given yours as an e-mail address to connect their account. To confirm this is correct, go to https://list.tokyoprogressive.org/accounts/confirm-email/MTE3:1iA44k:E7w7xh… Thank you from mailman3.emwd.com! mailman3.emwd.com" 6) And this is where it gets weird. If they send from their email program and get the welcome (Welcome to the "Discuss" mailing list! To post to this list, send your email to: discuss(a)list.tokyoprogressive.org You can unsubscribe or make adjustments to your options via email by sending a message to....... I then get a notice: "Your mail to 'discuss(a)list.tokyoprogressive.org' with the subject test notice Is being held until the list moderator can review it for approval. The message is being held because: The message is not from a list member" 7) While if they send FROM THE interface it goes through. 8) What I am confused about. Signing up for an account should make them a member, right? Subscribing should make them a non member, no? But here it seems that both make them a non member. And worse, someone with an account sending from the Interface has their message get through while the same person sending from an email program has me get a notice that a non member is trying to post. Before I can move my actual members over from another mailing list provider, I want to make sure what works and what doesn't. Thanks. paul!!!

5 years, 7 months

[MM3-users] Re: Error on timeout vs retry values

by Robert Moody

Thank you, that's fixed it. On 19/07/21 7:28 am, Abhilash Raj wrote: > > On Sat, Jul 17, 2021, at 11:23 PM, Robert Moody wrote: >> Hi everyone, >> >> I have recently setup a new mailman3 service using the venv method. >> Everything is functionally working however I keep getting an error sent >> to the mailman email every min which I can not seem to figure out why. >> >> /opt/mailman/venv/lib/python3.7/site-packages/django_q/conf.py:142: >> UserWarning: Retry and timeout are misconfigured. Set retry larger than >> timeout, >> failure to do so will cause the tasks to be retriggered before >> completion. >> See >> https://django-q.readthedocs.io/en/latest/configure.html#retry for details. >> See https://django-q.readthedocs.io/en/latest/configure.html#retry >> for details.""" >> >> I have edited the timeout and retry values in the >> /opt/mailman/venv/lib/python3.7/site-packages/django_q/conf.py file and >> restarted both mailman and mailman-web however this still keeps repeating. > You shouldn't edit files in `site-packages` directory as they are part > of the package itself and could cause issues. > > What you want to do is override it in your settings.py (or settings_local.py) file > for Django. I am not sure which guide you followed to install Mailman, but it > should be somewhere in /opt/mailman/ I suppose (not in venv/ for sure). > > It should look something like this > > https://mailman-web.readthedocs.io/en/latest/settings.html#mailman_web.sett… > > Just the value of retry should be just > than timeout as the error says, so > not necessarily same values, but so far 300 and 360 has been working > good for us and I haven't seen any report to increase them. > > >> Any ideas? >> >> Regards, >> >> Robert >> >> >> -- >> This message has been scanned for viruses and >> dangerous content by MailScanner, and is >> believed to be clean. >> >> _______________________________________________ >> Mailman-users mailing list -- mailman-users(a)mailman3.org >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> >

3 years, 9 months

[MM3-users] Easiest way to install a new mailman3 deployment?

by ieso＠johnwillson.com

So I've been trying for the last two weeks to get a new mailman3 server running on a virtualized server (any server), and I'm turning to this list after having failed many times and running out of holiday time. I started trying a non-docker installation on Ubuntu 18.04 (https://docs.google.com/document/d/1xIcSsoNFp2nHi7r4eQys00s9a0k2sHhu1V5Plan…) , which got me the closest. Except I had a problem with inbound email only being triggered when it came from certain accounts. But that clearly wasn't good enough for production, so after many attempts to figure out where it was failing, I decided to turn to docker as a solution that should be cleaner. A few attempts at doing a docker installation on digitalocean.com failed, which I realized might be due to it not routing private IP addresses, so I moved to AWS after checking that their VPC policy would fit mailman's docker requirements. I found a great but slightly outdated guide on how to do this (https://xiaoxing.us/2018/01/01/deploy-mailman-3-on-aws-using-docker/) By this point I knew enough to correct a number of places where the environment had changed since the procedure was written, but postorius still failed at the curl test. The challenge for me has been the difficulty to know how to troubleshoot the different different systems and network infrastructure that are used to keep mailman3 humming. I've tried about 7 different installation walkthroughs (there are no recent ones on Youtube by the way, in case anyone wants to seize that opportunity!), and the good guides provide ways to check each stage to try to help you a bit on that front. Nonetheless, I feel stuck and thought I'd ask the simple question... for a completely basic, barebones new installation, what's the easiest way to get a mailman3 installation up-and running? (e.g. Which server provider? Which operating system and version? Docker or otherwise?) Any pointers highly appreciated. Google Groups is clearly on its way out, as it no longer allows for people to easily join groups by sending an email or clicking a link, so that should be a big opportunity for mailman3 to step up and help give those mailing list migrants a new home... which is what we're looking for. We're just not quite as smart as you guys. ;-)

4 years, 3 months

[MM3-users] Ann: Mailman Core 3.3.5rc1 is out

by Abhilash Raj

Hey Everyone, I just tagged and released Mailman Core 3.3.5rc1 release. This is identical to 3.3.5b1 in code, the only changes that have been made are in utility scripts and testing configuration. You can install this version using: $ pip install --pre mailman==3.3.5rc1 If you want to download a release tarball, you can do so from here: https://pypi.org/project/mailman/3.3.5rc1/#files This is the first (and only) planned release candidate for 3.3.5 release. The stable release is planned for next week. thanks, Abhilash On 9/7/21 12:39 AM, Abhilash Raj wrote: > Hello Everyone, > > > I am pleased to announce that Mailman Core 3.3.5b1 is now out. It is a > pre-release for 3.3.5, which is slated to come out in 3 weeks from now. > > I am planning for a 2 week beta period, after which I'll release the > first RC and then stable a week after. Right now I am not planning to > release a second beta version, but if there are several changes in Core > in the next week or so, then I might. > > This release includes a lot of bug fixes and some new features. It also > includes a security enhancement that improves the authentication of REST > API by adding resistance to some timing channel attacks. > > A full list of changes in this version can be found here[1]. > > [1]: > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.… > > > We have also made improvements in the i18n workflow and now the email > templates are updated from the translated messages in the .po files that > we get from Weblate. Documentation for translators is available here[2] > if you want to help translate Mailman into your native language. > > [2]: https://docs.mailman3.org/en/latest/translation.html > > This is a pre-release version, if you want to test it with your setup, > you can install it using: > > $ pip install --pre mailman==3.3.5b1 > > If you want to download a release tarball, you can do so from here: > > https://pypi.org/project/mailman/3.3.5b1/#files > > > This release received a lot of contributions from community, which I am > really happy about! I'd like to thank everyone who made this release > possible! > -- thanks, Abhilash Raj (maxking)

3 years, 7 months

[MM3-users] Re: Speed/progress of update_index_one_list?

by Philip Colmer

We've now completed the indexing of our lists, some of which are quite large (100K+ messages per list). I wanted to share our findings of using Xapian to do this. Firstly, using "-v 2", e.g. "mailman-web update_index_one_list -v 2 <email address>", causes the command to print out the progress being made in batches of 1,000 messages. Secondly, Xapian is single-threaded. It only allows one writer at a time. Therefore, we focussed on speed of the hard drives and, on the advice received from the Xapian list, memory. Since we were using AWS EC2 with EBS storage, I elected to move the EBS storage from gp2 to gp3 - which has a higher base I/O figure - and I selected a r6i.8xlarge as this guaranteed 10Gb throughput to the EBS storage. It also delivered 32 vCPU and 256GB RAM. It was possibly overkill but it did the job (see below) and we've now switched back to a t3a.xlarge. Speed-wise, with the above configuration, the system indexed a list of nearly 85K messages in 46 minutes. I hope that is helpful information to anyone else who finds themselves migrating large Mailman 2 archives. Regards Philip On Fri, 14 Jan 2022 at 17:33, Mark Sapiro <mark(a)msapiro.net> wrote: > On 1/14/22 12:38 AM, Philip Colmer wrote: > > I have now switched to Xapian but, in a way, my original questions still > > stand. Is there a way of monitoring the progress of > > "update_index_one_list"? What can I do to the specification of the server > > to make that process go (much) faster? > > > > HyperKitty does the indexing by calling the Haystack update_index > command. See > > https://django-haystack.readthedocs.io/en/master/management_commands.html#u… > > This command has options such as --verbosity and --workers. I think > verbosity is set from the option provided to update_index_one_list and > if set to 2 will give some progress info. > > workers could increase parallelism but is unconditionally passed by > hyperkitty as 0 > > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

3 years, 3 months

[MM3-users] Do not display any lists on index the page (/postorius/lists/) by default (when you create the list it defaults to Hide) - Debian package install

by daniel.botting＠codethink.co.uk

Hi all, I am currently in the process of moving from mailman 2.1 to mailman 3, the basic functions are completed (mailing list and archiving). On our current mailman 2.1 install it advises there are no publicly advertised lists, when I visit mailman3-list.domain/postorius/lists/ I see all the lists that have been created, I require them not to be displayed. I know when you create a list you can on the 'Advertise this list' line select 'Hide this list in list index', but I would like it to default to this in case somebody forgets to select it. I am not a Python expert, though from the description at the top of the below file and the contents further down it looks like you are defining in base.py and then in default.py you are using what was defined, a class of 'Private' which should cause the lists not to be advertised, but they are? File: /usr/ib/python3/dist-packages/mailman/styles/base.py class Private: """Settings for private mailing lists.""" def apply(self, mailing_list): mlist = mailing_list mlist.advertised = False <-------------- mlist.archive_policy = ArchivePolicy.private mlist.subscription_policy = SubscriptionPolicy.confirm_then_moderate File: /usr/lib/python3/dist-packages/mailman/styles/default.py class PrivateDefaultStyle( Identity, BasicOperation, Bounces, Private, Discussion, Moderation): """Style for mailing-lists with private archives.""" name = 'private-default' description = _('Discussion mailing list style with private archives.') def apply(self, mailing_list): """See `IStyle`.""" Identity.apply(self, mailing_list) BasicOperation.apply(self, mailing_list) Bounces.apply(self, mailing_list) Private.apply(self, mailing_list) Discussion.apply(self, mailing_list) Moderation.apply(self, mailing_list) I added the below to /etc/mailman3/mailman.cfg: [styles] # The default style to apply if nothing else was requested. The value is the # name of an existing style. If no such style exists, no style will be # applied. default: private-default I took 'private-default' from /usr/lib/python3/dist-packages/mailman/styles/default.py, this means now when a new list is created the List style is 'Discussion mail list style with private archives so it is not displayed when you are logged out at /hyperkitty. Many thanks Daniel

2 years, 5 months

[MM3-users] Re: Running customized version of Mailman 3

by Alan Kelm

On 2/29/20 9:02 AM, Stephen J. Turnbull wrote: > Alan Kelm writes: > > > Since 2005, our organization has been running a customized version of > > Mailman 2 with extensions that meet our particular needs. The main > > novelty is support for /fixed subscription lists/, in which the list > > members are predetermined (corresponding to a "committee"). > > It's not clear to me that this can't be achieved with existing Mailman > 2 (or 3) features. That is, a committee's list could be populated > using mass subscribe, and unsubscriptions can be prevented by > requiring admin intervention. You are correct in pointing out that fixed lists could be implemented without customizations to Mailman, provided that one is willing to leave intact various references to users subscribing and unsubscribing themselves. Modifications were required in 6 separate places (for Mailman 2) to suppress such references for fixed lists. We had additional extensions to permit programmatic re-syncing of subscriber and permitted sender lists with data sources, an adjustment to suppress monthly subscription reminders for such lists, and lots of other minor tweaks that were not strictly necessary. > I'm not sure why this (or the "hybrid" list) requires patches. If I > am correct that the basic functionality is present, and requires only > small tweaks, it might be possible to include those tweaks in Mailman > 3 itself. (I can't promise; this requirement also seems quite rare, > and if substantial code changes, or multiple configuration options, > were required, we might prefer to avoid them.) Our hybrid lists were implemented using separate lists - an outer /opt-in/ list having a /fixed/ list as one of its subscribers (using a generic parent/child list framework). Quite a few customizations were needed to smooth over the rough points of this arrangement (for example to ensure that only the parent list's subject line prefix appears on message subject lines). Now that I reflect upon it, it would be likely be possible to maintain a mandated list of subscribers as a subset within an opt-in list (provided that one keeps a separate copy of the mandated subscribers, to enable detection of previously mandated addresses that need to be deleted). If I were to implement hybrid lists under Mailman 3, I would be inclined to look for simpler mechanisms that what we used under Mailman 2. > > We are considering moving to Mailman 3 and would like to do this in > > a way which makes it possible to periodically upgrade our > > installation to the newest Mailman version, without clobbering our > > customizations. Thus, I do not think that installing via "pip" is > > the right approach for us. While I'm relatively new to git and > > GitLab, they seem to be ideally suited to this situation. If I > > understand the developer documentation correctly, I start out by > > creating a "fork" and can later "fetch" changes from the master > > branch and "rebase" to bring them into our branch. > > Your understanding is correct in principle. If you don't have > experienced git users around, you might want to find somebody to help > you design a workflow. Eg, in general you want to rebase your changes > on top of the mainline, rather than the other way around. You also > need to be prepared for occasional cases where your patch conflicts > with something we do in the same place of the same file. Thank you for these helpful comments. When the time comes, we'll make sure to get the git configuration worked out properly. My investigations over the past few days have convinced me that we are better off sticking with Mailman 2 for a while longer. One feature which is essential for us is to have the list manager informed of bounces, so that failing addresses can be corrected and important missed messages can be resent by other means. The lack of bounce handling in Mailman 3.2 is therefore a showstopper for us. (One of our customizations for Mailman 2 ensures that bounce notifications are sent to the list manager, but this was just a small tweak to existing bounce handling code). I look forward to returning to take another look at Mailman 3 in the future. Thank you, once again, for your helpful comments. -Alan -- Alan Kelm Manager, Electronic Services Canadian Mathematical Society

5 years, 1 month

[MM3-users] installing mailman3 core

by Ralf Wiegand

Mark - What is the difference between to two install methods? What are my pros and cons? I have been using this: >>>> |# Note: if we are using a dedicated postgresql database server, all postgresql changes need to be done on the actual postgresql server.| |>>>| |sudo yum install postgresql-server postgresql-contrib| |sudo postgresql-setup initdb| |sudo systemctl start postgresql| |sudo systemctl enable postgresql| |sudo -u postgres psql -c ||"SELECT version();"| |sudo useradd -c ||"Mailman User"| |-s /bin/bash -m mailman| |sudo -u postgres create role mailman;| |sudo -u postgres createdb -O $USER mailman| |sudo -u postgres psql -c ||"grant all privileges on database mailman to mailman;"| |sudo -u postgres psql -c ||"ALTER ROLE mailman WITH PASSWORD 'secret123';"| |sudo -u postgres psql -c ||"ALTER ROLE "||mailman||" WITH LOGIN;"| |sudo echo ||"listen_addresses = '*' "| |>> /var/lib/pgsql/10/data/postgresql.conf| |sudo echo ||"# The user mailman will be able to access all databases from a trusted location (any) without a password"| |>> /var/lib/pgsql/data/pg_hba.conf| |sudo echo ||"host all mailman 0.0.0.0/0 trust"| |>> /var/lib/pgsql/data/pg_hba.conf| |sudo systemctl restart postgresql| |sudo ss -nlt | grep 5432| |<<<| |sudo yum install python36* --skip-broken -y| |# check make sure you have both versions| |sudo python --version| |Python 2.7.5| |sudo python3 --version| |Python 3.6.8| |sudo pip3 install psycopg2-binary| |sudo chown mailman.mailman /opt/mailman| |sudo chmod 755 /opt/mailman| |# NOTE: The first initial mailman3 core setup has to be done manually.| |# This is a important step to ensure the user "mailman" will run the actual mailman3 application and not root.| |# switch to user mailman| |mailman# cd /opt/mailman| |mailman# git clone https:||//gitlab.com/mailman/mailman.git| |mailman# cat <<EOT >> /opt/mailman/mailman/src/mailman/config/mailman.cfg| |[devmode]| |enabled: yes| |recipient: root@localhost| |[webservice]| |hostname: localhost| |port: 8001| |show_tracebacks: yes| |use_https: yes| |[mta]| |incoming: mailman.mta.postfix.LMTP| |outgoing: mailman.mta.deliver.deliver| |lmtp_host: localhost| |lmtp_port: 8024| |smtp_host: localhost| |smtp_port: 25| |[database]| |class||: mailman.database.postgresql.PostgreSQLDatabase| |url: postgres:||//mailman:secret123@<IP-ADDRESS or FQDN>/mailman| |EOT| |mailman# python3.6 -m venv venv-3.6| |>>| |mailman# source venv-3.6/bin/activate| |(venv-3.6) [mailman@empire mailman]$ cd mailman| |(venv-3.6) [mailman@empire mailman]$ python setup.py develop| |(venv-3.6) [mailman@empire mailman]$ mailman info| |<<| | | | | |So I was told I don't have to do this, I can just simple do (rhel7) | |yum install pyhton3| |pip install mailman| | | | |

4 years, 10 months

[MM3-users] Re: moving lists to another server

by Odhiambo Washington

On Sat, Nov 18, 2023 at 11:48 AM Stephen J. Turnbull < turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote: > Odhiambo Washington writes: > > > Hello Stephen et al, > > > > Since this topic is relevant to something I am faced with at the > moment, I > > believe it's okay if I piggyback on the thread. > > I think continuing the thread is appropriate, but I personally would > change the Subject to reflect the specifics. > > > I have a server that is already actively distributing Mailman list > traffic, > > and managing users and archives. > > I have offered to take over another community ML (also using MM3) from > > another server and host the list on my server. > > I am scratching my head bald trying to figure out how to bring in this > > other MM3 site and list into my existing server, especially with regard > to > > the DB backend. > > > > The list - let's call it users(a)domainX.name was using MySQL backend > and > > had a separate DB for Mailman core and Mailman web. > > I believe combining these into one isn't a problem. But getting to merge > > this DB into my current one is what I am not sure about. > > > > Any pointers on how to bring in this new site into my current setup?? > > I don't have pointers, in the sense of concrete advice I would not be > horrified if you followed it to the letter. Here's how I would attack > the problem. > > 1. Decide the List-Id and List-Post questions. I disagree with Mark > about List-Id (or maybe he mistyped List-Post?) -- the List-Id > should NOT be changed. I don't know how many users are taking > advantage of List-Id (my filters do!), but they will thank you for > keeping it the same. > > The List-Post question is contingent. Will you be serving their > domain, or will you change the List-Post address? The idea is to (if possible) not change anything at all. I will be serving the list using the original domain. I am hoping for a seamless migration where no one notices any change except for the IP address where the MM3 is running. > Note that with > some additional configuration to Mailman and your MTA they could > set up a forwarding configuration for their lists to your server, > Or if they don't have other email needs, set up an MX. Or they > can transfer their domain name's A and MX records to your host. > As it is, I have been given full control of the domain for this purpose so I can point A and MX records to my server. The ML itself is currently down. I have a dump of the /opt/mailman as well as the DBs used by Core and Web. Alternatively you can change the List-Post address.to your > domain. In that case you will have to edit the List-Post address > in your database. > I am hoping that should not be necessary. I actually had thought it wouldn't be complex doing this until I sat down and said wait - there is a DB involved :) > 2. There are similar issues for mailman-web (especially host names). > I have not thought about them at all, so I'll leave that for the > interested reader. ;-) > Hehee.. Wrapping my head around this has not been easy, to say the least. 3. (If the DB engines are different) Dump both databases in text > format (ie, scripts of SQL commands). Check to see how they > differ. > The DB is MySQL. My running setup uses MariaDB. I am not getting what you mean here by how they differ. 4. Adapt the foreign database to its new environment with search and > replace. (Be verbose about the search terms, include lots of > context to minimize the chance of inadvertantly replacing > something in the wrong context that you missed in step 3.) This > is where you would change the List-Post address, as well. You > will also want to change the 'mailmanweb' database to just 'mailman'. > If going with what I have mentioned above, is this step still necessary? > 5. (Optional if your name is "Daredevil") Create a new Mailman suite > install using your current configs, preferably in a new VM or set > of containers. Configure the MTA to save all outgoing mail to a > file, or maybe firewall off outgoing network traffic to port 25 > and leave all the posts in the MTA queues. Create a new > 'mailman-test' database. Copy all the dumps to the VM, and edit > the names of the databases in the SQL to 'mailman-test'. Restore > the dump of your system into the database, and test. Then do the > same, restoring the mailman and mailmanweb databases from the > foreign system into this new database. > I was thinking about this as one way of testing the situation, much as it is quite involving. 6. Adjust to address any issues discovered in 5. If none, go to 7, > Otherwise go back to 5. > > 7. Apply any changes made in 6 to your configs, and read the foreign > dumps into your system, and you should be ready to go for the > distribution part. Not sure about host and webserver > configuration for mailman-web, though. > Suppose I just do: 1. Create the new list on my server using the inherited domain and configure it appropriately, 2. Configure Django to handle the different domains (there is something that I am yet to understand here around SITE_ID when you have multiple sites!) 3. Extract the users from the list and subscribe them afresh (I can shoot an email to let them know the ML had to be moved so they can set their preferences again) 4. Import the archives of the list I should more or less achieve the migration, no? What about setting up on /opt/mailman2 a complete replica of the list? That'd be the real "Daredevil" :) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

1 year, 5 months

[MM3-users] Re: Speed/progress of update_index_one_list?

by Philip Colmer

I have now switched to Xapian but, in a way, my original questions still stand. Is there a way of monitoring the progress of "update_index_one_list"? What can I do to the specification of the server to make that process go (much) faster? By the way, just in case anyone else is looking to use Xapian, installing and setting up Xapian for HyperKitty can be boiled down to: 1. Download the git repo at https://github.com/notanumber/xapian-haystack 2. Run the "install_xapian.sh" script (supplying the version number of Xapian) 3. pip install xapian-haystack 4. Edit the MM3 settings.py file to switch to using Xapian Regards Philip On Fri, 14 Jan 2022 at 07:57, Philip Colmer <philip.colmer(a)linaro.org> wrote: > > > On Fri, 14 Jan 2022 at 07:50, Mark Dadgar <mark(a)pdc-racing.net> wrote: > >> On Jan 13, 2022, at 11:46 PM, Philip Colmer <philip.colmer(a)linaro.org> >> wrote: >> > Yesterday, I ran "mailman-web update_index_one_list" against a mailing >> > list. The command output "Indexing 328077 emails" and that is the last >> I've >> > heard from it. >> > >> > The process is still running but, looking in ~mailman/web/logs, I can't >> see >> > anything happening related to that list. >> > >> > Is there any way to determine how far this process has got, or what it >> is >> > actually doing? (Without stopping it) >> > >> > For future indexing, what makes the most difference to the speed? Does >> it >> > benefit from having multiple cores if I resize the AWS instance to a >> larger >> > processor? What is the bottleneck for this process? >> >> >> Which indexer are you using? Whoosh is interminably slow. Think “days to >> generate a list index” slow. >> > > Yeah, we're using Whoosh. Good to see it lives up to its name :) > > >> I recommend Xapien. It is ridiculously fast. >> > > Thank you. I had been put off using Xapian because the documentation, from > a MM3-perspective, seemed sparse and confusing, but I'll stick with it to > try and get it working if Whoosh is the root problem here. > > Philip > >

3 years, 3 months

[MM3-users] Re: dedicated mailman3 list server on Ubuntu 18.04 LTS - questions after installation

by Mark Sapiro

On 08/05/2018 03:37 PM, John Seifarth via Mailman-users wrote: > > The installation allows me to sign into the web interface, create a host, create mailing lists, and mass subscribe members. When members send email to the list address, the server sends the messages normally to the other members of the list. That's the good news. That's cool. You have raised a lot of questions and I don't have time now to deal with them all. Maybe others can help. I will say a few things now and maybe more to follow. > However, when I look at the list archives, there are no posts. I deliberately mass-subscribed an invalid address, and I can see in /var/log/mail.log that the message bounces, but I get no notification, nor do I see any bounce info when I look at the member info. About Archiving: Are you running django qcluster <http://docs.list.org/en/latest/config-web.html#running-the-task-queue-requi…> and all the scheduled tasks <http://docs.list.org/en/latest/config-web.html#scheduled-tasks-required>? About Bounce processing: Bounce processing is not functional in Mailman 3 yet. > 5) DKIM and DMARC > > I’ve got OpenDKIM running on inet:12345@localhost and OpenDMARC running on net:54321@localhost. > > How can I get mailman to use these milters and sign messages? How should DMARC munging be configured when signing? Your outgoing MTA should be doing the signing, not Mailman. Your signing outgoing messages won't help with DMARC if Mailman alters the message in any way, e.g. by subject prefixing, content filtering or addition of list headers and/or footers. Minimum DMARC settings would be DMARC mitigation action -> Replace From: with list address and DMARC Mitigate unconditionally -> No > 6) Is there an overview of how mailman3 works? > > Is there documentation which explains the path of an incoming post through the system, including initial reception, list processing, sending to members, processing bounces, and archiving posts? All the docs are at <http://docs.list.org/en/latest/> You may want to see <http://mailman.readthedocs.io/en/latest/src/mailman/docs/8-miles-high.html> amongst the many other things you'll find there. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years, 8 months

[MM3-users] 'str' object has no attribute 'token_type'

by Marco Marinello

Hi everybody, I'm sorry for being always the strange guy with strange errors here but for some reason... I am. On a Mailman installation that was doing just fine for more than a year, I started receiving error messages per email. It happens when mailman daemon archives a messages. Although the messages is correctly archived, this exception is thrown by Hyperkitty: Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/django/core/handlers/exception.py", line 47, in inner response = get_response(request) File "/usr/local/lib/python3.9/dist-packages/django/core/handlers/base.py", line 181, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python3.9/dist-packages/django/views/decorators/http.py", line 40, in inner return func(request, *args, **kwargs) File "/usr/local/lib/python3.9/dist-packages/hyperkitty/views/mailman.py", line 118, in _decorator return func(request, *args, **kwargs) File "/usr/local/lib/python3.9/dist-packages/django/views/decorators/csrf.py", line 54, in wrapped_view return view_func(*args, **kwargs) File "/usr/local/lib/python3.9/dist-packages/hyperkitty/views/mailman.py", line 164, in archive add_to_list(mlist_fqdn, msg) File "/usr/local/lib/python3.9/dist-packages/hyperkitty/lib/incoming.py", line 79, in add_to_list from_str = header_to_unicode(message['From']) File "/usr/lib/python3.9/email/message.py", line 391, in __getitem__ the values matching a header field name. File "/usr/lib/python3.9/email/message.py", line 471, in get for k, v in self._headers: File "/usr/lib/python3.9/email/policy.py", line 163, in header_fetch_parse return self.header_factory(name, value) File "/usr/lib/python3.9/email/headerregistry.py", line 601, in __call__ return self[name](name, value) File "/usr/lib/python3.9/email/headerregistry.py", line 196, in __new__ cls.parse(value, kwds) File "/usr/lib/python3.9/email/headerregistry.py", line 343, in parse [Address(mb.display_name or '', File "/usr/lib/python3.9/email/headerregistry.py", line 343, in <listcomp> [Address(mb.display_name or '', File "/usr/lib/python3.9/email/_header_value_parser.py", line 462, in display_name return self[0].display_name File "/usr/lib/python3.9/email/_header_value_parser.py", line 393, in display_name return self[0].display_name File "/usr/lib/python3.9/email/_header_value_parser.py", line 574, in display_name if res[-1][-1].token_type == 'cfws': Exception Type: AttributeError at /hyperkitty/api/mailman/archive Exception Value: 'str' object has no attribute 'token_type' Request information: USER: AnonymousUser I upgraded both to Debian 11 (from 10) and to latest versions of Hyperkitty, mailman-hyperkitty and mailman but still getting the error. Other installations with same specs I'm managing to do not encounter this problem. Any clue of what could have happened, why and how to fix it? Thanks in advance, Marco

3 years, 5 months

[MM3-users] Which folder should I share between pods with kubernetees persistentVolumeClaim ?

by Stephen J. Turnbull

antoine.depoisier--- via Mailman-users writes: > I use mailman with kubernetees, and I have four pods. Mailman is known to work in Docker, but the frequently used configuration is basically divided by application: all of the core mail- and list-handling processes in one container, the web administration processes in another, the archiver in another, and perhaps the RDBMS and MTA each have their own. So I don't understand what you mean by "with Kubernetes", and definitely not the node topology. Are you running any of the 5 applications just mentioned across multiple nodes? > I currently use persistentVolumeClaim for two folders in the > mailman configuration "cache" and "messages". > I share these folders, as mailman needs to be consistent for these > two folders, but I'm wondering if I should share other folders as > well, like locks and such. All of the processes need access to locks and to their relevant queue directories. I can't tell you offhand how which processes need access to which directories. > I ask this question because I often have a problem when creating a > mailing list. I think I have a problem with the locks, but I'm not > sure if the problem is due to the fact that the locks folder is not > shared. I would guess it's related. It's certainly very bad that a file that is supposed to be locked appears to be unlocked when the time comes for the lock owner to unlock it: > flufl.lock._lockfile.NotLockedError: Already unlocked [...] > When I check the lock folder, I have these two files, and I'm > wondering if I should delete these two files : > -rw-rw---- 2 mailman nogroup 97 Feb 6 2025 master.lck > -rw-rw---- 2 mailman nogroup 97 Feb 6 2025 master.lck|mailman-core-deployment-5f59557b66-bkkf4|16|4268829230172994873 Don't do that. If you want those to go away, stop Mailman. As Obi-wan Kenobi said, "These are not the locks you are looking for." The lock in your trace is specific to the Postfix routing files, The master.lck is used by the master process that controls all that others. > Do you have any answer for me ? Need to know the topology of the network of nodes hosting Mailman processes. Steve

2 months, 2 weeks

[MM3-users] Re: Mailman 3 Web Application Startup Time or Availability - 502 Bad Gateway and Postorius

by Mark Sapiro

On 5/3/21 3:27 PM, summersan(a)nclack.k12.or.us wrote: > Hi Mark, > > Thank you for your quick response! > > When setting up Mailman 3, I went through much of this "how to" guide here: https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10. Which uses gunicorn as the wsgi server for Django. > As far as I can tell, my current setup appears to be proxying from Nginx to Gunicorn, according to the related Nginx and mailman3-project-specific Gunicorn config files. Specifically, I checked the local address:port being used in the Nginx file and compared that to the mailman3-project-specific Gunicorn bind address. It looks like I might have "separated out" the REST API to a different server block in the Nginx config file, so I think I'm looking at the correct server block with this conclusion. Nginx has nothing to do with the REST API. The REST API is accessed by Postorius and HyperKitty via mailmanclient and django-mailmane which are installed by pip as dependencies of Postorius and Hyperkitty. > For what it's worth, I don't recall this type of "slow first visit startup" issue happening in the past when using web applications with uwsgi, and I've had the opportunity to use uwsgi with a variety of programs. I think Gunicorn is fairly new to me. I do, however, remember that something like this might have happened in the past when using Passenger (https://www.phusionpassenger.com/) a long time ago, but that might have been setting related on my part. > > Okay, thanks for the encouragement about the potential solution to periodically accessing the related "example.mailman.site/mailman3/lists/" type location (from my earlier email). Wanted to check on that and was also hoping that there might be some setting I'm missing in the overall Mailman 3 project setup related to the "slow first visit startup" issue. I'm not aware of anything you missed. I think this has to be a gunicorn issue or possibly related to Django, but I'm not specifically aware of any issues. There might be something you can set in gunicorn, but I don't know what. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years, 11 months

[MM3-users] Re: very few moderation capabilities for moderators

by Guillermo Hernandez (Oldno7)

On 18/9/21 14:29, Stephen J. Turnbull wrote: > Guillermo Hernandez (Oldno7) via Mailman-users writes: > > On 18/9/21 7:10, Stephen J. Turnbull wrote: > > > Guillermo Hernandez (Oldno7) via Mailman-users writes: > > > > > > > But? I'm talking about a moderator can put a member mails in hold when > > > > it is marked to accept. To prevent putting more gas in the fire. > > > > > > Not sure what you want here. I don't see how this is supposed to work > > > -- if a subscriber is set to accept, the moderator will never see the > > > posts until they're distributed to the subscribers. > > > > I'll try to explain: > > > > In my experience, the most useful tool to cool down a thread that has > > converted in a flame is to stop free distribution (hold) new mails > > coming from some suscribers > > That's what I was missing. You want moderators to be able to set the > moderation flag on a particular member, or several members. Yes. Exactly. That's the unique capability for moderators (that admins have) that I think mailman lacked ever. From my perspective: there is two complementary "sides" when it comes to managing a list: the admin tasks, configuring, setting rules of suscribing, acceptation mails, etc. AND the management of the flux of messages. Thats what I call "moderation". Warning suscribers when a thread overheat, and being capable to set off a member when his behaviour makes it neccesary. Good you have understand me despite muy poor explanations. In mailman2 I have this second part covered via python scripts (and login via special account from moderators), and I will do in mailman3. If I cannot succeed via python, I'm planning doing it via DDBB. Let's see. > > > A moderator doesn't have to be capable of configuring lists, but > > they would have to be capable to moderate the flux of mails when > > it's necessary. > > As I wrote before, I think this needs to be an option controlled by > the owner, which makes implementation more complicated. But we'll see > what the rest of the core thinks. > > Steve >

3 years, 7 months

[MM3-users] Re: A little stuck with installation of MM3 - ModuleNotFoundError: No module named 'flufl.lock'

by Odhiambo Washington

On Thu, 30 Jul 2020 at 00:13, Mark Sapiro <mark(a)msapiro.net> wrote: > On 7/29/20 12:15 AM, Odhiambo Washington wrote: > > > > Yes, the same problem. > > I created the file __init__.py in /opt/mailman/mm > > However, when I run using the wsgi.py I ended up (still do!) with the > error: > > > > [Wed Jul 29 10:08:51.869144 2020] [wsgi:info] [pid 79132] [remote > > 197.232.81.246:64709] mod_wsgi (pid=79132, process='mailman-web', > > application='mm3-lists.server.name|/mm3'): Loading Python script file > > '/opt/mailman/mm/wsgi.py'. > > What is the contend of the file /opt/mailman/mm/wsgi.py? > [wash@gw ~]$ cd /opt/mailman/mm/ [wash@gw /opt/mailman/mm]$ less wsgi.py """ WSGI config for HyperKitty project. It exposes the WSGI callable as a module-level variable named ``application``. For more information on this file, see https://docs.djangoproject.com/en/{{ docs_version }}/howto/deployment/wsgi/ """ import os # import sys # import site # For some unknown reason, sometimes mod_wsgi fails to set the python paths to # the virtualenv, with the 'python-path' option. You can do it here too. # # # Remember original sys.path. #prev_sys_path = list(sys.path) # # Add here, for the settings module #site.addsitedir(os.path.abspath(os.path.dirname(__file__))) # # Add the virtualenv #venv = os.path.join(os.path.abspath(os.path.dirname(__file__)), # '..', 'lib', 'python2.6', 'site-packages') # site.addsitedir(venv) # # Reorder sys.path so new directories at the front. # new_sys_path = [] # for item in list(sys.path): # if item not in prev_sys_path: # new_sys_path.append(item) # sys.path.remove(item) # sys.path[:0] = new_sys_path from django.core.wsgi import get_wsgi_application os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings") application = get_wsgi_application() > You previously posted the setting of WSGIDaemonProcess in your Apache > config as: > > > WSGIDaemonProcess mailman-web display-name=mailman-web > maximum-requests=1000 umask=0002 user=mailman3 \ > > group=mailman3 > python-path=/opt/mailman/mm/venv/lib/python3.7/site-packages:/opt/mailman/mm/venv/lib/python3.7 > \ > > python-home=/opt/mailman/mm/venv home=/opt/mailman/mm/var > > Is it still that? > This is from wsgi.conf in my Apache Includes/. It's a copy/paste of your version, with the only modification being the change pf python version to 3.7 -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

4 years, 8 months

[MM3-users] Re: integrating mm3 with postfix / lmtp

by Fabian A. Santiago

On October 25, 2017 9:13:20 PM EDT, Mark Sapiro <mark(a)msapiro.net> wrote: >On 10/25/2017 05:41 PM, Fabian A. Santiago wrote: >> >> No to the domains part and I think virtual alias maps is needed: >> >> mail postfix/trivial-rewrite[19445]: warning: do not list domain xxx >in BOTH virtual_mailbox_domains and relay_domains >> Oct 25 20:39:19 mail postfix/trivial-rewrite[19445]: warning: do not >list domain xxx in BOTH virtual_mailbox_domains and relay_domains >> Oct 25 20:39:19 mail postfix/submission/smtpd[19441]: NOQUEUE: >reject: RCPT from >pool-108-53-239-102.nwrknj.fios.verizon.net[108.53.239.102]: 550 5.1.1 ><test123@xxx>: Recipient address rejected: User unknown in virtual >mailbox table; from=<fsantiago@xxx> to=<test123@xxx> proto=ESMTP >helo=<[10.0.0.29]> > >(above slightly sanitized for return to the list) > >The issue here is your list domain cannot be a virtual_mailbox_domain >or >a virtual_alias_domain. > >I have the same issue on mail.python.org where we have both Mailman 2 >and Mailman 3 lists @python.org. and python.org is a >virtual_alias_domain. > >I have a group of MRs for a branch that implements "alias_domains" to >deal with this. > >Basically it works by creating another (virtual_alias_maps) >postfix_vmap >mapping to map list addresses @python.org to addresses @x.python.org. >Then the postfix_domains and postfix_lmtp maps have the x.python.org >domain and there is a patch to mailman/runners/lmtp.py to map the >x.python.org domain back to python.org. > >The various MRs are ><https://gitlab.com/mailman/mailman/merge_requests/202> for the core, ><https://gitlab.com/mailman/mailmanclient/merge_requests/18> for the >REST bindings used by Postorius and ><https://gitlab.com/mailman/postorius/merge_requests/186> to be able to >set an alias_domain via Postorius. > >That's not all implemented at mail.python.org. What's there is the >basic >Postfix config, a bash script the generate the new mappings from the >old >and a simple patch to mailman/runners/lmtp.py. > >So, the bottom line is it can be done and it's in process, but it won't >really be supported until Mailman 3.2. In the mean time, you just need >to have a separate domain for your lists that isn't a virtual_*_domain. Ok thanks. At least I have a clear answer now. Any ETA on 3.2 in mind yet? -- Thanks, Fabian S. OpenPGP: 3C3FA072ACCB7AC5DB0F723455502B0EEB9070FC

7 years, 5 months

[MM3-users] Re: Apple Mail encoding again

by David Newman

On 4/14/22 9:50 AM, Mark Sapiro wrote: > On 4/14/22 08:21, Stephen J. Turnbull wrote: >> Mark Sapiro writes: >> >> > That combination at least produces multipart/signed messages with >> > multipart/* parts containing a Content-Transfer-Encoding: header. >> > >> > The Python 3 email library considers this to be a >> > InvalidMultipartContentTransferEncodingDefect() message defect. >> >> Merely containing C-T-E is not an error per RFC 2045, although it >> seems to consider it a stupid thing to do. If present, it may be >> "binary", "7-bit", or "8-bit" (below summarized as "cleartext"). >> >> Did Apple Mail apply a non-cleartext C-T-E? Or has an RFC tightening >> that restriction been published since? (I don't know why it would, >> though.) > > > Yes. The C-T-E on all the multipart parts was quoted-printable, not > allowed under RFC 2045. > > >> Even with a non-cleartext C-T-E, it seems reasonable for Mailman to >> treat it as application/octet-stream and pass it through, unless the >> list is configured to do something like strip application/octet-stream >> or that requires decoding like spam-filtering (Postel principle, >> although one could argue it shouldn't be sent on on the same principle >> -- ah, the joys of being an e-mail Mediator). That would be >> especially true it if were quoted-printable, since any relevant MIME >> headers would (presumably) be pure ASCII and therefore unencoded. > > > The Python email library says the message is defective. Mailman's lmtp > runner in turn rejects the defective message with a 501 status > https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/runners/lmtp.p… > Yes. To be clear, this is an Apple Mail issue. Mailman 3 accepted PGP-signed messages from Apple Mail on versions prior to 15.0 running on MacOS 12.0.1. Something changed on Apple's part. > > It could accept a defective message or one with specific defects, but > this is problematic for further processing. For example, at least one > MUA, Thunderbird, renders such a message with an empty body. That's not my experience. Mailman 3 accepted without complaint a PGP-signed message I sent from Thunderbird 91.8.0 and forwarded it to list subscribers with the message body intact. dn

3 years

[MM3-users] Re: hyperkitty failed to create a thread

by Mark Sapiro

On 2/11/19 10:01 AM, Abhilash Raj wrote: > > Please create a bug report at https://gitlab.com/mailman/hyperkitty/issues with the affected messages. > > It would help us recreate the problem if you could attach a raw messages in the issue. I am > assuming the messages aren't sensitive since the archive are open. Otherwise, feel free to omit > any sensitive information. > > I haven't looked at the threading code in HK since a long time, but my impression is that it > looks at the 'In-Reply-To' header to figure out if the current email is a response to a previous > email that it received. > > In some cases, the order of the incoming messages can cause this to break, like when the > reply comes in first and the original messages comes in later. I am not sure exactly what is > happening here, but looking at the raw messages should help us reproduce and possibly > fix the issue. I downloaded the February mbox from <https://wlug.mailman3.com/hyperkitty/list/wlug@lists.wlug.org/export/wlug@l…> and imported it to a test list with 'django-admin hyperkitty_import and see the same issue. Examination of the mbox shows for the first message Message-ID: < <SN6PR01MB4719346533C08BF4FF62B880C36C0(a)SN6PR01MB4719.prod.exchangelabs.com>> This will be wrapped, so here's an excerpt Message-ID: < <SN6....prod.exchangelabs.com>> Note the folding of the header and the double angle brackets. The next message has (again excerpted to avoid wrapping) Message-ID: < <BL0....prod.exchangelabs.com>> In-Reply-To: SN6....prod.exchangelabs.com Note again the doubling of the angle brackets in the Message-ID and the absence of angle brackets in the In-Reply-To: The Message-ID: and In-Reply-To: headers of subsequent messages have the same issues. In looking at other mboxes downloaded from Hyperkitty, I see that the absence of References: and the dropping of the angle brackets from In-Reply-To: seem to be usual, and importing such a mbox preserves the threading. However, the doubling of the angle brackets is not usual and is almost certainly the cause of the problem. Do you have any of the original messages? What do the headers look like in those? -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years, 2 months

[MM3-users] Re: Permission denied for fulltext_index

by Jan Eden

On 2022-11-30 10:22, Mark Sapiro wrote: > On 11/30/22 08:41, Jan Eden via Mailman-users wrote: > > > > Changing the PATH value to a full path (/opt/mailman/fulltext_index) > > and restarting mailman-web generates a different error: > > > Yes, it should be a full path. I encountered yet another problem with emails not getting archived anymore: > ERROR 2022-12-01 09:18:00,111 1337915 hyperkitty.views.mailman Access to the archiving API endpoint was forbidden from IP XXX.XXX.XXX.XXX, your MAILMAN_ARCHIVER_FROM setting may be misconfigured where XXX.XXX.XXX.XXX is my server's public IP. This was probably caused by the switch to uwsgi_pass/a file socket, and could be fixed by adding MAILMAN_ARCHIVER_FROM = ('XXX.XXX.XXX.XXX', '::1') to /etc/mailman3/settings.py. > > > ERROR 2022-11-30 16:38:37,953 1028315 django.request Internal Server Error: /archives/list/testing@lists.eden.one/thread/T6MCILAVQYV5QIOGHJB3JNJ26Z7E2Z35/reattach-suggest > > > Traceback (most recent call last): > > > File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner > > > response = get_response(request) > > > File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response > > > response = wrapped_callback(request, *callback_args, **callback_kwargs) > > > File "/opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/lib/view_helpers.py", line 137, in inner > > > return func(request, *args, **kwargs) > > > File "/opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/views/thread.py", line 454, in reattach_suggest > > > sugg_thread = msg.object.thread > > > AttributeError: 'NoneType' object has no attribute 'thread' > > > This says that searching the archive for this list for messages matching the > current message subject returns None. It is probably a bug that we don't > test for this, but what exactly are you doing? > See the post at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > for the only way I've been able to create an error like this. There are currently 5 (test) threads in the archive. After changing the PATH in the HAYSTACK_CONNECTIONS dict to /opt/mailman/fulltext_index, and fixing the issue with MAILMAN_ARCHIVER_FROM above, /opt/mailman/web/logs/mailmanweb.log does not display new error messages, but I do not get any suggestions for reattaching threads, and a search for a specific thread subject ("Third test") does not return any results. So reattachment works in principle, but the suggest/search function does not. - Jan

2 years, 4 months

[MM3-users] Re: Mailman recovery test did not go as planned

by Mark Sapiro

On 8/5/22 09:38, bob B via Mailman-users wrote: > I have a test Mailman 3 environment. We are using docker, and we are running the database outside of docker in mysql. > I need to perform a DR recovery test. > > I shut down Mailman in docker (mailman-core & mailman-web, the only docker images I have running for Mailman). Does shutting down mailman-web also shut down Django qcluster? If not, this could be an issue. > While Mailman was stopped, I had our database team take a backup dump of the database. > I then restarted Mailman in docker. > I threw some more messages at a test list. > I validated that the new messages were delivered, and showed up in the archive. > I shut down Mailman again. > I had the database team restore the Mailman database from the backup. > I started Mailman > I logged into Mailman > I was expecting to see the messages in the archive prior to the backup and all the messages after the backup would be gone. I would expect that too. > When went into the list archive, I get a spinning icon and no messages showed up. I don't offhand know what the issue would be unless Django qcluster was running the whole time. > I quit, restarted my browser, and logged in again, and the same thing. > I then sent a test message to the list and the message was delivered to the list. > Then when I went to check the archive, the spinning icon was gone and that test message was there along with all the messages before the backup was taken. Possibly one of the Django periodic jobs, maybe cache_cleanup or daily_cleanup would fix it without a message. > Is this normal? After a restore did the new message cause a refresh of archived messages etc? > Is there a way to do this without sending a message to a list after a restoration of the database? > > Are there better steps I should take when restoring the database? I don't know what better you could do as far as the save and restore are concerned as long as shutting down mailman-web stops qcluster. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 8 months

[MM3-users] Re: Hipperkitty failing to archive messages in a list

by Mark Sapiro

On 1/5/22 4:01 AM, Guillermo Hernandez (Oldno7) via Mailman-users wrote: > > pip install --upgrade --force-reinstall --no-deps --no-cache-dir > django-mailman3 mailman mailman-hyperkitty mailmanclient postorius > django hyperkitty > > su -m mailman3 -c "python3 manage.py collectstatic" > su -m mailman3 -c "python3 manage.py migrate" > > In this last point I've received some warnings of this kind: > > *-*-*-*-*-*-* > > WARNINGS: > account.EmailAddress: (models.W042) Auto-created primary key used when > not defining a primary key type, by default 'django.db.models.AutoField'. > HINT: Configure the DEFAULT_AUTO_FIELD setting or the > AccountConfig.default_auto_field attribute to point to a subclass of > AutoField, e.g. 'django.db.models.BigAutoField'. > account.EmailConfirmation: (models.W042) Auto-created primary key used > when not defining a primary key type, by default > 'django.db.models.AutoField'. > > ... > > HINT: Configure the DEFAULT_AUTO_FIELD setting or the > SocialAccountConfig.default_auto_field attribute to point to a subclass > of AutoField, e.g. 'django.db.models.BigAutoField'. This is from Django >=3.2 can be fixed by adding DEFAULT_AUTO_FIELD = 'django.db.models.AutoField' to your settings. This has been done in HyperKitty's example_project/settings.py for the not yet released 1.3.6 version. These warnings are only warnings, they aren't fatal. ... > Running a 'su -m mailman3 -c "python3 manage.py makemigrations" shows > the same warning listed before and some new errors: > > *-*-*-*-*-*-* > > Migrations for 'postorius': > /usr/local/lib/python3.7/site-packages/postorius/migrations/0017_alter_emailtemplate_name.py > > - Alter field name on emailtemplate > Traceback (most recent call last): > File "manage.py", line 10, in <module> > execute_from_command_line(sys.argv) > File > "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", > line 419, in execute_from_command_line > utility.execute() > File > "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", > line 413, in execute > self.fetch_command(subcommand).run_from_argv(self.argv) > File > "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", > line 354, in run_from_argv > self.execute(*args, **cmd_options) > File > "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", > line 398, in execute > output = self.handle(*args, **options) > File > "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", > line 89, in wrapped > res = handle_func(*args, **kwargs) > File > "/usr/local/lib/python3.7/site-packages/django/core/management/commands/makemigrations.py", > line 190, in handle > self.write_migration_files(changes) > File > "/usr/local/lib/python3.7/site-packages/django/core/management/commands/makemigrations.py", > line 228, in write_migration_files > with open(writer.path, "w", encoding='utf-8') as fh: > PermissionError: [Errno 13] Permission denied: > '/usr/local/lib/python3.7/site-packages/postorius/migrations/0017_alter_emailtemplate_name.py' > > > *-*-*-*-*-*-* > > The file > '/usr/local/lib/python3.7/site-packages/postorius/migrations/0017_alter_emailtemplate_name.py' > does not exist at all makemigrations is trying to create that migration and you don't have permission to create the file. I don't know why it's trying to do that. Postorius 1.3.6 has migrations through 0016_auto_20210810_2157.py and shouldn't need more. To you have modificationd to Postorius? > Despite the error, I did a 'su -m mailman3 -c "python3 manage.py > compress"' and the subsequent "django-admin compilemessages" in > > /usr/local/lib/python3.7/site-packages/postorius, ../hyperkitty and > ../django_mailman3 > > And the log continues showing the same error when trying archive a new > message. > > ERROR 2022-01-05 12:27:31,406 6988 hyperkitty.views.mailman The > MAILMAN_ARCHIVER_KEY was not sent as the Authorization HTTP header. > ERROR 2022-01-05 12:27:31,418 6988 hyperkitty.views.mailman The > MAILMAN_ARCHIVER_KEY was not sent as the Authorization HTTP header. > ERROR 2022-01-05 12:27:31,685 6988 hyperkitty.views.mailman The > MAILMAN_ARCHIVER_KEY was not sent as the Authorization HTTP header. > ERROR 2022-01-05 12:27:32,045 6988 hyperkitty.views.mailman The > MAILMAN_ARCHIVER_KEY was not sent as the Authorization HTTP header. > ERROR 2022-01-05 12:27:32,070 6988 hyperkitty.views.mailman The > MAILMAN_ARCHIVER_KEY was not sent as the Authorization HTTP header. > ERROR 2022-01-05 12:27:32,093 6988 hyperkitty.views.mailman The > MAILMAN_ARCHIVER_KEY was not sent as the Authorization HTTP header. These messages come from some HyperKitty version between commit b415d29d6cc59b3270c35b03ba3313dd03450271 Mon Jun 21 00:11:48 2021 -0700 and commit c6272f3ef8375865382d0741d3d371a0dc41508a Fri Oct 8 06:32:06 2021 +0000 They do not come from HyperKitty 1.3.5 but from something later than 1.3.4 which makes be think your mailman-hyperkitty version is not 1.2.0 either. I suggest you do pip install --upgrade hyperkitty mailman-hyperkitty and ensure it installs hyperkitty 1.3.5 and mailman-hyperkitty 1.2.0 -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years, 3 months

[MM3-users] Re: Mailman sends Welcome emails, but does not send [List] postings emails

by Odhiambo Washington

On Wed, May 31, 2023 at 4:18 PM <justfixit(a)marilynstrother.com> wrote: > Very interestng! Upon restarting the system (down momentarily for taking a > snapshot) and starting Mailman again, all of these missing [List] messages > emerged from their timewarp and were delivered to the members / > subscribers. Just before shutting down the system I discovered that the > apache2 webserver had been off the air due to a module not being enabled > yet. > As of yet, I was unaware of any requirement for a web server to be online > (beyond possibly gunicorn) in order for Mailman to run its lists. Does that > speculation have a factual basis? Or do I risk starting a vicious rumor > amongst the impatient readers who are not still here once we deduce this > pair of observations were merely correlated in time, with no > cause-and-effect relationship? > Before this situation healed itself, the thought had crossed my mind to > peek and see if anything was stuck in the queues: > > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/… > While that could have provided peace of mind that email messages were > merely delayed instead of vanished, even in hindsight I do not see how that > would have led to repairing the situation. > Mailman3 is in two parts which are completely INDEPENDENT: 1. Mailman Core - Details at -> https://docs.mailman3.org/en/latest/install/virtualenv.html#installing-mail… 2. Maiman Web - Details at https://docs.mailman3.org/en/latest/install/virtualenv.html#installing-web-… Only Mailman Web requires a web server. To run a mailing list, you need Mailman Core+Database+MTA. No Webserver is required. In the absence of a hypothetical command: > mailman status --verbose > about which diagnostic tools or clues should I learn? > You should learn how to configure the logging for Mailman in mailman.cfg and how to read and interpret the information written to the logfiles :-) PS: I know MM3 is a big animal and isn't easy to understand. I have been through that - and still there. With time you'll just understand the bits so don't despair. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

1 year, 10 months

[MM3-users] Re: Mailman, etc. upgrade woes and persistent bugs

by Brian Carpenter

On 2/13/21 3:03 AM, Stephen J. Turnbull wrote: > How different is this from Mailman 2? (Just "not really different" is > fine, I'll be looking at your specific desiderata for more details > later, including potential GSoC tasks.) It seems to me it's basically > the same, except maybe for some Web 2.0 features like sorting on > fields. (I'm not deprecating those features: obviously sorting would > have helped Allan a lot, for example. It's just that Mailman 2 is > very much a Web 1.0, maybe even Web 0.9, application.) Am I missing > something big? I actually took at lot of ideas from the MM 2 Membership List page simply due to the fact that Postorius' was not helpful in inspiring anything. The one thing that I did do is take MM2's approach and modernized it. So here are more tasks a List Owner can do that MM2 could never offer (and Postorius doesn't even come close): -- Sort Members by Delivery mode (regular vs digest). (MM2 has no sorting abilities at all except by alphabet, Postorius doesn't even have that) -- Reveal which Members are registered with the UI (MM2, everyone was registered, in Postorius nothing is revealed about registration) -- Show in one List Member Management panel List Members, Non-Members, Moderators, and Owners (MM2 and Postorius requires this data to be seen via multiple pages) -- View Member's List Subscriptions to other Mailing Lists on the server and, if that List Owner is List Owner of those other lists, can even access the List Owner board (Affinity terminology) for those other Lists. -- Ban a member -- Promote a member to List Owner and/or Moderator -- Add a new List Member, List Owner, Moderator, and Non-Member from a Single List Member Membership panel. (Again both Postorius and MM2 requires multiple pages. Postorius is an improvement in this regard over MM2. The funny thing with Postorius is you can add a single Non-member, Moderator, and List Owner, but you cannot add a single List Member except through *Bulk* adding.) Features are not the only thing about a modern UI, UX design plays a MAJOR role in the modern UI. Postorius should have at least matched MM2 in features in regards to something as important as Member management but instead it took a step back into the dark ages. -- Brian Carpenter Harmonylists.com Emwd.com

4 years, 2 months

[MM3-users] Re: Again, after systemupgrade the log file in /var/log/mailman was owned by list:list instead of mailman:mailman and starting failed for this reason [support request]

by Odhiambo Washington

On Sun, Dec 24, 2023 at 3:35 PM Wikinaut <mail(a)tgries.de> wrote: > This is in > > /opt/mailman/web/logs# dir > total 3164 > 276 -rw-r--r-- 1 mailman mailman 274457 Dec 24 09:05 mailmanweb.log > 2056 -rw-r----- 1 mailman mailman 2100388 Dec 24 09:05 uwsgi-error.log > 832 -rw-r----- 1 mailman mailman 846536 Dec 24 09:10 uwsgi.log > > Looks okay. > > This is my /etc/systemd/system/mailman3.service: > > Unit] > Description=GNU Mailing List Manager > After=syslog.target network.target mysql.service > > [Service] > Type=forking > User=mailman > Group=mailman > PIDFile=/opt/mailman/venv/bin/master.pid > ExecStart=/opt/mailman/venv/bin/mailman start --force > ExecReload=/opt/mailman/venv/bin/mailman restart --force > ExecStop=/opt/mailman/venv/bin/mailman stop > > [Install] > WantedBy=multi-user.target > > > This mailman3web.service: > > [Unit] > Description=GNU Mailman Web UI > After=syslog.target network.target mysql.service mailman3.service > > [Service] > Environment="PYTHONPATH=/etc/mailman3/" > User=mailman > Group=mailman > ExecStart=/opt/mailman/venv/bin/uwsgi --ini /etc/mailman3/uwsgi.ini > KillSignal=SIGINT > > [Install] > WantedBy=multi-user.target > > > > > I do use the venv method, but the other issue may be a relict from former > setup, as I said, and difficult to find. > It so very strange, that the log-permisison issue stems from → systemctl > start mailman3. > Because you were referring to /var/log/, can you please share the output of the following: grep var_dir /etc/mailman3/mailman.cfg In my mailman.cfg, I have: var_dir: /opt/mailman/mm/var And in my settings.py I have /opt/mailman/mm/var/logs/mailmanweb.log specified for mailman-web. As you can see, mailman core and mailman-web both log to /opt/mailman/mm/var and given that /opt/mailman is owned by mailman:mailman, I don't see how a file owned by list:list would appear there. I also don't see how anything to do with /var/log/ is in your vocabulary :-) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

1 year, 3 months

[MM3-users] The future is here?

by ieso＠johnwillson.com

In my quest to figure out how even us idiots can successfully install a mailman3 server, I keep looking ways to reduce the process to the simplest possible set of instructions. So far, I haven't been doing very well with that, but I've been learning a ton. In particular, while I still haven't got a fully functioning mailman3 server yet, someone pointed me, off list, to Ansible, which I wasn't aware of before and which has completely blown me away by its power and simplicity. For those who aren't aware of it, it's an agentless, multi-platform automation technology. For our purposes, it replaces "how to install mailman3" guides with a single script that aims to actually do the entire thing for you. One of the particularly interesting points is that you can write complex scripts/data structures (or 'collections') that allow for a single collection to cover a broad range of use cases, such as [this person's](https://github.com/rivimey/) exim Ansible collection, which covers 3 very different use cases from a private domain email relay up to a full multi-domain email handler with IMAP integration. I haven't quite got the mailman3 one working yet, but I've just started playing with it. Here are some pointers in case anyone else is interested in trying, and I'd be very interested in whether or not I'm actually the last person to discover this technology, and if one of these solutions succeeds in delivering mailman3 "out of the box". 1. [Install Ainsible](https://docs.ansible.com/ansible/latest/installation_guide/intro_…, make sure you're on at least version 2.9, if not 2.10. Some distributions are still on 2.7, which is not compatible with more recent collections. The "Galaxy" collection is apparently only tested on Debian 10. 2. Pick your mailman3 Ansible collection of choice and install it (https://galaxy.ansible.com/ucr/mailman3, or https://github.com/rivimey/ansible-mailman3) Command to install official collection - ansible-galaxy install ucr.mailman3 3. Write playbook.yml with key variables (mailman3 example below) _________________________ mailman3-playbook.yml _________________________ - hosts: mailman3 vars: mailman3_domain: example.com # It will install and create a certificate using certbot, its nignx plugin and http validation (Public IP required). If false, it will install a self sign cert mailman3_install_certbot: true # install and create certificate # secrets, please change them mailman3_secret_key: "change_secretkey" mailman3_rest_api_pass: "change_api_pass" mailman3_archiver_key: "change_archiver_key" mailman3_db_pass: "change_db_pass" mailman3_web_db_pass: "change_web_db_pass" mailman3_web_admin_pass: "change_web_admin_pass" roles: - ucr.mailman3

4 years, 3 months

[MM3-users] Re: Issues with @gmail.com addresses

by Mohsen Masoudfar

Thank you Mark for your response as always! I checked the lists having the issue of sending emails to @gmail.com subscribers. They had already these settings (so no changes): DMARC Mitigate unconditionally : yes DMARC mitigation action: Replace From: with list address DMARC rejection notice: empty DMARC wrapped message text: empty I wonder three points: * What would be the next action to fix the issue? * This issue started around the beginning of August. Could it be due to some changes google made recently? * Give the number of Gmail users, this should annoy a bigger community, are you seeing that many upset people? Thanks, Mohsen ________________________________ From: Mark Sapiro <mark(a)msapiro.net> Sent: Tuesday, September 5, 2023 4:17 PM To: mailman-users(a)mailman3.org <mailman-users(a)mailman3.org> Subject: [MM3-users] Re: Issues with @gmail.com addresses [EXTERNAL EMAIL] On 9/5/23 12:59, Mohsen Masoudfar wrote: > Hi, > > since 8/1/2023 I get complains from users with @gmail.com addresses, that they are not getting any emails. > I checked the logs and I see that the emails are going out: > > to=<xxxx(a)gmail.com>, relay=email-smtp.us-east-1.amazonaws.com[35.169.171.20]:587, delay=0.46, delays=0.03/0.06/0.13/0.24, dsn=2.0.0, status=sent (250 Ok xxx ) > > But it seems the users are getting email delivered to them. > I’ve advised them to check their spam, promotions, and social folders and to add the list address to their email safe-senders lists, but still no success. > Is there any way to help this list members? Even though gmail publishes a DMARC policy of none, it has recently started enforcing a stricter policy for mail From: the gmail.com domain. You need to apply DMARC mitigations to mail From: the gmail.com domain. There are two ways to do this. One way is to set DMARC Mitigations -> DMARC Mitigate unconditionally to Yes (and set DMARC mitigation action appropriately) The other way requires upgrading Postorius and Mailman core to the heads of the GitLab branches (core=3.3.9b1, postorius=1.3.9b1). Doing this enables a DMARC Addresses setting which can be set to `^.*(a)gmail\.com$` to apply DMARC mitigatations to mail From: the gmail.com domain regardless of it's published policy. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mai…<https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/> Archived at: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mai…<https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…> This message sent to mmasoudf(a)aaas.org

1 year, 7 months

[MM3-users] Re: Default values for lists in Mailman 3

by Markus Grandpré

Thank you very much for your answer. Best regards, Markus Am 19.09.24 um 22:03 schrieb Mark Sapiro: > On 9/19/24 05:46, Markus Grandpré wrote: >> Dear list, >> >> I would like to activate the invitation setting for all lists as >> default. In Mailman 2.1 I have done this in the file </etc/mailman/ >> mm_cfg.py>: >> >> >> DEFAULT_SUBSCRIBE_OR_INVITE = Yes >> >> >> How can I do this in Mailman 3? > > There is a draft merge request at https://gitlab.com/mailman/ > postorius/-/merge_requests/930 which implements a framework for setting > Postorius defaults and creates a setting to set the default for pre- > verified on the mass subscription page. Once merged, this can easily be > extended to provide a default for Invitation. > > In the mean time, you can only do this by patching src/postorius/forms/ > list_forms.py like > ``` > --- a/src/postorius/forms/list_forms.py > +++ b/src/postorius/forms/list_forms.py > @@ -1283,7 +1283,7 @@ class ListMassSubscription(forms.Form): > > invitation = forms.BooleanField( > label=_('Invitation'), > - initial=False, > + initial=True, > required=False, > help_text=_( > 'If checked, the other checkboxes are ignored and the > users will ' > ``` >> Can I also transfer other default values like: >> >> ... >> DEFAULT_SERVER_LANGUAGE = 'en' >> DEFAULT_SEND_REMINDERS = 1 >> DEFAULT_SUBSCRIBE_OR_INVITE = Yes >> DEFAULT_PRIVATE_ROSTER = 2 >> DEFAULT_ARCHIVE = Off >> DEFAULT_ARCHIVE_PRIVATE = 1 >> ... >> >> from the very same configuration file to Mailman 3? Is there a >> documentation of the default values for lists in Mailman 3? > > This is done by creating a list style with the desired defaults. See > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/styles/ > docs/styles.html for documentation of styles and see https://gitlab.com/ > mailman/example-mailman-plugin for an example of creating a plugin in > Mailman core to make a persistent style. > > There is no specific doc for the list settings. You need to look at > https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/styles/ > base.py for the various settings and https://gitlab.com/mailman/ > mailman/-/blob/master/src/mailman/styles/default.py to see which default > classes are applied for various styles. > -- Markus Ludwig Grandpré Universität Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Abteilung IT-Dienste Forschung und Lehre, B803, Tel: ++49 7531 88 4342

7 months

[MM3-users] Re: User settings triggers error

by Mark Dadgar

On Jun 18, 2020, at 4:25 PM, Mark Dadgar <mark(a)pdc-racing.net> wrote: >>> Any thoughts on how to salvage ~20 years of list archives? >> >> >> The archives shouldn't be an issue. I don't think they are involved in >> the corruption, but anyway, hopefully you still have the archive mbox >> you imported. Then for the messages since then, you should be able to >> get a mbox from hyperkitty with something like >> >> https://example.com/hyperkitty/list/listname@example.com/export/arbitrary-n… >> >> with the obvious substitutions and appropriate start and end dates. >> >> The list is trickier. If you are going to start over with a clean >> install, and if you have a config.pck that you imported from MM 2.1, I >> suggest the following: >> >> Use the `mailman members` command to get a list of all members. Then >> depending on how much work you want to do, also get separate lists of >> plaintext and mime digest members and members with delivery disabled. >> >> Then on the new install, import the config.pck with `mailman import21` >> and import the archive mboxes with hyperkitty_import. >> >> Run `mailman members --sync` with the full current membership list, and >> record the [ADD]s. Then you can match the [ADD]s against the plain and >> mime digest members and those with delivery disabled and set their options. >> >> That's a lot to do, but it should get you close. > > I would probably be satisfied with just exporting the current list of member email addresses + names and letting people go back and change their list settings to switch to digest, etc. if necessary. > >> On the other hand, if you can wait, we might be able to figure out how >> to fix this, and it's even possible the underlying issue isn't database >> corruption. > > > Sure - I can wait. Thank you. I am probably going to go ahead with the list wipe and recreate in the next couple of days. Lack of access to the administration stuff is starting to become a problem. Sigh. So far, migrating from sqlite3 to postgres is looking like a bit of a bust. - Mark ----- mark(a)pdc-racing.net | 408-348-2878

4 years, 9 months

[MM3-users] Re: Question about message bounce stats

by Abhilash Raj

> On Aug 30, 2021, at 11:27 AM, Stephen J. Turnbull <stephenjturnbull(a)gmail.com> wrote: > > Ruth Ivimey-Cook writes: > >> If Abhilash has a starting point, > > IIRC it was quite primitive, and probably not at the the individual > message level, more like list level. We do keep a bounce score for Member (email, mailinglist pair) that is right now only available in Mailman’s database and is used for the purposes of un-subscriptions or disabling delivery. We haven’t had a chance to expose these values through REST API, but it is something I’d be happy to accept a patch for and isn’t too difficult if someone wants to pick up the task. It might be helpful at some point to expose some of those stats to the list owner or site admin as well in Postorius. > >> and there is acceptance from others to integrate something of this >> sort, then I could possibly help make it happen. > > If you're that interested, I'd say get started, at least on sketching > out requirements. There will probably be some nitpicking about code > style and the like, and there may be delays in integrating stuff due > to the peanut gallery contributing requirements that we decide we want > in the feature, but I don't think there's a problem with the basic > idea. Yep, +1. It would be good to first figure out what more stats do we need/want and if they just need to be exposed through the API or we need to setup some offline setup for calculating those metrics with some mechanism and then expose. > > We won't be putting tracking pixels and the like in outgoing posts, > but as long as it's internal data already generated by normal Mailman > operation, I would be somewhere between very surprised and shocked > (SHOCKED, I tell you!) if it weren't accepted. Yeah, anything that is possible to generate with regular Mailman operation is already possible for a site admin to get from logs with some scripting. So, the idea to expose them in a better UX I guess should be okay. Only thing we might want to consider is how much of that data is exposed to non-server-admins (list owners, mods etc) and the policy enforcement for that if users don’t like the default. -- thanks, Abhilash Raj (maxking)

3 years, 7 months

[MM3-users] Re: Issues with @gmail.com addresses

by Allan Hansen

Once this gets handled, Mohsen, Google has another issue with list servers: If a subscriber who uses GMail posts a message to a list, then that subscriber will not get his/her own message from the server - Google eats it. All other subscribers will get it, but not the author. My subscribers run into this all the time and are up in arms over it. I tell them to complain to Google. Yours, Allan On 9/5/23, 13:18, "Mark Sapiro" <mark(a)msapiro.net <mailto:mark@msapiro.net>> wrote: On 9/5/23 12:59, Mohsen Masoudfar wrote: > Hi, > > since 8/1/2023 I get complains from users with @gmail.com addresses, that they are not getting any emails. > I checked the logs and I see that the emails are going out: > > to=<xxxx(a)gmail.com <mailto:xxxx@gmail.com>>, relay=email-smtp.us-east-1.amazonaws.com[35.169.171.20]:587, delay=0.46, delays=0.03/0.06/0.13/0.24, dsn=2.0.0, status=sent (250 Ok xxx ) > > But it seems the users are getting email delivered to them. > I’ve advised them to check their spam, promotions, and social folders and to add the list address to their email safe-senders lists, but still no success. > Is there any way to help this list members? Even though gmail publishes a DMARC policy of none, it has recently started enforcing a stricter policy for mail From: the gmail.com domain. You need to apply DMARC mitigations to mail From: the gmail.com domain. There are two ways to do this. One way is to set DMARC Mitigations -> DMARC Mitigate unconditionally to Yes (and set DMARC mitigation action appropriately) The other way requires upgrading Postorius and Mailman core to the heads of the GitLab branches (core=3.3.9b1, postorius=1.3.9b1). Doing this enables a DMARC Addresses setting which can be set to `^.*(a)gmail\.com$` to apply DMARC mitigatations to mail From: the gmail.com domain regardless of it's published policy. -- Mark Sapiro <mark(a)msapiro.net <mailto:mark@msapiro.net>> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org <mailto:mailman-users@mailman3.org> To unsubscribe send an email to mailman-users-leave(a)mailman3.org <mailto:mailman-users-leave@mailman3.org> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ <https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/> Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org <mailto:mailman-users@mailman3.org>/message/WZY5IC3OVHVBHIGDT4TE3EWSSGMF6CYI/ This message sent to hansen(a)rc.org <mailto:hansen@rc.org>

1 year, 7 months

[MM3-users] Re: rDNS issue? Helo name contains an IP address (HELO was [127.0.0.1]) and not is valid

by Kyriakos Terzopoulos

Hi Steve, I'll check runq and let you know. As for the spaces it was just a copy/paste fail from my part since I want to obfuscate the true domain name. *Kyriakos Terzopoulos *Web developer / e-learning expert *Tel:*+30 211 213 9858 *Mobile:* +30 694 526 4512 * E-mail: *kyriakos.terzopoulos(a)gmail.com * Skype:* kyriakos.terzopoulos Find me on Facebook <http://www.facebook.com/cirrus3d> Follow me on Twitter <http://twitter.com/#%21/cirrus3d> On Wed, 25 Oct 2023 at 20:29, Stephen J. Turnbull < turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote: > Kyriakos Terzopoulos writes: > > > No more mail being marked as spam. However I face a (hopefully) new > > problem now. When I mass subscribe the users, they get an initial > > invitation email. However after replying I get this in the exim > > logs, the subsequent mails do not arrive and the users are not > > verified at all: > > > > 2023-10-25 13:36:41 1qvbFh-001kcb-Ig <= myname(a)myproject.eu > > H=localhost (webmail.myproject.eu) [127.0.0.1] P=esmtpa > > A=dovecot_login: myname(a)myproject.eu S=1726 > > id=32091a0e12cc23fa988380639a4c44a6@ myproject.eu > > 2023-10-25 13:36:41 1qvbFh-001kcb-Ig == partners(a)myproject.eu > > < > partners-confirm+6df62cd0f805b2b66b21a43a42ec8ee77487e8ad(a)myproject.eu> > > R=mailman3_router T=mailman3_transport defer (-54): > > retry time not reached for any host for ' myproject.eu' > > It looks like 'myname' has replied to the invitation, and was received > by this MTA (is it Mailman's MTA?). Then delivery to Mailman was > deferred because delivery to ' myproject.eu' has failed, and Exim is > queueing them for future delivery, until the next retry time. Try > using 'runq' then checking the log to see why mail is failing in the > first place. > > I do not understand why there are spaces before the host name in these > lines: > > > id=32091a0e12cc23fa988380639a4c44a6@ myproject.eu > > retry time not reached for any host for ' myproject.eu' > > The first never happens in my Exim instance. (I have no instances of > "retry time not reached" at all in my logs.) That suggests a > misconfiguration, in Exim, in Mailman, or Postorius. You should check > the mailman3_router and the mailman3_transport first because they're > easier. > > Steve > >

1 year, 5 months

[MM3-users] Re: Debian Package Install - Ownership and Permissions

by Mark Sapiro

On 4/7/23 14:58, Dave Hall via Mailman-users wrote: > On Fri, Apr 7, 2023 at 5:47 PM Mark Sapiro <mark(a)msapiro.net> wrote: > >> On 4/7/23 13:43, Odhiambo Washington wrote: >>> On Fri, Apr 7, 2023 at 8:09 PM Andreas Schamanek < >>> as2020+github(a)fam.tuwien.ac.at> wrote: >>>> >>>> I am running MM2 und 3 in parallel without problems, I do have a >>>> special MTA setup though (Sendmail). Can't speak for Postfix etc.. >>>> >>> >>> Your setup can handle list1(a)domain1.name via MM2 and MM3, >> simultaneously, >>> right? :-) >> >> It is possible to have an MM2 `list1` and an MM3 `list1(a)domain1.name` >> simultaneously on the same server, but what is the MTA going to do with >> incoming mail to `list1(a)domain1.name`. I suppose it could deliver it to >> both MM2 and MM3, but I don't see any practical use case for this. >> >> I run Exim4 on my servers so I can comment from this perspective. Exim > had a concept of 'routers' which are stacked and processed in order of > appearance. There is a router for MM2 and one for MM3 that pick off > incoming mail based on whether the email address matches a list name. WIth > this setup, you could have the same list name in both MM2 and MM3, but it > would go to whichever router appears first and the second one would never > see it. So I have the MM3 router first, and as I migrate my lists from > MM3 the email for those lists no longer gets to MM2. And mail.python.org which currently has 189 Mailman 3 lists and 223 Mailman 2.1 lists does something very similar in it's Postfix configuration and also prefers Mailman 3. However, maybe I misunderstood Odiambo's question about simultaneous support for a Mailman 2.1 and Mailman 3 list of the same name. We could certainly have that on mail.python.org, and even have both MM 2.1 and MM 3 web UIs for those lists, but as soon as the MM 3 list is created during migration, mail is only delivered to the MM 3 list. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years

[MM3-users] Re: ARC

by Marvin Gülker

Am 15. August 2019 um 02:08 Uhr +0900 schrieb Stephen J. Turnbull: > That doesn't help you one bit with your "rejection with no reason" > problem, though. It helps you with one specific reason (DMARC > p=reject). I don't have that problem anymore since I pay for my SMTP relay provider. For me, the issue is solved. Before my personal mail (I'm not talking about mailing lists, just simple direct e-mail) was pretty much always sorted into the recipient's spam folder or directly rejected at the SMTP level (this was the case with both Microsoft and AOL; GMail usually filed my e-mail als spam). I looked up the reasons they gave in the SMTP reject; it was always bad reputation of IPs. Tools like mail-tester on the other hand say my e-mail and IP was fine, and delivery to small providers always went nicely. My mail server has never sent a single spam e-mail. I run it since years, and I am very sure that this never has happened. This mail server only hosts my and my family's e-mail. I have no AOL contacts anymore, so I left that is it was. For Microsoft (i.e. outlook.com) I went through their IP unblocking formula (which at the time contained a CAPTCHA that I'm sure a computer could solve better than me simple human) multiple times, and I received "conditionally mitigated" as a response each time. After a few months, the rejections would start again. Thus, I say they were rejecting my e-mail for no reason. Might be a hyperbole, but from my point of view, there was no reason. Again, the problem is now past with my workaround, so there is no need to investigate any further. I've given up on it. I don't believe you can have a VPS that directly delivers e-mail to the large providers successfully. The workaround with the SMTP relay provider still allows me to self-host, which fulfills my needs. > The answer to that is "why shouldn't they?" I hope it goes like that. > I've never had a problem with any of the big services. There are > reasons why some hosts might have problems due to no misbehavior of > their own, and they wouldn't necessarily know about them either. But > mostly the mail does go through. Lucky you. My experience was the complete contrary, and it's not just me. Take a look: <https://lobste.rs/s/lvajly/google_is_eating_our_mail> > If the mail itself gets through, I see no reason why they wouldn't > trust your ARC fields, so you won't get DMARC rejections, as long as > nobody downstream of you breaks your ARC signatures. So, my situation currently is this. I have no (public) mailing lists on my own mail server, so there is no need for DMARC or ARC. Then, I'm involved in an open-source project that does have a mailing list and I'm considering to add ARC support for the list (one of the reasons why I constantly read this very mailing list). The project is small, so we'd like to avoid any effort that is fruitless anyway. Finally, I moderate the German ruby-de mailing list without any kind of influence on the software stack running it. That mailing list server runs Mailman 2.1.15 and this is not something I have any influence on. All of these list servers currently do not do ARC, hence I've not seen any rejections of ARC-signed messages (as there are none). I'm only evaluating how likely it'd be to see them. Please don't get me wrong; I appreciate all the effort that went into ARC! It looks like a good solution to the DMARC problems. > If you're seeing something different (ARC-authenticated messages > getting rejected for DMARC alignment failure), let us know and I'll > talk to a guy who knows this gal. This is a nice move. Thank you for this offer. When that happens, I'm going to post here! > But the day-to-day tuning of algorithms is targeted at users with > mailboxes, not the lists that fill those mailboxes. I am aware of that. See, when DMARC was spread over the world, ARC was not available. Even the Mailman wiki itself said and still says that DMARC has "negative consequences for mailing lists" and is "breaking long established mailing list norms, standards, and behaviors."[1] The problem still exists. On the ruby-de mailing list I moderate, there was a problem with a person sending e-mail from a p=reject domain to the list[2] (which is, remember, run by Mailman 2.1.15). Two subscribers were force-unsubscribed for the DMARC bounces. I had to manually re-subscribe them and had to explain the p=reject problem to the person sending the e-mails. This was... frustrating. And I think my statements on DMARC show my frustration. > Last I heard the Google folks and the IETF are very happy with ARC. I appreciate it. I hope it solves these problems. It's just that I'm reluctant to believe there's a quick solution available. It sounds to good to be true. > > because it is simply unfeasable to manually maintain a list of all > > open-source project mailing lists in the entire world. I could set > > a new one up tomorrow, how should you know? > > That's not how this works. I'm afraid it does. Fastmail does it exactly like that: manual white list, see <https://www.fastmail.com/help/technical/senderauthentication.html>. Quote of the relevant passage: > Until the list of ARC enabled forwarders grows we maintain lists of > known good email forwarders and mailing lists, which we use to > whitelist known-good mail forwarders, and apply this list when > evaluating DMARC policies for inbound mail. To do them justice, they say a little further down that a failed DMARC check at a p=reject policy domain does not cause an actual reject, but only increases the spam score. What worries me is that other providers might go that whitelisting approach, but don't follow the second part and still obey p=reject. If everything goes well, nobody is going to do that and ARC is successful. I'm all in for that result, but I lack the knowledge to judge what is likely and what not. Which is why I'm asking here. From your answer, I see that you think such behaviour is unlikely. That's good news, and it answers my question. Thanks! [1] https://wiki.list.org/DEV/DMARC [2] See July archive: https://lists.ruby-lang.org/pipermail/ruby-de/2019-July/thread.html -- Blog: https://mg.guelker.eu

5 years, 8 months

[MM3-users] Re: Problems after upgrading from bullseye to bookworm

by Odhiambo Washington

The virtualenv documentation works just fine with Bookworm AFAICT. I didn't need to do anything out of the documentation in my test VM. On Tue, 27 Jun 2023, 18:04 Eggert Ehmke via Mailman-users, < mailman-users(a)mailman3.org> wrote: > Hello Mark, > > thank you. I did change that entries, and now my mailman runs fine on > Debian Bookworm. > > I did a complete reinstall of the venv directory to have a clean start. > Maybe this was not > needed. What's missing in the documentation: > -the directiories web/logs have to be created manually with the ownership > mailman. > -The debian package pkg-config must be installed. > -the package psycopg2-binary can/must be installed without version > restriction on > Bookworm > Maybe this info helps some other people. > > Cheers, Eggert > > Am Montag, 26. Juni 2023, 22:22:37 CEST schrieb Mark Sapiro: > > On 6/26/23 12:35 PM, Ken Alker wrote: > > > I'm brand new to this, so take this with a grain of sale, but, I had a > > > similar problem a few days ago, however, mine was "flufl.i18n>=3.2". > > > Perhaps the same thing is happening with flufl.lock (but that didn't > bite > > > me). > > > > > > I solved my problem by following this: > > > <https://gitlab.com/mailman/mailman/-/issues/1085> and, ultimately > > > modifying > > > > "$VENV_ROOT/lib/$PYVERSION/site-packages/mailman-3.3.8.egg-info/requires. > > > txt" per that thread (see > > > <https://gitlab.com/mailman/mailman/-/issues/1085#note_1442207929> by > > > Thomas Ward section called "The WORKAROUND"). Maybe you need to do the > > > same thing, but for flufl.lock. > > Yes, the same issue now affects flufl.lock as well as flufl.i18n. You > > will need to change both flufl.lock->flufl_lock and > flufl.i18n->flufl_i18n. > > > > ___________________________________________ > Mailman's content filtering has removed the > following MIME parts from this message. > > Replaced multipart/alternative part with first alternative. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to odhiambo(a)gmail.com >

1 year, 9 months

[MM3-users] Re: Postorius 1.2.3 and MailmanClient 3.2.1 is now out

by Jonas Meurer

Jonas Meurer: > Abhilash Raj: >> On Mon, Jan 28, 2019, at 2:39 PM, Jonas Meurer wrote: >>> Abhilash Raj: >>>> On Sat, Jan 26, 2019, at 5:10 AM, Jonas Meurer wrote: >>>>> Hey Abhilash, >>>>> >>>>> Thanks a lot for your work on mailman3! >>>>> >>>>> Abhilash Raj: >>>>>> I am very pleased to announce (a bit late) that Postorius 1.2.3 and MailmanClient 3.2.1 is now out. >>>>>> >>>>>> The full set of changelog can be found in the documentation[1][2]. The most notable feature that >>>>>> I can think of is non-member management in Postorius. >>>>>> >>>>>> New releases are available on PyPI[3][4]. >>>>> >>>>> For earlier releases, you provided GnuPG signature files to >>>>> cryptographically verify that the release files were not tampered with. >>>>> >>>>> I couldn't find such signature files for the two latest releases. Would >>>>> you mind adding them? When packaging for Debian, I prefer to check the >>>>> GnuPG signatures before uploading new releases. Also, the signature >>>>> files get uploaded to the Debian archive as well so they can be checked >>>>> by everybody. That way we can cryptographically proof that the Debian >>>>> package is based on the original tarball that got packaged and signed by >>>>> upstream. >>>> >>>> >>>> Ah, sorry about that. I completely forgot about that. >>>> >>>> I am not sure if I can upload signatures after uploading the tarballs, I'll try to find out if that is possible. >>> >>> Thanks a lot, that would be awesome. I'll wait for it before doing the >>> Debian package uploads. >> >> If it is not possible to upload signatures, I'll bump up patch versions >> and push out new tarballs with signatures. > > Perfect, thanks a lot! For the record: Abhilash released mailmanclient 3.2.2 and postorius 1.2.4 with tarball signatures in the meantime. Thanks a lot for doing this, Abhilash! Both are uploaded to Debian already. Cheers jonas

6 years, 2 months

[MM3-users] Re: hello， Discussion on how to subscribe to the mailing list

by Mark Sapiro

On 1/16/23 18:55, 朱超 wrote: > Hi, When I subscribe the email list in Subscription Policy is confirm, Some problems happen in here： > 1.Some mail mua reject send the confirm the message, because the length of recipient is too long. for exmaple: requirements-confirm+9605556e0b35a5da280c1de0d34a3e6f770f8ea5@domain, and the mail mua reject to modify about this. The default English template says in part: Before you can start using GNU Mailman at this site, you must first confirm that this is your email address. You can do this by replying to this message. Or you should include the following line -- and only the following line -- in a message to $request_email: confirm $token I.e. It gives an alternative method if `reply` doesn't work. You can provide a custom version of this template with different wording if you think you can improve it. > 2.It is inconvenient for users to use. Many people will mistakenly think that after clicking the subscribe button, it will be successful, and they will not process the email. If you set Subscription Policy to Confirm, you are saying that the user must confirm that that is their email address. This requires sending an email requesting confirmation to that address. If you don't want that step, you can set Subscription Policy to Open, but then anyone can maliciously subscribe third parties to the list. > Regarding the discussion on the subscription method of the mailing list, should we consider optimizing the subscription method of the mailing list, for example: > 1. use the method of sending a verification code to subscribe by email，When you receive the verification code sent by the mailing list and fill it in again, you can subscribe successfully。 How is this different from the present method. If you want, you can make a custom template and include something like: Alternatively, you can confirm by going to the URL https://www.example.com/mailman3/lists/$list_id/confirm/?token=$token We don't include that in the default template because Mailman core doesn't know if you even have a web UI or how to access it if you do. > 2. use the email to send a connection to confirm the subscription，People click the link to be redirected to a confirmation page to subscribe to the email。 See above. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 3 months

[MM3-users]Installation issues

by Hellings Simon (EAST KENT HOSPITALS UNIVERSITY NHS FOUNDATION TRUST)

Hi folks I was speaking to someone last night on the IRC chat room @ http://webchat.freenode.net/ who suggested sending an email to this address to try and find a solution to my issues. I have been trying to get Mailman 3 installed on an Ubuntu 14.04 server. I thought I had followed the instructions from https://wiki.list.org/DEV/Mailman%203.0/Mailman%203.0%20Suite%20Install%20o… and http://mailman-bundler.readthedocs.org/en/latest/ but am getting an error 500 when I try and browse to http://x.x.x.x:8000/mailman3. I can open http://x.x.x.x:8000/archives and the login button seems to work, but when I click manage lists I get an error 500 again. The entire process I followed is below: * sudo apt-get update * sudo apt-get install git python3-dev python3-pip python-dev python-pip python-virtualenv * sudo apt-get install nodejs npm * sudo npm install -g less * sudo ln -s /usr/bin/nodejs /usr/bin/node * created new user 'mailmanuser' * 'mkdir /opt/mailman' * 'sudo chown mailmanuser:mailmanuser mailman' * 'su mailman' * 'cd /opt/mailman' * git clone https://gitlab.com/mailman/mailman-bundler.git * virtualenv -p /usr/lib/python2.7 venv * source venv/bin/activate * changed password in mailman_web/testing.py * cd mailman-bundler * pip install zc.buildout * buildout * sudo apt-get install ruby-sass * ./bin/mailman-post-update * ./bin/mailman-web-django-admin createsuperuser * ./bin/mailman start * ./bin/mailman-web-django-admin runserver 0.0.0.0:8000 I have attached the django and mailman logs and the top of the page I get back from the server I must have missed something but I don't really know where to go from here... Please can you review and advise if there is anything that I have missed? I look forward to hearing from you in due course. Any pointers greatly received. Kind regards Simon ******************************************************************************************************************** This message may contain confidential information. If you are not the intended recipient please inform the sender that you have received the message in error before deleting it. Please do not disclose, copy or distribute information in this e-mail or take any action in reliance on its contents: to do so is strictly prohibited and may be unlawful. Thank you for your co-operation. NHSmail is the secure email and directory service available for all NHS staff in England and Scotland NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and GSi recipients NHSmail provides an email address for your career in the NHS and can be accessed anywhere ********************************************************************************************************************

9 years

[MM3-users] Re: Timeout error on anonymous subscribe

by Mark Sapiro

On 10/23/24 23:36, vogt(a)spamcop.net wrote: > > I have tried anything I could think of but I cannot reproduce the issue using curl as user mailman (or any other for that matter) > > Going directly for localhost:8000, using some or all of the headers I can capture with tcpdump: > > ``` > $ curl -v -H 'Accept-Encoding: gzip, deflate' -H 'Connection: Keep-Alive' -H 'Host: lists.example.org' -A 'python-requests/2.32.3' http://localhost:8000/mailman3/api/templates/list/community.lists.example.c… > ``` > > as well as going through the local httpd: > > ``` > $ curl -v --http1.1 -H 'Accept-Encoding: gzip, deflate' -H 'Connection: Keep-Alive' -H 'Host: lists.example.org' -A 'python-requests/2.32.3' https://lists.example.org/mailman3/api/templates/list/community.lists.examp… > ``` > > Running tcpdump during those runs shows me it's sending the same headers as mailman does. Thus on http protocol level it's the same, but still the server returns immediately. > > What I have just noticed is that the Date header in the delayed response is the "late" time. I have even compared tcp headers from either capture. They look identical down to the frame sizes. > > The only difference the last times is that with curl, the server side sends the ACK for the request and milliseconds later a PSH,ACK with the response headers. When it comes internally from mailman it sends the ACK for the request and then 5 seconds later it sends the PSH,ACK with the response headers. I'm unable to explain what's going on. You could try one more thing in an attempt to duplicate this outside of Mailman. Invoke the same python3 that Mailman does - this may be in a venv or the system python3. If in a venv, you don't need to activate the venv, just run venv/bin/python. Then in response to Python's prompts do ``` >>> import requests >>> res = requests.get('https://lists.example.org/mailman3/api/templates/list/community.lists.examp…', timeout=5) ``` and see what that does? I think it should time out because this is exactly what Mailman core is doing tho get the template, and if it does, perhaps you can determine why from this simple example. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 months, 4 weeks

[MM3-users] Re: Timeout error on anonymous subscribe

by Gerald Vogt

On 24.10.24 22:09, Mark Sapiro wrote: > On 10/23/24 23:36, vogt(a)spamcop.net wrote: >> The only difference the last times is that with curl, the server side >> sends the ACK for the request and milliseconds later a PSH,ACK with >> the response headers. When it comes internally from mailman it sends >> the ACK for the request and then 5 seconds later it sends the PSH,ACK >> with the response headers. > > I'm unable to explain what's going on. You could try one more thing in > an attempt to duplicate this outside of Mailman. Invoke the same python3 > that Mailman does - this may be in a venv or the system python3. If in a > venv, you don't need to activate the venv, just run venv/bin/python. > > Then in response to Python's prompts do > ``` > >>> import requests > >>> res = requests.get('https://lists.example.org/mailman3/api/ > templates/list/community.lists.example.com/list:user:action:subscribe', > timeout=5) > ``` > and see what that does? I think it should time out because this is > exactly what Mailman core is doing tho get the template, and if it does, > perhaps you can determine why from this simple example. I have tried it using the fqdn url as well as the localhost:8000 url but in both cases requests.get immediately returns the response. The server side pushes the response within milliseconds. But that made me looking another time at the localhost tcpdump: I have noticed two connections to localhost:8000 where gunicorn with mailman_web.wsgi:application is running. The first one is POST /mailman3/lists/community.lists.example.com/anonymous_subscribe the second one is GET /mailman3/api/templates/list/community.lists.example.com/list:user:action:subscribe Both take 5 seconds. First the first one responds then shortly after the second one. I have noticed that only two gunicorn processes are running which looked like only one is a worker. Thus I have added workers=9 (2xncore+1) to /etc/mailman3/gunicorn.conf and restarted mailmanweb.service. Now it's working. So it must have been some kind of deadlock situation because it only had a single worker running. I have just followed the docs from https://docs.mailman3.org/en/latest/install/virtualenv.html#setting-up-guni… which doesn't mention anything about workers and I seriously don't have a clue about gunicorn. Maybe that could be added to the docs? Thanks for the help! Cheers, Gerald

5 months, 4 weeks

[MM3-users] Re: Multi-domain oddities in Hyperkitty and Postorius

by Jeremy Stanley

On 2023-02-15 18:35:35 -0800 (-0800), Mark Sapiro wrote: > On 2/15/23 07:52, Jeremy Stanley wrote: > > > > Thanks, I'm still doing my best to wrap my brain around how Django > > uses database migrations to create sites (seems like a very strange > > use of a migration, I'm only accustomed to seeing them applied for > > database schema updates). > > I don't think it does. There are a couple of migrations at > django/contrib/sites/migrations, but these are only the initial creation of > the model and an update to make the domain field unique. > > Why do you think it creates sites in this way. [...] Clearly a total misreading on my part of the Django documentation to which you referred earlier. It talks about having to feed Django multiple settings.py files with different SITE_ID values and running migrate to configure them, then links to the document on migrations which starts out explaining how to use makemigrations to create them. I took that to mean creating and configuring Django sites from the command line required supplying customized database migrations. The particularly misleading bits are where it says: "To set the correct name and domain for your project, you can use a data migration." "In order to serve different sites in production, you’d create a separate settings file with each SITE_ID" It doesn't seem to talk about using django_admin at all (or if it does, it's not mentioned by name, maybe assuming the reader knows to use it in order to perform the necessary steps). Thanks for the clear examples! If these are available somewhere in Django's documentation, I did a terrible job of searching for them. > you need to create a domain in Postorius. Look at the code behind > Postorius add new domain. Or you can create the domain in core via > REST, but you then have to modify it in Postorius to associate the > correct site. [...] Much appreciated, this was the other bit that was non-obvious to me. I guess you're saying Postorius's view of domains is independent from what's in Mailman Core, and there's no documented CLI/API for manipulating that in Postorius so I'll need to write one based on a reverse-engineering of its source code? Sounds doable now that I know where to look. Thanks again! -- Jeremy Stanley

2 years, 2 months

[MM3-users] Re: Mailman3 on AlmaLinux 9 - Python3.9 errors

by Willie Castillo

I saw them. OK, I must be missing several steps. I'll have to start from scratch. Let me carefully follow the link you send me, it should work for Almalinux 9.4 right? We recommend installing Pypi packages in a virtualenv. Doc is at https://urldefense.us/v3/__https://docs.mailman3.org/en/latest/install/virt… [cid:92ffd146-5b65-4e46-a2f8-021ea40371e9] Willie Castillo Linux Systems Administrator, IT Department - Defense & Security CAE USA T. +1-813-885-7481 E. Willie.Castillo(a)caemilusa.com A. 4908 Tampa West Blvd, Tampa, Fl. 33634 cae.com CONFIDENTIALITY NOTICE This e-mail message is intended only for the above named recipient(s) and may contain information that is confidential. If you have received this message in error or are not the named recipient(s), please immediately notify the sender, delete this e-mail message without making a copy and do not disclose or relay this e-mail message to anyone. ________________________________ From: Mark Sapiro <mark(a)msapiro.net> Sent: Thursday, July 18, 2024 2:55 PM To: mailman-users(a)mailman3.org <mailman-users(a)mailman3.org> Subject: [NON-INTERNAL] [MM3-users] Re: Mailman3 on AlmaLinux 9 - Python3.9 errors [You don't often get email from mark(a)msapiro.net. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. On 7/18/24 11:12 AM, willie.castillo(a)caemilusa.com wrote: > Mark first let me thank you for your assistance with this. I don't believe I mentioned that this is a fresh/new installation I'm performing on an AlmaLinux 9.4 VM. I have searched everywhere for a solid Mailman installation on AlmaLinux document/process but, I haven't had any luck. I tried following documentation on this site and I only found this https://urldefense.us/v3/__https://docs.mailman3.org/en/latest/install/dist… . Did you see my post at https://urldefense.us/v3/__https://lists.mailman3.org/archives/list/mailman… We recommend installing Pypi packages in a virtualenv. Doc is at https://urldefense.us/v3/__https://docs.mailman3.org/en/latest/install/virt… -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://urldefense.us/v3/__https://lists.mailman3.org/mailman3/lists/mailma… Archived at: https://urldefense.us/v3/__https://lists.mailman3.org/archives/list/mailman… This message sent to willie.castillo(a)caemilusa.com ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: Outlook-2rvyqd2h.png Replaced multipart/alternative part with first alternative.

9 months

[MM3-users] If I manually add email addresses to a list and then delete them, how do i permanent delete them from Users?

by Mark London

I added some email addresses manually to a mailing list and then removed them. But now I can't find how to totally remove them from mailman3. They show up in the Users list. When I go to the Django interface, I don't see them listed in any selection. I thought they would show up under Emails in Hyperkitty, but they are not there. Oddly, somehow I've managed to create multiple duplicates of the same address, under Emails. Figure that one out! Thanks. - Mark 21 mrl(a)psfc.mit.edu April 25, 2023, 4:54 a.m. [Mark] afasdfa mark.lists1.psfc.mit.edu 20 mailman(a)psfc.mit.edu April 25, 2023, 4:44 a.m. [Mark] Re: tet mark.lists1.psfc.mit.edu 19 london(a)mit.edu April 25, 2023, 4:31 a.m. [Mark] tet mark.lists1.psfc.mit.edu 18 mrl(a)psfc.mit.edu April 25, 2023, 3:28 a.m. [Mark] test mark.lists1.psfc.mit.edu 17 mailman(a)psfc.mit.edu April 25, 2023, 3:27 a.m. [Mark] tet mark.lists1.psfc.mit.edu 16 mailman(a)psfc.mit.edu April 25, 2023, 3:22 a.m. [Mark] Re: foobar mark.lists1.psfc.mit.edu 15 mrl(a)psfc.mit.edu April 25, 2023, 3:21 a.m. [Mark] test mark.lists1.psfc.mit.edu 14 mrl(a)psfc.mit.edu April 25, 2023, 2:05 a.m. [Mark] ;jklj mark.lists1.psfc.mit.edu 13 mrl(a)psfc.mit.edu April 25, 2023, 1:53 a.m. [Mark] foobar mark.lists1.psfc.mit.edu 12 mrl(a)psfc.mit.edu April 25, 2023, 1:52 a.m. [Mark] test mark.lists1.psfc.mit.edu 11 mailman(a)psfc.mit.edu April 25, 2023, 1:51 a.m. [Mark] Re: test mark.lists1.psfc.mit.edu 10 mrl(a)psfc.mit.edu April 25, 2023, 1:42 a.m. [Mark] test mark.lists1.psfc.mit.edu 9 mrl(a)psfc.mit.edu April 25, 2023, 1:38 a.m. [Mark] test mark.lists1.psfc.mit.edu 8 mrl(a)psfc.mit.edu April 25, 2023, 1:40 a.m. [Mark] test mark.lists1.psfc.mit.edu 7 mailman(a)psfc.mit.edu April 25, 2023, 1:33 a.m. [Mark] foobar mark.lists1.psfc.mit.edu 6 mailman(a)psfc.mit.edu April 25, 2023, 1:32 a.m. [Mark] test mark.lists1.psfc.mit.edu 5 root(a)psfc.mit.edu April 25, 2023, 1:21 a.m. [Mark] foobar mark.lists1.psfc.mit.edu 4 root(a)psfc.mit.edu April 25, 2023, 1:20 a.m. [Mark] test mark.lists1.psfc.mit.edu 3 root(a)psfc.mit.edu April 25, 2023, 1:11 a.m. [Mark] test mark.lists1.psfc.mit.edu 2 root(a)psfc.mit.edu April 25, 2023, 1:14 a.m. [Mark] test mark.lists1.psfc.mit.edu 1 mailman(a)psfc.mit.edu April 25, 2023, 1:22 a.m. [Mark] test mark.lists1.psfc.mit.edu

1 year, 11 months

[MM3-users] Re: Mailman web Stopped working after upgrade

by David Krantz

Hi, I tried psycopg2-binary and a few other rehashes of that. Now I got it working by downloading the source for psycopg2 manually and building it myself, then I installed it into my venv like any other self-built package. https://pypi.org/project/psycopg2/#files I run mod_wsgi in Apache so I thought it should be enough to restart Apache. I am quite convinced that I did that (and for that matter restart the machine at a few points). But I might have been sloppy and reversed the order in the end when I got it running, this process is a bit to manual. I will use another solution when I rebuild the server one day. The problem with rebuilding the server is that I'd have to install a MTA with all its modules, milters, setups and what not, and mailman. To just reinstall on a fresh machine would be easy, the trick is that I want to rebuild some things in a better way which will take me some time. E.g. using a separate wsgi service and a more modern spam filter than Spamassassin. Otherwise it is easy. But my situation now is a bit short on time so if I can postpone that a bit I will. Thanks. // David // David On Tue, Mar 3, 2020 at 10:02 PM Mark Sapiro <mark(a)msapiro.net> wrote: > > On 3/3/20 12:33 PM, David Krantz wrote: > > > > I think that this might be due to some kind of python misfeature where > > compiled code behaves slightly differently to interpreted code, that > > might explain the delay in it working initially before breaking. I > > have no idea of how to solve this. I have tried to uninstall and > > reinstlall psycopg2 in different versions without any luck. > > You need to activate your virtualenv and do > > pip install psycopg2-binary > > it must be psycopg2-binary, not just psycopg2. It's probably best to do > > pip uninstall psycopg2 > > first. > > As far as it working for a while, did you restart your wsgi service > (gunicorn, uwsgi, ??) after upgrading? > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

5 years, 1 month

[MM3-users] Re: Installing mailman 3

by Abhilash Raj

On Tue, Aug 1, 2017, at 02:29 PM, Rafael Mora wrote: > Hello Abhilash, > > I have this in my settings.py file: > > > EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' > EMAIL_HOST = 'localhost' > EMAIL_PORT = 25 > #EMAIL_HOST_USER = 'mailing(a)labbrands.com' > #EMAIL_HOST_PASSWORD = '******' This looks good to me. > > What should I use in user and password?? Does Django use postfix to send the email??? I think you don't need a password and username as long as your postfix is configured to relay local emails without authentication (which it is by default if you installed from distro packages and if you haven't changed the settings). > > Thanks > > > > > El mar., 1 ago. 2017 a las 16:20, Abhilash Raj (<raj.abhilash1(a)gmail.com>) escribió:>> Hi, >> >> >> >> On Tue, Aug 1, 2017, at 02:11 PM, Rafael Mora wrote: >> > It's ok now, but still I can't make it send the confirmation email>> > >> > This is from mailman.log: >> > >> > >> > [mailman@localhost logs]$ tail -f mailman.log >> > Aug 01 15:59:18 2017 (2327) lmtp runner started. >> > Aug 01 15:59:18 2017 (2333) virgin runner started. >> > Aug 01 15:59:18 2017 (2332) retry runner started. >> > Aug 01 15:59:18 2017 (2330) pipeline runner started. >> > Aug 01 15:59:18 2017 (2325) command runner started. >> > Aug 01 15:59:19 2017 (2328) nntp runner started. >> > Aug 01 15:59:19 2017 (2326) in runner started. >> > Aug 01 16:01:38 2017 (2331) 127.0.0.1 - - "GET >> > /3.0/lists?count=0&page=1&advertised=true HTTP/1.1" 200 90 >> > Aug 01 16:01:38 2017 (2331) 127.0.0.1 - - "GET >> > /3.0/lists?count=10&page=1&advertised=true HTTP/1.1" 200 90 >> > Aug 01 16:01:38 2017 (2331) 127.0.0.1 - - "GET /3.0/domains HTTP/1.1" 200>> > 90 >> > >> > >> > >> > >> > What should I do? >> >> If you are talking about Account confirmation emails, they usually are>> sent out from Django and not Mailman Core itself. >> >> To configure Django you need to edit the `settings.py` file, wherever>> that is one your installation. If you are following the guide mentioned>> at http://docs.mailman3.org/en/latest/prodsetup.html, it should be at>> `mailman-suite/mailman-suite_project/settings.py`. >> >> To configure your Django to send emails, have a look at >> http://docs.mailman3.org/en/latest/config-web.html#setting-up-email-required.>> >> That documentation page has pretty much everything you need in order to>> have a running Web Frontend, please go through it and make sure you have>> everything configured. >> >> Finally, the actual emails from Mailing lists are supposed to be handled>> by Core. You need to configure Postorius & Mailman both to communicate.>> The settings are mentioned >> http://docs.mailman3.org/en/latest/config-core.html#configuring-mta >> here. >> >> >> Hope that helps! >> >> > >> > >> > >> > >> > >> > >> > El mar., 1 ago. 2017 a las 15:36, Mark Sapiro (<mark(a)msapiro.net>)>> > escribió: >> > >> > > On 8/1/17 1:14 PM, Rafael Mora wrote: >> > > > >> > > > Why does it say Address already in use? >> > > >> > > >> > > Because you are starting lm tp runner and it's already running.>> > > >> > > Do 'mailman stop'. Then ensure no Mailman processes are running. kill>> > > any if necessary. Then do 'mailman start'. >> > > >> > > -- >> > > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,>> > > San Francisco Bay Area, California better use your sense - B. Dylan>> > > _______________________________________________ >> > > Mailman-users mailing list >> > > mailman-users(a)mailman3.org >> > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/>> > > >> > -- >> > >> > Atentamente / Best Regards >> > >> > Ing. Rafael Mora >> > _______________________________________________ >> > Mailman-users mailing list >> > mailman-users(a)mailman3.org >> > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/>> >> >> -- >> thanks, >> Abhilash Raj >> _______________________________________________ >> Mailman-users mailing list >> mailman-users(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/> -- > Atentamente / Best Regards > Ing. Rafael Mora -- thanks, Abhilash Raj

7 years, 8 months

[MM3-users] Re: cc'd e-mail addresses rewritten with @invalid.domain

by Mark Sapiro

On 9/21/20 10:49 AM, peterbw(a)gmail.com wrote: > Hi, > > We've recently noticed an issue where users who are cc'd in a message where a mailing list is also within the cc list are getting rewritten a user(a)invalid.domain. > > This issue looks like it may be related to Outlook and how user names are displayed specifically. Here is an example: > > To: test_user(a)gmail.com > cc: listserv(a)lists.test.org; Williams, Peter; > > "Williams, Peter" maps to a user e-mail in the Outlook Global Address book. When the message is sent the list will send the message with the first name and last name separated, for example the message will appear: > > to: test_user(a)gmail.com > cc: listserv(a)lists.test.org, Williams(a)invalid.domain, Peter(a)invalid.domain Because apparently Outlook is sending the message with the literal > Cc: listserv(a)lists.test.org; Williams, Peter; instead of something like > Cc: listserv(a)lists.test.org, "Williams, Peter" <peters_real_address(a)example.com> Then some MTA in the chain is seeing the unqualified addresses Williams and Peter and appending its own local domain. > This only seems to happen with users who are cc'd who are also members of the list that is being cc'd. For example in the scenario above my e-mail that is cc'd is also a member of listserv(a)lists.test.org. In this case, where do you see this? i.e., do other list members see it in the message they receive from the list? I don't know why that would be. I'm certain that Mailman is not involved in this and the Williams(a)invalid.domain and Peter(a)invalid.domain addresses are in the Cc: as received by Mailman. > This issue does not occur if the user e-mail address is listed before the mailing list in the cc field. So for example, if the cc in the scenario above contained Williams, Peter; listserv(a)lists.test.org; it would route properly. Again, I think this is Outlook. What's in the Cc: of the mail that test_user(a)gmail.com receives directly? What happens if you send mail in this way with To: test_user(a)gmail.com cc: other_user(a)gmail.com; Williams, Peter; Again, what's in the Cc: of the mail that test_user(a)gmail.com and other_user(a)gmail.com receive? Also, please see <https://wiki.list.org/DOC/Mailman%20is%20not%20Listserv>. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 7 months

[MM3-users] Re: MM attaches mime-attachments to mail as text file since update to 3.3.8

by Stefan Bauer

Dear Mark, thank you for your time. Here are the lists settings and an example mail attached. Filter content: NO Filter types: EMPTY Pass types: EMPTY Pass extensions: EMPTY Collapse alterantives: YES Convert html to plaintext: NO Thunderbird is showing the mail "correct". Outlook does show the attachments as attachments. Thank you. Stefan Am Mi., 18. Jan. 2023 um 18:02 Uhr schrieb Mark Sapiro <mark(a)msapiro.net>: > On 1/17/23 23:45, Stefan Bauer wrote: > > Dear Users & Developers, > > > > we're running MM3 on Ubuntu 22 and used the venv-installation method. > > Last week we updated via¹ to > > > > Mailman Core-Version GNU Mailman 3.3.8 (Tom Sawyer) > > Mailman Core API-Version 3.1 > > Mailman Core Python-Version 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC > > 11.3.0] > > > > and now mailman attaches mime-attachments to email as text-files. That > > breaks many mails in our environment. > > > > For example: Now we have attachments like > > > > smime.p7s + ATT000001.txt as well as what looks like MM footers > > "mailing list --- test@domain ... To unsubscripbe send an email to ..." > > > > How can we get the "old" processing back? > > I doubt this has anything to do with the upgrade per se. It has to do > with MIME multipart (in this case, signed) messages and how MM > headers/footers are added. The MM 2.1 FAQ article at > <https://wiki.list.org/x/4030707> has some info on this. > > In order to say more, I would need to see the list's settings for > Filter types, Filter extensions, Pass types, Pass extensions, Collapse > alternatives and Convert html to plaintext and the complete MIME > structure of the incoming message. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to cubewerk(a)gmail.com > ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Replaced multipart/alternative part with first alternative.

2 years, 3 months

[MM3-users] Re: Mixing Mailman 3 with AWS SES

by Andrew Hodgson

Ralph Weber <roweber(a)ieee.org> wrote: >I am moving the website www.t10.org from a private/corporate VM to AWS EC2s. >For many years, t10.org has had very happy results with using Mailman 2 for its email reflector, and I have no real concerns about moving to Mailman 3. My bugbear is the fact that I need the following mixed bag of incoming email addresses to work in concert with each other to receive and handle emails. [...] >My best guess is that the AWS Simple Email Service (SES) needs to sit in front of both Mailman and the home-grown code, to properly direct the incoming emails to the right places. The big worry is as follows... Why do you believe that is the case? In your current setup your mail server acts as primary MX, the server runs an MTA with an interface to Mailman. Why would you believe this to be different in the AWS environment? >All available evidence read to date suggests that the default AWS installation of Mailman 3 assumes that Mailman 3 is the *only* receiver of emails in the configuration. If true, then putting SES in front of Mailman could be a Herculean challenge. Which instructions are these? When I read about using Mailman on AWS, my experience is that getting email into the system is fine because AWS allows incoming connections on port 25, and so long as you have a static public IP address attached to the instance that routes traffic correctly incoming mail is fine. The challenge is with outbound mail from your server, as directly sending outbound email from AWS IP addresses is an issue because the AWS Ips are listed on many spam lists as spammers have been known to set up instances and abuse them by sending out bulk mail. Sending outbound emails via the SES service from Mailman is possible, but it comes with some caveats due to the way that SES validates email addresses on the platform. Some users have been able to get it working on this list, but have had to put the lists into an anonymous mode where the email of the sender is obfuscated so as not to trip up SES. For this reason I haven't moved my Mailman instances onto the public cloud, I instead use a VPS provider who's Ips have a good sending reputation. I am still interested in doing this work however, just need to work out the best option for sending out the email effectively. Thanks. Andrew.

4 years, 1 month

[MM3-users] Re: nginx configuration on a multitasking server

by David Newman

On 12/10/21 4:48 PM, Mark Sapiro wrote: > On 12/10/21 3:22 PM, David Newman wrote: >> On 12/10/21 12:57 PM, Mark Sapiro wrote: >>> >>> Are you using uwsgi or some other wsgi server. >> >> uwsgi, per the venv and PDF docs. Thanks for the fix. >> >> Regarding the uwsgi_pass docs, would 'uwsgi_pass_header' directives >> replace the 'proxy_set_header' ones in the venv doc? >> >> Looking at the uwsgi doc, it seems they might do different things. > > > If you are using uwsgi and proxy_pass http://127.0.0.1:8000 is working > for you, then I'm satisfied that the docs are OK now and we don't need > to say anything about uwsgi_pass. > > > >> It's not the situation I have to deal with, though. Short of changing >> a long-standing URL, what I'm looking to do is have "/" continue to >> default to roundcubemail, and have users type in longer URLs for MM3 >> and other services. > > > I think you can set the document root to the roundcubemail root and then > set locations for the Mailman destinations as, e.g. > > location /mailman3/ { > proxy_pass http://127.0.0.1:8000; > proxy_set_header Host $host; > proxy_set_header X-Forwarded-For $remote_addr; > } > location /archives/ { > proxy_pass http://127.0.0.1:8000; > proxy_set_header Host $host; > proxy_set_header X-Forwarded-For $remote_addr; > } > location /accounts/ { > proxy_pass http://127.0.0.1:8000; > proxy_set_header Host $host; > proxy_set_header X-Forwarded-For $remote_addr; > } > location /admin/ { > proxy_pass http://127.0.0.1:8000; > proxy_set_header Host $host; > proxy_set_header X-Forwarded-For $remote_addr; > } > location /user-profile/ { > proxy_pass http://127.0.0.1:8000; > proxy_set_header Host $host; > proxy_set_header X-Forwarded-For $remote_addr; > } > > I haven't tried anything like this, so I'm not sure it will work, but I > think so. Update: The above worked fine until MM3 informed a list owner of pending subscription requests, with a URL like this: https://lists.example.com/mailman/admindb/listname and that URL returned a 404 error. Adding a stanza for 'location /mailman/' to the nginx config above clears the 404 error, but Postorius returns "Page not found" with no entry in the web or MM3 logs, and a 200 OK in the Nginx logs. I think the URL is supposed to go to the list page in Postorius. How to configure this so that list owners and moderators get a valid URL for list management? Thanks! dn

3 years, 4 months

[MM3-users] Re: Question to add "alias" address in list by group

by Pierre Malard

Thanks, > Le 28 avr. 2022 à 15:17, Stephen J. Turnbull <stephenjturnbull(a)gmail.com> a écrit : > > Pierre Malard via Mailman-users writes: > >> We must add some (a lot) alias address for members and I don’t find >> any indications on doc to do that with « mailman » command. > > I have no idea what you mean by an "alias address" that can be added > for many members simultaneously. There is no concept of "alias" in > Mailman, and almost certainly no special command for this. These are authorized alternate email addresses. For example A@domain is subscribed to the Z list of which only members can contribute but we also want to authorize some other address of A (A@domain2, …) as an « alternative" email contributor. In the Mailman2 version this was called "accept_these_nonmembers" and it actually got added to that "non-members" list in mailman3. > > The general way to add a lot of addresses for a single list is the > mass add command in Postorius's users tab, or an equivalent script > using mailmanclient. If you need to do this at the user level (rather > than the subscription level), you will need to write a script for it, > and also to update subscriptions if that is what you have in mind. We would like to be able to add these "non-members" addresses with the "mailman" command line, indicating the assigned "role". But, obviously, the only available option managing the role is provided by "mailman members" which declares itself obsolete! :-( As far as the Python mailman libraries are concerned, we lack documentation. For example the possible parameters… Otherwise, working directly in the database seems dangerous to us. > > Steve > -- Pierre Malard « Je n'ai jamais séparé la République des idées de justice sociale, sans laquelle elle n'est qu'un mot » Jean Jaures - 1887 |\ _,,,---,,_ /,`.-'`' -. ;-;;,_ |,4- ) )-,_. ,\ ( `'-' '---''(_/--' `-'\_) πr perl -e '$_=q#: 3|\ 5_,3-3,2_: 3/,`.'"'"'`'"'"' 5-. ;-;;,_: |,A- ) )-,_. ,\ ( `'"'"'-'"'"': '"'"'-3'"'"'2(_/--'"'"' `-'"'"'\_): 24πr::#;y#:#\n#;s#(\D)(\d+)#$1x$2#ge;print' - --> Ce message n’engage que son auteur <-- ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Replaced multipart/alternative part with first alternative.

2 years, 11 months

[MM3-users] Re: Setup Question: Disk-Layout (Migrating from Mailman 2.1)

by Mihai Moldovan

* On 4/17/25 12:18, christian.schneider(a)tu-dortmund.de wrote: > I want to set up a new server (Ubuntu-24.04) and install Mailman-3 with the Virtualenv setup method: > https://docs.mailman3.org/en/latest/install/virtualenv.html > As I know from my old Mailman-2.1 the archives could get quite big... ;-) > So it seems very important to provide enough (expandable) disk space at the right place. > (I plan to use a seperate partition with LVM for the path in question) > > But I could find no information about the location where hyperkitty stores the archives on an Ubuntu-/Linux-system. > I assume the archives are stored underneath "/var/..." this is only a guess. > But is this guess right? Not quite. To the best of my knowledge, by default, a venv installation is fully contained under /opt/mailman, with archives and stuff stored at /opt/mailman/var/lib/... > Another question is: > How much space do I have to plan (roughly) for the "/opt"-partition for the mailman install if the archives are stored under "/var/..."? Depends on your archives, the way things are archived (you can choose different database backends, and these might make a difference in how efficiently or inefficiently data is stored) and whether you want to use the full-text search index, which adds another big chunk of data. I can give you a few pointers regarding my setup: my mailman 2.1 data directory takes up about 8.2 GB of disk space, most of which are archives, naturally (8.1 GB). I've imported most of the big archives to mailman 3 already, with a full-text search index, and that uses 5.2 GB disk space currently, but I'm still missing a few archives, so that'll still grow a bit. It will probably end up at about the same size, but I can come back to that once my import is fully done (which sadly will take a few days). Now, the 8.1 GB of mailman 2.1 archives are stored pretty inefficiently, because they are essentially duplicated (as a private mbox archive and public text archives). Mailman 3 doesn't do that - at least not directly - but instead stores archives once into the database, and queries the database whenever the archives are accessed via hyperkitty, so I would *assume* the size you need for the mailman 3 data after a successful migration is just a bit shy of what you needed for the mailman 2.1 data, maybe even a bit less. Mihai

6 days, 22 hours

[MM3-users] Re: Mailman3 throws 500 error (unicode encode) when loading held messages in Postorius

by Stephen J. Turnbull

Simon Handfield via Mailman-users writes: > I have been able to identify a file in the shunt queue that threw a > matching error when I performed mailman qfile on it. > I have run a loop over all remaining files in the shunt queue and > all the remaining files can be accessed by mailman qfile without > error. Good. But there are still a bunch of problems with those files since they got shunted. > I moved this file out of the queue in the hope that I would then be > able to load the held_messages page successfully. Unfortunately I > am still seeing a 500 error on loading the page. If this is a Mailman problem, I don't understand it. The shunt queue is not the moderation queue. The shunt queue is for messages that Mailman cannot handle without throwing an exception. Once a message is sent to the shunt queue, Mailman ignores it, so it shouldn't cause problems for the moderation page. Have you tried restarting Mailman since moving the qfile? If that doesn't work, and if it's the same error message and traceback, the easy suspect is browser caching (although we should be disabling that, I hope). Try a manual refresh (usually Ctrl-R or in the menu). If that doesn't work, try a private browsing window or another browser if you don't want to clear the cache or restart the browser. > I temporarily reenabled debug logging to confirm that the same > error was being thrown. > > Assuming that file was the cause are there other steps I need to > perform in addition to removing the file from the shunt queue > directory? The only one I can think of is the check for browser-side caching. Unfortunately, Mark's out of town (like, on top of a mountain with no bars on his phone), and I am not going to be able to do much with this until Thursday at the earliest. If you do get a new traceback, post it here (somebody else may have an idea). If you like you may send me the qfile at the address in From. I'll see if I can reproduce the bug, and I'm pretty sure I can tell you what's in the file so at least you'll know whether it's important to somebody who isn't a scammer/spammer. Steve

4 years, 7 months

[MM3-users] Re: dedicated mailman3 list server on Ubuntu 18.04 LTS - questions after installation

by Abhilash Raj

On Sun, Aug 5, 2018, at 3:37 PM, John Seifarth via Mailman-users wrote: > My goal is to build a stable dedicated list server using mailman3. It > should use all current email authentication techniques to clearly show > that it is an authentic non-spam mail list server. In the future, I may > want to use the REST interface, so that’s why mailman2 is not an option. > > My installation: > > Installed Ubuntu 18.04 LTS server, with postfix, mysql and postgres. I > then installed mail-stack-delivery, postfix-policyd-spf-python, > OpenDKIM, and OpenDMARC. I configured SPF, DKIM, and DMARC records on my > DNS server, and then configured postfix and dovecot to DKIM sign > outgoing mail from local users, and check SPF, DMARC, and DKIM on > incoming mail. > > I then installed the package mailman3-full, which provides version > 3.1.1-9, using postgres 10 as the database. I have looked in Ubuntu > bionic-backports and bionic-updates and do not see a more recent > version. I see that version 3.2 was released in July, are debian/ubuntu > mailman 3.2 packages available somewhere? > > I set up the /etc/mailman3 files: > > I added this to the end of mailman.cfg > [archiver.hyperkitty] > class: mailman_hyperkitty.Archiver > enable: yes > configuration: /etc/mailman3/mailman-hyperkitty.cfg > > In mailman-hyperkitty.cfg I copied the api_key: value from mailman- > web.py MAILMAN_ARCHIVER_KEY, but removed the single quotes surrounding > the value in the mailman-web.py file when writing the key in the > mailman-hyperkitty.cfg file. > > I modified /etc/postfix/main.cf with this appended to the end: > # Mailman-postfix integration > owner_request_special = no > # Support the default VERP delimiter. > recipient_delimiter = + > unknown_local_recipient_reject_code = 550 > owner_request_special = no > transport_maps = > hash:/var/lib/mailman3/data/postfix_lmtp > local_recipient_maps = proxy:unix:passwd.byname, $alias_maps, > hash:/var/lib/mailman3/data/postfix_lmtp > relay_domains = ${{$compatibility_level} < {2} ? {$mydestination} : {}} > hash:/var/lib/mailman3/data/postfix_domains > > I ran /usr/share/mailman3-web/manage.py createsuperuser > I ran a2enmod proxy_uwsgi > I made a symbolic link from /etc/mailman3/apache.conf to /etc/apache2/ > conf-available/mailman3.conf, then ran a2enconf mailman3 > > I have read the documentation in /usr/share/docs/mailman3* along with > what I could find concerning mailman3 on the web. > > The installation allows me to sign into the web interface, create a > host, create mailing lists, and mass subscribe members. When members > send email to the list address, the server sends the messages normally > to the other members of the list. > > However, when I look at the list archives, there are no posts. I > deliberately mass-subscribed an invalid address, and I can see in /var/ > log/mail.log that the message bounces, but I get no notification, nor do > I see any bounce info when I look at the member info. > > My questions: > > 1) logging > > A lot of my questions would probably be answered if I could figure out > hot to activate logging correctly. Currently, I see: > > root@mailman:/var/log/mailman3# ll -R > -rw-rw---- 1 list list 0 Jul 29 20:06 bounce.log > -rw-rw---- 1 list list 0 Jul 29 20:06 debug.log > -rw-r----- 1 list list 3298 Aug 4 00:00 mailman.log > -rw-rw---- 1 list list 34606 Jul 30 17:44 smtp.log > drwxr-xr-x 2 www-data www-data 4096 Aug 3 06:25 web/ > ./web: > -rw-rw-r-- 1 www-data www-data 1520 Aug 3 06:25 mailman-web.log > > The bounce log and debug log are empty. I’ve set propagate: yes in the > [logging.debian] section > > From /etc/mailman3/mailman.cfg > # In this section, you can define defaults for all loggers, which will be > # prefixed by 'mailman.'. Use subsections to override settings for specific > # loggers. The names of the available loggers are: > # > # - archiver -- All archiver output > # - bounce -- All bounce processing logs go here > # - config -- Configuration issues > <…snip…> > > Can anyone post an example of a ‘subsection’ to set archiver and bounce > to debug? Do I have to create /var/log/mailman3/mailman.archiver.log and > set ownership to list:list? [logging.archiver] level: debug [logging.bounce] level: debug > > > 2) Where are posts stored? > > For me, this is the biggest mystery in mailman3. Local mail is delivered > to ~/Maildir by dovecot. In the $var_dir hierarchy the directory > messages contains just 2 messages dating to my initial tests. Anything I > send to a list now, and which seems to be sent properly to the member > addresses, I cannot find on the filesystem. > > Any clues as to what may be going on here? Mailman by itself doesn't store anything, it just sends and receives. $var/queue directory should have all the posts that are currently in queue and are waiting to be processed. If you are talking about Archived messages, they should be in the database. > > 3) Archival - Hyperkitty > > How does this work? All the parts seem to be in place, and I can see > from ps > /usr/bin/python3 /usr/lib/mailman3/bin/runner --runner=archive:0:1 -C / > etc/mailman3/mailman.cfg > > How can I figure out why this is not working? In the web interface, I > see that each of my lists has zero posts. But posts mailed to the list > address do appear in the list members’ mailboxes! The runner above sends posts to Hyperkitty, which is a running Django app. You should check the logs if the messages are being successfully delivered to Hyperkitty. > > 4) Bounces > > I’ve deliberately put an erroneous email address as a list member. In > postfix’s /var/log/mail.log I see the bounces, but get no notification, > and /var/log/mailman3/bounce.log remains empty. > > 5) DKIM and DMARC > > I’ve got OpenDKIM running on inet:12345@localhost and OpenDMARC running > on net:54321@localhost. > > How can I get mailman to use these milters and sign messages? How should > DMARC munging be configured when signing? > > 6) Is there an overview of how mailman3 works? > > Is there documentation which explains the path of an incoming post > through the system, including initial reception, list processing, > sending to members, processing bounces, and archiving posts? > > Thanks in advance for help and pointers to any available documentation! > > > > John Seifarth > Words & Wires SPRL > Computer Consulting & Language Services > Brussels, Belgium > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ -- thanks, Abhilash Raj (maxking)

6 years, 8 months

[MM3-users] Various patches for mm3 stable 3.3.1

by Ruth Ivimey-Cook

Folks, For my own purposes I have created some patches which apply to the current mailman pip package. The first of them is for my purposes necessary, and has been discussed here before: being able to specify the address used to listen for tcp connections independently of the address the server considers it to be accessible at. A possible change to this patch would be to check whether this address was undefined but the old hostname was defined, and use that if so. For me this is not needed, and in any case it is a trivial change to make. The second patch adds a bunch of additional logging calls to various network-related parts of the package. I wrote these into the package because I was seeking the cause of template expansion causing messages to be silently dropped. I eventually found the cause was that I had an incorrect setting of the template root URL, but I regard the lack of anything in the logs as just as much of a bug. One important part of those changes was that I added a try/catch around two parts of the 'decorate' code, which is where the silent dropping occurred. If you only take one part of this patch, I encourage you to take this part. For the third patch, I discovered that the task runner was at one point busy-waiting to the tune of 100% cpu, and the cause was the way that the runner checked to see if there was work left in the queue. I am not quite sure what was the exact problem, but I think it was that there was something to do, but it was not achievable at that time, and the result was continual retry. The third patch changes the logic slightly so that this no longer occurs - retry will happen, but somewhat more gracefully. I hope you find some use in these three. Finally, I am currently experiencing problems with templates and would appreciate any thoughts: 1. If I do not set up 'individual' or 'full' personalisation, all is well and the default footer templates are expanded as expected. However setting up personalisation causes template expansion to fail completely, and messages get sent with a blank expansion. 2. I would find the postorius UI much easier to cope with if the default template was 'populated' into the UI wherever it was non-blank, rather than the UI starting off as if no templates were defined at all. Regards, Ruth

4 years, 6 months

[MM3-users]Re: Moderation Emails & Page

by Simon Hanna

> I’m running the current MM3 from the bundler on RHEL7, and most things seem to be working properly. > ERROR 2016-04-20 14:58:05,959 base 3256 140375419246336 Internal Server Error: /mailman3/lists/listname/held_messages > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response > response = wrapped_callback(request, *callback_args, **callback_kwargs) > File "/usr/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 22, in _wrapped_view > return view_func(request, *args, **kwargs) > File "/usr/lib/python2.7/site-packages/postorius/auth/decorators.py", line 77, in wrapper > return fn(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/postorius/views/list.py", line 413, in list_moderation > paginator_class=utils.MailmanPaginator) > File "/usr/lib/python2.7/site-packages/postorius/utils.py", line 89, in paginate > results = paginator.page(1) > File "/usr/lib/python2.7/site-packages/postorius/utils.py", line 62, in page > number = self.validate_number(number) > File "/usr/lib/python2.7/site-packages/django/core/paginator.py", line 39, in validate_number > if number > self.num_pages: > File "/usr/lib/python2.7/site-packages/django/core/paginator.py", line 86, in _get_num_pages > if self.count == 0 and not self.allow_empty_first_page: > File "/usr/lib/python2.7/site-packages/postorius/utils.py", line 75, in _get_count > result = self.function(count=1, page=1) > File "/usr/lib/python2.7/site-packages/mailmanclient/_client.py", line 681, in get_held_page > return Page(self._connection, url, HeldMessage, count, page) > File "/usr/lib/python2.7/site-packages/mailmanclient/_client.py", line 333, in __init__ > self._create_page() > File "/usr/lib/python2.7/site-packages/mailmanclient/_client.py", line 363, in _create_page > self._connection, entry['self_link'], entry) > KeyError: u’self_link' This one seems to be a new one. To my knowledge held messages should not work because of this issue: https://gitlab.com/mailman/mailmanclient/issues/10 Your issue seems to be with postorius' paginator. Could you open an issue in postorius? https://gitlab.com/mailman/postorius/issues Please include this trace, and some additional info. * What version of django are you using? * What version of postorius are you using? * What version of mailmanclient are you using? To actually confirm the issue is with postorius and not mailmanclient, you could try to access the held messages through mailmanclient directly The easiest way to do this, is start a django shell. You need to activate the python2 virtualenv for this Replace manage.py with whatever command you use to start the webserver python manage.py shell from postorius.utils import get_client lists = get_client().lists lists[0].held # pick a list that has held messages lists[0].get_held_page() # and you could try getting one specific message lists[0].get_held_message(<held id>) if you can do all this, then the issue is probably in postorius

9 years

[MM3-users] OperationalError: database is locked

by cyp＠trojina.si

On our installation, we are often gettting Internal Server Error (500). The logs look like this: ERROR 2018-01-09 13:52:57,640 1604 django.request Internal Server Error: /hyperkitty/list/test(a)seznami.cjvt.si/ Traceback (most recent call last): File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/exception.py", line 42, in inner response = get_response(request) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/HyperKitty-1.1.0-py2.7.egg/hyperkitty/lib/view_helpers.py", line 133, in inner mlist = MailingList.objects.get(name=mlist_fqdn) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 379, in get num = len(clone) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 238, in __len__ self._fetch_all() File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 1087, in _fetch_all self._result_cache = list(self.iterator()) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 54, in __iter__ results = compiler.execute_sql() File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/sql/compiler.py", line 835, in execute_sql cursor.execute(sql, params) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/utils.py", line 94, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/sqlite3/base.py", line 337, in execute return Database.Cursor.execute(self, query, params) OperationalError: database is locked Does this suggest I should switch from sqlite to postgres (https://stackoverflow.com/a/26864360)? I'm happy to do that, but how do I migrate the data between the databases (after installing postgres and creating the database)? I've found this doc (http://mailman.readthedocs.io/en/latest/src/mailman/docs/database.html#data…) but I don't think I understand this and I haven't used alembic: these instructions are just for when you modify the model within the same database, right? Thanks in advance!

7 years, 3 months

[MM3-users] rDNS issue? Helo name contains an IP address (HELO was [127.0.0.1]) and not is valid

by Kyriakos Terzopoulos

Hi all, I'm on the latest mailman installation in a HestiaCP server. The past few days I have been investigating some spam issues and by using dkimvalidator.com I found the following: 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager* 3.2 HELO_DYNAMIC_IPADDR Relay HELO'd using suspicious hostname (IP addr 1)* Note the last line on the above debug information which raises the spam score of the email. Some research showed me that I needed a rDNS entry in my server. I'm on a Hetzner cloud VM so I used the Hetzner console to change the rDNS entry from: static.XXX.XXX.XXX.XXX.clients.your-server.de to my initial server hostname for example: *hestia.myserver.com <http://hestia.myserver.com>* However, after some propagation, I found today that even though I can send email to any normal address created in the server's control panel, mailing list emails do not get sent. I see the following in the logs: Mailman SMTP log: Oct 24 14:17:47 2023 (3759611) <169814622178.3759638.4589434779084932533@localhost> *response exception: (550, b'Helo name contains an IP address (HELO was [127.0.0.1]) and not is valid'*, 'testlist-bounces+myusername=myproject.eu(a)myproject.eu') Oct 24 14:17:47 2023 (3759611) <169814622178.3759638.4589434779084932533@localhost> smtp to testlist(a)myproject.eu for 1 recips, completed in 45.07745146751404 seconds Oct 24 14:17:47 2023 (3759611) <169814622178.3759638.4589434779084932533@localhost> post to testlist(a)myproject.eu from testlist-confirm+07d3315fa11a75588857fc93d1832ba74fb09fdd(a)myproject.eu, 1633 bytes, 1 failures Oct 24 14:17:47 2023 (3759611) <169814622178.3759638.4589434779084932533@localhost> delivery to myusername(a)myproject.eu failed with code 550, b*'Helo name contains an IP address (HELO was [127.0.0.1]) and not is valid'* Exim4 log: 2023-10-24 14:17:47 H=([127.0.0.1]) [XXX.XXX.XXX.XXX] rejected MAIL <testlist-bounces+myusername=myproject.eu(a)myproject.eu>: *Helo name contains an IP address (HELO was [127.0.0.1]) and not is valid* The weird thing is that when I do a mailman-web sendtestemail this email is sent successfully. Also please note that before the rDNS change I did not have an issue with mail sending. Last but not least, after some search I set in Exim4 the following: PRIMARY_HOST_NAME = *hestia.myserver.com <http://hestia.myserver.com>* Any help would be greatly appreciated. *Kyriakos Terzopoulos *Web developer / e-learning expert *Tel:*+30 211 213 9858 *Mobile:* +30 694 526 4512 * E-mail: *kyriakos.terzopoulos(a)gmail.com * Skype:* kyriakos.terzopoulos Find me on Facebook <http://www.facebook.com/cirrus3d> Follow me on Twitter <http://twitter.com/#%21/cirrus3d>

1 year, 5 months

[MM3-users] Re: Archive Policy not reflecting in Hyperkitty

by Nathan Dixon

Update: I have manage to get this working. I had to delete all images in docker that were anything to do with mailman. Docker-compose rm didn't clean up enough stuff. Once there were no other mailman docker images, I then ran $ docker pull quay.io/maxking/mailman-web:0.2 $ docker pull quay.io/maxking/mailman-core:0.2 And docker-compose up didn't show any error messages relating to Hyperkitty this time and the archive policy is reflecting in Hyperkitty as it shows in Postorius. Thanks for your help Abhilash! Regards, On Mon, 13 May 2019 at 12:28, Nathan Dixon <nathan.dixon(a)evadon.com> wrote: > Hi, > I have removed the docker images and pulled the version tagged as 0.2. > > $ docker pull quay.io/maxking/mailman-web:0.2 > $ docker pull quay.io/maxking/mailman-core:0.2 > > Unfortunately I am still getting the error message when trying to run the > sync_mailman job manually: > > # python3 manage.py runjob sync_mailman > > Also running docker-compose up (without -d) I am seeing these errors for > each of the lists that currently exist: > > database_1 | ERROR: null value in column "description" violates > not-null constraint > database_1 | DETAIL: Failing row contains (2, > <listname>@<domain>.com, OMA-COM, null, [OMA-COM], 1, 2019-02-25 > 13:30:03.868072+00, <listuri>). > database_1 | STATEMENT: UPDATE "hyperkitty_mailinglist" SET "name" = > '<listname>@<domain>.com', "list_id" = '<listuri>', "display_name" = > '<LISTNAME>', "description" = NULL, "subject_prefix" = '[<LISTNAME>]', > "archive_policy" = 1, "created_at" = > '2019-02-25T13:30:03.868072+00:00'::timestamptz WHERE > "hyperkitty_mailinglist"."id" = 2 > database_1 | ERROR: null value in column "description" violates > not-null constraint > > All of the lists do have description that I can see in the Postorius > interface. > > Thanks > > On Tue, 7 May 2019 at 16:06, Abhilash Raj <maxking(a)asynchronous.in> wrote: > >> Are you using images with "rolling" tags? >> >> If yes, you should try to switch to "0.2" tag and the new release will be >> tagged with the same one. >> >> To update, you can do: >> >> $ docker pull ... >> $ docker pull .. >> >> $ docker-compose stop >> $ docker-compose rm >> $ docker-compose up -d >> >> This should basically get you up and running with the new image you >> pulled. >> >> >> -- >> thanks, >> Abhilash Raj (maxking) >> >> >> > > -- > > *Nathan Dixon* MEng > Senior Software Architect > m: +44 (0)7402 690311 > > t: +44 (0)20 81231252 > e: nathan.dixon(a)evadon.com > w: www.evadon.com > > This e-mail and any attachments are confidential and may be protected by > legal, professional or other privilege. If you are not the intended > recipient you should not store it, copy it, re-transmit it, use it or > disclose its contents, but should return it to the sender immediately and > delete your copy from your system. The views expressed are those of the > sender and not necessarily those of Evadon. Please note that whilst we scan > all e-mails for viruses we cannot guarantee that any e-mail is virus-free. > Please be advised that we expressly reserve the right to monitor email > content for the purposes of ensuring compliance with legal requirements and > company policies and your sending to, or receiving from, us of any email > constitutes your agreement to these terms. > > > Evadon Dynamics Limited, registered in England No. 09494097. Registered > Office: Electric Works, Sheffield Digital Campus, 3 Concourse Way, Sheffield, > S1 2BJ > -- *Nathan Dixon* MEng Senior Software Architect m: +44 (0)7402 690311 t: +44 (0)20 81231252 e: nathan.dixon(a)evadon.com w: www.evadon.com This e-mail and any attachments are confidential and may be protected by legal, professional or other privilege. If you are not the intended recipient you should not store it, copy it, re-transmit it, use it or disclose its contents, but should return it to the sender immediately and delete your copy from your system. The views expressed are those of the sender and not necessarily those of Evadon. Please note that whilst we scan all e-mails for viruses we cannot guarantee that any e-mail is virus-free. Please be advised that we expressly reserve the right to monitor email content for the purposes of ensuring compliance with legal requirements and company policies and your sending to, or receiving from, us of any email constitutes your agreement to these terms. Evadon Dynamics Limited, registered in England No. 09494097. Registered Office: Electric Works, Sheffield Digital Campus, 3 Concourse Way, Sheffield, S1 2BJ

5 years, 11 months

[MM3-users] Re: hello， Discussion on how to subscribe to themailing list

by Stephen J. Turnbull

朱超 writes: > Yes, I know why 404 is reported，The url should be postorius not be > mailman3, In the stock configuration both "mailman3" and "postorius" invoke the same view, and I'm fairly sure several templates invoke "mailman3" rather than "postorius". I believe the logic was that casual users will recognize "mailman3" more readily than "postorius", and similarly for "archive(s)" vs. "hyperkitty". > https://www.example.com/postorius/lists/$list_id/confirm/?token=$token, > and it report success. I am curious why there is no similar > processing logic for unsubscribing in postorius。 I don't understand what you're suggesting. A subscribed user necessarily has a confirmed address, and if a web administration app is running, they can log in and with a couple of clicks unsubscribe. Unsubscribe by mail does involve a confirmation step. As explained before, the reply confirmation is surely available to core; if you want I believe you can craft email templates to use the web confirmation either as an option or the preferred method. > Can we optimize for this？ What do you mean by "optimize"? If you mean fewer steps for the user, I believe the FAQ explains how to create a one-click unsubscription URL (at least that was possible with Mailman 2). There is a mild denial of service possibility, of course, but it doesn't involve third parties and is usually recoverable since the unsubscribe generates an email to the "victim" who can check archives in the majority of cases and easily resubscribe. > If you use mailman-web (include postorius and django-mailman3 and > django-allauth and so on) to set the template information, the > template obtained by mailman-core is obtained by requesting > mailman-web, No, the templates used by core and by Postorius are stored in different databases. It's true that you can use Postorius to set core's templates, but they're still stored in core's database, not in Django. They *must* be, because core needs to be able to send email even if there is no web interface at all. I suppose you could arrange that one could be set by the other. If you want to define a protocol so that any application that speaks the REST API can do this, I suppose it could be added to Mailman. But we can't depend on web administration being installed at all, and we can't depend on it being Postorius. Steve

2 years, 3 months

[MM3-users] Re: Apache+mod_wsgi issue

by Stephen J. Turnbull

Odhiambo Washington writes: > And I don't like this, so I think I will put all my configs into > /opt/mailman/mm/etc/ and link /etc/mailman3 to this path instead. Unfortunately by definition it's impossible to make the search path for the configuration file be configurable. Hm ... I think I know how to do this. ;-) Specifically, we could have a "install_defaults.py" file that lives next to bin/mailman or maybe in site-packages/mailman directory. bin/mailman would import specific variables from it, something like try: from .install_defaults import CONFIG_SEARCH_PATH, USE_CONF_D except ModuleNotFoundError: CONFIG_SEARCH_PATH = ['./mailman.cfg', ...] USE_CONF_D = False # this variable is probably a bad idea except ImportError: log'"your bootstrap.py is missing required variables'") raise Important: `mailman info` would report its location. > What bite me is probably related to this experience by Turnbull: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > I am on FreeBSD where user applications should have config files in > /usr/local/etc/ and not /etc/ so having /etc/mailman3 and > not /usr/local/etc/mailman3 kinda makes me feel weird. I sympathize. While I have no problem with the /etc/mailman3 approach, I think the *BSD approach of putting locally built configs in /usr/local/etc is equally reasonable. More than either, for third party and locally built software, I'm a fan of the /opt approach (ie, a self-contained tree for $APP, with /opt/$APP/bin, /opt/$APP/usr, /opt/$APP/var, /opt/$APP/etc or similar subdirectories). FWIW, Mailman 3's search path looks for mailman.cfg in the following places, in this order: 1. the value of the -C option # run-time option 2. $MAILMAN_CONFIG_FILE # login-time option 3. ./mailman.cfg # run-time option 4. ./var/etc/mailman.cfg # /opt-like 5. $HOME/.mailman.cfg # for developers? 6. /etc/mailman.cfg # traditional 7. /etc/mailman3/mailman.cfg # distro-friendly (Linux, at least) # good place for conf.d (see issue#424) 8. ../../etc/mailman.cfg # /opt-like # (relative to Mailman 3's bin directory) IMO this list is way too long, and ./var/etc is kind of strange. But the thing about creating the file even though it finds one of the other ones is really annoying. BTW the config file creation issue was reported about two years ago: https://gitlab.com/mailman/mailman/-/issues/797 and there are a couple of loosed related issues (ie, if we're going to thoroughly review the configuration files): - Separate config for secrets: https://gitlab.com/mailman/mailman/-/issues/466 - Support conf.d directory: https://gitlab.com/mailman/mailman/-/issues/424 Comments welcome, not really sure what I want to do here. Steve

2 years, 3 months

[MM3-users] Re: Installing mailman 3

by Rafael Mora

Postfix is listening on 127.0.0.1:25 is it ok? or should listen on 0.0.0.0:25?? El mar., 1 ago. 2017 a las 16:34, Abhilash Raj (<raj.abhilash1(a)gmail.com>) escribió: > > > > On Tue, Aug 1, 2017, at 02:29 PM, Rafael Mora wrote: > > Hello Abhilash, > > I have this in my settings.py file: > > > EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' > EMAIL_HOST = 'localhost' > EMAIL_PORT = 25 > #EMAIL_HOST_USER = 'mailing(a)labbrands.com' > #EMAIL_HOST_PASSWORD = '******' > > > This looks good to me. > > > What should I use in user and password?? Does Django use postfix to send > the email??? > > > I think you don't need a password and username as long as your postfix is > configured to relay local emails without authentication (which it is by > default if you installed from distro packages and if you haven't changed > the settings). > > > Thanks > > > > > El mar., 1 ago. 2017 a las 16:20, Abhilash Raj (<raj.abhilash1(a)gmail.com>) > escribió: > > Hi, > > > > On Tue, Aug 1, 2017, at 02:11 PM, Rafael Mora wrote: > > It's ok now, but still I can't make it send the confirmation email > > > > This is from mailman.log: > > > > > > [mailman@localhost logs]$ tail -f mailman.log > > Aug 01 15:59:18 2017 (2327) lmtp runner started. > > Aug 01 15:59:18 2017 (2333) virgin runner started. > > Aug 01 15:59:18 2017 (2332) retry runner started. > > Aug 01 15:59:18 2017 (2330) pipeline runner started. > > Aug 01 15:59:18 2017 (2325) command runner started. > > Aug 01 15:59:19 2017 (2328) nntp runner started. > > Aug 01 15:59:19 2017 (2326) in runner started. > > Aug 01 16:01:38 2017 (2331) 127.0.0.1 - - "GET > > /3.0/lists?count=0&page=1&advertised=true HTTP/1.1" 200 90 > > Aug 01 16:01:38 2017 (2331) 127.0.0.1 - - "GET > > /3.0/lists?count=10&page=1&advertised=true HTTP/1.1" 200 90 > > Aug 01 16:01:38 2017 (2331) 127.0.0.1 - - "GET /3.0/domains HTTP/1.1" 200 > > 90 > > > > > > > > > > What should I do? > > If you are talking about Account confirmation emails, they usually are > sent out from Django and not Mailman Core itself. > > To configure Django you need to edit the `settings.py` file, wherever > that is one your installation. If you are following the guide mentioned > at http://docs.mailman3.org/en/latest/prodsetup.html, it should be at > `mailman-suite/mailman-suite_project/settings.py`. > > To configure your Django to send emails, have a look at > > http://docs.mailman3.org/en/latest/config-web.html#setting-up-email-required > . > > That documentation page has pretty much everything you need in order to > have a running Web Frontend, please go through it and make sure you have > everything configured. > > Finally, the actual emails from Mailing lists are supposed to be handled > by Core. You need to configure Postorius & Mailman both to communicate. > The settings are mentioned > http://docs.mailman3.org/en/latest/config-core.html#configuring-mta > here. > > > Hope that helps! > > > > > > > > > > > > > > > El mar., 1 ago. 2017 a las 15:36, Mark Sapiro (<mark(a)msapiro.net>) > > escribió: > > > > > On 8/1/17 1:14 PM, Rafael Mora wrote: > > > > > > > > Why does it say Address already in use? > > > > > > > > > Because you are starting lm tp runner and it's already running. > > > > > > Do 'mailman stop'. Then ensure no Mailman processes are running. kill > > > any if necessary. Then do 'mailman start'. > > > > > > -- > > > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > > > San Francisco Bay Area, California better use your sense - B. Dylan > > > _______________________________________________ > > > Mailman-users mailing list > > > mailman-users(a)mailman3.org > > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > > > > -- > > > > Atentamente / Best Regards > > > > Ing. Rafael Mora > > _______________________________________________ > > Mailman-users mailing list > > mailman-users(a)mailman3.org > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > > -- > thanks, > Abhilash Raj > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > -- > > Atentamente / Best Regards > > Ing. Rafael Mora > > > -- > thanks, > Abhilash Raj > > -- Atentamente / Best Regards Ing. Rafael Mora

7 years, 8 months

[MM3-users] Re: Running mailman3 on AWS and dealing with SES email verification

by Mohsen Masoudfar

Thank you Mark, this is really very helpful. I am going to work on the details. It would be breat, if there were a way to set some of this options as default, like DMARC mitigation is default to no, I changed it to unconditionally and replace From to with list address and got good results, but the default is no. How may I changed the default, do that the list owner/creator does not need to update these fields. Thanks Mohsen -----Original Message----- From: Mark Sapiro <mark(a)msapiro.net> Sent: Thursday, April 30, 2020 1:15 PM To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: Running mailman3 on AWS and dealing with SES email verification [EXTERNAL EMAIL] On 4/30/20 9:25 AM, Mohsen Masoudfar wrote: > Hi, > > This seems to be a very basic question and/or issue, but I could not find a reliable for it: > > Assume a list member with the email address like (just as an example) > joe.student(a)university.edu sends an email to a list, Mailman gets the > email, processes it and would try to send that email to all members > with a sender field as: 'joe.student(a)university.edu'. plus > listname(a)listserver.TLD Are you referring to the envelope sender or to the From: header? The envelope sender will be listname-bounces(a)listserver.TLD or if VERP is enabled, listname-bounces+recipient=recipient.domain(a)listserver.TLD. The From: header will be unchanged from the incoming message. If yo want the From: to be the list, you can apply DMARC "Replace From with list address" unconditionally and the From: will look like From: joe.student via listname <listname(a)listserver.TLD> ... > I found some solution: > > > * Asking the members to add their email to the signature > * manipulating the from field and add the name of the sender as a comment, not that easy, needs some Python code > * manipulating the from files and replacing the @ with something else, not that easy, needs some Python code > * . . . The DMARC mitigations will do these thing for you. > More important: None of these would allow a simple click on the reply bottom to send the email to the original sender. And they will add the poster's original From: address to Reply-To: -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mai…

4 years, 11 months

[MM3-users] Re: Mailman, etc. upgrade woes and persistent bugs

by Stephen J. Turnbull

Andreas Barth writes: > I'd even say: these are different flags. They already effectively are, as Abhilash explained earlier (more precisely, four values for one variable). > So basically I'd consider it rather (as seperate boolean ones): I don't think separate booleans is a good idea, since they're basically mutually exclusive states, and in most scenarios you want the user to be able to reenable from any of the disabled states, so having both "member disabled" and "admin disabled" would have no special semantics. > - "disabled by member" (could be set by an admin as well of course) Currently this is determined by whether the logged in user is the subscriber or not. It's an interesting idea to allow it to be set by an admin. Then we could have the semantics that the only transition allowed from this state would be to enabled, so that admins would know the user had intentionally disabled and know not to enable without permission. > - "disabled for administrative reasons" (only by admin but user > should be able to see it) I don't see why a user would need to see the difference. Unless you're suggesting that the user would not be able to reenable without asking the admin? What is the use case for this? > - "bounced" (user and admin could reset it but not set it) Reenable but not set "bounced" is the current situation. > Or is this too complex for normal admins and users? I don't really understand the use case for administrative disable, except at the request of the user. (The only thing I can think of offhand is the "surprise party" scenario, which seems like a stretch. In Mailman 2, you could do it with a topic that only the planners were subscribed to, and I would think that more likely to pass without notice by the guest of honor.) In the current situation, where the state is determined by who is logged in, not by what the user wants, I think it would be confusing for some users to see "disabled by admin" if they requested it. I think that's the overwhelmingly common case for administrative disable. So I'm against showing the user the difference. In your scheme, the question is whether admins would properly distinguish between user requests and the case where the admin disables for some reason. If they do, it's harmless, but it seems like a small burden on the admins. Steve

4 years, 2 months

[MM3-users] Mixing Mailman 3 with AWS SES

by Stephen J. Turnbull

Ralph Weber writes: > I am moving the website www.t10.org from a private/corporate VM to > AWS EC2s. > > For many years, t10.org has had very happy results with using > Mailman 2 for its email reflector, and I have no real concerns > about moving to Mailman 3. My bugbear is the fact that I need the > following mixed bag of incoming email addresses to work in concert > with each other to receive and handle emails. > > > t10(a)t10.org (Primary Mailman Reflector) > > chair(a)t10.org (Mailman Reflector to direct messages to the T10 > > chair and his designates) > * docs(a)t10.org (document posting mechanics - serviced by a small > mountain of home-grown code code > * bbs(a)t10.org (other publicly accessible services - serviced by > code that recalls the days when everything as done on a Bulletin > Board System, BBS) This isn't a Mailman question, to be frank. As Abhilash points out, Mailman doesn't care about other mailboxes served by its host (unless they conflict with list names or adinistrative addresses derived from list names, which evidently yours do not). It's a question about the mail service at AWS. I've not seen any documentation in our project about setting up at AWS; I don't know anybody who has done so. (I haven't looked for either though.) I took a quick look for AWS SES documentation. They don't seem to have any real documentation online for the general public, only lists of features. They also seem to be very focused on sending rather than receiving. Much talk of how much mail you can send, bulk emailing, statistics on responses, and the like. None about setting up for incoming mail. Probably I could get access to online help if I signed up for a free account, but that's above my pay grade. > My best guess is that the AWS Simple Email Service (SES) needs to > sit in front of both Mailman and the home-grown code, I don't know if AWS provides an alternative Complicated Email Service, but yes, something like SES is going to have to sit in front of the various mailboxes. Mailman itself has a certain amount of capability to deal with the main email protocols, but it is definitely not capable of replacing the main mail server. For security reasons I would definitely advise against any Mailman 3 presence directly on the Internet. It was designed on the assumption that it would talk to the Internet via a standard mail server, Postorius (for administration), and HyperKitty (for mail archives)[1], and occasionally to admins logging in with shell accounts. > All available evidence read to date URLs, please. As I wrote above, Amazon's documentation is at best not very discoverable. > suggests that the default AWS installation Whose default installation of Mailman? AWS's? Typical 3rd party configurations? > of Mailman 3 assumes that Mailman 3 is the *only* receiver of > emails in the configuration. That's possible, but it's likely an artifact of the application -- I would expect that many who host Mailman on AWS do so to isolate Mailman from other services they provide. So it seems reasonable that in many such installations Mailman is *the* service for that instance. But mail servers are designed to be many inputs, many outputs services. If SES itself is as restricted as you fear, surely AWS provides an alternative, more powerful mail service. I can imagine quite a few restrictions, other than the backing application, on the SES that Amazon *might* impose. But without access to documentation, we can't say much. > Or... Hercules may have already cleaned the Augean Stables, and > left behind a AWS Lambda function that can serve as a keystone > puzzle piece for solving my dilemma. Good luck on that. I don't know of any, but that would be a quick solution for you. I hope somebody here knows more. As evidenced by Andrew's post, there are folks thinking about it, so maybe somebody's actually tried -- anybody? Successes? Pitfalls to watch for? The other issue Andrew mentions could be important. Since Solar Winds I'm seeing daily attempts to access my O365 server (nonexistent) from AWS. Given the size of AWS, I think they probably actually do a good job of keeping felonious behavior to a minimum[2], but a lot of folks I know are happy to cut off huge netblocks until somebody they know actually complains. :-( Footnotes: [1] Or more precisely, some webservice providing those services mediating between the Internet and Mailman sitting cozily behind a firewall. [2] DigitalOcean, on the other hand, is accumulating blocked /24s, and even /16s, at my firewall. But that's another story.

4 years, 1 month

[MM3-users] Re: 500 error on Held Messages page

by Mark Sapiro

On 9/21/22 10:39, Dan Caballero wrote: > I figured out the correct import based on what was done previously. It worked. Why do you think you need to import anything? > ">>> from mailman.app.moderator import handle_message `handle_message` is not referenced in the following and doesn't need to be imported here. ``` >>>> requestdb = IListRequests(m) >>>> for req in requestdb.held_requests: > ... if req.request_type == RequestType.held_message: > ... if not requestdb.get_request(req.id): > ... print(f'Bad request id: {req.id}, deleting') > ... requestdb.delete_request(req.id) > ... > Bad request id: 1694, deleting > Bad request id: 2509, deleting > Bad request id: 2727, deleting > Bad request id: 3251, deleting > Bad request id: 3265, deleting > Bad request id: 3371, deleting > Bad request id: 4844, deleting > Bad request id: 6058, deleting > Bad request id: 6201, deleting > Bad request id: 6432, deleting > Bad request id: 6502, deleting >>>> commit ()" ``` I assume after doing this, you were able to access the held messages view for the list. Note that the only names referenced in the above and not defined there are `IListRequests` and `RequestType`, but these are among the many names implicitly imported by `mailman shell` in interactive mode. ``` $ mailman shell Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. >>> for name in dir(): ... print(name) ... ALL_TEMPLATES ALT_TEMPLATE_NAMES ALWAYS_REPLY AcceptEvent AcceptOwnerEvent Action AddressAlreadyLinkedError AddressError AddressNotLinkedError AddressVerificationEvent AlreadySubscribedError ArchivePolicy ArchiveRenderingMode BadDomainSpecificationError BounceContext ChainEvent ClobberDate ConfigurationUpdatedEvent ContinueProcessing DMARCMitigateAction DatabaseError DeliveryMode DeliveryStatus DigestFrequency DiscardEvent DiscardMessage DomainCreatedEvent DomainCreatingEvent DomainDeletedEvent DomainDeletingEvent DuplicateStyleError EmailError ExistingAddressError FilterAction FilterType HoldEvent IAPI IAcceptableAlias IAcceptableAliasSet IAddress IArchiver IAutoResponseRecord IAutoResponseSet IBan IBanManager IBounceEvent IBounceProcessor ICLISubCommand ICacheManager IChain IChainIterator IChainLink IConfiguration IContentFilter IDatabase IDatabaseFactory IDomain IDomainManager IEmailCommand IEmailResults IEmailValidator IHandler IHeaderMatch IHeaderMatchList ILanguage ILanguageManager IListArchiver IListArchiverSet IListManager IListRequests IMailTransportAgentAliases IMailTransportAgentDelivery IMailTransportAgentLifecycle IMailingList IMember IMembershipManager IMessage IMessageStore IMutableChain IOneLastDigest IPendable IPended IPendedKeyValue IPendings IPipeline IPlugin IPreferences IRoster IRule IRunner IStyle IStyleManager ISubscriptionManager ISubscriptionService ISwitchboard ITemplate ITemplateLoader ITemplateManager IUser IUserManager IWorkflowState IWorkflowStateManager InvalidConfigurationError InvalidEmailAddressError InvalidListNameError LinkAction ListAlreadyExistsError ListCreatedEvent ListCreatingEvent ListDeletedEvent ListDeletingEvent MailmanError MemberRole MembershipChangeEvent MembershipError MembershipIsBannedError MissingConfigurationFileError MissingPreferredAddressError MissingUserError NewsgroupModeration NoSuchListError NotAMemberError PasswordChangeEvent PendType Personalization RejectEvent RejectMessage ReplyToMunging RequestRecord RequestType Response ResponseAction RunnerCrashEvent RunnerInterrupt SomeRecipientsFailed SubscriptionConfirmationNeededEvent SubscriptionEvent SubscriptionInvitationNeededEvent SubscriptionMode SubscriptionPendingError SubscriptionPolicy TokenOwner TooManyMembersError UnrecognizedBounceDisposition UnsubscriptionConfirmationNeededEvent UnsubscriptionEvent UnverifiedAddressError __builtins__ abort commit config getUtility m >>> ``` -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 7 months

[MM3-users] Re: Counter-intuitive header filter processing

by Mark Sapiro

On 09/18/2018 01:04 PM, Jonas Meurer wrote: > > Maybe then the description for chain "accept" in postorius needs to be > adjusted. Currently it says "Accept immediately (bypass other rules)". > This gives the impression, that it will overrule the default action. Yes, I agree it's confusing. > A related followup question: what configuration is recommended for an > announcement list where only specific senders (e.g. sender(a)example.org) > should be allowed to post and mail from all (other) subscribers should > be rejected? > > The only way I see to achieve this currently is: > > * Set default action for member posts to "Default processing" > * Add a header filter to accept "sender(a)example.org" > * Add a header filter to reject ".*" > > In my eyes, that's very counter-intuitive and cumbersome to configure. > But maybe I just missed a more obvious and elegant solution to achieve > the same? The way to do that is to set the default action for member posts to Reject and set Member Options -> Administration options -> Moderation for the authorized posters to Accept or Default processing as desired. However, this allows anyone to post by spoofing an authorized address. To avoid that, leave everyone's Moderation setting at List default and post using "pre-approval" (see <https://mailman.readthedocs.io/en/latest/src/mailman/rules/docs/approved.ht…>. The downside of this is it requires the poster to send a cleartext password in a header or body line of the post. It will be removed from the post delivered to list members, but sending it in cleartext in an email may be an issue. We will eventually do something with PGP signatures instead, but that isn't here yet. > And one more comment: if I make the list an "anonymous list" (i.e. "hide > the sender of a message, replacing it with the list address"), then > header filters for the "from" header don't work any longer. That's kind > of unexpected as well. I would have expected mailman to process the > header filters at reading the incoming mail, not after rewriting headers > of the outgoing mail? Your expectation is correct. All the applicable rules are applied to the incoming message before any message alterations such as anonymizing are applied. Possibly you were confused because some other rule's action took precedence over your header rule. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years, 7 months

[MM3-users] Re: moving from sqlite3 to postgresql

by Michael Richardson

Stephen J. Turnbull <steve(a)turnbull.jp> wrote: >> I installed via debian packages, and I'd really prefer to stay with that. >> (Not what many maintainers would say, I know. Essential that it work if >> mailman3 is going to take over) > That's really Debian's problem, and it's a hard one since the whole > point of using Debian packages for mission-critical applications is > that they're stable. Unfortunately, the email and web environments > are not, and dealing with changes in them is often quite impractical > given the Debian release cycle. Yes, I'm aware of this tusscle. There are some ways to split this up. >> I installed with sqlite3, since it seemed if all work was going to >> go through the mailman3 daemon, that it would handle all the >> concurrency issues. > The problem is that sqlite3 is intended to be a single-threaded > system, but Mailman 3 by default spawns around 15 daemons that may try > to access the database. I would expect you would have warnings that > the database is locked frequently. Not sure what you mean by > "Sometimes this goes way on it's own, but sometimes it does not". Meaning, sometimes whatever has the lock releases it everything proceeds. Sometimes, it's still stuck a few days later, and I restart the mailman daemons. > Note that our primary purpose for the sqlite3 configuration is so unit > tests don't have to spin up a full-scale PostgreSQL or MySQL database. Fair enough. It was not obvious when installing that I should not use sqlite3. >> So I tried to switch last week from sqlite3 to postgresql. >> I used pgloader with a configuration file attached below. > Can't help you with that offhand, never used pgloader. Whatever > method you use to load up the pg database, I would recommend doing > some psql cave-diving into the PostgreSQL database to confirm it's > sane before trying to configure Mailman to use that database. Yes, I'm gonna have to do that. At least I'll start by counting number of records in each table, and then see if there is some systematic lack. >> ps: I'd pay some $$ money for email support consulting. > See https://wiki.list.org/COM/Mailman%20consulting%20services > Per my .sig, I'm on the list above but please go through channels > linked there rather than talking to me personally about business. Understood.

4 days, 17 hours

[MM3-users] Re: How To Install Mailman 3 on Debian 10 (Complete Guide)

by Stephen J. Turnbull

Bottom line, I think Abhilash is right: we should emphasize the pip install (probably into a venv) as the "best supported" install. Perhaps we can prioritize being more systematic about documenting the various approaches, not in term of the details (although that's a "nice to have"), but with respect to the advantages and disadvantages of the various install methods. Comments welcome. A few comments on the distro installs for the Mailman nerds: Mark Dadgar writes: > 2) The package installs are non-standard with respect to the way > the mailman3 community tends to install them, To be fair, *most* of the differences with Linux distros is that Mailman uses a "classic" $prefix-style install into $prefix/usr and $prefix/var, while the distros follow the FHS (File Hierarchy System) layout. The FHS layout in many ways is more closely-specified, and more maintainable at the distro and system levels, but it requires more effort than just throwing files into a set of more or less reasonably organized directories. Mailman 3 now provides an option for (better) FHS conformance at installation, which should reduce this problem over time. Most of the rest have to do with protecting config files from overwriting by distro upgrades. That is annoying, because location of config files is a frequent question in support, and also can lead to duplicate config files (and "great fun had by all" :-/ ) if the admin follows the recommendation of the "wrong" documentation. > > Distribution packages tend to be useful for really basic things > > like libraries. For constantly evolving projects where you need > > to be current, they're more of a detriment. > > So much this. I agree. But I prefer to phrase it as "mission-critical features are evolving" rather than "constantly evolving". I suspect there are a lot of people who have a casual need for a few mailing lists, and will do fine with Mailman 3 from $DISTRO stable or LTS. :-) The thing is, I think those folks will find "apt-get mailman-suite" or "yum mailman-suite" with no help from us. And I agree that the differences between the organic evolution of Mailman setup within the project and the distros' FHS-based installs are a real consideration if Mailman is a mission-critical application for you. On the other hand, Brian's "greenfield setup of a server specialized to Mailman" is probably a minority interest (ie, in terms of the number of site admins involved -- it probably serves a majority of subscriptions!) Steve

4 years, 1 month

[MM3-users] Re: UnicodeEncodeError: 'ascii' codec can't encode character

by Marvin Gülker

Hi, Am 11. Dezember 2019 um 13:49 Uhr -0800 schrieb Mark Sapiro: >OK. I have verified that this is due to ><https://bugs.python.org/issue32330> and has actually been reported as ><https://gitlab.com/mailman/mailman/issues/441> and worked around in >Mailman by <https://gitlab.com/mailman/mailman/merge_requests/350> which >is included in Mailman 3.2.0 and up. Thanks, and sorry for bugging you about old versions then. I'm going to forward the problem to Ubuntu's bugtracker then. But I now have a different problem. I've removed all offending messages (it were more than just one that caused the UnicodeEncodeError), but I still can't access the page for the held messages queue. It gives an error 500, now with a different exception in the log. I've included it below. It looks to me that the messages haven't properly been deleted. What I did was this: for each offending message hash 1. I ran the DELETE query 2. I deleted the file from the file system 3. I ran the delete_message() function I originally wanted to run only the delete_message() function, but a quick check on the DB and the file system after running it for one hash told me that it neither deletes the message from the DB nor from the file system. So I did everything at once. I do have a backup of both the DB and the filesystem from before I started this, in case this broke something. Is there something left that I have to do now? Here's the exception: Dec 12 08:34:30 2019 (9570) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3.6/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 214, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 215, in falcon.api.API.__call__ (falcon/api.c:2872) File "falcon/api.py", line 189, in falcon.api.API.__call__ (falcon/api.c:2419) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 167, in on_get resource = self._make_collection(request) File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 157, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 78, in _make_resource resource['msg'] = msg.as_string() AttributeError: 'NoneType' object has no attribute 'as_string' -- Blog: https://mg.guelker.eu

5 years, 4 months

[MM3-users] Re: Docker Support Sucks

by Brian Carpenter

On 7/21/20 8:55 PM, Mark Sapiro wrote: > On 7/21/20 5:40 PM, Brian Carpenter wrote: > > Abhilash (@maxking) is one person and he has a $DayJob. We're all doing > the best we can. Well, I still need support. > This is not a Docker issue per se. It is a consequence of upgrading > Mailman core to 3.3.1+ and processing many old, stale, previously > unprocessed bounces. See > <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…> I am not worried about the bounce issues at this point. I am just trying to get a membership list exported for a client. > >> So, I want to totally move away from the use of Docker (mainly because I have found it buggy and the support is simply lacking) and install Mailman 3 for this client from source. However I need to get an export of their list members. But of course, I can't. Rest API times out and I don't know how to apply the instructions I was given to fix it and I don't have days to way for one. So please help. >> >> I increased the gunicorn workers to 4, hoping that would allow me to do the CSV export. No. So was given these instructions: >> >> ### >> The issues seems to be due to timeout not due to high load. You want to increase the timeout for gunicorn and you can do that by specifying a gunicorn.ini config file: >> >> [webservice] >> configuration: /path/to/gunicorn.cfg >> >> and then in the gunicorn.cfg you can add >> >> [gunicorn] >> timeout = 90 >> graceful_timeout = 30 >> ### >> >> I applied the above changes to /opt/mailman/core/mailman-extra.cfg and created the gunicorn.cfg file. Restarted the docker container (mailman-core) and no access to Postorius due to no REST api. >> >> After restarting docker-mailman, I can't access Postorius due to the following error: >> >> ERROR 2020-07-20 20:16:27,351 15 postorius Mailman REST API not available > Is Mailman core running? If not, whats in core's mailman.log as to why not? It looks like but no error message in mailman.log. However I noted that the gunicorn process did not start. This is with the configuration line enabled in mailman-extra.cfg: # docker-compose restart mailman-core Restarting mailman-core ... done [root@nicu-net docker-mailman]# tail -f /opt/mailman/core/var/logs/mailman.log Jul 22 01:25:17 2020 (22) in runner caught SIGTERM. Stopping. Jul 22 01:25:17 2020 (25) out runner caught SIGTERM. Stopping. Jul 22 01:25:17 2020 (25) out runner exiting. Jul 22 01:25:17 2020 (21) command runner exiting. Jul 22 01:25:17 2020 (22) in runner exiting. Jul 22 01:25:17 2020 (24) nntp runner caught SIGTERM. Stopping. Jul 22 01:25:17 2020 (24) nntp runner exiting. [2020-07-22 01:25:19 +0000] [27] [INFO] Shutting down: Master Jul 22 01:25:21 2020 (1) Master stopped Jul 22 01:25:26 2020 (1) Master started Jul 22 01:25:44 2020 (27) rest runner started. Jul 22 01:25:44 2020 (26) pipeline runner started. Jul 22 01:25:44 2020 (20) bounces runner started. Jul 22 01:25:45 2020 (22) in runner started. Jul 22 01:25:45 2020 (28) retry runner started. Jul 22 01:25:46 2020 (29) virgin runner started. Jul 22 01:25:46 2020 (25) out runner started. Jul 22 01:25:46 2020 (23) lmtp runner started. Jul 22 01:25:46 2020 (24) nntp runner started. Jul 22 01:25:47 2020 (30) digest runner started. Jul 22 01:25:47 2020 (19) archive runner started. Jul 22 01:25:47 2020 (21) command runner started. This is with the configuration line disabled in mailman-extra.cfg: # docker-compose restart mailman-core Restarting mailman-core ... done [root@nicu-net docker-mailman]# tail -f /opt/mailman/core/var/logs/mailman.log Jul 22 01:27:49 2020 (1) Master stopped Jul 22 01:27:53 2020 (1) Master started Jul 22 01:28:10 2020 (30) digest runner started. Jul 22 01:28:10 2020 (29) virgin runner started. Jul 22 01:28:10 2020 (24) nntp runner started. Jul 22 01:28:10 2020 (25) out runner started. Jul 22 01:28:10 2020 (22) in runner started. Jul 22 01:28:11 2020 (28) retry runner started. Jul 22 01:28:11 2020 (27) rest runner started. [2020-07-22 01:28:11 +0000] [27] [INFO] Starting gunicorn 19.9.0 [2020-07-22 01:28:11 +0000] [27] [INFO] Listening at: http://172.19.199.2:8001 (27) [2020-07-22 01:28:11 +0000] [27] [INFO] Using worker: sync [2020-07-22 01:28:11 +0000] [32] [INFO] Booting worker with pid: 32 [2020-07-22 01:28:11 +0000] [33] [INFO] Booting worker with pid: 33 [2020-07-22 01:28:11 +0000] [34] [INFO] Booting worker with pid: 34 [2020-07-22 01:28:11 +0000] [35] [INFO] Booting worker with pid: 35 Jul 22 01:28:11 2020 (19) archive runner started. Jul 22 01:28:11 2020 (21) command runner started. Jul 22 01:28:12 2020 (23) lmtp runner started. Jul 22 01:28:12 2020 (20) bounces runner started. Jul 22 01:28:12 2020 (26) pipeline runner started. -- Please let me know if you need further assistance. Thank you for your business. We appreciate our clients. Brian Carpenter EMWD.com -- EMWD's Knowledgebase: https://clientarea.emwd.com/index.php/knowledgebase EMWD's Community Forums http://discourse.emwd.com/

4 years, 9 months

[MM3-users] Dropdown problem in postorius due to bootstrap

by Greg Newby

(My first post) As has been observed, the Ubuntu 24.04 LTS packaging for native Mailman3 can be challenging. I've been solving all kinds of issues, and I'm writing here about something I've so far been unable to solve. The problem: *In Postorius, dropdown menus are not working*. I.e., as a list admin you can click on a menu option like "Users" or "Mass operations," and nothing happens. As a regular user, the top-right icon to manage settings and logout does not do anything. The same behavior occurs regardless of which browser I use. The dropdown in HyperKitty works, and HyperKitty uses a different code base. In the browser developer console (Firefox & others), the error is: “Uncaught TypeError: n.createPopper is not a function.” I've dug deeply into this. The relevant <script> is bootstrap.min.js. Postorius (via django) is using version 5.3.1 via package libjs-bootstrap5: ** This is what django is using: /usr/share/bootstrap-html/js/bootstrap.bundle.min.js -- version 5.3.1, linked to: /usr/share/javascript/bootstrap5/js/bootstrap.bundle.min.js /usr/share/python3-django-postorius/static/postorius/libs/bootstrap/js/bootstrap.bundle.min.js Here is detail of the error from the Firefox developer console: Uncaught TypeError: n.createPopper is not a function _createPopper dropdown.js:241 show dropdown.js:139 toggle dropdown.js:121 <anonymous> dropdown.js:446 i event-handler.js:118 R event-handler.js:184 on event-handler.js:216 <anonymous> dropdown.js:444 <anonymous> bootstrap.bundle.min.js:6 <anonymous> bootstrap.bundle.min.js:6 By temporarily replacing /usr/share/python3-django-postorius/static/postorius/libs/bootstrap/js/bootstrap.bundle.min.js I have tried moving back to 4.6.1 (from libjs-bootstrap4). I have also retrieved the latest 5.3.1 from the CDN, which is slightly different but doesn't change the error: https://cdn.jsdelivr.net/npm/bootstrap@5.3.1/dist/js/bootstrap.bundle.min.js Here's the JavaScript section from the bottom of a sample page from my Postorius site (view-source: https://lists.ogf.org/mailman3/postorius/lists/gfsg.lists.ogf.org/held_mess… ): <script src="/mailman3/static/postorius/libs/jquery/jquery-3.6.0.min.js"></script> <script src="/mailman3/static/postorius/libs/bootstrap/js/bootstrap.bundle.min.js"></script> <script src="/mailman3/static/django-mailman3/js/main.js"></script> <script src="/mailman3/static/postorius/js/script.js" defer></script> <script> // Add the .js-enabled class to the body so we can style the elements // depending on whether Javascript is enabled. $(document).ready(function(){ $("body").addClass("js-enabled"); $(".gravatar").addClass("rounded-circle"); }); </script> I would appreciate suggestions about how to solve this problem. Thanks to everyone who is doing a great job building and supporting Mailman3. I've been using Mailman2 for a long time, and have dozens of lists on multiple servers. ~ Greg Newby

9 months, 2 weeks

[MM3-users] Settings for prevention of spam obfuscation

by Stephen J. Turnbull

Richard Rosner writes: > Are there any settings that may help with making it easier seeing > spam as spam? I gather you mean that you are letting spam go through to your subscribers, and you want to give them more information? For practical purposes, you've already lost, though. Studies show that users are not very good at picking up on such details. > E.g. some mail programs, including Outlook, allow the > user to set a from mail address that most mail clients will show > instead of the true sender address, but the senders mail address is > still present in the sender header field. While some people and > companies carelessly abuse that functionality, hiding their true > mail address, those bad practices also are abused my spammers and > others. What you consider "abuse" or "bad practice" is often very useful. For example, I have validated four different addresses that I can set in Gmail, each of which I use for specific recipients. Of course, all are "public" identities, if you know me you likely know more than one of those addresses. Of course all such messages have my Gmail address as sender. I would be very angry if you dug that out and substituted it for my public identity. If you are seriously considering such a strategy, you should do a study to see whether you'd catch more spam than ham this way. In any case, competent spammers are spoofing *all* of the identities involved in creating and injecting the email. The only way to effectively authenticate email is to use cryptographic signatures. Since very few users do this themselves, you're left with DKIM. Technically speaking DKIM only authenticates the sending host. But on the assumption that the sending host authenticates its users, if the From field is included in that host's DKIM signature, you can trust the authenticity of the author identity in the From field. Mailman does not perform DKIM verification, but your MTA almost certainly does already. It probably also already performs the DMARC "From alignment" check, which is even more strict than the DKIM test I described above. From alignment requires a valid DKIM signature including the From field, and the domain in the From address must be the same as the domain that signed the message. Your spam filter should be using those before forwarding to Mailman. > Does mailman have any settings that would remove any such > obfuscations, e.g. in this case detect the manipulated from address > and replace it with the sender address? No. In fact, the only optional manipulations of From that Mailman performs are (a) to remove the address for anonymized lists, and (b) to change the address to that of the list in cases where changes to the message performed by the list would invalidate the DKIM signature. If you are enabling features that invalidate the original DKIM signature, you can provide more information to your subscribers via the ARC (Authenticated Received Chain) protocol. It's better to have this implemented in the MTA, but as long as the MTA is configured to provide verification of incoming SPF and DKIM results in the Authentication-Results field, Mailman can optionally participate in ARC for you. > While I do run my own spam filter, it's virtually getting harder > every day to make sure you detect all spam while not having too > many false positives. If it was possible to reverse at least the > most obvious manipulations, that would probably already help. Also, > it would make it much easier for the users to see at first glance > who actually wrote an email. It's always less trustworthy if the > senders address is shown as user3168(a)suspiciousdomain.gg instead of > it being manipulated to say support(a)yourprovider.com. I think the odds are good that you'll annoy some of the more sophisticated subscribers, but not save anyone from spam. I haven't collected statistics on this, but anomolous message IDs are often noticable, often in messages "from support(a)someprovider.com". Of course almost none of your users will be checking that, but your spam filter may be able to use failed alignment between Message-ID and DKIM signer. You haven't described your use case in any detail, so the following advice may not apply for your lists. But generically the strategies that have high success rates are 1. Human moderation, preferably after a highly accurate filter, of course. 2. Members-only list. 3. Crank up the spam points for invalid or missing DKIM signature and for failed From alignment (but this may cause problems for some of your posters). There are other strategies specific to the list. One list I know had success by giving negative spam points to mentions of the list's topic. Steve

1 year, 1 month

[MM3-users] Upgrade

by Mark Sapiro

On 5/22/19 2:07 PM, David Krantz wrote: > Hi all, > > I am running a relatively old version of mailman 3, namely 3.2.0a1 > from late september 2017. This is installed using the old > mailman-bundler method. I use postfix and postgres. > > I really have two questions: > > First: What is the normal way to upgrade such a setup? I guess that I > just can get the updated versions of all packages through git, set up > new venv's to simplify a rollback, back up the database and then just > restarting with the new version hoping for the best. That's more or less correct. I did this on mailman3.org a little over a year ago. I suggest you see <https://wiki.list.org/DOC/Mailman%203%20installation%20experience> which describes much of what I did. you can also see <https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=re…> for what I had with Mailman Bundler. The biggest hurdle is now Postorius and Hyperkitty are Python3 so you need Python3 versions of various dependencies. Here's roughly what I did. git clone everything from GitLab. As a developer, I've always been running off the heads. There are occasional glitches, but I can work around these. You can also install from PYPI if you want more stability. Everything is mailman, mailman-hyperkitty, mailmanclient, django-mailman3, hyperkitty and postorius. But, I'm getting ahead. Create a new python3.5+ virtualenv. Python3.5 is the minimum, but 3.6 or 3.7 would be even better. Activate that virtualenv and if you are using the branches from GitLab, in each branch directory, do python setup.py clean --all python setup.py install or just pip install everything. You also need to do the things indicated in the Preliminaries and Installation sections of <https://wiki.list.org/DOC/Mailman%203%20installation%20experience>. Also see the Migration section for scripts I use to update from GitLab and build 'everything' and the rest of the document too. Then once you have things set up, it's pretty much a matter of just updating the various scripts you use to run things to point to the new command files in, in my case, /opt/mailman/mm/bin/. One caveat to note is if you are using the Whoosh backend. The Python2 version is incompatible with Python3 so you need to remove the fulltext_index directory and run the django management rebuild_index command to recreate it. You should be able to just run rebuild_index or maybe clear_index followed by update_index without removing the old index first, but there are reports that this doesn't work. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years, 11 months

[MM3-users] Re: Mailman 3 stops delivering to a domain after too many bounces

by Tim Cutts

I agree with this analysis. I have had issues in the past where some big email providers allowed the first few through and then started returning an error for everyone else in the same domain. It was fairly obvious from the MTA logs (postfix in my case) when this was happening — fixing it was rather trickier, since ultimately it’s overzealous anti spam policies at the mail provider which are the cause of the trouble. Postfix is a bit of a pain for configuring domain-specific delivery behaviours; it has global settings smtp_client_connection_count_limit and smtp_client_connection_rate_limit, which might be helpful, but it’s hard to make them different for different domains. Exim would be much simpler to configure to do that. One day when my bag of round tuits gets bigger I might switch back to Exim. Tim > On 12 Jul 2022, at 13:52, Stephen J. Turnbull <stephenjturnbull(a)gmail.com> wrote: > > Julian Kippels writes: > >> This was not a problem on Mailman 2, but has become a problem on >> Mailman 3. It seems to me that for example Mailman 3 sends mails to >> the first 5 existing member addresses from domain A, then >> encounters several (20 or so seems to be the threshold) "dead" >> member addresses from domain A that generate a bounce and then does >> not even attempt to reach the remaining 800 member adresses from >> domain A and only sends to members on domain B and domain C. > > "Seems to you"? Do you have logs and empty MTA queues etc as evidence > for this, or just mail not arriving at the subscription addresses? > What are the configuration settings for the list, most importantly, > are messages personalized? > > In the default configuration without personalization, I think this is > very unlikely to be Mailman's problem. Mailman will send a reasonably > large number (specifically, 500) of recipients for the message, and > then send the message itself once. In this case, if only a couple > dozen recipients of 800 receive the message, it's either the recipient > MTA or some interaction between your MTA and theirs, not Mailman. > Check the MTA outgoing queue and MTA logs. > > If the list has enabled personalization, so that the footer (usually) > contains subscriber-specific information, then there is only one > recipient per message. However, the interaction you describe is > unlikely to be due to Mailman's behavior in that case as well, as > Mailman tracks bounces by individual address, not by domain. It's not > useful to do so, as almost all bounces that reach are > address-specific. If there is a problem in connecting to a remote > host, that is almost always handled by the MTA and Mailman never finds > out about it. Again, check the queue and the logs for the MTA as well > as for Mailman. > > On the other hand, I was talking with the postmaster of a rather large > site recently, who was complaining about freemail G. What happens is > that with a very large list with many bouncing addresses, the sending > MTA builds up a backlog of undelivered messages. The MTA tries to > resend them at intervals (or for some MTAs the next time a message > goes out to that domain). Of course since it's a mailing list, the > same addresses bounce, and now you have *twice as many* undelivered > messages waiting to be delivered. Pretty soon you have a very large > number of attempts within a very short time (in the case of my > postmaster acquaintance, an average of 3/second), and freemail G > treats that as a DOS attack and starts refusing or delaying > connections. > > That sounds a lot like what's happening to you. Without the log and > configuration information it's not possible to be sure, but your > hypothesis isn't very consistent with the way Mailman actually does > its work. > >> Is there a way to globaly ignore bounces and > > No. That's one of the better ways I can think of to acquire a > reputation as a spammer and generally bad actor. > >> send out mails to all members of a list, no matter what? > > That is what Mailman does. Only after it has received "enough" > bounces for a particular user does it disable delivery to that user > only. Mailman does not keep records of bounces per domain. > > Steve > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

2 years, 9 months

[MM3-users] Re: Archive Policy not reflecting in Hyperkitty

by Nathan Dixon

Hi, I have removed the docker images and pulled the version tagged as 0.2. $ docker pull quay.io/maxking/mailman-web:0.2 $ docker pull quay.io/maxking/mailman-core:0.2 Unfortunately I am still getting the error message when trying to run the sync_mailman job manually: # python3 manage.py runjob sync_mailman Also running docker-compose up (without -d) I am seeing these errors for each of the lists that currently exist: database_1 | ERROR: null value in column "description" violates not-null constraint database_1 | DETAIL: Failing row contains (2, <listname>@<domain>.com, OMA-COM, null, [OMA-COM], 1, 2019-02-25 13:30:03.868072+00, <listuri>). database_1 | STATEMENT: UPDATE "hyperkitty_mailinglist" SET "name" = '<listname>@<domain>.com', "list_id" = '<listuri>', "display_name" = '<LISTNAME>', "description" = NULL, "subject_prefix" = '[<LISTNAME>]', "archive_policy" = 1, "created_at" = '2019-02-25T13:30:03.868072+00:00'::timestamptz WHERE "hyperkitty_mailinglist"."id" = 2 database_1 | ERROR: null value in column "description" violates not-null constraint All of the lists do have description that I can see in the Postorius interface. Thanks On Tue, 7 May 2019 at 16:06, Abhilash Raj <maxking(a)asynchronous.in> wrote: > Are you using images with "rolling" tags? > > If yes, you should try to switch to "0.2" tag and the new release will be > tagged with the same one. > > To update, you can do: > > $ docker pull ... > $ docker pull .. > > $ docker-compose stop > $ docker-compose rm > $ docker-compose up -d > > This should basically get you up and running with the new image you pulled. > > > -- > thanks, > Abhilash Raj (maxking) > > > -- *Nathan Dixon* MEng Senior Software Architect m: +44 (0)7402 690311 t: +44 (0)20 81231252 e: nathan.dixon(a)evadon.com w: www.evadon.com This e-mail and any attachments are confidential and may be protected by legal, professional or other privilege. If you are not the intended recipient you should not store it, copy it, re-transmit it, use it or disclose its contents, but should return it to the sender immediately and delete your copy from your system. The views expressed are those of the sender and not necessarily those of Evadon. Please note that whilst we scan all e-mails for viruses we cannot guarantee that any e-mail is virus-free. Please be advised that we expressly reserve the right to monitor email content for the purposes of ensuring compliance with legal requirements and company policies and your sending to, or receiving from, us of any email constitutes your agreement to these terms. Evadon Dynamics Limited, registered in England No. 09494097. Registered Office: Electric Works, Sheffield Digital Campus, 3 Concourse Way, Sheffield, S1 2BJ

5 years, 11 months

[MM3-users] Re: AttributeError: 'MailingList' object has no attribute 'get_requests_count'

by Mark Sapiro

On 10/26/20 6:10 PM, Shashikanth Komandoor wrote: > > How to find out the versions of the each and every component > of the mailman 3 like mailman core, mailmancilent, postorius, > hyperikitty, mailman-hyperkitty, django-mailman3, mailman-suite and so, > after installation? cd to the directory in which you cloned the gitlab project for the component and do `git log` > If I am running an old mailmanclient or older version of any > other mailman component, please suggest how to upgrade either one or > multiple components based on the requirement. Is there any link that > would help me to upgrade ? > > I have used the > link https://docs.mailman3.org/en/latest/devsetup.html to install the > mailman 3 components. I expected that the procedure followed in this > link would install the latest versions of the mailman 3 components. > Please correct me if I am wrong. Is there any latest version > compatibility table ? If you installed each of mailman, mailmanclient, django-mailman3, postorius and hyperkitty according to instructions like $ cd ~/mailman $ git clone https://gitlab.com/mailman/mailmanclient.git Then set up mailmanclient: $ cd mailmanclient $ python setup.py develop $ cd .. from that document, you should have the latest version. If that is the case, `git log` in the mailmanclient directory should begin like commit aaa529808e2d61899f90509da4ea802a70d2c8ff (HEAD -> master, origin/master, origin/HEAD) Merge: ff39dd8 8670751 Author: Abhilash Raj <maxking(a)asynchronous.in> Date: Thu Oct 15 01:09:32 2020 +0000 Merge branch 'fix-ci' into 'master' Use GHCR for the CI image See merge request mailman/mailmanclient!130 commit 8670751caff689733751ef323ef521c4a3fa1237 Author: Abhilash Raj <raj.abhilash1(a)gmail.com> Date: Wed Oct 14 17:36:58 2020 -0700 Use GHCR for the CI image commit ff39dd8f03a820f442f2251553c444370df3b914 Merge: 80ebed4 76d44e1 Author: Abhilash Raj <maxking(a)asynchronous.in> Date: Fri Aug 7 19:35:05 2020 +0000 Merge branch 'invite' into 'master' Add support for inviting an email address to join a list. See merge request mailman/mailmanclient!129 or if you did this before Oct 15, maybe it starts with commit ff39dd8f03a820f442f2251553c444370df3b914 The particular commit you need for your issue is commit 86eb0da3cdb48e253b0d36138ab7cf91259a7583 Merge: c8dcb1e 28d1c3a Author: Abhilash Raj <maxking(a)asynchronous.in> Date: Sat May 16 08:02:21 2020 +0000 Merge branch 'held-count' into 'master' Add a new get_held_count() API to get a count of held messages. See merge request mailman/mailmanclient!122 Note that unless you intend to do actual development of Mailman 3, you should be following <https://docs.mailman3.org/en/latest/install/virtualenv.html> rather than <https://docs.mailman3.org/en/latest/devsetup.html>. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 5 months

[MM3-users] Re: Installation Questions [Installation}

by Odhiambo Washington

On Mon, Nov 4, 2024 at 12:36 PM Gerald Vogt <vogt(a)spamcop.net> wrote: > On 04.11.24 09:13, Odhiambo Washington via Mailman-users wrote: > > On Mon, Nov 4, 2024 at 10:34 AM Gerald Vogt <vogt(a)spamcop.net> wrote: > >> On your server it looks like this: > >> > >> # ls -la /etc/mailman3 > >> total 28 > >> drwxr-xr-x. 2 root mailman 95 Oct 25 08:12 . > >> drwxr-xr-x. 99 root root 8192 Oct 29 07:42 .. > >> -rw-r--r--. 1 root mailman 266 Oct 25 07:37 gunicorn.conf > >> -rw-r-----. 1 root mailman 92 Nov 21 2023 mailman-hyperkitty.cfg > >> -rw-r-----. 1 root mailman 797 Sep 9 11:20 mailman.cfg > >> -rw-r-----. 1 root mailman 3015 Oct 25 08:12 settings.py > >> > >> and it works just fine. > >> > > > > True, but making the mailman user own the files makes life easier when > you > > operate from the virtualenv - > > you do not have to exit the virtualenv to edit the files in > /etc/mailman3, > > and then re-enter the virtualenv. > > The virtualenv doesn't change the current uid. That doesn't make a > difference. > > > You do not have to give the mailman user sudoer rights. That's the whole > > point about the below: > > > > ``` > > sudo mkdir /etc/mailman3 > > sudo chown mailman:mailman /etc/mailman3 > > sudo chmod 755 /etc/mailman3 > > ``` > > Well, that essentially was my question: why does the mailman user > require sudo rights? Mark did not say that the mailman user required sudo access. Please re-read what he said. > Why does it need to be able to write or change > those files/directories? Except for the convenience which isn't a reason > to weaken security. > Make me understand how the security is weakened. The Mailman user account has no password so cannot login from outside. Even if it was able to login, it will still end up only accessing $home and /etc/mailman3, no? > In respect to security, i.e. separation of the service user from write > access to it's core configuration files, it should not be done unless > absolutely necessary. Sorry, I didn't understand that. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

5 months, 2 weeks

[MM3-users] Weird response to postings after subscriptions and accounts

by Paul Arenson

I accidentally posted to a test account I had not actually used. Am reposting to my correct account. Sorry to the moderators. In my test of the two forms of signup, I am getting these unexpected results. Is it my settings or is something weird within the program itself? 1) https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres… 2) Choose "You can also subscribe without creating an account. If you wish to do so, please use the form below" 3) Then I have to approve it. They get "Welcome to the "Discuss" mailing list! To post to this list, send your email to: discuss(a)list.tokyoprogressive.org 4) But I have to approve it.I assume this is because they do not yet have a real account. And they should be listed as a NON MEMBER? 5) But even when they sign up here: https://list.tokyoprogressive.org/accounts/signup/?next=%2Fpostorius%2Flist… they get an email from my provider (NOT MY LIST????) as follows: "Hello from mailman3.emwd.com! You're receiving this e-mail because user iCloud has given yours as an e-mail address to connect their account. To confirm this is correct, go to https://list.tokyoprogressive.org/accounts/confirm-email/MTE3:1iA44k:E7w7xh… Thank you from mailman3.emwd.com! mailman3.emwd.com" 6) And this is where it gets weird. If they send from their email program and get the welcome (Welcome to the "Discuss" mailing list! To post to this list, send your email to: discuss(a)list.tokyoprogressive.org You can unsubscribe or make adjustments to your options via email by sending a message to....... I then get a notice: "Your mail to 'discuss(a)list.tokyoprogressive.org' with the subject test notice Is being held until the list moderator can review it for approval. The message is being held because: The message is not from a list member" 7) While if they send FROM THE interface it goes through. 8) What I am confused about. Signing up for an account should make them a member, right? Subscribing should make them a non member, no? But here it seems that both make them a non member. And worse, someone with an account sending from the Interface has their message get through while the same person sending from an email program has me get a notice that a non member is trying to post. Before I can move my actual members over from another mailing list provider, I want to make sure what works and what doesn't. Thanks. PS I am running out of test accounts to use to test my own setup. Thanks. paul!!!Subscribe to this list still shows up at the bottom.

5 years, 7 months

[MM3-users] Re: mailman3 not working for mailman21 lists

by Stephen J. Turnbull

blup(a)fortytwo.ch writes: > First I did create the list with the old name and imported the old > configs with mailman import21. As the list did not work then, I > deleted it and created it again (with the same manual configuration > as for the new list that's working). OK. So as far as you know the configurations of list1 and list2 on Mailman 3 are identical, at least in the case where you have deleted list1 and recreated by hand. > I don't have a qfiles directory. Try queue/, especially look for the out[going], pipeline, and shunt subdirectories. > > Is the 'outgoing' qrunner still running? (ps ax | grep outgoing) > > no Sorry, I got that wrong. Mark recommends this: ps -fwwA|grep runner=|sed s/.*runner=// and gave this as sample output: archive:0:1 bounces:0:1 command:0:1 in:0:1 lmtp:0:1 nntp:0:1 out:0:1 pipeline:0:1 rest:0:1 retry:0:1 task:0:1 virgin:0:1 digest:0:1 rest:0:1 rest:0:1 so it looks like the process name of the runner is "out", not "outgoing". Besides checking for the "out" runner, also check for "pipeline", which has been known to fail in the past. > But once more: list1 was working fine as long as mailman21 was used > and stopped working when switching to mailman3. I don't expect spam > filters that worked with mailman21 to stop working with mailman3. Mailman 3 is a complete rewrite, essentially no Mailman 2 code was left untouched because of the pervasiveness of the change of the fundamental string type str from encoded bytes to abstract characters. I don't expect behavior to change, but I certainly am not going to rule anything out when trying to diagnose problems based on so little information. > I have more or less no Python knowledge (i.e. I can read and > understand Python but can't write it). Could you please give me > detailed command by command instructions. Thanks. I'll take a look at it tomorrow, but I can't promise results until the weekend if I can't write a script in 30 minutes. Normally Mark Sapiro would handle this kind of thing, but he's offline for a couple weeks so you've got me. My knowledge is mostly on the email protocols side of things (Internet message format, authentication protocols), not the internal processing. I'll do what I can, but if I can't do it real quick it's probably going to take me several hours, which I'm not going to have until the weekend. Steve

2 years, 8 months

[MM3-users] Re: Avoiding non-members to be saved.

by Stephen J. Turnbull

Guillermo Hernandez (Oldno7) via Mailman-users writes: > I find it very convenient too, but it concerns me that spammers, > really "profesional" spammers and senders of phising mail change > the forged address and sender domain too rapidly to track them. As > of it, the non-member database is growing more and more. It could > be a "monster" in two or three years... Well, let's think that out. I think we allocate a 255-byte text field for email addresses, another 255 bytes for display names, and I can't imagine that including the rest of the data fills as much as 1k, even including indicies and other database optimization overhead. So if you get a million spam messages to your lists, each from a unique address, that's only one gigabyte. To get a million spam messages in three years, you'd need to get almost a thousand per day that get through your MTA's filters to Mailman's moderation queue, which is the only place this automatic construction of nonmembers takes place. I think your moderators would let you know about that long before your disk fills up. I'm not making fun of you, I'm explaining why I don't think this is a high priority for us, and why you don't need to worry about monster databases, either: the human resources just for looking at the queue, let alone taking action, will be a constraint long before the disk or CPU consumption for the address database is. If there's something wrong with my analysis, let me know, but as far as I can see, if the spam flood to your lists gets to that scale, the first worries are denial of service and admin exhaustion. As usual, the best answer is to stop the barbarians at the gate (ie, the MTA). I guess if the list is configured to automatically discard all posts from non-members, we could optimize away the non-member creation. But this seems like a relatively rare case, and if there were non-local bans on addresses, it might not be a good idea anyway. In theory, we could even arrange to do the lookup at moderation time, although we currently do it at mail receipt time. That would allow a moderator to ban the addressee on list A and then not even see the post when they go to check on list B. I suspect even professional spammers don't bother to create a unique sender address for each recipient! Obviously, this kind of service would require putting the nonmember addresses in the database, at least temporarily. Steve

3 years, 8 months

[MM3-users] Re: Gmail problems?

by turnbull

2023-04-24 02:12 に Stephen Daniel さんは書きました: > When I check a message that I received from the list I am told: > ARC-Authentication-Results: i=2; mx.google.com; > dkim=neutral (body hash did not verify) header.i=(a)bellsouth.net > header.s=s2048 header.b=nkq1aBRe; This DKIM failure is normal. To get the body hash to verify, you need to set the body header and body footer to empty (beware, you can't see whitespace, but if there's whitespace, it's not *empty*). The header usually is empty, but by default we do set the footer, so you'd have to clean that out. Of course, if you have legal or organizational requirements to set the footer or header, you're stuck. > arc=pass (i=1); ARC is a protocol designed for mailing lists and other intermediaries that may change messages in ways that invalidate cryptographic signatures. It works by having each host that may munge messages (1) check all the previous ARC authentication results and (2) providing its own signature on those results if the most recent one is authentic and validates. (This is like a hash chain such as in git.) > spf=pass (... > > I also see this: > ARC-Authentication-Results: i=1; pb-mx22.pobox.com; > arc=none (no signatures found); This says that at the first ARC host in the path no ARC headers were found. Of course not, there were no previous participating hosts. > bimi=skipped (DMARC did not pass); > dkim=fail (message has been altered, 2048-bit rsa key sha256) Note it has already failed DKIM here. This says that some domain (presumably the one providing the Mailman host) (a) altered the message, invalidating the DKIM signature, and (b) based on arc=none above, does not participate in ARC. > header.d=bellsouth.net header.i=(a)bellsouth.net header.b=nkq1aBRe > header.a=rsa-sha256 header.s=s2048 x-bits=2048; > dmarc=none policy.published-domain-policy=none > > How do I fix this DKIM error? > Has the body changed because I add a footer to emails as they pass > through? Yes. There are two ways to fix it. (1) Don't add a header or footer to the body, and don't munge the subject with [list: serial#] annotations. This is guaranteed to work, in the sense that the DKIM signature will validate, and presumably DMARC "from alignment" will pass. (2) Use the ARC protocol. Mailman 3 provides the basic functionality, but exactly how you you hook that up to your site's crypto credentials depends on your site. Probably you can just reuse the DKIM key-pair. ARC may be less reliable, depending on recipient host policy (including whether it participates in ARC at all). Steve

2 years

[MM3-users] Re: list footer questions

by Mark Sapiro

On 5/8/20 12:52 PM, Mohsen Masoudfar wrote: > Hi, > > This seems to me like basic simple questions, I have tried FAQ and other resources, with not much success. > > I have two questions: > > 1. When I send an email to a list, in gmail and yahoo I see the following text with the links at the end of the email body: > _______________________________________________ > Testlist_dmark1 mailing list - listname(a)domain.tlb<mailto:testlist_dmark1@newlistserv.aaas.org> > To unsubscribe send an email to listname-leave(a)domain.tlb > > In<mailto:listname-leave@domain.tlb%0b%0bIn> outlook this is added as an attachment in the form of a TEXT file, when you open it, you see the same text. > Question: Is there a way to change this, that it looks in outlook the same way as it appears in gmail/yahoo? There are a few things going on here. First, the actual footer is _______________________________________________ Testlist_dmark1 mailing list - listname(a)domain.tlb To unsubscribe send an email to listname-leave(a)domain.tlb The <mailto:...> links are added by the Mail client displaying the mail. > In outlook this is added as an attachment in the form of a TEXT file, when you open it, you see the same text. > Question: Is there a way to change this, that it looks in outlook the same way as it appears in gmail/yahoo? > > I know MS does it own stuff sometime and we have no impact on that, but I was thinking maybe we might change the footer in a way that outlooks sees that as part of email body. Yes, this is an outlook thing, but you do have some control. See the FAQ article at <https://wiki.list.org/x/4030707> for a more detailed explanation, but the issue is that after any content filtering has been applied, the message is still multipart so Mailman adds the footer as a separate MIME text/plain part at the end of the message. You can avoid this by doing heavy content filtering on the messages. Namely, accept only multipart, text/plain and text/html, collapse alternatives and convert html to plain text. Note that some of this involves updating the list's pass_types which is not exposed in Postorius before 1.3.3 which is not yet released. > 2. How may I change the format and the text in footer? > I was expecting it under list > setting, in postorius, but I could not find it. It's the top level Templates tab in Postorius (just left of Settings). It was added in Postorius 1.2.1. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 11 months

[MM3-users] Re: AH01102: error reading status line from remote server 172.19.199.3:8000

by Abhilash Raj

Hi Henrick, On Tue, Jul 18, 2017, at 04:54 AM, Henrik Rasmussen wrote: <snip> > raise ImproperlyConfigured("No Facebook app configured: please" > ImproperlyConfigured: No Facebook app configured: please add a SocialApp > using the Django admin This is a known issue, it happens because the Facebook is added as a default service provider but it crashes the login page if it is not configured. As a get around, you can either remove the Facebook as a service provider from the Django's INSTALLED_APPS variable, or you can login into the Django's admin console and setup Facebook as a login provider. Documentation should have instructions on both, please ask again if anything isn't clear there. > > When I do sign in, I get the following from uwsgi-error.log <snip> > File > "/usr/local/lib/python2.7/site-packages/django/core/mail/backends/smtp.py", > line 100, in send_messages > new_conn_created = self.open() > File > "/usr/local/lib/python2.7/site-packages/django/core/mail/backends/smtp.py", > line 58, in open > self.connection = connection_class(self.host, self.port, > **connection_params) > File "/usr/local/lib/python2.7/smtplib.py", line 256, in __init__ > (code, msg) = self.connect(host, port) > File "/usr/local/lib/python2.7/smtplib.py", line 316, in connect > self.sock = self._get_socket(host, port, self.timeout) > File "/usr/local/lib/python2.7/smtplib.py", line 291, in _get_socket > return socket.create_connection((host, port), timeout) > File "/usr/local/lib/python2.7/socket.py", line 575, in > create_connection > raise err > socket.error: [Errno 111] Connection refused This is because you haven't setup Django to send emails. When you register an email with Django, it needs to be able to verify that it is a valid email address for which it should be configured to send email. See the documentation on how to do so at docs.mailman3.org asynchronous.in/docker-mailman/ > Henrik > > -----Oprindelig meddelelse----- > Fra: Abhilash Raj [mailto:raj.abhilash1@gmail.com] > Sendt: 17. juli 2017 18:04 > Til: mailman-users(a)mailman3.org > Emne: [MM3-users] Re: AH01102: error reading status line from remote > server 172.19.199.3:8000 > > Hi Henrik, > > On Mon, Jul 17, 2017, at 02:31 AM, Henrik Rasmussen wrote: > > I've just installed Mailman 3.1 on RHEL7 (Docker-Mailman3) and to get the > > right CSS layout I installed Apache on the Mailman server outside Docker, > > as advised by Abhilash Raj and Alexander Klotz. > > > > My Mailman server is on a secured network and the https connection from > > Internet will be handled by proxy servers, so I will be using http (port > > 80) between proxy-servers and Mailman server. > > > > But when I sign in (currently via an SSH tunnel to the Mailman server > > from my workstation) I get "Internal error: [client ::1:41912] AH01102: > > error reading status line from remote server 172.19.199.3:8000, referer: > > http://localhost/accounts/login/?next=%2Fpostorius%2Flists%2F" > > > Output from logs might be more helpful to debug what is wrong. They are > present at `/opt/mailman/web/logs/mailmanweb.log` for Django. You may > also find some other log files there which are related to uwsgi, the > WSGI server that is used to interface Django with Web Servers like > Apache. > > > If I type a wrong password, I am just warned about that, so the problem > > appears when log in is accepted. > > > > I expect that it is because I connect to localhost:80 through SSH tunnel. > > What do I miss in my Apache configuration on the Mailman server to avoid > > this problem? > > > > <VirtualHost *:80> > > ErrorLog /var/log/httpd/error_log > > LogLevel warn > > CustomLog /var/log/httpd/access_log combined > > > > ServerAdmin root(a)example.foo > > ServerName list.foo.bar > > Alias /static /opt/mailman/web/static > > Alias /favicon.ico /opt/mailman/web/static/hyperkitty/img/favicon.ico > > > > ProxyPassMatch ^/static/.* ! > > ProxyPass / http://172.19.199.3:8000/ > > ProxyPassReverse / http://172.19.199.3:8000/ > > > > <Directory /> > > Allow from all > > Require all granted > > </Directory> > > > > <Directory /opt/mailman/web/static/hyperkitty/img/> > > Allow from all > > Require all granted > > </Directory> > > > > # ssl config here ... > > </VirtualHost> > > > > > > Thanks, > > Henrik Rasmussen > > > > _______________________________________________ > > Mailman-users mailing list > > mailman-users(a)mailman3.org > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > > -- > thanks, > Abhilash Raj > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ -- thanks, Abhilash Raj

7 years, 9 months

[MM3-users] Re: How To Install Mailman 3 on Debian 10 (Complete Guide)

by Allan Hansen

My 2 cents’ worth: When moving to MM3 from MM2, the latter of which I (relatively) easily installed myself, I read what I could find and went on installing MM3. I soon ran into trouble, namely somewhere around the hooking up of the web server. I ran into a thicket that I could not decipher. Then I asked the person who now hosts the server. He soon ran into trouble at around the same place (as far as I remember). I then asked someone who had been very helpful with MM2 to do the installation. He's a professional. He soon ran into trouble. I don’t know where, but he tried over several weeks to get it going. The trouble? Finding the correct documentation. Understanding the documentation, which is not exactly clear. Doing what the documentation says, if it says it, and things not working as they should. I then asked Brian, who has helped a lot of people (like many of you have). He did the installation very quickly and with no trouble. Don’t tell him this, but it was money well spent! :-) Yours, Allan Hansen hansen(a)rc.org > On Feb 25, 2021, at 18:38 , Brian Carpenter <brian_carpenter(a)emwd.com> wrote: > > On 2/25/21 8:28 PM, Abhilash Raj wrote: >> Thanks Brian, this looks quite comprehensive in the details. The only >> thing I am a bit concerned about is granting sudo privileges to the >> Mailman user. It really shouldn't have sudo given that Mailman and >> Django are supposed to run as mailman user. Any compromise of >> the Django application will provide the attacker root on the machine. > > Thanks for pointing that out. I have removed that step. It really wasn't needed. > >> Is there a specific reason that you chose to go with an entirely new doc > > Yes. Just from observing the communications on this list, there are multiple ways to install Mailman 3 which produces installation issues, database problems, outdated apps, etc. My document produces a VERY stable installation of Mailman 3 and one that is easy to maintain and update. It is also very comprehensive covering not just Mailman Suite, but the configuration of an entire server environment. Personally I think the choice of Debian, NGINX, Postfix, and PostgreSQL makes for a long and loving relationship with Mailman 3 but I do understand everyone has their own environment that they want to work with. My guide represents a very stable and easily maintained environment for me and hopefully others. > >> rather than helping to improve the existing one? Several parts of it (at >> least the ones that official guide covers) seems similar to me and is >> duplicate information that at least two people are going to spend time >> writing and maintaining in future. > > Is there an official guide? See that is the problem. I did not know there was an official guide. Because every time I tried finding documentation in the past using Google searches, I got fed outdated information. So for me the guide that I came up with, works great for me and if no one else uses it, I certainly will. > > Your guide, by nature has to be minimalistic to accommodate all the various server environments out there. My is meant to be narrow but comprehensive. > > >> Is it something about the contribution process to the official documentation >> that makes it hard for people to contribute? Most of the pages at >> docs.mailman3.org or come from this[1] repo and use Sphinx to build and >> REsT formatting (.rst). >> >> [1]:https://gitlab.com/mailman/mailman-suite-doc > > Well I don't know much about any of the above approach to documentation. So I would say that is a hindrance. > > >> I am just trying to understand how can we lower the barrier for community >> members to help contribute to existing docs instead of them having to create >> new ones. Specifically around installation, since that tends to get stale often >> when depedent packages change or a new dependency is added that >> breaks the installation. > > I had to update my guide twice due to recent changes made to the installation process such as the addition of the Dart Sass section and the problem with the cryptography module when installing Hyperkitty. While my guide covers these new hurdles to a new installation, I don't think yours says anything about them. The Dart Sass section came from Mark's comments on this list. > > -- > Brian Carpenter > Harmonylists.com > Emwd.com > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

4 years, 1 month

[MM3-users] Re: How to store additional information about lists?

by Stephen J. Turnbull

Andreas Barth writes: > public: anyone can subscribe and read the archive. This is already possible, more or less the default list configuration. "Subscribe and read archive" involves multiple separate list attributes, enforced by three separate applications. Subscription policy is controlled by whether you need approval of an admin or not, and whether you need to verify your email or not, plus the ban lists. Archive access (assuming you have one at all) is contrlled by a different list attribute, archive_policy. Core has final say on subscriptions, but Postorius could do it if email subscriptions are disabled. HyperKitty handles the archive permissions. These various aspects are currently quite loosely coupled. > restricted: for employees (or members - or whatever) only. So they > could subscribe themself. The archive and posting may be restricted > to subscribers or to anyone in that group - both would be working > for me. I think this can be implemented with the controls we already have. The basic idea is to handle the "anyone in that group" case by mass subscribing and setting the subscription to no-mail. Obviously, this is most practical with a fairly static membership (eg, students at a school). A fluid membership (for example, a volunteer organization) would be made easier to manage with a "member adapter" to an external database to handle the "do I know this person" questions. ("Member adapter" is Mailman 2 terminology, I don't know if it's exactly the same in Mailman 3; it does or *should* exist in Mailman 3, though.) It's possible that we could make this easier to configure with some changes to core. However, if this is as close to what you need as I think it is, I would advise against vendoring Mailman, and instead writing some scripts to help manage mass subscription, or create the member adapter. > (How to know who is allowed to subscribe is another question - having > an internal mailing list which contains all the members just for this > purpose would be ok for me.) That's helpful to know. This sounds very similar to the Systers approach, where the Mailman core User database is the authoritative membership database for the organization. > private: hand-selected people, think about e.g. board or steering > committees internal discussions. Of course, the archive (if there is > any) should be restricted to the actual members. Posting to these > lists is currently not restricted, so no derivation needed from the > current set in postorius. This is already possible by requiring admin approval, or by setting the ban list to something that matches everything like "^.".

3 years, 10 months

[MM3-users] Re: using SSH/TLS with external MTA

by Roland Giesler

On 2024/07/28 17:06, Mark Sapiro wrote: > On 7/28/24 06:26, Roland Giesler via Mailman-users wrote: >> On 2024/07/28 00:39, Mark Sapiro wrote: >> A mail was sent to root on the localhost with the following content: >> >> Return-Path: <root(a)giesler.za.net> >> X-Original-To: root@localhost >> Delivered-To: root@localhost >> Received: from mailman.fast.za.net (localhost [IPv6:::1]) >> by mailman.localdomain (Postfix) with ESMTP id 2174A120A5B >> for <root@localhost>; Sun, 28 Jul 2024 13:11:03 +0000 (UTC) >> Content-Type: text/plain; charset="utf-8" >> MIME-Version: 1.0 >> Content-Transfer-Encoding: quoted-printable >> Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: >> /accounts/login/ >> From: root(a)giesler.za.net >> To: root@localhost >> Date: Sun, 28 Jul 2024 13:11:03 -0000 >> Message-ID: <172217226313.2150.10895652368679013629(a)mailman.fast.za.net> >> >> Internal Server Error: /accounts/login/ >> >> SMTPRecipientsRefused at /accounts/login/ >> {'roland(a)giesler.za.net': (454, b'4.7.1 <roland(a)giesler.za.net>: >> Relay acce= >> ss denied')} > > This is mail from django-allauth, not from Mailman. Your settings in > mailman.cfg do not affect how this mail is sent. See > https://docs.djangoproject.com/en/5.0/topics/email/#smtp-backend for > the Django settings for this. These settings from the Django docs look like the ones in mailman.cfg. Do I have to set them somewhere else too? SMTP backend¶ <https://docs.djangoproject.com/en/5.0/topics/email/#smtp-backend> /class/|backends.smtp.||EmailBackend|(/host=None/,/port=None/,/username=None/,/password=None/,/use_tls=None/,/fail_silently=False/,/use_ssl=None/,/timeout=None/,/ssl_keyfile=None/,/ssl_certfile=None/,/**kwargs/)¶ <https://docs.djangoproject.com/en/5.0/topics/email/#django.core.mail.backen…> This is the default backend. Email will be sent through a SMTP server. The value for each argument is retrieved from the matching setting if the argument is|None|: * |host|:|EMAIL_HOST| <https://docs.djangoproject.com/en/5.0/ref/settings/#std-setting-EMAIL_HOST> * |port|:|EMAIL_PORT| <https://docs.djangoproject.com/en/5.0/ref/settings/#std-setting-EMAIL_PORT> * |username|:|EMAIL_HOST_USER| <https://docs.djangoproject.com/en/5.0/ref/settings/#std-setting-EMAIL_HOST_…> * |password|:|EMAIL_HOST_PASSWORD| <https://docs.djangoproject.com/en/5.0/ref/settings/#std-setting-EMAIL_HOST_…> * |use_tls|:|EMAIL_USE_TLS| <https://docs.djangoproject.com/en/5.0/ref/settings/#std-setting-EMAIL_USE_T…> * |use_ssl|:|EMAIL_USE_SSL| <https://docs.djangoproject.com/en/5.0/ref/settings/#std-setting-EMAIL_USE_S…> * |timeout|:|EMAIL_TIMEOUT| <https://docs.djangoproject.com/en/5.0/ref/settings/#std-setting-EMAIL_TIMEO…> * |ssl_keyfile|:|EMAIL_SSL_KEYFILE| <https://docs.djangoproject.com/en/5.0/ref/settings/#std-setting-EMAIL_SSL_K…> * |ssl_certfile|:|EMAIL_SSL_CERTFILE| <https://docs.djangoproject.com/en/5.0/ref/settings/#std-setting-EMAIL_SSL_C…> The SMTP backend is the default configuration inherited by Django. If you want to specify it explicitly, put the following in your settings: EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend" If unspecified, the default|timeout|will be the one provided by|socket.getdefaulttimeout()| <https://docs.python.org/3/library/socket.html#socket.getdefaulttimeout>, which defaults to|None|(no timeout).

8 months, 3 weeks

[MM3-users] Re: Mailman sends Welcome emails, but does not send [List] postings emails

by Stephen J. Turnbull

Nelson Strother writes: > I did not intend to imply that `mailman start` (predictably) > consumes more time than `mailman restart`. Both are lengthy and > variable; You did say something about "30min" I believe. I don't understand why it would take anywhere near that long. For example, I'm working with a site with ~20,000 lists, and it only takes a few seconds. Oh ... wait ... are you using Postfix as the MTA? The supplied Postfix MTA interface rebuilds the list aliases database on every (re)start, and that's an O(#lists^2) process the way Mailman does it. Postfix doesn't actually use the file to route mail, it always goes to lmtp:127.0.01 or something like that. So I configured Postfix to probe for existence of the list in Mailman's database. I think Postfix has plugins for all Mailman-supported databases (PostgreSQL, MySQL/MariaDB, SQLite3). I can provide the changes needed for this for PostgreSQL right away, if you're a MySQL site I can give you the PostgreSQL config to give you the idea, but I may be sslow if you want advice about the MySQL config. It's pretty straightforward, though. > I do not yet understand how to make use of these clues, but at > least one can see an epitaph from each deceased process. I don't understand them either. The only thing I can think of is that a runner may try to access Mailman core, timeout, and crash during the long startup. At least the MTA reconfig would make the restart cycle a lot less painful. Note: this is NOT a Mailman-supported config (or even contrib quality) yet, but I'm pretty confident of the theory and it is working well in QA testing. Regarding the "alpha" status of the config, one Mailman 3 instance can support multiple domains, but the Postfix docs say not to hit the database for the "domain exists" check. If you aren't going to add additional Mailman domains, then this doesn't matter (the domain list file produced by the stock Mailman postfix config will work), but if you do add or remove domains, you either need to configure Postfix to hit the database for the "domain exists" check too contrary to Postfix's advice, or you'd need to manage the domain list file by hand. In any case, you could use this config for debugging, and go back to the tried and true aliases file approach for production. And as I indicated I'm somewhat available on a best-effort basis for trouble-shooting.

1 year, 10 months

[MM3-users] running a script for the mailman-shell (to discard held messages) gives errors about a package argument for relative import

by stephan.edel＠gmx.net

Our organization has been been communicating through about a dozen mailing lists, running on mailman, with ~200 subscribers, for 15-odd years. After Debian 10 LTS ran out, I had to upgrade our organisations Debian server, forcing me into mailman3. I haven't managed, yet, to get the web interface running, so I need to do the most urgent tasks in the shell for now. I have found out that, for example, to discard all held messages from the list "motor", I need to > sudo -u list mailman shell -l motor.our-server.de and in the shell, >>> from mailman.app.moderator import handle_message >>> rdb = IListRequest(m) >>> for req in rdb.held_requests: ... if req.request_type == RequestType.held_message: ... handle_message(m, req.id, Action.discard) > commit() Now, obviously, I put these exact lines in a script, so I don't have to type it out every single time, and try to run it using > sudo -u list mailman shell -l testlist.our-server.de -r ./mm_discard_held.py but that invariably gives me ------------------------ Traceback (most recent call last): File "/usr/bin/mailman", line 33, in <module> sys.exit(load_entry_point('mailman==3.3.8', 'console_scripts', 'mailman')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 69, in invoke return super().invoke(ctx) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/commands/cli_withlist.py", line 294, in shell r = call_name(dotted_name, m, *run_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/utilities/modules.py", line 69, in call_name named_callable = find_name(dotted_name) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/utilities/modules.py", line 52, in find_name module = import_module(module_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/importlib/__init__.py", line 121, in import_module raise TypeError(msg.format(name)) TypeError: the 'package' argument is required to perform a relative import for './mm_discard_held' ------------- I've tried a PYTHONPATH=/root/bin , where mm_discard_held.py lives, as suggested. I've tried moving it to the path where the mailman binary lives, as suggested. I always get the same result. What else can I do? Thank you.

10 months, 1 week

[MM3-users] Re: Error on timeout vs retry values

by Robert Moody

Thanks I will give that a go a bit later today and see what happens. ⁣Get BlueMail for Android On 19 Jul 2021, 07:34, at 07:34, Abhilash Raj <maxking(a)asynchronous.in> wrote: > > >On Sat, Jul 17, 2021, at 11:23 PM, Robert Moody wrote: >> Hi everyone, >> >> I have recently setup a new mailman3 service using the venv method. >> Everything is functionally working however I keep getting an error >sent >> to the mailman email every min which I can not seem to figure out >why. >> >> /opt/mailman/venv/lib/python3.7/site-packages/django_q/conf.py:142: >> UserWarning: Retry and timeout are misconfigured. Set retry larger >than >> timeout, >> failure to do so will cause the tasks to be retriggered >before >> completion. >> See >> https://django-q.readthedocs.io/en/latest/configure.html#retry for >details. >> See https://django-q.readthedocs.io/en/latest/configure.html#retry > >> for details.""" >> >> I have edited the timeout and retry values in the >> /opt/mailman/venv/lib/python3.7/site-packages/django_q/conf.py file >and >> restarted both mailman and mailman-web however this still keeps >repeating. > >You shouldn't edit files in `site-packages` directory as they are part >of the package itself and could cause issues. > >What you want to do is override it in your settings.py (or >settings_local.py) file >for Django. I am not sure which guide you followed to install Mailman, >but it >should be somewhere in /opt/mailman/ I suppose (not in venv/ for sure). > > >It should look something like this > >https://mailman-web.readthedocs.io/en/latest/settings.html#mailman_web.sett… > >Just the value of retry should be just > than timeout as the error >says, so >not necessarily same values, but so far 300 and 360 has been working >good for us and I haven't seen any report to increase them. > > >> >> Any ideas? >> >> Regards, >> >> Robert >> >> >> -- >> This message has been scanned for viruses and >> dangerous content by MailScanner, and is >> believed to be clean. >> >> _______________________________________________ >> Mailman-users mailing list -- mailman-users(a)mailman3.org >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> > > >-- > thanks, > Abhilash Raj (maxking) >_______________________________________________ >Mailman-users mailing list -- mailman-users(a)mailman3.org >To unsubscribe send an email to mailman-users-leave(a)mailman3.org >https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

3 years, 9 months

[MM3-users] Please help - Held message is crashing mailman.

by Alex King

Hi, I've used mailman a lot in the past, and decided to install mailman3 in a recent install. I'm unimpressed so far, it seems buggy. Running on Ubuntu 18.04.2 LTS from packages, mailman3 3.1.1-9, mailman3-web 0+20170523-14, python-django-postorius 1.1.2-3. After a few messages, I now can't go to the "Held Messages" in postorius, I get a 500 error. I tried to interact on the command line, but could not work out how. Reading https://mailman.readthedocs.io/en/latest/src/mailman/docs/install.html and https://mailman.readthedocs.io/en/latest/src/mailman/commands/docs/shell.ht…, I tried: # mailman shell Welcome to the GNU Mailman shell >>> command = cli('mailman.commands.cli_withlist.shell') Traceback (most recent call last): File "/usr/lib/python3.6/code.py", line 91, in runcode exec(code, self.locals) File "<console>", line 1, in <module> NameError: name 'cli' is not defined I did a lot of reading of the manual and googled, but I have no idea where the cli object is supposed to come from.....? The documentation is confusing. Anyway, after some trial and error, i found: # mailman shell Welcome to the GNU Mailman shell >>> list_manager = getUtility(IListManager) >>> m=list_manager.get("committee(a)[redacted].org") >>> from mailman.interfaces.requests import IListRequests >>> requests = IListRequests(m) >>> [x for x in requests.held_requests][0].id 8 >>> requests.get_request(8) Traceback (most recent call last): File "/usr/lib/python3.6/code.py", line 91, in runcode exec(code, self.locals) File "<console>", line 1, in <module> File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/requests.py", line 120, in get_request result.data_hash, expunge=False) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/pending.py", line 138, in confirm value = json.loads(keyvalue.value) File "/usr/lib/python3.6/json/__init__.py", line 354, in loads return _default_decoder.decode(s) File "/usr/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.6/json/decoder.py", line 355, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 1 (char 0) It seems something was written out to disk that can't be read in for some reason (assuming a json object on disk?) How do I debug this further? How do I find the json being decoded? Any help would be appreciated. (I found mailman2 just worked, I was happy with that.) (My colleague posted a bug about this I believe but I don't have the link to that.) Thanks, Alex

6 years

[MM3-users] Re: How to delete non-users

by Allan Hansen

Thank you, Stephen. Yes, I have verified that the users will be subscribed once the moderator accepts their subscription requests. The tricky time is between the non-member record is created and the moderator accepts the application. Many of them will try to post before their subscription is approved (some moderators are good at procrastinating). During this time the user thinks that the subscription has been entered, but is not able to send/receive messages. Then they complain to me... While some of the users and moderators may have visual impairments (well, it's possible that even a majority wears glasses - I do), the bigger issue is that most are not techies, and some are very much not techies. They are brilliant in other ways and Mailman has been a wonderful tool for them all to communicate and organize, but it's also a bit quirky. Yours, Allan On 9/16/22, 23:28 , "Stephen J. Turnbull" <stephenjturnbull(a)gmail.com> wrote: Mark Sapiro writes: > My original point is the presence of a nonmember role should not prevent > the user from subscribing. If there is an issue with subscribing it is > probably because of the outstanding subscription request waiting > moderator approval. I don't think it is likely, but is it possible that the moderator is not just refusing the subscription, but banning the address from subscribing? > I understand that you would like the nonmember role to disappear, but > that isn't going to happen in the distribution, and I can't undertake > the extensive effort required to figure out how you can do it for your site. It seems to me that explicitly displaying the nonmember role is an internal and/or administrative need, and most ordinary users are only interested in whether they are subscribed or not. If the list appears in "their" list of list, they're going to assume they're subscribed. (We've even had that confusion about moderators and owners in the past.) I haven't done any deep thinking about this, but perhaps we could provide a more streamlined view tuned to this specific need? That is, the role won't go away, but at least in some contexts you'll have to ask explicitly for that user-list relationship to be displayed. Also, IIRC Allan's community contains a substantial group of folks with visual impairments, presumably including admins and moderators. Especially to serve that circumstance I think it makes sense for us to provide streamlined views that are more easily grasped as a whole where we can. _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

2 years, 7 months

[MM3-users] Re: Installing mailman 3

by Abhilash Raj

Hi Rafael, On Tue, Jul 11, 2017, at 07:32 AM, Rafael Mora wrote: > Hello, > > I want to install mailman 3 in a centos 7 virtual machine. I'm following > these steps and I cannot make it work: I am wondering if you are interested to setup a development environment or use Mailman 3? These instructions below are for working on Mailman, if you just need to deploy, instructions here[1] would be the best. > 1. `cd ~` > 2. `git clone https://gitlab.com/mailman/mailman.git` > 3. `cd mailman` > 4. `virtualenv -p python3 venv` > 5. `source venv/bin/activate` > 6. `python setup.py develop` > 7. I am pretty sure this setup command will place a configuration file > into > `~/mailman/var/etc/mailman.cfg`. Edit this file. > https://mailman.readthedocs.io/en/latest/src/mailman/config/docs/config.html > 8. `mailman info` can confirm which configuration file is in use > 9. `mailman start` will start up Mailman 3 Core. This is what the parent > background process command looks like in `ps -ef`: > `/home/ubuntu/mailman/venv/bin/python > /home/ubuntu/mailman/venv/bin/master > -C /home/ubuntu/mailman/var/etc/mailman.cfg` Looks like you are working as the ubuntu user here and the the virtualenv that you created exists at `/home/ubuntu/mailman/venv`. > I also tried the steps on the website > https://wiki.list.org/DEV/SetupDevEnvironment > but it doesn't work either > > What do you suggest? > > This is what I get from console: > > > [root@localhost ~]# cd dev/ > > [root@localhost dev]# cd mailman/ > > [root@localhost mailman]# source venv3/bin/activate Why do you try to user a different venv as a root user here? If you'd just try: $ source /home/ubuntu/mailman/venv/bin/activate and then try $ mailman start things would work! However, this would be a different instance of mailman and would create the new configuration (and the entire var directory) at the directory you run this command from. If you want to inhibit this, you can either use the -C flag, or set MAILMAN_CONFIG_FILE environment variable to point to your one single configuration file. > > (venv3) [root@localhost mailman]# cd mailman/ > > (venv3) [root@localhost mailman]# ls > conf.py COPYING data README.rst src > unittest.cfg > contrib coverage.ini MANIFEST.in setup.cfg template.py > copybump.py cron port_me setup.py tox.ini > > (venv3) [root@localhost mailman]# cd .. > > (venv3) [root@localhost mailman]# mailman start > Starting Mailman's master runner > /root/dev/mailman/venv3/bin/python: can't open file '/sbin/master': > [Errno > 2] No such file or directory > > (venv3) [root@localhost mailman]# > > Thank you > -- Hope that helps! [1]: http://docs.mailman3.org/en/latest/prodsetup.html -- thanks, Abhilash Raj

7 years, 9 months

[MM3-users] Re: Mails are stuck in Mailman Pipeline Queue

by Shashikanth Komandoor

Thank you Mark. This solution worked fine for me. On Sun, Mar 27, 2022 at 9:58 AM Mark Sapiro <mark(a)msapiro.net> wrote: > On 3/26/22 21:01, Shashikanth Komandoor wrote: > > > > But the below trace back is observed during mailman service stopping: > > > This doesn't say much ... > > > ...> Mar 27 09:01:22 2022 (5864) Uncaught runner exception: > > Mar 27 09:01:22 2022 (5864) Traceback (most recent call last): > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py", > > line 173, in _one_iteration > ... > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py", > > line 114, in signal_handler > > raise RunnerInterrupt > > mailman.interfaces.runner.RunnerInterrupt > > Mar 27 09:01:22 2022 (5864) SHUNTING: 1648351882.6693938+ > > c2e6ed709646a7680c73059b52d2f0451836e591 > > Mar 27 09:01:22 2022 (5864) digest runner exiting. > > Above, digest runner was interrupted in processing and shunted the > message it was working on. > > and below, pipeline runner was interrupted in processing and shunted the > message it was working on. > > > Mar 27 09:01:22 2022 (5860) Uncaught runner exception: > > Mar 27 09:01:22 2022 (5860) Traceback (most recent call last): > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py", > > line 173, in _one_iteration > > self._process_one_file(msg, msgdata) > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py", > > line 266, in _process_one_file > > keepqueued = self._dispose(mlist, msg, msgdata) > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/runners/pipeline.py", > > line 37, in _dispose > > process(mlist, msg, msgdata, pipeline) > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/core/pipelines.py", > > line 50, in process > > handler.process(mlist, msg, msgdata) > > File "/var/lib/mailman/venv/lib64/python3.6/site-packages/ > > mailman/handlers/member_recipients.py", line 84, in process > > for member in mlist.regular_members.members > > File "/var/lib/mailman/venv/lib64/python3.6/site-packages/ > > mailman/handlers/member_recipients.py", line 83, in <genexpr> > > recipients = set(member.address.email > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/model/roster.py", > > line 239, in members > > yield from self._get_members(DeliveryMode.regular) > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/model/roster.py", > > line 226, in _get_members > > if member.delivery_mode in delivery_modes: > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/model/member.py", > > line 205, in delivery_mode > > return self._lookup('delivery_mode') > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/model/member.py", > > line 167, in _lookup > > pref = getattr(self.address.preferences, preference) > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/sqlalchemy/orm/attributes.py", > > line 287, in __get__ > > return self.impl.get(instance_state(instance), dict_) > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/sqlalchemy/orm/attributes.py", > > line 723, in get > > value = self.callable_(state, passive) > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/sqlalchemy/orm/strategies.py", > > line 760, in _load_for_state > > session, state, primary_key_identity, passive > > File "<string>", line 1, in <lambda> > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/sqlalchemy/orm/strategies.py", > > line 850, in _emit_lazyload > > session.query(self.mapper), primary_key_identity > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/sqlalchemy/ext/baked.py", > > line 603, in _load_on_pk_identity > > setup, tuple(elem is None for elem in primary_key_identity) > > The above says the runner was doing a database lookup. This may be a > coincidence or there may be some database locking issue that underlies > the issue with the pipeline runner. > > > > File > "/var/lib/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py", > > line 114, in signal_handler > > raise RunnerInterrupt > > mailman.interfaces.runner.RunnerInterrupt > > Mar 27 09:01:22 2022 (5860) SHUNTING: 1648351882.8537986+ > > 9bf59cb23a71a4bae65536bfdbec0abb261adaa0 > > Mar 27 09:01:22 2022 (5860) pipeline runner exiting. > > > I suggest doing the following assuming your Mailman database is not > sqlite3 (if it is, just do step 2): > > 1) Stop mailman core and maybe other mailman services too. > 2) Assuming the two shunted messages are still in Mailman's shunt queue, > run `mailman unshunt` to unshunt them. If there are more shunted > messages, inspect them with `mailman qfile` to see if you want them, > They may be spam. > 3) Restart your database server. > 4) Start Mailman core and any other stopped services. > > Also, look in the database server logs for any issues. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- Thanks & Regards, Shashi Kanth.K 9052671936

3 years

[MM3-users] Intermittent: pymysql.err.InternalError: Packet sequence number wrong - got 102 expected 77

by Torge Riedel

|Hi list, sometimes I get the following error in mailman core's log: "pymysql.err.InternalError: Packet sequence number wrong - got 102 expected 77". I attached the full log after this mail. I check the provided link in the log (|||http://sqlalche.me/e/2j85|) and also searched for it on the internet and found the following possible reasons: 1. DB user does not have permissions at all -> I doubt that, because in general mailman is working ok 2. DB user is not allowed to connect from remote -> I doubt that too, because everything is on one host and DB user is allowed to connect from localhost 3. Multi-Threading -> pymysql is mentioned to be not thread-safe and each thread needs its own connection May this be caused by multi-threading? What do I have to do (is there anything I can do?) to solve this on my own?||||| Kind regards Torge || |Full log follows: [2021-05-23 07:31:45 +0200] [8244] [ERROR] Error handling request /3.1/lists?advertised=true&count=10&page=1 Traceback (most recent call last): File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1284, in _execute_context cursor, statement, parameters, context File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute cursor.execute(statement, parameters) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 170, in execute result = self._query(query) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 328, in _query conn.query(q) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 517, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 732, in _read_query_result result.read() File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1082, in read self._read_result_packet(first_packet) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1151, in _read_result_packet self._get_descriptions() File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1222, in _get_descriptions field = self.connection._read_packet(FieldDescriptorPacket) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 671, in _read_packet % (packet_number, self._next_seq_id)) pymysql.err.InternalError: Packet sequence number wrong - got 102 expected 77 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 274, in falcon.api.API.__call__ File "falcon/api.py", line 269, in falcon.api.API.__call__ File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/lists.py", line 322, in on_get resource = self._make_collection(request) File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/helpers.py", line 172, in _make_collection request, self._get_collection(request)) File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/helpers.py", line 167, in _paginate return list_start, total_size, collection[list_start:list_end] File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/utilities/queries.py", line 42, in __getitem__ return self._query[index] File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3149, in __getitem__ return list(res) File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3481, in __iter__ return self._execute_and_instances(context) File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3506, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1020, in execute return meth(self, multiparams, params) File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_clauseelement distilled_params, File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1324, in _execute_context e, statement, parameters, cursor, context File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1518, in _handle_dbapi_exception sqlalchemy_exception, with_traceback=exc_info[2], from_=e File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_ raise exception File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1284, in _execute_context cursor, statement, parameters, context File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute cursor.execute(statement, parameters) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 170, in execute result = self._query(query) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 328, in _query conn.query(q) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 517, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 732, in _read_query_result result.read() File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1082, in read self._read_result_packet(first_packet) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1151, in _read_result_packet self._get_descriptions() File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1222, in _get_descriptions field = self.connection._read_packet(FieldDescriptorPacket) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 671, in _read_packet % (packet_number, self._next_seq_id)) sqlalchemy.exc.InternalError: (pymysql.err.InternalError) Packet sequence number wrong - got 102 expected 77 [SQL: SELECT mailinglist.list_id AS mailinglist_list_id, mailinglist.preferred_language AS mailinglist_preferred_language, mailingli$ FROM mailinglist WHERE mailinglist.advertised = true ORDER BY mailinglist.list_id LIMIT %(param_1)s] [parameters: {'param_1': 10}] (Background on this error at: http://sqlalche.me/e/2j85) |

3 years, 11 months

[MM3-users] Re: My list has been switched from public to private archives, but they are still visible

by Enrique Terrazas

I am having a very similar issue. Here is what I have found: There are basically three archive settings we can select via the postorious web front end for archives. These can be found in: Settings > Archiving and they are: Public archives Private archives Do not archive this list In the Django administrative interface we can find those same(?/similar?) settings here: Main Page > Hyperkitty > Mailing lists > Mailing List object(x) > Archive policy: never private public I’m not sure which of the two takes precedence. My guess is postorius since that is a list owner accessible setting and the Django administrative interface would typically not be accessible to list owners. We have noticed the following odd behavior, not sure if it is a bug. For existing lists with private archives, when switching to “Do not archive this list” in postorious, the desired effect of no longer archiving the list is instantaneous. However, after the daily cron job that syncs with Mailman Core happens, the existing archives now become publicly accessible. Is this behavior based on the assumption that the list will no longer be archived and will therefore be purged? Enrique > On Sep 19, 2019, at 1:08 PM, Abhilash Raj <maxking(a)asynchronous.in> wrote: > > On Thu, Sep 19, 2019, at 3:31 AM, Paul Arenson wrote: >> Can anyone help with this? >> >> I set my list to private archives today. Up until then, it was pubic. >> And yet, if you go to the list page, you will see that if you click >> archives (top left), you can see them. (Filled with a bunch of tests >> posts for now) > > This change takes a bit of time to sync up Hyperkitty. THere is a daily > cron job which syncs this information from Mailman Core. > > I have made a change which should fix this problem and the process would > become synchronous. > > You can run `python manage.py runjobs --daily` command to force sync > that information. > >> >> https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist.toky… >> >> Am I misunderstanding what a private archive is? > > Not really, your expectation is right. > >> >> Is something wrong with my setup (which is hosted)? >> >> Thank you. >> _______________________________________________ >> Mailman-users mailing list -- mailman-users(a)mailman3.org >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org >> https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mai… >> > > -- > thanks, > Abhilash Raj (maxking) > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mai…

5 years, 7 months

[MM3-users] Re: Fwd: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive

by David Partain

Hi Mark, Thank you for your kind answer. While I've been running a mailman site for many years, I'm just getting going with mailman3 (and migrating my 1400 lists...) and I feel lost occasionally :) I wrote: > I have the same issue, and have changed the configuration in /etc/mailman3/mailman-web.py to: > > DATABASES = { > 'default': { > 'ENGINE': 'django.db.backends.mysql', > 'NAME': 'mailman3web', > 'USER': 'mailman3web', > 'PASSWORD': 'xxxxxxxxxxxxxxxxxx', > 'HOST': 'localhost', > 'PORT': '', > 'OPTIONS': { > 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", > 'charset': 'utf8mb4', > }, > } > } > > and restarted stuff, but I cannot tell that it has changed anything. You responded: > That should do it. I think the original message should have been saved > as a queue entry in Mailman's var/archives/hyperkitty/spool/ directory. That directory has all of the messages sent to that list: # ls /var/lib/mailman3/archives/hyperkitty/spool/ 1611827925.8030825+925c996b9b6020ee18e9a689ce23384855159f77.pck 1611827926.1923075+e728f3c29c1fe6eeeca641314cdba7007cb6e132.pck 1611827925.8917305+d16b7004bdbbef98ce40fb8ee70e492715d1607d.pck 1611827926.2623458+1aac13b50d8e7f32eb14d13e6559656aa8f3bd10.pck 1611827925.9594557+7ad2a910b5ee9142c88f0bf26c60c6df960d4262.pck 1611827926.3488555+04a540651c5f829751518433e1c54168ab63a38b.pck 1611827926.0404096+a5e745f51a9f7d541aeb770a89604be17019f3c5.pck 1611827926.4206145+de861eec6404d0c8f9f8bdb180e29bff2de7c52b.pck 1611827926.1167526+0aa63639a4ec102bf63522827637e75e6a2b092a.pck > If this is the case, it would have been retried and produced the same > error with each new post, and after you made the above change, it would > have been archived and the errors stopped. > Are you saying you are still seeing these errors? If so, it may be a > different error. What is the exact error message? We don't need all the > Django stuff, just the part equivalent to > Internal Server Error: /mailman3/hyperkitty/api/mailman/archive > OperationalError at /hyperkitty/api/mailman/archive > (1366, "Incorrect string value: '\xF0\x9F\xA4\xA3\xF0\x9F...' for > column mailman3web.hyperkitty_email.content at row 1") Yes, I'm still seeing the errors. When I send mail to a list with a cute emoji, I see the following in /var/log/mailman3/web/mailman-web.log: ERROR 2021-01-28 10:58:45,708 16923 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive (bunch of python errors omitted) DataError: (1366, u"Incorrect string value: '\\xF0\\x9F\\x98\\x8A\\x0A\\x0A...' for column 'content' at row 1") Also, a bunch of mail is sent to root, presumably one for every message that has previously failed, with the same error information, plus information about the POST method done. (If something there would be helpful, please let me know.) Finally, I'm doing all of this on Ubuntu, with the packages available in the distro, which means: mailman3 3.1.1-9 mailman3-doc 3.1.1-9 mailman3-full 3.1.1-9 mailman3-web 0+20170523-14 python-django-hyperkitty 1.1.4-4 python-django-mailman3 1.1.0-4 python-django-postorius 1.1.2-3 python-mailmanclient 3.1.1-5 python3-mailman-hyperkitty 1.1.0-4 Thanks very much for any help you can offer. Cheers, David

4 years, 2 months

[MM3-users] Re: Long-shot cry for help

by Mark Sapiro

On 6/3/21 6:34 AM, David Partain via Mailman-users wrote: > Hi, > > I'm having severe operational issues with my mailman3 site. My site is built on standard Ubuntu packages (18.04), and they're ancient. I'm having two big issues... > > 1. I have the old smiley problem that I've had from the very beginning and never solved. I have added 'charset': 'utf8mb4' to the relevant config file (/etc/mailman3/mailman-web.py), but that hasn't "taken", in the sense that the calls to mysql do not include the charset stuff. You may need to set the character set and collation for the DB or the server to utf8mb4. See https://dev.mysql.com/doc/refman/5.7/en/charset-applications.html for MySQL (I don't know about MariaDB. > /usr/share/mailman3-web/settings_local.py links to /etc/mailman3/mailman-web.py and there is no file /usr/share/mailman3-web/settings_local.pyc. I moved it out of the way hoping it'd get re-built, but it has not been in the months since. Python3 does not create .pyc files in the same directory as the .py. They are created in a __pycache__ sub-directory. See https://www.python.org/dev/peps/pep-3147/ for details. However, you seem to be still running a Python 2.7 version of Django et al. In this case, if it's not creating a .pyc, it is likely because it doesn't have permission. That is not a problem. If there's no .pyc, it just runs the .py. > The result of this that I get hundreds of mail messages to root (thousands, really), rapidly filling / with mail in /var/mail/root and logging in /var/log/mail.log. The mail has something like "Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive" and "DataError at /hyperkitty/api/mailman/archive (1366, u"Incorrect string value: '\\xF0\\x9F\\x98\\x8A\\x0D\\x0A...' for column 'content' at row 1")" Tell your users that smiley faces and other non-base-plane unicode emojis in email subjects add no value ;) > I have tried to trace to figure out what file is _actually_ controlling the calls to mysql. There's a file called /var/lib/ucf/cache/:etc:mailman3:mailman-web.py as well, but I have no idea what that is or if it might be used. I don't know what that is either. The underlying database access is via Django. See, e.g., https://docs.djangoproject.com/en/3.2/ref/databases/ However, it is not the MySQL queries that are the issue. > Is there something like 'mailman info' for mailman-web so I can see what files are being used? I'd love to fix this... > > 2. There's this one curious cron job... > > * * * * * www-data [ -f /usr/bin/django-admin ] && python2.7 /usr/bin/django-admin runjobs minutely --pythonpath /usr/share/mailman3-web --settings settings > > (yes, I know python2.7 is a problem... see the last paragraph below). > > These jobs start piling up. They run & run & run, and eventually the machine screeches to halt when it runs out of memory and goes belly up. This is a relatively recent development. After a while, I'll have 5 or 6 or more of these all running, looking something like this: > > www-data 6366 6364 99 15:00 ? 00:12:27 python2.7 /usr/bin/django-admin runjobs minutely --pythonpath /usr/share/mailman3-web --settings settings > > What this means in practice right now is that I'm forced to go in and kill these on a regular basis or the machine _will_ die and people _will_ shout at me. > > Does anyone have any idea why these might not be finishing? Can I comment this out completely & live happier? Any help whatsoever? You don't NEED to run this job. In the latest releases there are no minutely jobs anyway. Prior to HyperKitty 1.3.0 the update_index job was run minutely to update the search indexes with new posts, but now it's hourly. You probably have a huge backlog of unindexed posts causing this job to run for a long time (Whoosh is particularly slow at indexing, Xapian is much better). I suggest for now, you quit running the minutely jobs and just try manually running ``` python2.7 /usr/bin/django-admin runjob update_index --pythonpath /usr/share/mailman3-web --settings settings ``` which will probably run for hours if not days. Then after it completes, you can add that to cron to run hourly. > Finally, I should add that I know my current installation is fundamentally flawed because I chose to go with distro packages. I simply cannot fix it. My strong recommendation to others who follow me is that they do NOT do this. Bite the bullet and install via pip... I'm now in the process of setting up a new server that is completely based on the latest via pip installation. I have to get away from what I have or lose my sanity. Good plan. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years, 10 months

[MM3-users] Re: Post-installation documentation?

by David Newman

On 11/23/21 4:40 AM, Stephen J. Turnbull wrote: > David Newman writes: > > > I am looking for a high-level "do these steps next to get a list > > working" document rather than an API command reference. > > There's no such documentation. "Get a list working" depends on what > you mean by "working". Thanks for getting back to me. IMO post-installation docs for list admins would be a big help in Mailman3 adoption. I'm willing to help with this. When I say "working": I'm looking to migrate Mailman2 mailing lists to Mailman3, but am struggling with even basic setup tasks before we attempt list migtration. For example, on admin page I've set up two lists, one public and one private (each with unique list IDs), but neither appear in the site's list index. And I don't know where to add list members. And clicking the site archives link throws this error: > The basic answer is > > 1. Install Mailman 3 suite. > 2. Log in to Postorius as an admin user and create the list. > > After that, it's case-specific. > > > - why do the venv docs and admin FAQ give different nginx configs for > > the reverse proxy? > > Because the FAQ is providing a short snippet to solve a specific > problem, the venv docs provide what I guess is somebody's full working > config. Setting up a Mailman server is complex; usually any example > you see in the docs is somebody's "works for me" configuration. > > > - Does the 'Secret_Hyperkitty_API_Key' need to go anywhere else besides > > /opt/mailman/mm/mailman-hyperkitty.cfg? > > It needs to match MAILMAN_ARCHIVER_KEY in HyperKitty's settings.py. > There are a couple of reasonable ways to organize HyperKitty and > Postorius configs, I'm not sure where the appropriate settings.py > would be on your server. > > > - For Django (presumably in /etc/mailman3/settings.py, but the docs > > don't say), what values to use with Postfix to set up the email back end? > > The answer is "that depends on your mail environment". As far as I > know, normally it just works with the default settings, but if your > Postfix requires secure, you need to set that up: > https://docs.djangoproject.com/en/3.2/ref/settings/#std:setting-EMAIL_USE_T… > and possibly the following 3 or 4 variables if authentication is needed. > > There was a thread on this a couple of weeks ago. > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

3 years, 4 months

[MM3-users] Re: Turn off social logins?

by Jonas Meurer

Hi Stephen, Stephen J. Turnbull: > Jonas Meurer writes: > > > But still, adding so-called "social" authenticators to the mailman3 > > django application will expose users to the risk of clicking on > > those ID provider buttons. > > I can see this point for Debian users. I suspect Ubuntu users are > somewhat less paranoid. :-) Don't know ;) I don't think that Debian users are particularly paranoid. But I consider it a great value of Debian that privacy is considered as something important. > Since that risk holds even for sites that enable explicitly (assuming > we adopt the same policy) I will take a look at making that risk hard > to realize (more distance from anything else clickable, smaller > buttons with visible and accurate boundaries. Mh, I didn't mean that people click on the buttons by accident. But that they will do so, because it is so convenient (which it is, admittedly). But in my eyes, software developers, package maintainers and system administrators all should do their best to protect their users from privacy and/or security threats. Probably we have a different opinion on whether providing usage of authentication providers which earn their money by collecting users' data exposes users to privacy threats. > > > Personally, I consider it a major privacy issue if one central > > instance (e.g. Facebook) is able to track on which platforms and > > services you authenticate. > > Sure, but that ship has pretty well sailed AFAICT. Most users use > unconfigured versions of IE (or Edge) and Safari, which means they're > subject to all manner of webbugs. My employer just asked me to stop > using Firefox because it's too pedantic for their website development > vendors. :-( GDPR enforcement seems to primarily be an arm of the EU > trade offensive against large American services (that's the > Economist's recent opinion, not mine), while Europe-based globals are > undoubtedly doing the same crap. I have a different opinion on this. You're correct that most users know next to nothing about protecting themselves against all kinds of threats on the internet. But to me this shows even more, that we as those who build the tools have a responsibility to protect our users against shooting themselves in their foot. And I don't agree that any ship has sailed here. If you look at the mainstream development, then you're probably right. But this should not serve as the only rule, no? It's well known that the big corporates make their money with users data. That's how they survive in capitalism. But that doesn't mean that we as independent and free software developers have to agree on the dangerous interfaces and standards those corporates develop (for their business!). Don't get me wrong: I like it that Mailman3 actually supports this wide range of authentication providers. There's probably good use cases for it. And if your users like it and you consider it a win for them, then by all means go on with providing the feature. But it's a different question what a good default and the criteria for choosing it are. > > > And to be honstes, I'm a bit irritated that those tracking features > > from big corporates like Facebook and Google, whose main business > > model is to aggregate as as much private data points as possible > > about as much users as possible are enabled by default in Mailman3 > > upstream. > > Your irritation is not our problem, though, since you can use Debian's > version, and as I mentioned earlier, as far as I know most of our > sites are happy to have social auth. I will be paying attention to > the list to see what others think. My irritation was mainly about what I wrote above: that the privacy concerns against authentication providers from big corporates seem to be of low priority when deciding about defaults in Mailman3 upstream. > > I agree with Torge that those social auth providers should be > > disabled per default. IMHO, A sane default would be to list them in > > the settings.py but have them commented out. > > "Those" social auth providers? Are there social auth providers who > provide what you consider acceptable privacy guarantees (a la > Duck-Duck-Go in search engines)? If so, we could make those higher in > the list/easier to use. To be honest, I've never evaluated the privacy implications of OpenID. From a first glance, it looks *way* better. But for sure, Github, Google, Facebook, Stackexchange and Twitter all rely on collecting users data for their business model. So it's probably not a good idea to share with them the information which platforms and services you authenticate against, *if* you care about privacy. Anyway, all that's just *my* opinion. But it's good to have this discussion :) Cheers, jonas

6 years, 2 months

[MM3-users] Re: Edit archived email? Is it possible?

by Prasanth Nair

Hi Mark, Many thanks for your help. I have managed to modify some of the messages after following your instructions. Kind regards, Prasanth On Wed, 29 Sept 2021 at 08:08, Prasanth Nair <prasanth.nair(a)linaro.org> wrote: > Hi Mark, > > Thank you for your help. Let me try this. > > Kind regards, > Prasanth > > On Tue, 28 Sept 2021 at 18:03, Mark Sapiro <mark(a)msapiro.net> wrote: > >> On 9/28/21 6:02 AM, Prasanth Nair wrote: >> > Hello, >> > >> > Does anyone know how to edit an archived email in Mailman3? It is >> possible >> > in Mailman2 ( >> > >> https://wiki.list.org/DOC/How%20do%20I%20edit%20the%20archives%20of%20a%20M… >> ). >> > Is there any option similar to this in Mailman3? >> >> >> In some sense the process is the same but possibly even simpler in >> Mailman 3. You need to modify the underlying data which requires >> appropriate access to the server. >> >> In Mailman 3, the data are in the hyperkitty_email table in Mailman's >> database. You can get the Message-ID hash from the URL to the archived >> message, e.g. for the message to which I'm replying, the URL is >> >> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… >> and the hash is 7PVJL23DLSWMXXL44ALQUO6VSYBRZ4OA. Then if you have >> access, you can do database queries like >> ``` >> SELECT subject, content FROM hyperkitty_email WHERE message_id_hash = >> '7PVJL23DLSWMXXL44ALQUO6VSYBRZ4OA'; >> ``` >> and >> ``` >> UPDATE hyperkitty_email SET content = '<new content>' WHERE >> message_id_hash = '7PVJL23DLSWMXXL44ALQUO6VSYBRZ4OA'; >> UPDATE hyperkitty_email SET subject = '<new subject>' WHERE >> message_id_hash = '7PVJL23DLSWMXXL44ALQUO6VSYBRZ4OA'; >> ``` >> If the message has attachments, they are in the hyperkitty_attachment >> table and can be referenced by `email_id` which is the corresponding >> `id` from the hyperkitty_email entry. >> >> You can also do similar things via `django-admin shell`. E.g. >> ``` >> $ django-admin shell >> Python ... >> (InteractiveConsole) >> >>> from hyperkitty.models.email import Email >> >>> x = >> Email.objects.filter(message_id_hash='7PVJL23DLSWMXXL44ALQUO6VSYBRZ4OA') >> >>> print(x[0].content) >> ``` >> but this requires some knowledge of HyperKitty's models and Django's ORM. >> >> -- >> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, >> San Francisco Bay Area, California better use your sense - B. Dylan >> _______________________________________________ >> Mailman-users mailing list -- mailman-users(a)mailman3.org >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> >

3 years, 6 months

[MM3-users] Re: Badsignature on uwsgi-error.log

by Helio Loureiro

Hi, I was wrong. It keeps happening. So the base_url didn't fix the issue. And the keys are the same in all configurations. Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro On Fri, 22 Mar 2024 at 09:37, Helio Loureiro <helio(a)loureiro.eng.br> wrote: > Hi Mark, > > Thanks for your response. I found what the issue was. > > I actually misspelled hyperkitty as hyPperkitty on the url. > > So the error on mailman-hyperkitty.cfg was: > > base_url: http://localhost:8000/hypperkitty/ > > I moved to: > > base_url: http://localhost:8000/hyperkitty/ > > and it was solved. > > Problem was the wrong link was getting a generic page from Django instead > of error. And it was failing to do the parsing probably, which was taking > wrongly as key mistake. > Best Regards, > Helio Loureiro > https://helio.loureiro.eng.br > https://github.com/helioloureiro > https://mastodon.social/@helioloureiro > > > On Thu, 21 Mar 2024 at 01:13, Mark Sapiro <mark(a)msapiro.net> wrote: > >> On 3/20/24 14:25, Helio Loureiro wrote: >> > Hi, >> > >> > I keep receiving a lot of errors like this: >> > >> > --- Logging error --- >> > Traceback (most recent call last): >> ... >> > >> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py", >> > line 55, in unsign >> > raise BadSignature('Signature "%s" does not match' % sig) >> > django.core.signing.BadSignature: Signature >> "F6V0Dr_SvBsx5-cY8vcvLXrX8tA" >> > does not match >> ... >> > >> > The api_key on mailman-hypperkitty.cfg, SECRET_KEY and >> MAILMAN_ARCHIVE_KEY >> > on setting.py are the same. >> >> >> Of those three, the api_key on mailman-hypperkitty.cfg and >> MAILMAN_ARCHIVE_KEY must match. SECRET_KEY is different. It is a Django >> thing and has nothing to do with the others. See >> https://docs.djangoproject.com/en/5.0/ref/settings/#secret-key >> >> If you changed it, that's the issue. If you know what it was previously, >> you could set the previous value in SECRET_KEY_FALLBACKS, see >> https://docs.djangoproject.com/en/5.0/ref/settings/#secret-key-fallbacks >> >> -- >> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, >> San Francisco Bay Area, California better use your sense - B. Dylan >> >> _______________________________________________ >> Mailman-users mailing list -- mailman-users(a)mailman3.org >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> Archived at: >> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… >> >> This message sent to helio(a)loureiro.eng.br >> >

1 year, 1 month

[MM3-users] Mailman 3 questions (purge archives, list metrics, and auto discard held messages)

by Stephen J. Turnbull

bob B via Mailman-users writes: > --Just wanted to check that there is still no way for mailman3 to > automatically purge archived emails after a set amount of time? Pedantically, Mailman itself can't do that, and never will be able to. That's up to the archiver in use. I assume you mean HyperKitty. > it looks like you have to export the database, purge the messages, > then re-import, is this still the only/best way to purge old > archived messages? For HyperKitty, this is still true. This is a pretty hairy operation, and infrequently requested. A bug could be extremely destructive. I expect it will be a while before it is implemented and part of the standard distribution. I guess there may be potential applications where there's something like a max 90 days document retention policy, so there's some room for it. Of course, a volunteer might speed that up. (Money is unlikely to do so, the currently active core developers don't have a lot of time or appetite for consulting.) > --I noticed on the list info page you have metrics, which is > straightforward except for "volume" My volume number is not > increasing? What is that counting exactly? Volume number refers to the digest volume. Each digest mail distributed constitutes a numbered "issue". Every once in a while we increment the volume number and reset the issue to 1 so that the issue numbers don't get huge. The options for reset frequency are Daily, Weekly, Quarterly, Monthly, Yearly. If it's one of the longer ones, you're not going to see increments for a long time. And if you have digests disabled, you won't see it increment at all. > --Is there any way to auto discard held messages after x amount of > time? I assume not but had a user ask if that was an option. There currently is no such option. I'm not sure it's a great idea because if x is large, it's not very useful, and if x is small, moderators will go on vacation or get busy at work, and a bunch of valuable messages will be auto-discarded. If you're not going to be reasonably prompt about moderation, you may as well set the list to discard, not hold. So I'm not inclined to work on this feature myself. I wouldn't oppose another core developer who wanted to add it, and I might even apply a 3rd party patch if it came with a plausible story that I'm at least occasionally wrong about the risk/reward balance I expect.

2 years, 8 months

[MM3-users] Re: mailman3-web and the django middleware problem

by Thomas Stein

Am 2024-04-26 00:52, schrieb Mark Sapiro: > On 4/25/24 14:01, Thomas Stein wrote: >> Am 2024-04-25 22:03, schrieb Mark Sapiro: >>> >>> I don't know how things are arranged in the Debian/Ubuntu package, >>> but you may have a settings.py which at the end does something like >>> ``` >>> try: >>> from settings_local import * >>> except ImportError: >>> pass >>> ``` >>> If that is the case, adding the above snippet to the end of >>> settings_local.py should work. >> >> Yes, that's the case. I added the snippet in question to >> settings_local.py but this leads to another error: >> >> File "/usr/share/mailman3-web/settings.py", line 503, in <module> >> from settings_local import * >> File "/usr/share/mailman3-web/settings_local.py", line 194, in >> <module> >> x = list(MIDDLEWARE) >> ^^^^^^^^^^ >> NameError: name 'MIDDLEWARE' is not defined >> dpkg: error processing package mailman3-web (--configure): > > Yes. I understand. In your case, MIDDLEWARE is only defined in > settings.py and not in settings_local.py. As I understand it originally > you added > ``` > x = list(MIDDLEWARE) > x.insert(0, 'allauth.account.middleware.AccountMiddleware') > MIDDLEWARE = x > ``` > to settings.py and you say it didn't help. I don't know why that > wouldn't work as long as there is no setting for MIDDLEWARE in > settings_local.py and you added that after the MIDDLEWARE setting in > settings.py. > When you say it didn't help, do you mean you still get the > > django.core.exceptions.ImproperlyConfigured: > allauth.account.middleware.AccountMiddleware must be added to > settings.MIDDLEWARE > > message or something else? I note that the original post in this thread > refers to /etc/mailman3/mailman-web.py which is presumably where the > package ultimately puts the Django settings, and I don't know what > `dpkg --configure` is doing. > > One thing you could try is in settings_local.py, put > ``` > from settings import MIDDLEWARE > x = list(MIDDLEWARE) > x.insert(0, 'allauth.account.middleware.AccountMiddleware') > MIDDLEWARE = x > ``` > to work around the NameError. After that I get: File "/usr/share/mailman3-web/urls.py", line 20, in <module> from django.conf.urls import include, url ImportError: cannot import name 'url' from 'django.conf.urls' (/usr/lib/python3/dist-packages/django/conf/urls/__init__.py) dpkg: error processing package mailman3-web (--configure): installed mailman3-web package post-installation script subprocess returned error exit status 1 Errors were encountered while processing: I guess I'll wait for the new package for now. thanks again! t.

11 months, 4 weeks

[MM3-users] Re: A little stuck with installation of MM3 - ModuleNotFoundError: No module named 'flufl.lock'

by Odhiambo Washington

On Thu, 30 Jul 2020 at 22:02, Mark Sapiro <mark(a)msapiro.net> wrote: > On 7/30/20 11:24 AM, Odhiambo Washington wrote: > > > This works - I mean changes the error message to > > > > [Thu Jul 30 18:11:50.269667 2020] [wsgi:error] [pid 7627] [remote > > 197.232.81.246:63962] > > > /opt/mailman/mm/venv/lib/python3.7/site-packages/readme_renderer-26.0-py3.7.egg/readme_renderer/markdown.py:38: > > UserWarning: Markdown renderers are not available. Install > > 'readme_renderer[md]' to enable Markdown rendering. > > [Thu Jul 30 18:11:50.269700 2020] [wsgi:error] [pid 7627] [remote > > 197.232.81.246:63962] warnings.warn(_EXTRA_WARNING) > > > I'm not sure why you are getting this warning, and it's non-fatal, but > you can get rid of it by > > pip install cmarkgfm > > in your virtualenv. > (venv) [mailman3@gw ~]$ pip install cmarkgfm Requirement already satisfied: cmarkgfm in ./mm/venv/lib/python3.7/site-packages/cmarkgfm-0.4.2-py3.7-freebsd-12.1-RELEASE-p7-amd64.egg (0.4.2) Requirement already satisfied: cffi>=1.0.0 in ./mm/venv/lib/python3.7/site-packages/cffi-1.14.0-py3.7-freebsd-12.1-RELEASE-p7-amd64.egg (from cmarkgfm) (1.14.0) Requirement already satisfied: pycparser in ./mm/venv/lib/python3.7/site-packages/pycparser-2.20-py3.7.egg (from cffi>=1.0.0->cmarkgfm) (2.20) (venv) [mailman3@gw ~]$ > > [Thu Jul 30 21:11:50.573233 2020] [authz_core:error] [pid 10846] [client > > 197.232.81.246:63962] AH01630: client denied by server configuration: > > /var/spool/mailman-web, referer: > > http://mm3-lists.my.server/mm3/mailman3/lists/ > > > > There is NO /var/spool/mailman-web anywhere. It has not been created. > > You mentioned that it should be created automatically, IIRC. > > > No. I think I told you that the references to > /var/spool/mailman-web/static in your Apache config should be changed to > /opt/mailman/mm/static and that directory would be created/populated by > mailman-post-update. > > I think you still have > > Alias /static "/var/spool/mailman-web/static" > > in your Apache config and it needs to be > > Alias /static "/opt/mailman/mm/static" > > That may be my fault as when I recently edited > < > https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=At… > >, > I missed that line. After changing the Alias line and running the mailman-post-update, all errors are gone and I have a web UI. Yayi!!! I have been able to create a domain, a list for testing. Now I just need to go through the analysis of what I have been doing so that I can clean up my notes. Then I can come back on migration issues. A BIG THANK YOU for all the support up to this point. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

4 years, 8 months

[MM3-users] mailman procesing small amount of mail, then stop process them like 30 seconds, and then loop back

by Stephen J. Turnbull

mathias.bedu(a)epitech.eu writes: > I am currently trying to optimize my mailman installation, which is > working. The only issue is that i see that mailman seems to process > a few number of messages (around 50 each times), then stopping, and > then processing few next messages. What makes you think that this is Mailman stopping? Mailman's default behavior is to send messages to the MTA as fast as it will accept them, and let the MTA take care of any throttling or optimization. Mailman even tries to parallelize this process. So if you're seeing delays I believe that is more likely to be the MTA delaying acceptance rather than Mailman. The other possibility is lists that have DMARC mitigation on only for p=reject domains. Then there may be delays due to DNS lookups to check for the From domain's DMARC policy. > I precise that the machine itself is largely more powerful than > what mailman is using. This is probably true, but CPU and memory are not usually what limits mail delivery. Aside from MTA and DNS delays, and very underpowered hosts, there's no reason why Mailman would be be stopping for more than a second or two even if some of the "be nice to the MTA" parameters below are given restrictive values. > So my question is : could you give me some leads for potentials > parameters or stuff like this that i could modify to optimize the > installation ? You might look at these parameters in your configuration file, or in the definition file schema.cfg (do not change this file because it is overwritten by upgrades). The defaults are set pretty much as fast as possible. # Ceiling on the number of recipients that can be specified in a single SMTP # transaction. Set to 0 to submit the entire recipient list in one # transaction. max_recipients: 500 # Ceiling on the number of SMTP sessions to perform on a single socket # connection. Some MTAs have limits. Set this to 0 to do as many as we like # (i.e. your MTA has no limits). Set this to some number great than 0 and # Mailman will close the SMTP connection and re-open it after this number of # consecutive sessions. max_sessions_per_connection: 0 # Maximum number of simultaneous subthreads that will be used for SMTP # delivery. After the recipients list is chunked according to max_recipients, # each chunk is handed off to the SMTP server by a separate such thread. If # your Python interpreter was not built for threads, this feature is disabled. # You can explicitly disable it in all cases by setting max_delivery_threads # to 0. max_delivery_threads: 0

4 years, 7 months

[MM3-users] Re: Database running VERY hot

by Darren Smith

Well, we haven't tackled the issue if importing old archives yet - we're trying to stabilize things first, trying to get the 504 gateway timeouts to stop happening. Once that happens, we have over 40 million old emails to import. (some of the emails date back to 1987 I think.) We'll see how it goes. :) -Darren On Fri, Mar 9, 2018 at 10:20 AM, Abhilash Raj <maxking(a)asynchronous.in> wrote: > On Fri, Mar 9, 2018, at 9:11 AM, Darren Smith wrote: > > Abhilash, > > Thanks for the response. I'm going over to lists.fedoraproject.org to see > if they have done any fine tuning. > > Our installation is at https://lists.rootsweb.ancestry.com, if you're > interested in seeing an installation with over 32,000 mailing lists. As I > mentioned, it was big enough that we have disabled the browse experience in > Postorius and are hosting our own pages for browsing to a new list. > > > Please keep us posted if you have other unique challenges for operating > this high number of lists, other than this particular one. We'd love to be > able to support high number of lists efficiently :) > > If we get time to work on this we will certainly create a pull request. :) > > > On Thu, Mar 8, 2018 at 11:49 AM, Abhilash Raj <maxking(a)asynchronous.in> > wrote: > > Hi Darren, > > > On 3/8/18 9:43 AM, Darren Smith wrote: > > Hello all, > > We had an old installation of Mailman 2.1 across three servers with over > 32,000 mailing lists on them. Due to circumstances beyond my control, we > had to shut them down and quickly set up Mailman 3 on a new server. > > We have Mailman3 set up on a single server this time, as we were having > problems with a 3-server setup and Hyperkitty and archives. Long story > short, it is up and running, and our users are starting to come back. > > However, we do not have very many users hitting Postorius and Hyperkitty > yet, and we are seeing a distressing number of 504 gateway timeouts. > > We noticed that there is a significant number of queries to the API > whenever hitting Postorius's front page to load the mailing lists, and then > on every load. It looks as if it makes a single call to the API to get > information about a single list, and does this 10 times - once for each > list - to render the page. This gets much worse when users set the view to > 200 per page, as they are inclined to do with 32,000 mailing lists... > > What we've done, actually, is to override the front page and point the > users to a different set of static pages where we have all of our lists > categorized and searchable, and when they find the list they are looking > for, we have a link back to postorius. It has taken a lot of the stress off > of the server. > > However, I still see where the database at times can run VERY hot. > > We have set up this server to run with PostgreSQL. When I do a ps aux | > grep postgres, I find a lot of processes that have been spawned to take > care of mailman - a dozen or so. However, one of the threads is taking up > about 85% of the load, another thread about 14%, and little dribbles here > and there from the rest of the threads. > > Here are the questions: > > 1) Has anyone run into problems like these? I would think that maybe this > is an issue having so many mailing lists, but honestly there aren't that > many people using the system yet. It is going to ramp up as the users > realize that we are live again, but we aren't getting that much traffic. > > > I can say that in general, Postorius doesn't really optimize REST API > queries and so loading a single page can sometimes make things bad. > > Currently, the largest installation of Mailman 3 that we know of runs at > lists.fedoraproejct.org and the administrators of the list would know how > well does our Web UI scale. > > However, I don't think we have anyone actively working on trying to fix > this particular issue. Pull Requests and fixes are welcome :) > > > 2) Is there any configuration that I can set, especially with regards to > the database, so that it can be used more efficiently and not throttle > through one or two threads? > > > There isn't any configuration that would change how databases are used in > Mailman Core. Django *may* have some, but I don't know about any of them of > the top of my head. > > thanks, > Abhilash > > > -- > Abhilash Raj > maxking(a)asynchronous.in > > >

7 years, 1 month

[MM3-users] Re: Hyperkitty not receaving messages / empty archive [Django] ERROR (EXTERNAL IP): Internal Server Error: /hyperkitty/api/mailman/urls

by Helge Baumgart

Hello Mark, THANK You for identifying MY typos.... how embarrassing. :-) Now ist running like a charm. It's always the same... These Variables are setting the internal Values via the docker-compose.yml. If they do not match, the internal variables are not set. Therefor my typo H*A*YPERKITTY_API_KEY =! HYPERKITTY_API_KEY caused, that the mailman-web section of the docker-compose.yml had NO Hyperkitty API- KEY. I suspected something like that, that's why i set the key via the .env variabels to ensure identical values in all sections oft the docker.compose.yml. I didn't check the name of the variables... Thx again! Best wishes Helge Ps.: That docker-compose.yml with the settings_locale.py and mailman_extra.cfg basically setts a complete deployable environment INCLUDING the MTA, which is independent of the host... Am 22.01.2025 um 06:36 schrieb Mark Sapiro: > On 1/21/25 05:44, Helge Baumgart wrote: > >> Hyperkitty-Api-Key in the docker-compose.yml is set via .env >> Variabel, so it should be identical. > > > I don't know how this works in the Docker image, but ... > >> Traceback (most recent call last): >> File >> "/usr/lib/python3.11/site-packages/django/core/handlers/exception.py", >> line 56, in inner >> response = get_response(request) >> File >> "/usr/lib/python3.11/site-packages/django/core/handlers/base.py", >> line 197, in _get_response >> response = wrapped_callback(request, *callback_args, >> **callback_kwargs) >> File >> "/usr/lib/python3.11/site-packages/hyperkitty/views/mailman.py", line >> 109, in _decorator >> not hmac.compare_digest( >> >> Exception Type: TypeError at /hyperkitty/api/mailman/urls >> Exception Value: unsupported operand types(s) or combination of >> types: 'str' and 'NoneType' > > The above code is doing > > auth = authorization.split() > if (len(auth) != 2 or auth[0] != 'Token' or > not hmac.compare_digest( > auth[1], settings.MAILMAN_ARCHIVER_KEY)): > > > and the exception says either auth[0] is None or auth[1] or > settings.MAILMAN_ARCHIVER_KEY is None. > > But > >> META: > ... >> HTTP_AUTHORIZATION = 'Token (value)' > and > >> Settings: >> Using settings module settings > ... >> MAILMAN_ARCHIVER_KEY = '********************' > > So The Authorization header is the appropriate `Token value` which > should result in auth[0] being 'Token', auth[1] being the value and > MAILMAN_ARCHIVER_KEY is set > >> That is my docker-compose.yml: > ... >> mailman-web: > ... >> - HAYPERKITTY_API_KEY=${MM3_HYPERKITTY_API_KEY} >> #someapikey_longAndTrustworty > > > I don't really understand why the above typo (HAYPERKITTY_API_KEY, not > HYPERKITTY_API_KEY) would cause this error, but it needs to be fixed. >

3 months

[MM3-users] base64 re-encoding and line endings

by Stephen J. Turnbull

Quoted parts are reordered. Executive summary: Mail occasionally arrives at Mailman, Mailman reassembles it with a git patch using CRLF line endings encoded as BASE64, and git can't apply the patch without "fixing" the line endings. I don't think this is a Mailman problem. Analysis below. It would be very helpful to have a copy of such a mail (verbatim as delivered to Mailman's MTA would be best, but at least all Content-Type, Content-Disposition, and Content-Transfer-Encoding header fields). David Bremner writes: > Occasionaly people send it with Content-Transfer-Encoding 8bit. > I guess with 8bit transfer encoding there probably isn't a clear > distinction between the line-endings on the wire, and the intended > line-endings for delivery, It's clear from RFC 2045: line endings on the wire must be CRLF.[1] What they are at either end is ambiguous because it's up to the MTAs, *and will be different on different receiving hosts if the native conventions of the hosts are different.* This means that as far as Mailman is concerned those line endings are *supposed* to be CRLF and nothing else. (Not clear what should happen if 8bit conflicts with the MIME content type or the content of the patch body, see below, but that's the sending system's issue.) I don't think this is a Mailman 3 problem (except possibly in the archive). The canonical [content transfer] encoding model[2] implies that since Mailman receives the message "off the wire" via LMTP (rather than as a system file), if the MIME content type is text/*, (1) the line breaks Mailman sees in the message body are always CRLF after content transfer decoding, (2) the line breaks Mailman emits to its MTA are always CRLF "inside" the BASE64, and (3) the receiving mail system (usually the MTA) is responsible for decoding the BASE64 and then transforming the CRLFs to the local convention. (1) and (2) seem to be consistent with your report, and would indicate that Mailman is conforming to the relevant standards. I wonder if the sending systems are using a binary MIME content type such as application/patch? That would be incompatible with the 8bit content transfer encoding, and might confuse the systems on either end as to what encoding should be done. (I guess it might also confuse Mailman, I'll have to check, but it doesn't point *only* at Mailman.) In that case, the MTA will decode the BASE64 but *not* transform the CRLF to LF, which is consistent with the observed behavior as well. Of course it's *possible* that Mailman is receiving LF and transforming that to CRLF, but that implies sender behavior inconsistent with the 8bit content transfer encoding, there's nothing in the email processing model that suggests Mailman might do that under the reported conditions (in particular, the 8bit content transfer encoding is the identity, implying that even if Mailman "notices" that it is 8bit, it should do nothing about it except maybe complain about NUL, solo CR or LF, and lines longer than 998 bytes), and I think it would show up in other contexts with similarly distressing consequences if Mailman were doing it. > I have a local workaround involving re-formatting the patches, but > I'd really like anyone to be able to apply the patches from the > list archive. If your archive is hosted on an LF system and the content-type applied to the patch is text/*, it "should" just work and the archived patches being impossible to apply "as is" is probably a Mailman issue. > This seems somewhat related to > https://gitlab.com/mailman/mailman/-/issues/269 Not the same. The problem there is that for some reason Mailman is altering the text (removing trailing whitespace). That's unambiguously wrong regardless of MIME content type or content transfer encoding, and damaging in the case of format=flowed. I'm going to have to revisit that issue. > I guess with 8bit transfer encoding there probably isn't a clear > distinction between the line-endings on the wire, and the intended > line-endings for delivery, but it could be inferred from the > "X-Mailer: git-send-email" header. No, *you* as receiver can make that inference, but *we* as a middleman can't. git is used on both LF and CRLF platforms. If we're receiving CRLF, we have to send it back out. This implies that if you want to, you can write a plug-in handler to make this transformation inside Mailman, and insert it into the processing pipeline before the archive handler. Footnotes: [1] RFC 2045, Section 2.8. 8bit Data "8bit data" refers to data that is all represented as relatively short lines with 998 octets or less between CRLF line separation sequences [RFC-821]), but octets with decimal values greater than 127 may be used. As with "7bit data" CR and LF octets only occur as part of CRLF line separation sequences and no NULs are allowed. That is, Content-Transfer-Encoding: 8bit is not binary, does not admit non-text, and requires that line-endings be CRLF. [2] RFC 2049, Section 4. Canonical Encoding Model

4 years, 8 months

[MM3-users] Re: username

by Stephen J. Turnbull

bryan.kartzman(a)yu.edu writes: > will do this. The university does use SSO for many applications and > I was asked by upper management of the IT department to try and > implement it here. I'm still trying to understand your reply to my > SSO question from last week (my SSO knowledge is limited and I'll > have to read up on Shibboleth, SAML, as well as Django) so for now > I'm thinking to not use SSO If they want to use it, pass the buck to them. I'm happy to help with the Mailman side (on a time-available basis or as a paid consultant -- time available is probably a much better deal, though!) But I can't know what the system in use is, so maybe you should get them to write an outline of the system (what software, what requirements, what ID-related attributes are available). > then transition to SSO later. I don't think that will be a problem technically, as long as the user ids are the same for the same people. There will be UX issues though in the configuration I use. I think people will accept them quickly, but the UX will be different, which makes folks nervous if you don't warn them and explain why it's OK. > If you have any pointers as to > where to read up on SSO, SAML, Shibboleth and also on Django please > point me in the right direction. For django, search "site:djangoproject.com REMOTE_USER". For shibboleth, the relevant documentation is on the atlassian site. Here's the main documentation for the "service provider" component (this is the part that Postorius would be using indirectly): https://shibboleth.atlassian.net/wiki/spaces/SP3/overview search there for "REMOTE_USER" for the most relevant documentation. The SAML protocol is above our pay grade. :-) There's an introduction to the SAML architecture linked from the service provider documentation. That's way more than you need for this task, I think, so just skim it. Another guy did the Shibboleth config, so I'm a little fuzzy on this, but again I think that's mostly IT/NOC responsibility. (IIRC the main thing was getting the ID provider credentials and our host certs from the NOC, you can't generate those yourself!) The parts that hook into mailman are a list of translations of ID attributes to Apache variables, some email addresses for error reporting, and in Apache some translations of Apache variables to HTTP headers. But you need to make sure it's SAML. I'm no expert on this stuff, but it seems like there are a bunch of these protocols, like OpenID and OAuth2 out there. Django knows how to do those, too, but we need to know which one! Steve

1 year, 10 months

[MM3-users] Re: moving list from mm3 -> mm3

by Mark Sapiro

On 12/8/23 08:01, Jens Günther wrote: > > This is a nice collection of the configuration of a certain list. Does > anybody know how I would get this data back to the newly created list in > the second working mm3-instance? Shouldn't it be possible via POST > command? I thought of something like: > > curl -u restadmin:restpass -d @short.domain -H "Content-Type: > application/json" localhost:8001/3.1/lists/&short.&domain/config > > But it returns {"title": "405 Method Not Allowed"} and I don't know > where to look in the docs, since it shows almost always a way through > mailman shell. You need a PUT command, not POST. Try the above with `-X PUT`. > ## Members, moderators, owners > > To export all members I normally run within mm-core-container as user > mailman the following command: > > mailman members &short.&domain > &short.txt > > To fill them back in, it can be done in Postorius (or is there a way > through API?). You can use Postorius -> Mass operations -> Mass subscribe to add the members, but you can't do that for owners and moderators. Also you lose information about the members such as Delivery status, Delivery mode and other user settings. > What I would love to know is how to get out Owners of the corresponding > lists and reimport them in the new one (which will be the same at the > end :-D but different instance). You can use the --role option with `mailman members` to get owners and moderators, but short of a custom script, you have to add them manually. > ## registered users > > This part is somehow tricky, since I have no clue how to do it. Some of > the mailinglist-admins don't want to bother their list-owners again with > registering newly on the other system. Is there a way to export all > users (I think I found them in the DB in the table auth_users, somehow > in combination with account_emailaddress) and get them over to the new > instance (actually I just need part of them, but that's another question > that I might solve lateron on my own). > > Maybe the answer is already here: > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/… > -> but can somebody help to get this working (import/export)? That is not the answer. Those are Mailman core users, not Django (web login) users. You need to get the relevant entries from the auth_users and account_emailaddress tables and add them to the new DB. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 4 months

[MM3-users] Re: mailman 2.1 or 3.0 (or another MLM)

by David Newman

+1 to Stephen's comments. I too installed and ran MM2 lists for many years, and found the MM3 setup to be challenging. It took me four or five full installations on test VMs before I was comfortable going into production, and even then I ran into several hiccups, well documented on this list. A few things may help: - don't install from packages on Debian/Devuan. The package version isn't current and has several issues. For now, use Python/pip installation instead. (I'd much prefer to use the distro package any day, and hope the need for this bullet point goes away soon.) - start from the venv and web setup docs: https://docs.mailman3.org/en/latest/install/virtualenv.html https://docs.mailman3.org/en/latest/config-web.html There's some overlap between these but they're the closest thing to an installation manual we have. - Document every step. For example, I documented around six different config changes needed for MariaDB, which my server runs instead of PostgreSQL. You may have similar experiences with Apache instead of Nginx. - For web and email setup, note that the above URLs assume MM3's web components (Django, Postorious, Hyperkitty) are accessible through "/" in Apache's DocumentRoot directive. If you have other web services on the same server (e.g., if you're also running Roundcube), you'll need to tweak your Apache config. - AFAIK there's currently no sysadmin manual for the care and feeding of MM3 and friends once it's up and running. I have some installation and config notes from my setup, and I'd be glad to share them, but they don't cover list management tasks. Hope all this is useful. MM3 is bigger and more complex and has more stuff to break. Once it's running, though, it's been pretty solid. /drn On 1/24/22 4:54 PM, Stephen Daniel wrote: > Nate -- > > A few months ago I, a total novice with Mailman, set out to install > Mailman3 on a Debian server. > I used these instructions: DOC/Howto_Install_Mailman3_On_Debian10 > <https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10> > > It took me several tries, but with some help from people on the list it is > now up and running. It was also a *much *bigger investment of my time than > I expected. > > I cannot say I completely understood what I was doing every step of the > way, and some of the steps are like: go edit the config file so that it is > right, which can be daunting. However, even for someone who's never > touched mailman before, it can be done. > > Best of luck > > -- Stephen > > On Mon, Jan 24, 2022 at 7:19 PM nate <mailman-list(a)linuxpowered.net> wrote: > >> Hello mailman users out there - >> >> I have been a casual user of mailman for 21 years now I think it is. >> Most of my >> lists were super small (my only current lists have no more than 8 >> members each), >> super low volume. It's "just worked" over the years, no issues. I have >> been >> using Linux since 1996, not a newbie here. 21 years ago I deployed >> mailman as >> part of our corporate email and then there were a lot more users. Since >> that >> time I just use mailman on my personal server. >> >> I did a apt-get dist-upgrade on my system and was surprised to see >> mailman had >> vanished. Wasn't expecting that.. not a big deal, or so I thought. >> >> I am running Devuan 4, which is based on Debian. Investigating further >> it seems >> Debian at some point dropped Mailman 2.1 and started including Mailman >> 3. OK, >> not too bad I thought.. >> >> I was/am prepared to start my mailing lists from scratch again, manually >> copy/paste over settings/users/etc. Don't care about archives etc. I >> setup mailman >> 2.1 on another Devuan 3.x system and copied my mailman configs over to >> it, and >> can view them fine via the mailman cgi scripts through Apache, I was >> planning >> on just copy/pasting list settings between the two systems for each >> list. >> >> Roughly 10-15mins into trying to get mailman3 off the ground I just find >> myself >> overwhelmed. That doesn't happen often. I'm honestly completely shocked >> at >> how it seems the complexity of the system has increased by orders of >> magnitude. >> >> I have been browsing around >> https://docs.mailman3.org/en/latest/pre-installation-guide.html >> (and other links on that documentation site) >> >> But as much information as there is, to me there really isn't any >> information >> there. If that makes sense. I'm sure it doesn't make sense so let me >> give an >> example. >> >> On this page: >> https://docs.mailman3.org/en/latest/config-web.html >> >> I assume that would lead me to how do I configure the web interface so I >> can do stuff with it. >> >> I setup an admin account, then it goes into scheduled tasks(which appear >> to >> be handled by the Debian package already but don't know what these tasks >> would have to do with a web interface). I tried sending a test email but >> that failed, I looked at the setting up email portion but it does not >> give any indication where that config would be stored, a recursive grep >> on /etc for EMAIL_BACKEND turns up nothing (there is a /etc/mailman3 >> directory with some files in there). >> >> Then it goes into other things like task queues, full text searching, >> here I am just looking for info on where this web interface is, where >> is the config, what port does it listen on? Is it a cgi like mailman2? >> It seems like it could be a standalone service. >> >> Then it shows a bunch of other settings, not indicating what file these >> are stored in(or what directory the file is in), etc etc etc. Debian's >> own documentation is severely lacking in this department as well, >> normally >> I am used to going to /usr/share/doc/<package>/ and there is some useful >> stuff. There is stuff for mailman3 but it's minimal at best. >> >> So I'm thinking mailman3 is not the package for me. I've really >> appreciated >> mailman 2.1 (and earlier versions) as I said they always just worked, >> never >> had to think about them. I'm sure a lot of good hard work has been put >> into 3, I just don't recall the last time I looked at a project and just >> honestly didn't even know where to begin which surprised me so much >> given >> my 20+ year history of mailman as a user and as an admin. >> >> Is there some document(s) out there that would be more beneficial to >> me in setting up mailman3? I see that mailman 2 looks like it's still >> being maintained, I assume mailman3 is the result of python3 code and >> mailman2 perhaps uses python 2.7? I am unsure on the ability to run >> mailman2 on a modern Debian-based distro since of course Python 2.7 >> is pretty obsolete at this point (or maybe Mailman 2.1 doesn't use >> python I don't recall). I could try to install the older mailman 2.1 >> package on Devuan 4 and see what happens I don't know if it would >> be that simple or not. >> >> Or is there a better MLM for simple use cases? I used Majordomo back in >> the 90s and was happy to get onto Mailman from that. Something similar >> to >> Mailman 2.1 in features/functionality. Worst case I could just use >> aliases >> in my postfix config at the end of the day that is all that is really >> needed, though I do like some level of access control on who can email >> the lists(even if nobody outside the list knows the list exists), I've >> had some of these lists for 2 decades, old habits die hard right? >> >> I did some searches for mailing list software on the Debian package >> archives and saw a few options, nothing really stood out at me as >> something super interesting yet though. >> >> I don't mean to come across as harsh or complaining or anything I'm >> just here in shock kind of. Thinking about what the best solution for >> me would be. My mailing lists have been broken for a couple of weeks >> but that's really not a big deal. >> >> thanks >> >> nate >> _______________________________________________ >> Mailman-users mailing list -- mailman-users(a)mailman3.org >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

3 years, 2 months

[MM3-users] re-sending confirmation mails (was Re: Re: pending subscriptions lost?)

by IEM Network Operation Center (IOhannes m zmölnig)

On 3/14/22 18:36, Mark Sapiro wrote: > On 3/14/22 09:18, IEM Network Operation Center (IOhannes m zmölnig) wrote: >> >> this is with "Postorius Version 1.2.4" (as installed by the mailman3 >> Debian package). > > The Debian packages are OK for people who want to install things via a > package manager, but they are quite out of date. i know. the problem is, that right now i have a running (Debian-based, thus outdated) mailman3 installation, and I still haven't found a proper strategy to upgrade and keep staying up-to-date. so for the time being i'm stuck with what i have. > > Something like the following in `mailman shell` can do it. thanks a lot. afaiu, this snippet allows me to purge outdated pending subscriptions. modifying it slightly, also allows me to just list the pending subscriptions along with their tokens and print them out. is it still possible to just use these tokens to confirm the pending subscription (even though it is long overdue)? if so, i could just send out a reminder to those 50 or so people who have not confirmed yet. while investigating this a bit, i also noticed that it seems that with mailman3 the only option to confirm a pending subscription is via email ("leaving the subject intact") - at least that is the only option that is listed in the confirmation mail. now i would like to massage this into what i think is a bit more user-friendly. first of all: most email confirmation systems allow me to either "reply by email" or "click this link". is there a way to do this for mailman3? that is: is there a (hidden) URL that would accept the token in order to confirm a subscription? also: the confirmation mail says that the subject must be kept intact. however, i noticed that the actual "From" address of the confirmation mail already contains the token. would it be enough if people just sent an email to this special email-address (that contains the token), regardless of the subject? i'm thinking of sending a rather personalized email that has a more attractive subject than "confirm flkdsajfsalkjflsajfsoisjdl". finally: what else is required to approve the confirmation? will the sender's email-address be taken into account? (e.g. if they subscribed "foo+mylist(a)example.com" but send the confirmation-reply from "foo(a)example.com", will that still work? sorry for so many questions. i *did* search the documentation, but simply haven't found anything that would help me. i've probably looked in the wrong places, but mailman3 is rather huge so it's easy to miss the imprtant parts. gmdsa IOhannes

3 years, 1 month

[MM3-users] Re: migrate MM3 list from an old server to a new one

by IEM Network Operation Center (IOhannes m zmölnig)

On 8/7/24 12:55, Odhiambo Washington via Mailman-users wrote: > On Wed, Aug 7, 2024 at 1:05 PM IEM Network Operation Center (IOhannes m > zmölnig) <noc(a)iem.at> wrote: >> - ./data: empty > > With a Postfic MTA situation, that would have had some files. So you don't > use Postfix. > >> - ./lists: empty directories named after each list (i think these are >> used by the MTA to determine whether it should hand over a mail to the >> mm3-ltmp) > > True, at least according to Exim MTA. indeed, i'm using exim. i guess these directories are named like they are for backwards compatibility. if the sysadmin is supposed to ever do something with them, it would be nice if there was some (discoverable) documentation (like a README in Mailman's var; or the yet-to-be-written mm3-to-mm3 migration guide; even if those files/directories are mentioned in the Bible for installing Mailman, i don't think this can be found by a normal administrator (it's not reasonable to assume that one should look up each file/directory in Mailman's var to see if it is needed). > >> - ./locks: plenty of lock-files, most of them rather stale (as in: 3 >> years old) > > So MM3 on that host hasn't been running or? it *is* running. i do not know why it didn't cleanup. > - ./messages: a couple of old messages (i think all of them have been >> held for approval) >> - ./queue: dirtree without files >> - ./templates: more templates for the existing mailinglists, though they >> don't seem to be used (are these the fallbacks when I remove a template >> via the webinterface?) >> > > Yes. so can i just ignore them? > So out here we focus so much on the following as the Bible for installing > and running MM3: https://docs.mailman3.org/en/latest/install/virtualenv.html > probably. sorry for being obtuse. afaict, the documentation is written for people setting things up from scratch, where the paths are mostly setup automatically, and you only need to configure them manually for interoperability with other software (e.g. when using postfix as the MTA, the docs tell the admin to ensure that both mm3 and postfix communicate via the same paths). i guess i'm confused because it is unclear to me, which of these paths hold persistent data, and which are just for ephemeral things (like sockets). mgfasdr IOhannes ___________________________________________ Die Inhaltsfilterung von Mailman hat die folgenden MIME-Teile aus dieser Nachricht entfernt. Content-Type: application/pgp-keys Name: OpenPGP_0xB65019C47F7A36F8.asc

8 months, 2 weeks

[MM3-users] Re: How To Install Mailman 3 on Debian 10 (Complete Guide)

by Lars Schimmer

On 02/03/2021 18:31, Quanah Gibson-Mount wrote: > > > --On Tuesday, March 2, 2021 9:09 AM +0100 Lars Schimmer > <l.schimmer(a)cgv.tugraz.at> wrote: > >> On 01/03/2021 13:15, Brian Carpenter wrote: >>> On 3/1/21 3:10 AM, Lars Schimmer wrote: >>>> If it should be a gold standard, it should mention the default setup >>>> of a standard debian system (which is exim4, btw) and not assuming a >>>> special setup debian system. >>> >>> So why don't you write up your own how-to for your so-call ''standard >>> debian system". Remember this is a forum for Mailman 3, not a forum for >>> the users of a "standard debian system". So far your comments are not >>> helpful at all. >> >> Install debian >> >> apt-get install mailman3-full > > As someone who is stuck using the ancient version of mailman3 in debian, > I have to disagree. 99.9999% of the time I hit an issue, it turns out > it was fixed years ago, but because I use the debian maintained version, > I don't have those fixes. It's horrible. Intereting view for a software poackage taking a few years to get a way to migrate from mailman2 to mailman3. Or in other words: if a stable version is not useable/to old, the software is not production ready. Sure, bugs do happen, developement does happen. But for a production server you do not want to update every month just for the latest feature. If a software reaches stable, it gets a package which is able to run the lifetime of debian stable. Bugs will be iron out with debian security packages. All new features will not be production critical, mostly add new features. > Distribution packages tend to be useful for really basic things like > libraries. For constantly evolving projects where you need to be > current, they're more of a detriment. Nope. Vote against this view. You need a stable system to rely on and setup without hassle in a nonce. Not to check every package you installed for latest update and run updates every week with latest bugs, issues while upgrading a package. Debian stable has a reason to exist for admins with less time and a wish to run production ready software. Not latest bleeding edge testing software. > --Quanah > > -- > > Quanah Gibson-Mount > Product Architect > Symas Corporation > Packaged, certified, and supported LDAP solutions powered by OpenLDAP: > <http://www.symas.com> MfG, Lars Schimmer -- ------------------------------------------------------------- TU Graz, Institut für ComputerGraphik & WissensVisualisierung Tel: +43 316 873-5405 E-Mail: l.schimmer(a)cgv.tugraz.at Fax: +43 316 873-5402 PGP-Key-ID: 0x4A9B1723

4 years, 1 month

[MM3-users] Re: SMTPSenderRefused - howto enable STARTTLS for local mail drop

by Stefan Bauer

Unfortunately i need this, as the postfix is listening on all interfaces and we have the policy to enforce TLS only. Nothing special in the logs: Dec 07 08:14:14 2021 (12303) Using agent: <mailman.mta.bulk.BulkDelivery object at 0x7fbc29b1c4a8> Dec 07 08:14:14 2021 (12303) Connecting to localhost:25 Dec 07 08:14:14 2021 (12303) envsender: v-test-bounces@domain, recipients: ['recipient(a)own.domain'], size(msgtext): 5809 Dec 07 08:14:14 2021 (12303) <kcEE.aBI3BVc2RCi6kAxA4PJ0hg.ADrcCDrr1wE(a)mailsrv103.my.domain> smtp to v-test@domain for 1 recips, completed in 0.008081912994384766 seconds Dec 07 08:14:14 2021 (12303) <kcEE.aBI3BVc2RCi6kAxA4PJ0hg.ADrcCDrr1wE(a)mailsrv103.my.domain> post to v-test@domain from v-test@domain, 5336 bytes Dec 07 08:14:47 2021 (12297) Peer: ('127.0.0.1', 43210) Dec 07 08:14:47 2021 (12297) ('127.0.0.1', 43210) handling connection Dec 07 08:14:47 2021 (12297) b'220 ml01.my.domain GNU Mailman LMTP runner 2.0\r\n' Dec 07 08:14:47 2021 (12297) ('127.0.0.1', 43210) EOF received Dec 07 08:14:47 2021 (12297) Connection lost during _handle_client() Dec 07 08:14:47 2021 (12297) ('127.0.0.1', 43210) connection lost Dec 07 08:15:00 2021 (12297) Peer: ('127.0.0.1', 43516) Dec 07 08:15:00 2021 (12297) ('127.0.0.1', 43516) handling connection Is my mailman version maybe too old for this setting? I'm running GNU Mailman 3.1.1 (Between The Wheels) Thank you. Am Mo., 6. Dez. 2021 um 19:04 Uhr schrieb Mark Sapiro <mark(a)msapiro.net>: > On 12/6/21 2:51 AM, Stefan Bauer wrote: > > Need to bring this up again. Django now sends with STARTTLS, but mailman > > itself, does still only drop mails in cleartext. > > Dec 6 11:47:33 al01 postfix/lmtp[7598]: A6422600510: > > to=<remote-party@remote>, relay=127.0.0.1[127.0.0.1]:8024, delay=0.01, > > delays=0.01/0/0/0.01, dsn=2.0.0, status=sent (250 Ok) > > Dec 6 11:47:33 al01 postfix/qmgr[29156]: A6422600510: removed > > Dec 6 11:47:35 al01 postfix/smtpd[8092]: disconnect from localhost[::1] > > ehlo=1 mail=2 rcpt=2 data=2 commands=7 > > > > No starttls=1 at all :( > > > > Does the above settings work for anyone? > > > > etc/mailman3/mailman.cfg > > [mta] > > incoming: mailman.mta.postfix.LMTP > > outgoing: mailman.mta.deliver.deliver > > > > smtp_host: localhost > > smtp_port: 25 > > smtp_user: > > smtp_pass: > > smtp_verify_hostname: false > > smtp_verify_cert: false > > smtp_secure_mode: starttls > > > This should work. What do you see in Mailman's smtp.log if you add > ``` > [logging.smtp] > level: debug > ``` > to mailman.cfg. > > However, do you really need this. It will only affect delivery from > Mailman to Postfix via the loopback interface on the localhost. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

3 years, 4 months

[MM3-users] Re: Docker Support Sucks

by Allan Hansen

Hi Brian, You have been one of the good and generous people providing support. I hope i can repay a bit with this: These are one-line bash scripts I wrote after moving to MM3, so I could do things not yet in MM3, such as letting my moderators request a list’s members from me. I need one more script that will sync a text file to an existing list, deleting and adding as necessary. The scripts are for public use, reuse, misuse and overuse. I claim no rights and no responsibilities. mm3Members.bash #!/bin/bash # mm3Members # # Obtain a list of members of a given list # # Usage: # mm3Members.bash <listName> # # # Revisions: # Date Name Reason for change # 28-Mar-2020 Allan Hansen Initial # exec /opt/mailman/mm/venv/bin/mailman -C "/opt/mailman/mm/mailman.cfg" members $1 |grep -v -e "has no members"|sed -e 's/.*/\L&/‘ mm3ASddMembers.bash #!/bin/bash # mm3AddMembers # # Add members from a given list. # # Usage: # mm3AddMembers.bash <fileName> <listName> # <fileName>: text file with addresses to add. Same legal formats as in mass subscribe # # # Revisions: # Date Name Reason for change # 28-Mar-2020 Allan Hansen Initial # exec /opt/mailman/mm/venv/bin/mailman -C "/opt/mailman/mm/mailman.cfg" members --add $1 $2 mm3DeleteMembers.bash #!/bin/bash # mm3DeleteMembers # # Deletes members from a given list # <fileName> List of members to delete. Legal format the same a formats used for mass subscribes # # Usage: # mm3DeleteMembers.bash <fileName> <listName> # # Revisions: # # Date Name Reason for change # 28-Mar-2020 Allan Hansen Initial # exec /opt/mailman/mm/venv/bin/mailman -C "/opt/mailman/mm/mailman.cfg" members --delete $1 $2 mm3MemberCount.bash #!/bin/bash # mm3MemberCount # # Obtain count of the members of a given list # # Usage: # mm3MemberCount.bash <listName> # # Revisions: # Date Name Reason for change # 28-Mar-2020 Allan Hansen Initial # /home/hansen/scripts/mm3Members.bash $1|wc Yours, Allan Hansen hansen(a)rc.org > On Jul 21, 2020, at 18:31 , Brian Carpenter <brian_carpenter(a)emwd.com> wrote: > > On 7/21/20 8:55 PM, Mark Sapiro wrote: >> On 7/21/20 5:40 PM, Brian Carpenter wrote: >> >> Abhilash (@maxking) is one person and he has a $DayJob. We're all doing >> the best we can. > > Well, I still need support. > >> This is not a Docker issue per se. It is a consequence of upgrading >> Mailman core to 3.3.1+ and processing many old, stale, previously >> unprocessed bounces. See >> <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…> > > I am not worried about the bounce issues at this point. I am just trying to get a membership list exported for a client. > >> >>> So, I want to totally move away from the use of Docker (mainly because I have found it buggy and the support is simply lacking) and install Mailman 3 for this client from source. However I need to get an export of their list members. But of course, I can't. Rest API times out and I don't know how to apply the instructions I was given to fix it and I don't have days to way for one. So please help. >>> >>> I increased the gunicorn workers to 4, hoping that would allow me to do the CSV export. No. So was given these instructions: >>> >>> ### >>> The issues seems to be due to timeout not due to high load. You want to increase the timeout for gunicorn and you can do that by specifying a gunicorn.ini config file: >>> >>> [webservice] >>> configuration: /path/to/gunicorn.cfg >>> >>> and then in the gunicorn.cfg you can add >>> >>> [gunicorn] >>> timeout = 90 >>> graceful_timeout = 30 >>> ### >>> >>> I applied the above changes to /opt/mailman/core/mailman-extra.cfg and created the gunicorn.cfg file. Restarted the docker container (mailman-core) and no access to Postorius due to no REST api. >>> >>> After restarting docker-mailman, I can't access Postorius due to the following error: >>> >>> ERROR 2020-07-20 20:16:27,351 15 postorius Mailman REST API not available >> Is Mailman core running? If not, whats in core's mailman.log as to why not? > > It looks like but no error message in mailman.log. However I noted that the gunicorn process did not start. > > This is with the configuration line enabled in mailman-extra.cfg: > > # docker-compose restart mailman-core > Restarting mailman-core ... done > [root@nicu-net docker-mailman]# tail -f /opt/mailman/core/var/logs/mailman.log > Jul 22 01:25:17 2020 (22) in runner caught SIGTERM. Stopping. > Jul 22 01:25:17 2020 (25) out runner caught SIGTERM. Stopping. > Jul 22 01:25:17 2020 (25) out runner exiting. > Jul 22 01:25:17 2020 (21) command runner exiting. > Jul 22 01:25:17 2020 (22) in runner exiting. > Jul 22 01:25:17 2020 (24) nntp runner caught SIGTERM. Stopping. > Jul 22 01:25:17 2020 (24) nntp runner exiting. > [2020-07-22 01:25:19 +0000] [27] [INFO] Shutting down: Master > Jul 22 01:25:21 2020 (1) Master stopped > Jul 22 01:25:26 2020 (1) Master started > Jul 22 01:25:44 2020 (27) rest runner started. > Jul 22 01:25:44 2020 (26) pipeline runner started. > Jul 22 01:25:44 2020 (20) bounces runner started. > Jul 22 01:25:45 2020 (22) in runner started. > Jul 22 01:25:45 2020 (28) retry runner started. > Jul 22 01:25:46 2020 (29) virgin runner started. > Jul 22 01:25:46 2020 (25) out runner started. > Jul 22 01:25:46 2020 (23) lmtp runner started. > Jul 22 01:25:46 2020 (24) nntp runner started. > Jul 22 01:25:47 2020 (30) digest runner started. > Jul 22 01:25:47 2020 (19) archive runner started. > Jul 22 01:25:47 2020 (21) command runner started. > > This is with the configuration line disabled in mailman-extra.cfg: > > # docker-compose restart mailman-core > Restarting mailman-core ... done > [root@nicu-net docker-mailman]# tail -f /opt/mailman/core/var/logs/mailman.log > Jul 22 01:27:49 2020 (1) Master stopped > Jul 22 01:27:53 2020 (1) Master started > Jul 22 01:28:10 2020 (30) digest runner started. > Jul 22 01:28:10 2020 (29) virgin runner started. > Jul 22 01:28:10 2020 (24) nntp runner started. > Jul 22 01:28:10 2020 (25) out runner started. > Jul 22 01:28:10 2020 (22) in runner started. > Jul 22 01:28:11 2020 (28) retry runner started. > Jul 22 01:28:11 2020 (27) rest runner started. > [2020-07-22 01:28:11 +0000] [27] [INFO] Starting gunicorn 19.9.0 > [2020-07-22 01:28:11 +0000] [27] [INFO] Listening at: http://172.19.199.2:8001 (27) > [2020-07-22 01:28:11 +0000] [27] [INFO] Using worker: sync > [2020-07-22 01:28:11 +0000] [32] [INFO] Booting worker with pid: 32 > [2020-07-22 01:28:11 +0000] [33] [INFO] Booting worker with pid: 33 > [2020-07-22 01:28:11 +0000] [34] [INFO] Booting worker with pid: 34 > [2020-07-22 01:28:11 +0000] [35] [INFO] Booting worker with pid: 35 > Jul 22 01:28:11 2020 (19) archive runner started. > Jul 22 01:28:11 2020 (21) command runner started. > Jul 22 01:28:12 2020 (23) lmtp runner started. > Jul 22 01:28:12 2020 (20) bounces runner started. > Jul 22 01:28:12 2020 (26) pipeline runner started. > > -- > Please let me know if you need further assistance. > > Thank you for your business. We appreciate our clients. > Brian Carpenter > EMWD.com > > -- > EMWD's Knowledgebase: > https://clientarea.emwd.com/index.php/knowledgebase > > EMWD's Community Forums > http://discourse.emwd.com/ > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

4 years, 9 months

[MM3-users] Re: Newbie question 2: Rewriting issue for bounced emails

by William Oliver

On Sun, 2021-12-26 at 18:23 -0800, Mark Sapiro wrote: > On 12/26/21 6:19 PM, Mark Sapiro wrote: > > > > Otherwise, you can do this in mailman shell, e.g. > > ``` > > $ mailman shell > > > > Welcome to the GNU Mailman shell > > Use commit() to commit changes. > > Use abort() to discard changes since the last commit. > > Exit with ctrl+D does an implicit commit() but exit() does not. > > > > >>> dm = getUtility(IDomainManager) > > >>> d = dm.get('example.com') > > >>> d.alias_domain = x.example.com > > >>> commit() > > ``` > > > Ooops. That should be > > d.alias_domain = 'x.example.com' > > I.e., quoted > > Thanks. I am *almost* there. Adding the vmap in mailman fixed *almost* everything. Now I can add users, get receive mails to the list, archive messages correctly, and send emails. Except... The list sends emails to local recipients fine. I have three users on my test list: fp145(a)libertyfp.org (local) billo(a)billoblog.com (not local) oliver(a)billoblog.com (not local) When I send a message to the list from *any* of these three, the mailman3 accepts it and attempts to broadcast it out. It is delivered sucessfully to the local email address, but it looks like there's a relaying problem with external addresses. In the syslog snippet below, note that the fp145(a)libertyfp.org mail is delivered, but the mail to the recipients at billoblog.com get an "Access denied" Dec 27 15:33:25 libertyfp postfix/smtpd[419593]: connect from mail.libertyfp.org[2.56.57.28] Dec 27 15:33:25 libertyfp postfix/smtpd[419593]: DC203421BE: client=mail.libertyfp.org[2.56.57.28] Dec 27 15:33:25 libertyfp postfix/cleanup[420001]: DC203421BE: message- id=<9785b5a56581104fe079221bb947eb3a03813d97.camel(a)billoblog.com> Here it is getting delivered locally: Dec 27 15:33:25 libertyfp postfix/cleanup[420001]: DC203421BE: warning: header Subject: [Testlist] bbbbb from mail.libertyfp.org[2.56.57.28]; from=<testlist-bounces+fp145=libertyfp.org(a)libertyfp.org> to=<fp145(a)libertyfp.org> proto=ESMTP helo=<mail.libertyfp.org> Dec 27 15:33:25 libertyfp postfix/qmgr[418758]: DC203421BE: from=<testlist-bounces+fp145=libertyfp.org(a)libertyfp.org>, size=2651, nrcpt=1 (queue active) Dec 27 15:33:25 libertyfp postfix/virtual[420016]: DC203421BE: to=<fp145(a)libertyfp.org>, relay=virtual, delay=0.03, delays=0.01/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to maildir) Dec 27 15:33:25 libertyfp postfix/qmgr[418758]: DC203421BE: removed Here it is getting bounced for billo(a)billoblog.com and oliver(a)billoblog.com: Dec 27 15:33:26 libertyfp postfix/smtpd[419593]: NOQUEUE: reject: RCPT from mail.libertyfp.org[2.56.57.28]: 554 5.7.1 <billo(a)billoblog.com>: Recipient address rejected: Access denied; from=<testlist-bounces+billo=billoblog.com(a)libertyfp.org> to=<billo(a)billoblog.com> proto=ESMTP helo=<mail.libertyfp.org> Dec 27 15:33:26 libertyfp postfix/smtpd[419593]: disconnect from mail.libertyfp.org[2.56.57.28] ehlo=1 mail=2 rcpt=1/2 data=1 rset=1 quit=1 commands=7/8 Dec 27 15:33:26 libertyfp postfix/smtpd[419593]: connect from mail.libertyfp.org[2.56.57.28] Dec 27 15:33:26 libertyfp postfix/smtpd[419593]: NOQUEUE: reject: RCPT from mail.libertyfp.org[2.56.57.28]: 554 5.7.1 <oliver(a)billoblog.com>: Recipient address rejected: Access denied; from=<testlist-bounces+oliver=billoblog.com(a)libertyfp.org> to=<oliver(a)billoblog.com> proto=ESMTP helo=<mail.libertyfp.org> This error occurs whether the account posting to the list is in the libertyfp.org domain or billoblog.com domain. Note that "regular" mail from libertyfp.org does get delivered to outside addresses just fine, originating from inside the domain or through an email client outside the domain. I tried changing the smtp port in mailman.cfg to 587, but that didn't change anything. In my reading, there seems to be differing discussions about what should be in the mydestinations value, but I don't know what it should be. Here's the current last bit of my main.cf, along with the mydestinations setting: mydestination = localhost.org, localhost # add to the end (add virtual users) # if specify multiple domains, specify comma or space separated virtual_mailbox_domains = libertyfp.org virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/virtual-mailbox virtual_uid_maps = static:20000 virtual_gid_maps = static:20000 # mailman3 changes owner_request_special = no always_add_missing_headers = yes transport_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp local_recipient_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp relay_domains = hash:/opt/mailman/mm/var/data/postfix_domains default_destination_recipient_limit = 30 default_destination_concurrency_limit = 15 virtual_alias_maps = hash:/opt/mailman/mm/var/data/postfix_vmap header_checks = regexp:/etc/postfix/header_checks Here's the current values in /opt/mailman/mm/var/data/postfix_domains: # AUTOMATICALLY GENERATED BY MAILMAN ON 2021-12-27 21:10:32 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. x.libertyfp.org libertyfp.org here's postfix_lmtp: # AUTOMATICALLY GENERATED BY MAILMAN ON 2021-12-27 21:10:32 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. # Aliases which are visible only in the @x.libertyfp.org domain. testlist(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-bounces(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-confirm(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-join(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-leave(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-owner(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-request(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-subscribe(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 testlist-unsubscribe(a)x.libertyfp.org lmtp:[mail.libertyfp.org]:8024 Heres the postfix_vmap that was (finally) successfully created: # AUTOMATICALLY GENERATED BY MAILMAN ON 2021-12-27 21:10:32 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. # Virtual mappings for the @libertyfp.org domain. testlist(a)libertyfp.org testlist(a)x.libertyfp.org testlist-bounces(a)libertyfp.org testlist-bounces(a)x.libertyfp.org testlist-confirm(a)libertyfp.org testlist-confirm(a)x.libertyfp.org testlist-join(a)libertyfp.org testlist-join(a)x.libertyfp.org testlist-leave(a)libertyfp.org testlist-leave(a)x.libertyfp.org testlist-owner(a)libertyfp.org testlist-owner(a)x.libertyfp.org testlist-request(a)libertyfp.org testlist-request(a)x.libertyfp.org testlist-subscribe(a)libertyfp.org testlist-subscribe(a)x.libertyfp.org testlist-unsubscribe(a)libertyfp.org testlist-unsubscribe(a)x.libertyfp.org For what it's worth, here's the extended header info for the mail that *was* delivered: Return-Path: <testlist-bounces+fp145=libertyfp.org(a)libertyfp.org> X-Original-To: fp145(a)libertyfp.org Delivered-To: fp145(a)libertyfp.org Received: from mail.libertyfp.org (mail.libertyfp.org [2.56.57.28]) by mail.libertyfp.org (Postfix) with ESMTP id 226F6421BA for <fp145(a)libertyfp.org>; Mon, 27 Dec 2021 16:27:17 -0500 (EST) Received: from [10.112.157.251] (unknown [92.60.40.252]) by mail.libertyfp.org (Postfix) with ESMTPSA id F3608421BA for <testlist(a)libertyfp.org>; Mon, 27 Dec 2021 16:27:11 -0500 (EST) Message-Id: <562c2f8aaeb3c9e2ccf198aff197dbd299fc002a.camel(a)libertyfp.org> From: fp145(a)libertyfp.org <fp145(a)libertyfp.org> To: testlist(a)libertyfp.org Date: Mon, 27 Dec 2021 16:27:07 -0500 In-Reply-To: <eb3380eecc734e240f65ddf974d81ee7dac6ca20.camel(a)libertyfp.org> References: <eb3380eecc734e240f65ddf974d81ee7dac6ca20.camel(a)libertyfp.org> User-Agent: Evolution 3.40.0-1 Mime-Version: 1.0 Message-Id-Hash: FKEDFEFMXBD6YUDWKL235PQRIMUYYPEQ X-Message-Id-Hash: FKEDFEFMXBD6YUDWKL235PQRIMUYYPEQ X-Mailfrom: fp145(a)libertyfp.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember- moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.5 Precedence: list Subject: [Testlist] Re: test from fp145 List-Id: Test list <testlist.libertyfp.org> Archived-At: <https://libertyfp.org/archives/list/testlist@libertyfp.org/message/FKEDFEFM… > List-Archive: <https://libertyfp.org/archives/list/testlist@libertyfp.org/> List-Help: <mailto:testlist-request@libertyfp.org?subject=help> List-Owner: <mailto:testlist-owner@libertyfp.org> List-Post: <mailto:testlist@libertyfp.org> List-Subscribe: <mailto:testlist-join@libertyfp.org> List-Unsubscribe: <mailto:testlist-leave@libertyfp.org> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Evolution-Source: 14004b3d9af2d67c898fd6d6c16b487f796088f5 I'm beginning to hate postfix. billo

3 years, 3 months

[MM3-users] Re: templates

by Odhiambo Washington

If you don't have any templates in the file system and none in the DB table, then you must go to Jerusalem and pray, because you don't have any and nothing will work the way you want. It's either or! On Thu, May 4, 2023, 21:14 Christian via Mailman-users < mailman-users(a)mailman3.org> wrote: > In March when I started troubleshooting my lists, you stated: > > "Also, one other thing you might check. Do these non-working lists have > templates defined in postorius, in particular, one for > list:member:regular:footer, and if not, are there any in Mailman's > var/templates/lists/redlatinleg(a)lists.ccalternatives.org/ derectory that > Mailman doesn't have permission to find and read?" > > There are no other templates listed in either Postorius nor in Mailman's > var/templates (as I stated, I do not even lave a 'lists' subdirectory under > Mailman's var/templates). My subscribers are not receiving message > distributions from my lists, even while some of them are listed in the > archives. > > > > > -----Original Message----- > From: Mark Sapiro <mark(a)msapiro.net> > Sent: Thursday, May 4, 2023 10:58 AM > To: mailman-users(a)mailman3.org > Subject: [MM3-users] Re: templates > > On 5/4/23 10:33, Christian via Mailman-users wrote: > > I'm having issues with my lists. For each list I have created a > > template [ list:user:action:subscribe ]which has a prospective > > subscriber have to consent to list rules before they are subscribed. > > This template I created is listed in Postorius for each list. I have > > noticed that there are no other templates listed in Postorius. > Furthermore, there is no 'lists' > > subdirectory under Mailman's var/templates/. > > > When you set a template in Postorius it saves it in the > postorius_emailtemplate table in the database and tells Mailman core to get > the template from Postorius via a URI to a Postorius API. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe > send > an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/ > QIHCMEBWANR67HV3NKBK5MCQNSNZRF2A/ > <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…> > > This message sent to csa(a)web-analysts.net > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to odhiambo(a)gmail.com >

1 year, 11 months

[MM3-users] Re: Apache+mod_wsgi issue

by Mark Sapiro

On 12/25/22 01:14, Odhiambo Washington wrote: > On Sat, Dec 24, 2022 at 9:40 PM Mark Sapiro <mark(a)msapiro.net> wrote: >> >> How about https://mm3-lists.kictanet.or.ke/user_profile/ ? >> > > I have uncommented the line "WSGIScriptAlias /user-profile > /opt/mailman/mm/wsgi.py", > I don't see any definition for it in urls.py as well. Perhaps that's the > issue? It is defined in django_mailman3/urls.py which is referenced by re_path(r'', include('django_mailman3.urls')), > When I uncomment /user-profile, I end up with an ugly URL: > https://mm3-lists.kictanet.or.ke/user-profile/user-profile/ > > So instead of aliasing the /user-profile, I have decided to add another > definition to urls.py: > re_path(r'^user-profile/', include('allauth.urls')), > Which I think gives me a cleaner URL: > https://mm3-lists.kictanet.or.ke/user-profile/login/?next=/user-profile/ It shouldn't include `allauth.urls`. That URL is appropriate if you aren't logged in, but what do you get when you are? Go to https://lists.mailman3.org/user-profile/ and log in if necessary and you'll see what it should look like. > I also had to alias /accounts, which then brings me to: > https://mm3-lists.kictanet.or.ke/accounts/mailman3/lists/ - which I believe > is fine because one gets the > option to either login or signup. > > And I think that "/account" and "/user-profile" are synonymous. They are in your instance because of re_path(r'^user-profile/', include('allauth.urls')), but they should not be. >> It's a mod_wsgi thing. I haven't used mod_wsgi for Mailman for some time >> (I now use gunicorn on all the sites I manage), but when I did I think I >> had > > >> WSGIScriptAlias /mm3 /opt/mailman/mm/wsgi.py >> >> and all the URLs were like https://example.com/mm3/mailman3/..., >> https://example.com/mm3/archives/..., etc. >> > > I already had gunicorn working too, but I just need to also figure out this > mod_wsgi. Why? Intellectual curiosity or do you have an actual need. The bottom line is with mod_wsgi, whatever you have in your WSGIScriptAlias is going to be prepended to the URL path. If you don't want anything prepended, you need ``` WSGIScriptAlias / /opt/mailman/mm/wsgi.py ``` but then in order for user-profile to work, you need to replace ``` re_path(r'', include('django_mailman3.urls')), ``` in urls.py with the urlpatterns from django_mailman3/urls.py ``` re_path(r'^user-profile/delete$', delete_account, name='mm_user_account_delete'), re_path(r'^user-profile/$', user_profile, name='mm_user_profile'), ``` and also add ``` from django_mailman3.views.profile import delete_account, user_profile ``` to the imports in urls.py -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 3 months

[MM3-users] Re: Database running VERY hot

by Abhilash Raj

On Fri, Mar 9, 2018, at 9:11 AM, Darren Smith wrote: > Abhilash, > > Thanks for the response. I'm going over to lists.fedoraproject.org to > see if they have done any fine tuning.> > Our installation is at https://lists.rootsweb.ancestry.com, if you're > interested in seeing an installation with over 32,000 mailing lists. > As I mentioned, it was big enough that we have disabled the browse > experience in Postorius and are hosting our own pages for browsing to > a new list. Please keep us posted if you have other unique challenges for operating this high number of lists, other than this particular one. We'd love to be able to support high number of lists efficiently :) > If we get time to work on this we will certainly create a pull > request. :)> > > On Thu, Mar 8, 2018 at 11:49 AM, Abhilash Raj > <maxking(a)asynchronous.in> wrote:>> Hi Darren, >> >> >> On 3/8/18 9:43 AM, Darren Smith wrote: >> >>> Hello all, >>> >>> We had an old installation of Mailman 2.1 across three servers >>> with over>>> 32,000 mailing lists on them. Due to circumstances beyond my >>> control, we>>> had to shut them down and quickly set up Mailman 3 on a new server.>>> >>> We have Mailman3 set up on a single server this time, as we were >>> having>>> problems with a 3-server setup and Hyperkitty and archives. Long >>> story>>> short, it is up and running, and our users are starting to come >>> back.>>> >>> However, we do not have very many users hitting Postorius and >>> Hyperkitty>>> yet, and we are seeing a distressing number of 504 gateway >>> timeouts.>>> >>> We noticed that there is a significant number of queries to the API>>> whenever hitting Postorius's front page to load the mailing lists, >>> and then>>> on every load. It looks as if it makes a single call to the API >>> to get>>> information about a single list, and does this 10 times - once >>> for each>>> list - to render the page. This gets much worse when users set the >>> view to>>> 200 per page, as they are inclined to do with 32,000 mailing >>> lists...>>> >>> What we've done, actually, is to override the front page and >>> point the>>> users to a different set of static pages where we have all of our >>> lists>>> categorized and searchable, and when they find the list they are >>> looking>>> for, we have a link back to postorius. It has taken a lot of the >>> stress off>>> of the server. >>> >>> However, I still see where the database at times can run VERY hot.>>> >>> We have set up this server to run with PostgreSQL. When I do a ps >>> aux |>>> grep postgres, I find a lot of processes that have been spawned >>> to take>>> care of mailman - a dozen or so. However, one of the threads is >>> taking up>>> about 85% of the load, another thread about 14%, and little >>> dribbles here>>> and there from the rest of the threads. >>> >>> Here are the questions: >>> >>> 1) Has anyone run into problems like these? I would think that >>> maybe this>>> is an issue having so many mailing lists, but honestly there >>> aren't that>>> many people using the system yet. It is going to ramp up as the >>> users>>> realize that we are live again, but we aren't getting that much >>> traffic.>> >> I can say that in general, Postorius doesn't really optimize REST API >> queries and so loading a single page can sometimes make things bad.>> >> Currently, the largest installation of Mailman 3 that we know of >> runs at lists.fedoraproejct.org and the administrators of the list >> would know how well does our Web UI scale.>> >> However, I don't think we have anyone actively working on trying to >> fix this particular issue. Pull Requests and fixes are welcome :) >> >>> >>> 2) Is there any configuration that I can set, especially with >>> regards to the database, so that it can be used more efficiently >>> and not throttle through one or two threads?>> >> There isn't any configuration that would change how databases are >> used in Mailman Core. Django *may* have some, but I don't know about >> any of them of the top of my head.>> >> thanks, >> Abhilash -- Abhilash Raj maxking(a)asynchronous.in

7 years, 1 month

[MM3-users] Re: Relaying email from external SMTP server (office365 or Exchange)

by Waqas Ahmed Khan

Hi, I have successfully configured Office365 to relay emails from Mailman3. I used the below configuration. change following lines in mailman.cfg: smtp_host: your-domain.com.mail.protection.outlook.com smtp_port: 25 And then configure SMTP Relay Connector in Office365, so it can recognize your server. SMTP relay doesn't use a specific Office 365 mailbox to send email. Option3 of Link: https://docs.microsoft.com/en-us/exchange/mail-flow-best-practices/how-to-s… Could you Please confirm, what should be hardware configuration (ram, processor, disk) to run mailman3 server for medium size organization? This server will have 10-15 lists and some lists having 10k to 20k members. Also what should be DISK SPACE of the server for Archiving emails? Regards, *Waqas Ahmed Khan* p: +923212608044 Skype: waqas-ahmed00 <waqas.ahmed0(a)gmail.com> waqas.ahmed0(a)gmail.com [image: LinkedIn] <http://pk.linkedin.com/pub/waqas-ahmed-khan/15/96a/39a/> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_cam…> Virus-free. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_cam…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Tue, Jun 23, 2020 at 8:23 AM Mark Sapiro <mark(a)msapiro.net> wrote: > On 6/22/20 7:58 AM, Waqas Ahmed Khan wrote: > > Hi, > > > > I have installed mailman3 with postfix. The server is working fine. To > > protect the IPs from blacklisting, I want to relay all emails from > > postfix/mailman to external SMTP servers like O365, Gmail, or our > exchange > > server. > > > I would not recommend using O365 or Gmail as outgoing MTAs. I don't > really know about )365, but I don't think Gmail likes you to send mail > via a gmail server if it is not From: (or maybe envelope from or both) a > gmail or googlemail address. > > > > Could you please confirm is this possible. And how to do its > configuration. > > > It is possible and there are two ways to do it. You can configure this > in Postfix to relay all outgoing mail via some other server. > > > You can configure it in Mailman by setting in the [mta] section of > mailman.cfg > > smtp_host: > smtp_port: > > and presumably for authentication > > smtp_user: > smtp_pass: > > Also see > > https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/config/schema.… > and following for other relevant settings. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

4 years, 10 months

[MM3-users] Re: Installing mailman 3

by Kyle Richardson

It seems odd it would attempt to find the master script in /sbin. Could you please provide the outputs of `mailman info` and `mailman conf` to assist in troubleshooting? -Kyle On Tue, Jul 11, 2017 at 2:00 PM, Rafael Mora <rafael.mora.guti(a)gmail.com> wrote: > Hi Abhilash, > > I'm trying to use Mailman 3 on my centos 7 virtual machine. > > Those steps I wrote are from websites, the last console instructions are > mine. > > I don't use ubuntu, I use centos 7. > > When I run the start command I get this: > (venv3) [root@localhost mailman]# mailman start > > Starting Mailman's master runner > > /root/dev/mailman/venv3/bin/python: can't open file '/sbin/master': > > [Errno > > 2] No such file or directory > > What can I do? > > Thank you > > El mar., 11 jul. 2017 a las 12:31, Abhilash Raj (<raj.abhilash1(a)gmail.com > >) > escribió: > > > Hi Rafael, > > > > On Tue, Jul 11, 2017, at 07:32 AM, Rafael Mora wrote: > > > Hello, > > > > > > I want to install mailman 3 in a centos 7 virtual machine. I'm > following > > > these steps and I cannot make it work: > > > > I am wondering if you are interested to setup a development environment > > or use Mailman 3? > > These instructions below are for working on Mailman, if you just need to > > deploy, instructions here[1] would be the best. > > > > > 1. `cd ~` > > > 2. `git clone https://gitlab.com/mailman/mailman.git` > > <https://gitlab.com/mailman/mailman.git> > > > 3. `cd mailman` > > > 4. `virtualenv -p python3 venv` > > > 5. `source venv/bin/activate` > > > 6. `python setup.py develop` > > > 7. I am pretty sure this setup command will place a configuration file > > > into > > > `~/mailman/var/etc/mailman.cfg`. Edit this file. > > > > > https://mailman.readthedocs.io/en/latest/src/mailman/ > config/docs/config.html > > > 8. `mailman info` can confirm which configuration file is in use > > > 9. `mailman start` will start up Mailman 3 Core. This is what the > parent > > > background process command looks like in `ps -ef`: > > > `/home/ubuntu/mailman/venv/bin/python > > > /home/ubuntu/mailman/venv/bin/master > > > -C /home/ubuntu/mailman/var/etc/mailman.cfg` > > > > Looks like you are working as the ubuntu user here and the the > > virtualenv that you created exists at `/home/ubuntu/mailman/venv`. > > > > > I also tried the steps on the website > > > https://wiki.list.org/DEV/SetupDevEnvironment > > > but it doesn't work either > > > > > > What do you suggest? > > > > > > This is what I get from console: > > > > > > > > > [root@localhost ~]# cd dev/ > > > > > > [root@localhost dev]# cd mailman/ > > > > > > [root@localhost mailman]# source venv3/bin/activate > > > > Why do you try to user a different venv as a root user here? If you'd > > just try: > > > > $ source /home/ubuntu/mailman/venv/bin/activate > > > > and then try > > > > $ mailman start > > > > things would work! > > > > However, this would be a different instance of mailman and would create > > the new configuration (and the entire var directory) at the directory > > you run this command from. If you want to inhibit this, you can either > > use the -C flag, or set MAILMAN_CONFIG_FILE environment variable to > > point to your one single configuration file. > > > > > > > > (venv3) [root@localhost mailman]# cd mailman/ > > > > > > (venv3) [root@localhost mailman]# ls > > > conf.py COPYING data README.rst src > > > unittest.cfg > > > contrib coverage.ini MANIFEST.in setup.cfg template.py > > > copybump.py cron port_me setup.py tox.ini > > > > > > (venv3) [root@localhost mailman]# cd .. > > > > > > (venv3) [root@localhost mailman]# mailman start > > > Starting Mailman's master runner > > > /root/dev/mailman/venv3/bin/python: can't open file '/sbin/master': > > > [Errno > > > 2] No such file or directory > > > > > > (venv3) [root@localhost mailman]# > > > > > > Thank you > > > -- > > > > Hope that helps! > > > > [1]: http://docs.mailman3.org/en/latest/prodsetup.html > > > > -- > > thanks, > > Abhilash Raj > > _______________________________________________ > > Mailman-users mailing list > > mailman-users(a)mailman3.org > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > > -- > > Atentamente / Best Regards > > Ing. Rafael Mora > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

7 years, 9 months

[MM3-users] Re: Turn off social logins?

by Torge Riedel

Am 16.02.19 um 13:50 schrieb Stephen J. Turnbull: > Mark Sapiro writes: > > On 2/14/19 11:10 PM, Stephen J. Turnbull wrote: > > > Torge Riedel writes: > > > > > > > - there is a way to do this now via settings_local.py, but feels > > > > a little bit risky if default configuration changes on update of > > > > mailman > > > > > > This should not be true. If it is, we need to fix that. Will check. > > > > > > It is more or less true [...] the only way settings_local.py can > > change something like INSTALLED_APPS is to redefine INSTALLED_APPS > > in it's entirety which leads to the possibility of INSTALLED_APPS > > changing in settings.py and those changes not being picked up in > > settings_local.py. > > That's a different issue from the one I believe Torge is worried > about, which is an update overwriting settings_local.py. But it's not > good. Well, Mark got it what I was meaning. Maybe my explanation was not precise enough. Hopefully an update is not overwriting settings_local.py in any case. Is it part of delivery and then changed afterwards? Haven't checked that yet. I personally like the way other services are importing custom settings by including all config files from a given directory. Maybe it is not worth for mailman cause custom settings will not grow very big - so no need to split it in tiny files. > > > The way around this is to remove the import of settings_local from > > settings.py and then put > > > > from settings import * > > > > at the beginning of settings_local.py and then point Django at > > settings_local rather than settings. > > I don't understand the ramifications of making this change (presumably > Barry or somebody had a reason for the change from Mailman 2's > method), but at first glance it looks like a big step in the right > direction. > > > Then you can do things like > > > > try: > > INSTALLED_APPS.remove('allauth.socialaccount.providers.google') > > except ValueError: > > pass > > > > in settings_local.py except there are still issues because, e.g., > > INSTALLED_APPS is defined as a tuple (immutable) and not a list in > > settings.py. > > Given the concerns of privacy advocates, I would say that it would be > better to have a separate > > AUTH_PROVIDERS = ['allauth.socialaccount.providers.google', ...] > > in settings_local.py, and merge that in to INSTALLED_APPS. Of course > removing specific providers would be OK too, but I think that this is > a place where "deny all / accept what you want" is appropriate. > > Steve >

6 years, 2 months

[MM3-users] Help on removing moderators

by sean_sullivan＠rhoworld.com

Hello Mailman-users. I was hoping someone more experienced than myself would be able to help me with an issue I am currently experiencing. I have created a function to add moderators which works great, but I have also been trying to remove moderators as another function, but I am running into some issues. When I use the mailman shell based upon the documentation, it works great: https://docs.mailman3.org/projects/mailmanclient/en/latest/src/mailmanclien… everything works as expected. However, if I try and bring that in as a function that can be used with mailman withlist, I am consistently getting errors, and being that I am not very experienced in python, it has me scratching my head to say the least. from mailmanclient import Client from mailman.interfaces.member import MemberRole, AlreadySubscribedError from mailman.testing.helpers import subscribe addresses = ['test1(a)example.com', 'test2(a)example.com', 'test3(a)example.com'] client = Client('http://localhost:8001/3.1', 'restadmin', 'restpass') def set_owners(mlist): for email_addr in addresses: try: member = subscribe(mlist, '', email=email_addr, role=MemberRole.owner ) print (member) except AlreadySubscribedError as err: print (err) def remove(mlist): my_list = client.get_list(mlist) # print(my_list.fqdn_listname) # for email_addr in addresses: # my_list.remove_owner(email_addr) The errors I am getting are: ./bin/mailman withlist -r set_owners.remove -l it-test-b(a)lists.rhoworld.com Traceback (most recent call last): File "./bin/mailman", line 8, in <module> sys.exit(main()) File "/opt/mailman3/lib/python3.6/site-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/opt/mailman3/lib/python3.6/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/opt/mailman3/lib/python3.6/site-packages/mailman/bin/mailman.py", line 68, in invoke return super().invoke(ctx) File "/opt/mailman3/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/opt/mailman3/lib/python3.6/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/opt/mailman3/lib/python3.6/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/opt/mailman3/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func return f(get_current_context(), *args, **kwargs) File "/opt/mailman3/lib/python3.6/site-packages/mailman/commands/cli_withlist.py", line 293, in shell r = call_name(dotted_name, m, *run_args) File "/opt/mailman3/lib/python3.6/site-packages/mailman/utilities/modules.py", line 70, in call_name return named_callable(*args, **kws) File "/opt/mailman3/bin/set_owners.py", line 24, in remove my_list = client.get_list(mlist) File "/opt/mailman3/lib/python3.6/site-packages/mailmanclient/client.py", line 347, in get_list 'lists/{0}'.format(fqdn_listname)) File "/opt/mailman3/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 112, in call error_msg, response, None) urllib.error.HTTPError: HTTP Error 404: 404 Not Found So, I am wondering, is it possible to do something like this or am I completely misunderstanding the documentation, and why do I get a 404 here, when in the shell everything works as expected. I know I must be missing something, but cannot for the life of me figure out what that is. Any help would be greatly appreciated. Thanks, Sean

5 years, 1 month

[MM3-users] Re: Digests not working correctly

by Joel Lord

The May 4th digest that went out was _also_ size-triggered, so this may have nothing to do with periodic digests at all, and possibly my periodic digests aren't working. I'm not on any of my own lists in digest mode, I'm slowly extracting diagnostic information out of people who are. Also, since this is a ~2 month cycle, it's really difficult to get data points to work with. I'll need to remember to go in and look when this settles down again (new cycle of activity started last night) to see if there's anything left pending. (venv) root@host2:/home/lists/mailman/venv/bin# pip freeze | grep -i hyper HyperKitty==1.3.7 On 6/4/2023 10:05 PM, Mark Sapiro wrote: > On 6/4/23 18:35, Joel Lord wrote: >> >> The periodic digests do seem to be coming out. I also now have >> confirmation that the one message in this morning's digest that was >> from May 4th was also included in the last digest back on May 4th, so >> it seems that the one message was left behind in the digest queue when >> the periodic digest was sent. > > I don't see how that can happen. The process that sends a digest renames > the var/lists/<list-id>/digest.mmdf mailbox file in which the messages > are accumulated to var/lists/<list-id>/digest.<volume>.<issue>.mmdf, > where <volume> and <issue> are the volume and issue numbers of that > digest, and then queues a message in the `digest` queue to tell the > digest runner to create the digest from the messages in that mbox and > send it. Thus, it leaves no var/lists/<list-id>/digest.mmdf mailbox file > behind and that is created anew when the next post arrives. Further, if > there is a non-empty digest.mmdf file, its messages should be sent no > later than the next 11 PM `cron digests`. > > >> There was one earlier message to the list back on May 4th, before the >> one that got duplicated, but I can't tell if that triggered a >> size-based digest to be sent: the logs aren't clear enough on that >> detail for me to tell > > > OK > > >> Just to inform things: >> >> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep mailman >> django-mailman3==1.3.9 >> mailman==3.3.8 >> mailman-hyperkitty==1.2.1 >> mailman-web==0.0.6 >> mailmanclient==3.3.5 >> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep hyper >> mailman-hyperkitty==1.2.1 > > Actually, it's HyperKitty, not hyperkitty, but I assume HyperKitty is up > to date as are the others. > >> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep post >> postorius==1.3.8 >> >> > -- Joel Lord

1 year, 10 months

[MM3-users] Re: messages stuck in the bad queue

by Mark Sapiro

On 6/25/23 1:31 PM, Ken Alker wrote: > > I moved the .psv file from the bad queue into the shunt queue. I then > ran "mailman unshunt" (as user 'mailman' while in the virtual > environment). I tailed mailman.log during this process and no logs were > spit out. The date stamp on the .psv file never changed (maybe it does > not when being moved between queues?) and, AFAICT, the file never moved > from the shunt queue. I waited maybe five minutes, tops. I forgot that the extension in the `bad` queue is `.psv` (for preserve). You have to rename it to `.pck`. >>> Is there a way to reprocess the bad queue? >> >> You could just move the messages to the 'in' queue. Changing .psv to .pck in the process. this can be done, e.g., by ``` for file in `ls var/queue/bad/*.psv;do mv /var/queue/bad/$file /var/queue/in/${file/psv/pck};done ``` > I just now tried moving the same message into the 'in' queue but, again, > nothing happened. I left it in there for five minutes. Do I have to > run a program to get it to act on the 'in' queue (I presume that there > is a "runner" that is always looking and taking care of this already as > I presume this is the queue where all 'normal' traffic is handled). Again, you need to change the extension to .pck. > Here are the (obfusacted) results of "mailman qfile > /opt/mailman/mm/var/queue/shunt/1675389793.6945386+2aeaf0015558c9d8380c96142c3fa9d03a8142bc.psv" (the message I was experimenting with): > > [----- start pickle -----] > <----- start object 1 -----> > MIME-Version: 1.0 > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: 7bit > Subject: THE-list subscription notification > From: email(a)obfuscated.com > To: the-list-owner(a)lists.obfuscated.com > Message-ID: <167538979369.2390432.521344217673230839(a)obfuscated.net> > Date: Thu, 02 Feb 2023 18:03:13 -0800 > Precedence: bulk > > user(a)domain.tld has been successfully subscribed to THE-list. > > <----- start object 2 -----> > { '_parsemsg': False, > 'envsender': 'email(a)obfuscated.com', > 'listid': 'the-list.obfusacted.com', > 'nodecorate': True, > 'recipients': {'person(a)domain.tld'}, > 'reduced_list_headers': True, > 'version': 3} > [----- end pickle -----] I was assuming these were list welcome messages which they aren't. They are owner notifications, so they were sent to the -owner address and processed through the owner-pipeline, but unless the Debian package changed the default-owner-pipeline, it doesn't contain mime-delete so how they ended up in the `bad` queue is still a mystery. However, what would be the point of resending these at this time? -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 9 months

[MM3-users] Issue with Docker Postfix and Mailman3 Configuration

by Stephen J. Turnbull

recida8902--- via Mailman-users writes: > When I send an email from my configured email address, What "configured email address"? Do you mean the administrator email address for the Mailman applications? If so, that doesn't matter; the only things Mailman uses it for are (1) as the return address for some automatically generated email and (2) as a place to send certain error messages. It is not specially routed because it is mentioned in the Mailman configuration. It is specially routed because of the host it originates on, and perhaps filtered differently according to some component of the address. > it is successfully delivered to the intended recipients. However, > when someone else sends an email to the same address, it does not > reach the Mailman application, and no error messages are generated. The first thing you need to do is find logs of various messages. For mail transfer agents (MTAs or "mail servers") it's usually in /var/log/postfix/mainlog (I think) or /var/log/mail.log, sometimes in /var/log/syslog or /var/log/messages. Most likely you just need to trace one example each of a message from the "configured address" to Mailman and a message from "outside" to Mailman. Find where the messages that don't get through have a receipt but no transmission. This will most likely be from your border MTA to Mailman, or from your border MTA to your Dockerized Postfix. > 3. Checked the Mailman configuration: I reviewed the configuration > files of Mailman and ensured that all necessary settings are > properly configured. However, I couldn't identify any specific > variables or options related to incoming email handling. There aren't any. Once it gets to Mailman, either it's related to a list or one of the sitewide administrative addresses, or it gets rejected as incorrectly addressed. Incoming mail is handled by the MTA which sorts out Mailman mail vs any other mail. All interesting configuration is in the MTA, Postfix in your case. However, the problem is most likely at the network level, not the mail level. Configuration of Postfix for Mailman is a long-solved problem. First, your Docker host needs to be correctly configured to talk to the Internet side of your network and to the containers' network, and to function as a gateway between the two networks. The issue is that the containers use so-called private IP addresses, which cannnot be reached from the public internet. My guess is that the most likely problem is that your border MTA is misconfigured in trying to talk to the next hop (either the Postfix MTA or Mailman itself). To figure that out, you need to trace messages until they disappear, find out what the actual IP addresses being used by each host are, and what addresses the MTAs are configured to send each mind of mail to.

1 year, 7 months

[MM3-users] Re: MM3 templates

by Odhiambo Washington

On Thu, Feb 16, 2023 at 10:26 AM Mark Sapiro <mark(a)msapiro.net> wrote: > On 2/15/23 23:04, Odhiambo Washington wrote: > > > > The last time I referred to the link, I deliberately duplicated the <LC> > > folder in two locations: > > [09:51 ~ ]$ ls -al /opt/mailman/mm/var/templates/site/en/ > > total 48 > > drwxr-xr-x 2 mailman mailman 512 Feb 14 08:53 . > > drwxr-xr-x 3 mailman mailman 512 Feb 14 08:53 .. > > -rw------- 1 mailman mailman 615 Jan 17 13:57 > domain:admin:notice:new-list.txt > > -rw------- 1 mailman mailman 341 Jan 17 13:54 > list:admin:action:post.txt > > -rw------- 1 mailman mailman 158 Dec 17 13:32 > list:member:digest:footer.txt > > -rw------- 1 mailman mailman 452 Jan 17 13:54 > list:member:digest:masthead.txt > > -rw------- 1 mailman mailman 137 Jan 17 13:55 > list:member:generic:footer.txt > > -rw------- 1 mailman mailman 1219 Dec 8 15:12 > list:member:regular:footer.txt > > -rw------- 1 mailman mailman 944 Jan 17 15:50 > list:user:action:subscribe.txt > > -rw------- 1 mailman mailman 247 Jan 17 13:56 > list:user:notice:post.txt > > -rw------- 1 mailman mailman 802 Jan 17 13:56 > list:user:notice:probe.txt > > -rw------- 1 mailman mailman 639 Jan 17 13:57 > list:user:notice:welcome.txt > > > > Yet, it is NOT being picked. So definitely something is wrong somewhere. > > > If you have the same list:* file in both > /opt/mailman/mm/var/templates/site/en/ and > /opt/mailman/mm/var/templates/lists/kictanet.lists.kictanet.or.ke/en/ > the ones in the lists directory will take precedence for that list over > the ones in the site directory. > In my case, I had the exact same files in the two paths and the list:user:action:subscribe.txt was still not used in my tests. Why? I have now created the directory: mkdir -p /opt/mailman/mm/var/templates/domains/lists.kictanet.or.ke/en/ And I have placed the same files in there. Gearing up for a test to see if the template will be used this time around. > See > > https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/utilities/i18n… > for a discussion of the search order. > Will do. > >> > >> Are the other templates such as list:member:regular:footer.txt used? > >> > > > > I do see that in posts to the list, but I believe that is perhaps only > > working because the list had that footer when it was imported from MM2. > > A template from import21 of the kictanet(a)lists.kictanet.or.ke list will > be stored in the > /opt/mailman/mm/var/templates/lists/kictanet.lists.kictanet.or.ke/en/ > directory. > It wasn't a template per se. It was the regular nondigest msg_footer from MM2. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)

2 years, 2 months

[MM3-users] Re: Setup Question: Disk-Layout (Migrating from Mailman 2.1)

by Christian Schneider

Thank you for your instant reply! Mihai Moldovan wrote: > > But I could find no information about the location where hyperkitty stores the archives on an Ubuntu-/Linux-system. > > I assume the archives are stored underneath "/var/..." this is only a guess. > > But is this guess right? > > Not quite. > To the best of my knowledge, by default, a venv installation is fully contained > under /opt/mailman, with archives and stuff stored at /opt/mailman/var/lib/... This information is very helpful! Actually I have about 45GB of archives and these archives only hold messages of the last year... I run a clean-up script every first day of a month which deletes posts older than 365 days... So there is lot of disk space needed... ;-) (and even more if I plan to keep posts around for a longer time) > > Another question is: > > How much space do I have to plan (roughly) for the "/opt"-partition for the mailman install if the archives are stored under "/var/..."? > > Depends on your archives, the way things are archived (you can choose different > database backends, and these might make a difference in how efficiently or > inefficiently data is stored) and whether you want to use the full-text search > index, which adds another big chunk of data. > I can give you a few pointers regarding my setup: my mailman 2.1 data directory > takes up about 8.2 GB of disk space, most of which are archives, naturally (8.1 > GB). I've imported most of the big archives to mailman 3 already, with a > full-text search index, and that uses 5.2 GB disk space currently, but I'm still > missing a few archives, so that'll still grow a bit. It will probably end up at > about the same size, but I can come back to that once my import is fully done > (which sadly will take a few days). > Now, the 8.1 GB of mailman 2.1 archives are stored pretty inefficiently, because > they are essentially duplicated (as a private mbox archive and public text > archives). Mailman 3 doesn't do that - at least not directly - but instead > stores archives once into the database, and queries the database whenever the > archives are accessed via hyperkitty, so I would *assume* the size you need for > the mailman 3 data after a successful migration is just a bit shy of what you > needed for the mailman 2.1 data, maybe even a bit less. Perfect! :-) If I mount an extra partition of 250Gigs at /opt this should carry me for a longer time... And - if I setup a logical volume with LVM - there is an option to stack up the space with ease... ;-) > Mihai Thank you very much for your help! Chris

6 days, 22 hours

[MM3-users] Re: Deleting A List

by Mark Sapiro

On 5/3/24 03:59, Odhiambo Washington wrote: > On Thu, May 2, 2024 at 8:11 PM Mark Sapiro <mark(a)msapiro.net> wrote: > >> On 5/2/24 03:42, Odhiambo Washington wrote: >>> I have deleted a list (list(a)nog.ke) - list_id = list.nog.ke. >>> I can see from Django Admin UI that the archives are still intact. >>> I created a new list (list(a)lists.nog.ke) - list_id = list.lists.nog.ke >>> Is there a way to move the archives from the old list to the new list? >> >> >> Change the `name` and `list_id` in the list's entry in the >> `hyperkitty_mailinglist` table in the database. If there is already an >> entry there for the new list, delete it >> >> > ``` > mysql> select id, name, list_id from hyperkitty_mailinglist; > +----+----------------------------+----------------------------+ > | id | name | list_id | > +----+----------------------------+----------------------------+ > | 1 | skunkworks(a)lists.my.co.ke | skunkworks.lists.my.co.ke | > | 2 | list(a)nog.ke | list.nog.ke | > | 3 | eanog(a)lists.my.co.ke | eanog.lists.my.co.ke | > | 4 | isoc(a)lists.my.co.ke | isoc.lists.my.co.ke | > | 5 | kazi(a)lists.my.co.ke | kazi.lists.my.co.ke | > | 6 | kenog(a)lists.my.co.ke | kenog.lists.my.co.ke | > | 7 | mechanics(a)lists.my.co.ke | mechanics.lists.my.co.ke | > | 8 | science(a)lists.my.co.ke | science.lists.my.co.ke | > | 9 | security(a)lists.my.co.ke | security.lists.my.co.ke | > | 10 | vmware_tech(a)lists.my.co.ke | vmware_tech.lists.my.co.ke | > | 11 | water(a)lists.my.co.ke | water.lists.my.co.ke | > | 12 | wordpress(a)lists.my.co.ke | wordpress.lists.my.co.ke | > | 13 | list(a)lists.nog.ke | list.lists.nog.ke | > +----+----------------------------+----------------------------+ > 13 rows in set (0.00 sec) > > mysql> delete from hyperkitty_mailinglist where id = 13; > ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key > constraint fails (`mailmansuite`.`hyperkitty_mailinglist_owners`, > CONSTRAINT `hyperkitty_mailingli_mailinglist_id_a47e04ec_fk_hyperkitt` > FOREIGN KEY (`mailinglist_id`) REFERENCES `hyperkitty_mailinglist` (`id`)) > ``` > > Is there an easier way? > I need to delete id=13 then rename id=2 > There are several tables that reference the hyperkitty_mailinglist table by id. These are hyperkitty_email, hyperkitty_mailinglist_owners, hyperkitty_mailinglist_moderators and hyperkitty_thread. Ideally, you wouldn't have created the list(a)lists.nog.ke before updating the hyperkitty_mailinglist entry, but now, do the following: ``` update hyperkitty_email set mailinglist_id = 2 where mailinglist_id = 13; update hyperkitty_mailinglist_owners set mailinglist_id = 2 where mailinglist_id = 13; update hyperkitty_mailinglist_moderators set mailinglist_id = 2 where mailinglist_id = 13; update hyperkitty_thread set mailinglist_id = 2 where mailinglist_id = 13; delete from hyperkitty_mailinglist where id = 13; update hyperkitty_mailinglist set name = 'list(a)lists.nog.ke', list_id = 'list.lists.nog.ke' where id = 2; ``` If there have been no posts to the list(a)lists.nog.ke list the updates of hyperkitty_email and hyperkitty_thread won't do anything, and likewise the update of hyperkitty_mailinglist_moderators if the list has no moderators -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

11 months, 2 weeks

[MM3-users] Re: Postorius fails to show list because of a large message being held

by Mark Sapiro

On 7/14/20 3:54 AM, Brian Carpenter wrote: > On 7/14/20 6:49 AM, Gilles Filippini wrote: >> Mailman Core logs: >> [2020-07-14 10:41:21 +0000] [36] [CRITICAL] WORKER TIMEOUT (pid:49) >> [2020-07-14 10:41:21 +0000] [49] [INFO] Worker exiting (pid: 49) >> [2020-07-14 10:41:22 +0000] [50] [INFO] Booting worker with pid: 50 > > I am pretty sure that timeout is related to gunicorn. If it is, then > increasing the worker out timeout setting for gunicorn should work. You > tend to get such errors when doing an export of a large membership > roster. This is the first time I have seen it related to a held message. It seems weird that a gunicorn worker would run for more that the default 30 seconds because of a 23MB message, but I think Brian is correct. > I have this in my setup: > > mailman.cfg: > > [webservice] > configuration: /opt/mailman/mm/gunicorn.cfg > > gunicorn.cfg: > > [gunicorn] > workers = 4 > timeout = 900 > > You will need to restart gunicorn after adjusting the timeout setting. It's confusing, but after adjusting that setting, you need to restart Mailman core, not gunicorn. If you are running a gunicorn service, that is what is providing WSGI support for Django (Postorius and/or HyperKitty). You may be using gunicorn for this or uWSGI or Apache mod_wsgi, but is any case, that is not the gunicorn we are considering here. Regardless of what you use to provide WSGI support for Django, Mailman core uses gunicorn to support the REST API, and that is the gunicorn affected by the settings in the gunicorn.cfg pointed to by [webservice] configuration: /opt/mailman/mm/gunicorn.cfg If, with the above configuration, you do `ps -fwwA|grep runner=rest` you will see something like: mailman 20582 20561 0 01:21 ? 00:00:16 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/deployment/mailman.cfg --runner=rest:0:1 mailman 20722 20582 0 01:21 ? 00:01:19 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/deployment/mailman.cfg --runner=rest:0:1 mailman 20725 20582 0 01:21 ? 00:01:18 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/deployment/mailman.cfg --runner=rest:0:1 mailman 20726 20582 0 01:21 ? 00:01:18 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/deployment/mailman.cfg --runner=rest:0:1 mailman 20727 20582 0 01:21 ? 00:01:50 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/deployment/mailman.cfg --runner=rest:0:1 The first of these, pid 20582, is Mailman's actual REST runner. The other 4 with parent pid 20582 are the 4 gunicorn worker processes forked from the REST runner. It's the REST runner or Mailman core that needs to be restarted to pick up that change. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 9 months

[MM3-users] Re: Settings for prevention of spam obfuscation

by Thomas Ward

I think we forget DMARC can operate with SPF only mode. If you truly want stronger antispam controls then deny all posts from nonmembers, and/or implement SpamAssassin at the postfix level with extended OSINT pattern sets and policies. That will catch MOST stuff but also likely have a higher false positive moderation / reject rate Sent from my Galaxy -------- Original message -------- From: Richard Rosner <rrosner5(a)gmail.com> Date: 2/24/24 10:57 (GMT-05:00) To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: Settings for prevention of spam obfuscation Well, I don't want to give users additional information, just rectify the manipulated information. And yes, while there may be edge cases where these bad practices do come in handy, but the vast majority of mails (at least from what I can tell from my lists) is either spammers or unnecessary abuse of features. I can't tell if it's because Microsoft sets bad defaults for Outlook or admins hired are incompetent and the competent ones either retire or quit, but there's just too much going on in mails that's just nonsense. E.g. encoding all text - including stuff like the subject - with base64 is just ridiculous, when what was sent is only plain text that uses a small subset of UTF-8 characters. And that's just a very common bad practice and far from the only one. And sure, competent spammers may be able to evade much more, I've seen several emails that must have had manipulated sender data, but that could only be concluded from context, not from analyzing the headers. But thankfully most spammers are far from competent, so spam filters can easily detect far over 50 % of the spam. It's only the slightly more competent spammers that use tactics abused by many non-spammers that give me a headache. I could easily crank up the spam points for various indications, so there would be pretty much no spam left in the false negative category. But sadly, I know from experience that this will lead to a much larger amount of false positives. For the most common senders that won't be sending spammers, I already created whitelisting rules, but that's about it. And until someone can create an ML model that's not gobbling up resources for training and that doesn't need thousands of emails to train with, there's only so much you can do. While DKIM and DMARC are great, they are useless when the vast majority of senders don't have any DKIM signatures. For ARC it's not as uncommon to be present, but the situation isn't that much better. _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… This message sent to teward(a)thomas-ward.net

1 year, 1 month

[MM3-users] Ann: Mailman Core 3.3.5 is out

by Abhilash Raj

Hey Everyone, I just tagged and uploaded Mailman Core 3.3.5 release. This release includes both bug fixes and some new features. A full list of changes is available here: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.… You can install this version using pip: $ pip install mailman==3.3.5 The release tarball is available on PyPI as well: https://pypi.org/project/mailman/3.3.5/#files Thanks to everyone who made this release possible! Abhilash > On Sep 19, 2021, at 11:02 AM, Abhilash Raj <maxking(a)asynchronous.in> wrote: > > Hey Everyone, > > I just tagged and released Mailman Core 3.3.5rc1 release. > > This is identical to 3.3.5b1 in code, the only changes that have been made are in utility scripts and testing configuration. > > You can install this version using: > > $ pip install --pre mailman==3.3.5rc1 > > If you want to download a release tarball, you can do so from here: > > https://pypi.org/project/mailman/3.3.5rc1/#files > > This is the first (and only) planned release candidate for 3.3.5 release. The stable release is planned for next week. > > thanks, > Abhilash > > > On 9/7/21 12:39 AM, Abhilash Raj wrote: >> Hello Everyone, >> I am pleased to announce that Mailman Core 3.3.5b1 is now out. It is a pre-release for 3.3.5, which is slated to come out in 3 weeks from now. >> I am planning for a 2 week beta period, after which I'll release the first RC and then stable a week after. Right now I am not planning to release a second beta version, but if there are several changes in Core in the next week or so, then I might. >> This release includes a lot of bug fixes and some new features. It also includes a security enhancement that improves the authentication of REST API by adding resistance to some timing channel attacks. >> A full list of changes in this version can be found here[1]. >> [1]: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.… We have also made improvements in the i18n workflow and now the email templates are updated from the translated messages in the .po files that we get from Weblate. Documentation for translators is available here[2] if you want to help translate Mailman into your native language. >> [2]: https://docs.mailman3.org/en/latest/translation.html >> This is a pre-release version, if you want to test it with your setup, you can install it using: >> $ pip install --pre mailman==3.3.5b1 >> If you want to download a release tarball, you can do so from here: >> https://pypi.org/project/mailman/3.3.5b1/#files >> This release received a lot of contributions from community, which I am really happy about! I'd like to thank everyone who made this release possible! > > > -- > thanks, > Abhilash Raj (maxking) -- thanks, Abhilash Raj (maxking)

3 years, 6 months

[MM3-users] Re: ARC

by Stephen J. Turnbull

Marvin Gülker writes: > However, I thought the entire point in ARC was fixing automatic > rejections caused by DMARC invalidation by mailing lists, which are > vital for so many open-source projects (including Mailman iself). Yes. s/fixing/making it possible to fix/ and s/mailing lists/indirect mail/ (the REALLY big losers in April 2014 were small businesses which had a large percentage of their invoices become undeliverable). That doesn't help you one bit with your "rejection with no reason" problem, though. It helps you with one specific reason (DMARC p=reject). > If ARC falls short of solving that problem, because the large > providers do not automatically trust ARC, then supporting ARC on my > mailing list server is kind of void. Why should almighty Microsoft > trust my small personal mailing list server? The answer to that is "why shouldn't they?" I've never had a problem with any of the big services. There are reasons why some hosts might have problems due to no misbehavior of their own, and they wouldn't necessarily know about them either. But mostly the mail does go through. If the mail itself gets through, I see no reason why they wouldn't trust your ARC fields, so you won't get DMARC rejections, as long as nobody downstream of you breaks your ARC signatures. If you're seeing something different (ARC-authenticated messages getting rejected for DMARC alignment failure), let us know and I'll talk to a guy who knows this gal. My experience is good, the people I talk to are happy. If your experience is different, the ARC developers should know about it, otherwise we can't do anything about it. > ARC can only achieve its goal with black-listing untrusted relays > rather than white-listing known good ones, The goal of ARC is to keep users at p=reject sites happy. A lot of the people involved both at the IETF and at the DMARC Consortium are old-timers who are very sympathetic to mailing lists. But the day-to- day tuning of algorithms is targeted at users with mailboxes, not the lists that fill those mailboxes. Sad to say, users care more about spam (that's their provider's fault) than about mailing lists (which they assess as the mailing list's fault). Last I heard the Google folks and the IETF are very happy with ARC. > because it is simply unfeasable to manually maintain a list of all > open-source project mailing lists in the entire world. I could set > a new one up tomorrow, how should you know? That's not how this works. All of these services use huge databases of spam, ham, hosts, and mailing lists, and apply machine-learning algorithms to that data to create classifiers. It may take them some time to tune them for ARC, but the principle is there's no black or white here, it's all grey. Steve

5 years, 8 months

[MM3-users] Re: Mailman not able to process "big" requests

by Jonas Felber

Thank you for your support, I got informed that we are now able to send out emails again. The problem was that the crashes we got while mass subscribing led to database corruptions. (From what I know, the problem was on our side, as we kubernetes did not properly restore some files after restarting so the database got corrupted (apparently we are running now a cleanup script on each restart)) There are still some problems, but I just wanted to send out a quick update. Thanks for your support, Jonas Jonas Felber <jonas.felber(a)vis.ethz.ch> Aktuar & CTF Präsident VIS - Verein der Informatik Studierenden an der ETH Zürich CAB E 31, Universitätstr. 6, ETH Zentrum, CH-8092 Zürich https://www.vis.ethz.ch > On 24 Apr 2018, at 17:21, Darren Smith <silas.crutherton(a)gmail.com> wrote: > > Our installation at https://lists.rootsweb.ancestry.com has around 33,000 > lists and somewhere around 1.5 million members. We have lists with 5,000+ > members. > > Note that I haven't tried any mass subscriptions. When we upgraded from > mailman 2 to mailman 3, we used the old .pck files and ran the import on > each of the lists. It did take about 6 weeks to do the full import all > 33,000 lists. I'm not joking. > > As for sending messages - there isn't any problem sending that many > messages out, but then we are also not trying to subscribe list members at > the time. We are, though, in the process of importing the old archives from > our old mailman 2 system. This doesn't seem to affect the rest of the > system. > > We have seen some inefficiencies in the new system, such as the front page > of postorius - loading all of the information for 200 lists (if the user > set the "lists per page" to 200) was bringing down the system. We have our > own metadata system to be able to search for lists by state/region/surname > (ours is a genealogy mailing list system), so we had to replace the front > page with our own "find a list" functionality. > > The other main problem was the page where a user can see what lists they > subscribe to. That page was also a denial of service if the user had > several hundred lists they are subscribed to. And for a genealogy mailing > list system, this is common. > > We have actually ended up creating our own API layer (only accessible on > the local machine) and make certain calls to it for the most painful of the > issues. One of these days (when we get a moment to breathe) we are going to > post some of our changes to the project so that we can ditch our own API > and just use the internal REST API. > > To be fair, the inefficiencies are not necessarily with mailman or the rest > API itself - but with Postorius making hundreds of calls to load a single > page. Though if the API supported loading multiple lists at once instead of > having to load them serially, that might help as well. > > So while we have not seen any timeout issues on the call to mass subscribe > users, we HAVE found several other calls that are effectively a denial of > service and have gotten around them ourselves, with plans to contribute to > the project in the near future. > > -Darren > > >> On Tue, Apr 24, 2018 at 8:47 AM, Mark Sapiro <mark(a)msapiro.net> wrote: >> >>> On 04/24/2018 12:36 AM, Abhilash Raj wrote: >>> On Mon, Apr 23, 2018, at 10:41 PM, Jonas Felber wrote: >> >>>> We also have a problem with sending out emails to "large" (>50) groups, >>>> as they seem to not getting sent out at all. Do you have any idea what >>>> the problem here could be? >>> >>> Which MTA are you using? What do you see in mailman's logs and MTA's >> logs? >>> >>> I haven't seen this kind of problem before with Mailman 3. This very >> list is hosted using Mailman 3 and I believe has more than 50 subscribers. >> >> >> Nor have I seen this issue. This list currently has 158 non-digest >> members. There are several @python.org Mailman 3 lists with over 100 >> non-digest members including the active yt-users(a)python.org list with >> 301 members, and I have gotten no reports of an issue like this. >> >> Also, there are over 600 lists at <https://lists.fedoraproject.org/>, >> and I don't think this is an issue there either. >> >> -- >> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, >> San Francisco Bay Area, California better use your sense - B. Dylan >> _______________________________________________ >> Mailman-users mailing list >> mailman-users(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

6 years, 11 months

[MM3-users] Re: Missing Base64 encoding from control message reply

by Jana Berger

On 22.09.2023 18:32, Stephen J. Turnbull wrote: > lists-mailman3(a)janaberger.de writes: > > > I have a setup with GNU Mailman 3.3.8 where a user tried to > > subscribe. When replying to the "confirm" message, the user > > received a completely garbled response, like so: > > > > N?z?????zf???&???j?????u?[zZ0:???v???,j?z???k?e?0 > > > 4D???r?^b??r?????*'???y????{ax?L?X????0:???v???,j?z???k?d?m?-???zz?rn??l]y+???-E?( > > > ?^N??'??M???t???Mz?????^i?????}?o?\????6sN?zf??????r??z?????????b?{h?????f?r?ne== > > To diagnose, we really need to see that in context of the raw message > as received by the user. You can see the same or similar result by opening the EML I quoted. Does the list support attachments? Then I will attach the file directly. > > > The user then forwarded the .eml file to me, which clearly shows > > the contents of the mail as plain text while being announced as > > base64 (Content-Transfer-Encoding: base64): > > It also clearly doesn't show anything like the mojibake above. What > is the relation between the two? Sorry, I should have made that clearer - when opening the EML file with e.g. Thunderbird, the contents (message text) of the mail as displayed is the garbled string. I assume it is the result of Thunderbird trying to process the contents as base64-encoded, which it is not. > > While the problems could be related to Mailman, it could also be any > MTA along the way or the user's mail client. > > I suspect that the .eml file has been significantly altered by > intervening software. As far as I know, the Python email package does > not MIME-encode plain ASCII text as you see in the email's subject. I have multiple control messages, all received from my instance which all contain the "Content-Transfer-Encoding: base64" marker, their contents is plain text (Content-Type: text/plain; charset="utf-8"), but they are still encoded using base64. Of course that is not proof, but an indication that it is either done by the mail server itself or widely across MUAs. > > The only other thing I can think of is it might have something to do > with your internationalization settings. What is the default language > of the list? (This is available in Postorius.) Have you changed the > default encoding for English on your site? (I think you need to look > in mailman.cfg for this.) What is the default language of the site? Global "Preferred language" is "English (USA)", which is not overridden by the list in question. I have not changed the default encoding. Is there a surefire way to detect this? My mailman.cfg has no section pertaining to encoding, and all related files also do not mention encoding.

1 year, 7 months

[MM3-users] Re: Process on migrating mailman

by Mark Labeste

Just like to confirm when doing this dumping process. Nothing will change on the current mailing list? On Tue, Jan 23, 2024 at 2:01 PM Odhiambo Washington <odhiambo(a)gmail.com> wrote: > > > On Tue, Jan 23, 2024 at 4:22 AM Mark Labeste <marqueejen13(a)gmail.com> > wrote: > >> About 500 list. Not sure how to check size of the archives. >> >> On Thu, Jan 18, 2024 at 4:44 PM Odhiambo Washington <odhiambo(a)gmail.com> >> wrote: >> >>> >>> >>> On Thu, Jan 18, 2024 at 8:49 AM Mark Labeste <marqueejen13(a)gmail.com> >>> wrote: >>> >>>> Any suggestion on how to migrate mailing list and archirve with >>>> diffirent server and domain. For the mailing list from >>>> sample.domain1.example.com to sample.domain2.example.com >>>> >>>> Old Mailing list >>>> Mailman Core Version GNU Mailman 3.3.5 (Tom Sawyer) >>>> Mailman Core API Version 3.1 >>>> Mailman Core Python Version 3.6.8 (default, Sep 26 2019, 11:57:09) >>>> [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] >>>> Domain Domain1.example.com >>>> >>>> New Mailing list >>>> Mailman Core Version GNU Mailman 3.3.9 (Tom Sawyer) >>>> Mailman Core API Version 3.1 >>>> Mailman Core Python Version 3.9.18 (main, Sep 22 2023, 17:58:34) >>>> [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] >>>> Domain domain2.example.com >>> >>> >>> >>> How many lists? How big are the archives? >>> >>> Looking at >>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…, >>> it seems like it's not a trivial exercise. >>> >>> However, I am wondering about the consequences of: >>> 1. dumping the DBs and doing 's/DOMAIN1/DOMAIN2/g' in the dumps, then >>> re-importing >>> 2. editing settings.py and webserver config to reflect the DOMAIN2 >>> 3. reconfiguring your MTA to handle list mails using the DOMAIN2 >>> >>> I am imagining a virtualenv type of installation. >>> >> > The archives are in the database so nothing to worry about, > Now that I think about this again, after dumping and importing the > database as shown above, you need to rerun the 'mailman-web migrate` once > again to import any DB schema changes between the versions. > Actually, just run all the options from > https://docs.mailman3.org/en/latest/install/virtualenv.html#run-database-mi… > . > > -- > Best regards, > Odhiambo WASHINGTON, > Nairobi,KE > +254 7 3200 0004/+254 7 2274 3223 > In an Internet failure case, the #1 suspect is a constant: DNS. > "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) > [How to ask smart questions: > http://www.catb.org/~esr/faqs/smart-questions.html] > -- Regards, mARK

1 year, 3 months

[MM3-users] Re: Verification Failed Error

by Abhilash Raj

On Fri, Dec 21, 2018, at 12:18 PM, Odhiambo Washington wrote: > > > On Fri, 21 Dec 2018 at 20:19, Abhilash Raj <maxking(a)asynchronous.in> wrote: >> On Fri, Dec 21, 2018, at 4:00 AM, Stephen J. Turnbull wrote: >> > Abhilash Raj writes: >> > >> > > Exim experts would be able to help you better with this. >> > >> > I'm the one who set up the original sample Exim .conf, because (a) I >> > didn't want to migrate all my bots to Postfix, and (b) the one-real- >> > host .conf turned out to be trivial. I'm an Exim expert only for my >> > own configuration, which is a (pre-global-warming) glacially-moving >> > target. I've never seen anything like the problems described here, >> > sorry. Can't help. >> > >> > Since it's looking like we don't have any real Exim experts hanging >> > around here, perhaps we should move the sample containers and other >> > default configs to assume Postfix and document "Exim should work but >> > you're kinda on your own, we'll do what we can but it's limited." At >> > least Mark is maintaining such systems, and they're probably better >> > documented than Exim for the single-real-host configuration, though >> > they would probably run into similar problems with the containerized >> > setup. WDOT? >> >> >> Originally, the reason I went with Exim was because the postmap + hash >> files were hard to generate for Mailman from within the container when >> the postfix was running on the host. Exim checks existence of certain >> paths to verify that the Mailing List exists, which was easier to do. >> >> However, now Mailman Core supports regex maps for Postfix, which >> can be used with containers. >> >> Images already support using Postfix, it is just not the default in setup >> scripts and documentation. I have opened an issue[1] to maybe change >> that if some initial testing doesn't goes fine. >> >> [1]: https://github.com/maxking/docker-mailman/issues/282 > > > Hello Maxking, > > It is importantest to maintain Exim support :-) I didn't mean to say that we are going to stop supporting Exim, it is just that I'll make Postfix default in Container given that most of the Mailman Devs are more comfortable helping out people with Postfix issues, as they use it for their own personal setups. Switching between default and non-default would be easy and both of them will be supported, to the best my own abilities :) > > -- > Best regards, > Odhiambo WASHINGTON, > Nairobi,KE > +254 7 3200 0004/+254 7 2274 3223 > "Oh, the cruft.", grep ^[^#] :-) -- thanks, Abhilash Raj (maxking)

6 years, 4 months

[MM3-users] Re: Importing mbox files into archive defect with lines with From

by Alex Schuilenburg

Hi again. On 20/09/2023 17:59, Stephen J. Turnbull wrote: > [...] > I suspect that "DROP DATABASE mailmanweb;" and then loading the dumped > database, followed by `mailman-web migrate` will work. (Usual caveat > of you should have a backup onsite, a backup in a bank vault, and a > backup store on the dark side of the moon before trying this!) That exactly what I did in my initial attempt. It was a straight dump and restore which does as its first cmd "DROP DATABASE mailmanweb;" As per https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…, the migrate error was the table django_content_type already exists. Indeed it exists in the old database, but other tables do not (list owner & moderator, and other social media tables). So if a mailman-web dpkg update also updated the database, then Debian's update clearly did not properly log the changes. Or maybe they were logged elsewhere like in /var/lib/mailman3 in the old machine which also needed to be carried over to the new - so updating on a new machine involved copying over not just the database, but some other files that provide db state? [...] > > After all, I would expect that an export of the archive to mbox, > > followed by a delete of the archive, followed by a > > hyperkitty_import of the archive, should leave you at the same > > place. > > You would expect, for sure. You would be wrong, because mbox is a > lossy format by design. (Or by lack of design, if you prefer.) I'll stick with "whatever mechanism you provide to produce an export should include a mechanism that can import it". It's like: Here is a way you can back up your data (through an mbox export), but by the way, the format we back it up in is broken so restores (mbox import) are not guaranteed. That's just broken logic IMHO, especially since the "fix" in this case is so simple. Yes, mbox is broken in a multitude of ways, but if hyperkitty is going to produce an mbox export of a list that is stored in a database (i.e. it has to construct the mbox), hyperkitty ought to do so in a format that it can at least re-import. My 2 pence worth... Of course this may all work for you, if you are using other backend combinations... I should mention that I had to re-apply and rework some of my old patches back into Debian's libs. For example, update_index_one_list fails when using xapian with "Key too long" errors, so I had to re-apply the patch I contributed in https://gitlab.com/mailman/hyperkitty/-/issues/322. Sad that I am now of the age where I search for an issue only to discover I provided a patch for it 2 years ago ;-( -- Alex

1 year, 7 months

[MM3-users] Re: Postorius claims 2 messages held when there are none - out of sync

by Mark Sapiro

On 6/26/23 11:53 AM, Ken Alker wrote: > > Is the var/messages hierarchy also where all the archived emails are > stored for future viewing via the web interface? No. They are stored in the hyperkitty_email and hyperkitty_attachment tables in the database. >> Also, there is a long thread at >> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/threa >> d/UWCUQYU7Q5X5LWNII57ZZ4DDXE4DEFX7/ and maybe others that might be >> relevant. > > I have never used the mailman shell before, but based on the referenced > thread I learned to do this (note that the thread also taught me how to > rewrite it to narrow down to just the list with issues so I didn't > delete held messages in other lists, if my case): > > for verification: > > venv) mailman@speedy:~/web/logs$ mailman shell -l > sbarc-list(a)lists.netlojix.com > Welcome to the GNU Mailman shell > Use commit() to commit changes. > Use abort() to discard changes since the last commit. > Exit with ctrl+D does an implicit commit() but exit() does not. > The variable 'm' is the sbarc-list(a)lists.netlojix.com mailing list > >>> requests = IListRequests(m) > >>> first = True > >>> for request in requests.held_requests: > ... key, data = requests.get_request(request.id) > ... if first: > ... first = False > ... print(m.list_id) > ... print (f"""\ > ... Sender: {data['_mod_sender']} > ... Subject: {data['_mod_subject']} > ... Date: {data['_mod_hold_date']} > ... Reason: {data['_mod_reason']} > ... """) > ... > and then to delete the two messages: > > (venv) mailman@speedy:~/web/logs$ mailman shell -l > sbarc-list(a)lists.netlojix.com > Welcome to the GNU Mailman shell > Use commit() to commit changes. > Use abort() to discard changes since the last commit. > Exit with ctrl+D does an implicit commit() but exit() does not. > The variable 'm' is the sbarc-list(a)lists.netlojix.com mailing list > >>> requests = IListRequests(m) > >>> first = True > >>> for request in requests.held_requests: > ... key, data = requests.get_request(request.id) > ... if first: > ... first = False > ... print(m.list_id) > ... print(f'request id: {request.id}, deleting') > ... requests.delete_request(request.id) > ... > sbarc-list.lists.netlojix.com > request id: 27, deleting > request id: 28, deleting > >>> > > The result of this, however, was that the web interface still claimed I > had two held messages. I wasn't sure if there was some cron-type > "sweeper" process I had to wait to auto-run or if I had to manually do > something to create an update. I then realized that the sample script > had a ' >>> commit ()" ' at the end, which I'd missed. So, I re-entered > the script (but it didn't spit out that it deleted anything this time) > and got an error when I did the commit. Was this in the same `mailman shell` interaction or a new one? What was the error? > So, I figured the messages HAD > been delete and the commit wasn't really necessary. When I went back to > the web interface, the number of held messages was at 0 and I no longer > get the original error when clicking on "Held messages", so my issue is > solved. But this does leave me with some educational questions: > > 1) Did I affect something somehow with my "commit", or was there a job > that cleaned things up (if so, what job)? I'm not sure of the sequence of events, but I think you probably looked in Postorius before you quit `mailman shell` and then you quit `mailman shell` with ctrl-D which did an implicit commit. > 2) Is the "commit" even necessary (I saw it in someone else's script in > the thread)? It is necessary to explicitly commit() or quit with ctrl-D to actually update the database. > 3) Is there a way to save a script and then execute it via shell rather > than typing it in every time (it appears from the examples that people > are typing them in every time, which is super tedious; I didn't paste > for fear of doing something damaging, but maybe that would work)? Yes, you can save a script in the bin/ directory in your venv and run it with `mailman shell -r`. See `mailman shell --details`. > 4) Are the request.id's of the messages dynamic/changing, or are they > fixed for life? I thought I'd delete based on exact request.id without > looping but I was afraid maybe they were changing over time and that > this wouldn't be a good idea. Might be nice to know for the future. The request.id is constant for that request object. The request object is fixed for a specific held message request. I.e., a specific held message's request object and its request.id and other attributes are fixed until that request is handled. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 9 months

[MM3-users] Re: mailman 2.1 or 3.0 (or another MLM)

by nate

(In case the list admin/moderator sees this message I sent this email again from the wrong FROM: address a few minutes ago, re-sending with the right one now, please discard that bad message(have not gotten confirmation it was held for moderation) -- side note with MM2 it gave the user the option to discard their own message if it was held for moderation if I recall right. My very first post to this list was held for moderation (sent from correct FROM: address), not sure why it was held(it got through eventually), was assuming that perhaps I triggered a bug in that the fact I am using the text mailman in my email address maybe tripped something I don't know, but I was seemingly unable to view the status/discard that message.) On 2022-01-26 7:22, David Newman wrote: > Yes, this is possible. The IETF, the body that defines Internet > technical standards, is still on MM2. I’m with them; given my druthers > I’d run MM2 forever because it Just Works. > > Then again, I put a high priority on keeping the base OS and > applications patched against current security vulnerabilities. That’s > getting harder to do with MM2. > > Debian and FreeBSD have deprecated Python 2. MM2 still has maintenance > releases but no new development. MM2 also has some archaic design > decisions such as sending user passwords in plaintext email. > > Like it or not, MM3 and related components are current, and currently > maintained. I wasn’t crazy about the learning curve to get MM3 > working, but I pressed on because it will provide a platform I can > keep running much farther into the future. Thanks for the insight, I read another post from someone yesterday saying they had 35 years of experience and were having a bunch of issues, much more serious situation(real users impacted) than mine. I think I will go the route of MM2, probably setup a new VM and get postfix to just route the email for MM2 to the other system for processing. Then I guess I would need some proxypass rules in Apache to forward the cgi requests to the other system. I don't recall any serious security issues over the years with MM2 (at least ones that made tech news), grand scheme of things for my use case I think it would be ok, just stick with Devan 3.x for MM2 for the foreseeable future. Feels super weird building a dedicated system for just this but I have plenty of hardware resources to do it. I already have an LXC container at home for a legacy software product (TV Mobili, AFAIK last version came out in 2015) on Devuan 3 which I never plan to upgrade just to be safe. Time for one more, well this case would be a VM running at my colo. Hopefully can work on it today, though am not in any rush. thanks nate

3 years, 2 months

[MM3-users] Re: Archive not updated

by Marc SCHAEFER

Hello, On Mon, Sep 02, 2024 at 09:08:09AM -0000, mailman3--- via Mailman-users wrote: > Debian bookworm Archival works here. Here is my simplified auto-deployement script (some might not be useful to you, I am using a local Postfix): PS: replace services_install by apt-get install djangoauth=@@@REPLACE # this is a bash array: server, user, pw psqlauth=@@@REPLACE services_install postfix (while read line do case $line in *=*) postconf -e "$line";; *) echo "TOSSING: $line" >&2;; esac done) <<EOF myhostname = $lists mydestination = $lists mynetworks = 127.0.0.0/8 inet_protocols = ipv4 inet_interfaces = all transport_maps = hash:/var/lib/mailman3/data/postfix_lmtp local_recipient_maps = hash:/var/lib/mailman3/data/postfix_lmtp relay_domains = hash:/var/lib/mailman3/data/postfix_domains relayhost = @@@REPLACE EOF cat >> /etc/aliases <<EOF postorius: root www-data: root root: ${djangoauth[1]} EOF newaliases services_install dbconfig-pgsql cat > /etc/dbconfig-common/mailman3.conf <<EOF dbc_install='true' dbc_upgrade='true' dbc_remove='false' dbc_dbtype='pgsql' dbc_dbuser='${psqlauth[1]}' dbc_dbpass='${psqlauth[2]}' dbc_dbserver='${psqlauth[0]}' dbc_dbname='mailman' dbc_dbadmin='postgres' EOF sed < /etc/dbconfig-common/mailman3.conf \ > /etc/dbconfig-common/mailman3-web.conf \ "s/^dbc_dbname=.*/dbc_dbname='mailmanweb'/" chmod 600 /etc/dbconfig-common/mailman3.conf \ /etc/dbconfig-common/mailman3-web.conf services_install mailman3-full mailman3-web sed --in-place "s/^site_owner: .*/site_owner: ${djangoauth[1]}/" \ /etc/mailman3/mailman.cfg # archiving mails from mailing-lists automatically cat >> /etc/mailman3/mailman.cfg <<'EOF' [archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /etc/mailman3/mailman-hyperkitty.cfg [styles] default: private-default EOF # enable archiver key sed -i "s%^api_key:.*%api_key: $(egrep MAILMAN_ARCHIVER_KEY /etc/mailman3/mailman-web.py | awk '{print $NF;}' | sed "s/'//g")%" /etc/mailman3/mailman-hyperkitty.cfg # work-around: SQLAlchemy no longer supports "postgres" sed --in-place "s%^url: postgres://%url: postgresql://%" \ /etc/mailman3/mailman.cfg sed --in-place "s/.*'Mailman Suite Admin', 'root@localhost').*/('Mailman Suite Admin', '${djangoauth[1]}'),/;s/^EMAILNAME\s*=.*/EMAILNAME = '$lists'/;/django_mailman3.lib.auth.fedora/d" /etc/mailman3/mailman-web.py # aliases mailman <-> postfix mkdir -p /var/lib/mailman3/data chown list:list /var/lib/mailman3/data for i in postfix_lmtp postfix_lmtp postfix_domains do touch /var/lib/mailman3/data/$i postmap /var/lib/mailman3/data/$i done chown list:list /var/lib/mailman3/data/postfix_* # work-around for django_q # (does not seem to work, and DEFAULT_AUTO_FIELD should be also # configured to django.db.models.BigAutoField) dpkg-divert --rename /usr/lib/python3/dist-packages/django_q/conf.py sed < /usr/lib/python3/dist-packages/django_q/conf.py.distrib \ > /usr/lib/python3/dist-packages/django_q/conf.py \ 's%TIMEOUT = conf.get("timeout", None)%TIMEOUT = conf.get("timeout", 59)%' pkill -e -2 --full 'python3 manage.py qcluster' || true # fix example.com PGPASSWORD=${psqlauth[2]} psql -h ds-services-psql -U ${psqlauth[1]} mailmanweb -c "UPDATE django_site SET domain = '$lists', name = '$lists' WHERE id = 1" # workaround required? #chgrp list /var/lib/mailman3/locks/ #chmod g+rwx /var/lib/mailman3/locks/ chgrp list /var/log/mailman3/*.log # https://docs.djangoproject.com/en/3.0/ref/django-admin/#django-admin-create… env DJANGO_SUPERUSER_PASSWORD=${djangoauth[3]} \ DJANGO_SUPERUSER_USERNAME=${djangoauth[2]} \ DJANGO_SUPERUSER_EMAIL=${djangoauth[1]} \ mailman-web createsuperuser --noinput \ || echo "django super user already exists in DB, ignored" # then I migrate the lists

7 months, 2 weeks

[MM3-users] Re: A little stuck with installation of MM3 - ModuleNotFoundError: No module named 'flufl.lock'

by Odhiambo Washington

On Sun, 26 Jul 2020 at 00:17, Mark Sapiro <mark(a)msapiro.net> wrote: > On 7/25/20 12:54 PM, Odhiambo Washington wrote: > > > > Finally, I have adapted your init script to get some rudimentary one > that I > > could use on FreeBSD. > > I have to change /opt/mailman to be owned by mailman3:mailman3 (because I > > have "live" MLs on this server, using mailman-2.1.34. > > I am not sure if they can co-exist. I suppose they could, but what might > be > > the security implication, if any?? > > I manage more than one server that supports both Mailman 2.1 and Mailman > 3 running as user `mailman`. I'm not aware of any security issues. > > > > This is what the init script looks like (rudimentary!!): > > (venv) [root@gw /usr/local/etc/rc.d]# less mailman3 > > ### BEGIN INIT INFO > > # Provides: GNU Mailman > > # Short-Description: Mailman3 Service > > # Description: service control for Mailman3 > > ### END INIT INFO > > > > > PATH=/opt/mailman/mm/bin:/opt/mailman/mm/venv/bin:/usr/sbin:/usr/bin:/bin:/sbin: > > DESC="GNU Mailman service" > > DAEMON=/opt/mailman/mm/bin/mailman > > NAME=mailman > > USER=mailman3 > > GROUP=mailman3 > > > > # Needed by click > > export LANG=en_US.UTF-8 > > > > # Exit if the package is not installed > > [ -x "$DAEMON" ] || exit 0 > > > > # Load the VERBOSE setting and other rcS variables > > #. /lib/init/vars.sh > > > > # Define LSB log_* functions. > > # Depend on lsb-base (>= 3.2-14) to ensure that this file is present > > # and status_of_proc is working. > > #. /lib/lsb/init-functions > > > > case "$1" in > > start) > > [ "$VERBOSE" != no ] && echo "Starting $DESC" "$NAME" > > # use --force to remove a stale lock. > > /usr/local/bin/sudo -u $USER $DAEMON start --force > > T don't recommend using --force in init scripts. It *shouldn't* matter > because even with --force the master shouldn't break the lock if the pip > that set it still exists, but I prefer not to use it. > > > > ;; > > stop) > > [ "$VERBOSE" != no ] && echo "Stopping $DESC" "$NAME" > > /usr/local/bin/sudo -u $USER $DAEMON stop > > ;; > > status) > > /usr/local/bin/sudo -u $USER $DAEMON status > > ;; > > reopen) > > /usr/local/bin/sudo -u $USER $DAEMON reopen > > ;; > > restart) > > log_daemon_msg "Restarting $DESC" "$NAME" > > /usr/local/bin/sudo -u $USER $DAEMON restart > > ;; > > *) > > echo "Usage: $SCRIPTNAME {start|stop|status|reopen|restart}" >&2 > > exit 3 > > ;; > > esac > > > > It does start mailman3 for sure, but also complains after a few seconds > > with the message: > > > > (venv) [root@gw /usr/local/etc/rc.d]# > > > */opt/mailman/mm/venv/lib/python3.7/site-packages/mailman-3.3.2b1-py3.7.egg/mailman/rest/wsgiapp.py:180: > > DeprecatedWarning: Call to deprecated function __init__(...). API class > may > > be removed in a future release, use falcon.App instead.* > > * **kws)* > > > Hmm... strange, My various MM 3 installs are running falcon 1.4.0, 2.0.0 > and 3.0.0a1 and I don't see this warning, but it is only a deprecation > warning. It doesn't mean it won't work. > > > > But: > > > > (venv) [root@gw /usr/local/etc/rc.d]# ps ax | grep mailman > ... > > It seems Mailman core and the rest runner and its workers are all running. > > > > > > Assuming that I am using mod_wsgi with Apache, and that I have configured > > apache right using > > > https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=At… > > .. > > should I be able to access the MM3 web UI?? > > Yes, I think so. > > > > At that point, I am feeling somewhat confused as to what to try next. > > > > (venv) [root@gw /opt/mailman/mm/var/logs]# less mailman.log > > Jul 25 22:53:16 2020 (40670) Master started > > Jul 25 22:53:18 2020 (42036) bounces runner started. > > Jul 25 22:53:18 2020 (45205) out runner started. > > Jul 25 22:53:19 2020 (41191) archive runner started. > > Jul 25 22:53:19 2020 (46670) retry runner started. > > Jul 25 22:53:19 2020 (44567) nntp runner started. > > Jul 25 22:53:19 2020 (43069) in runner started. > > Jul 25 22:53:19 2020 (46960) virgin runner started. > > Jul 25 22:53:20 2020 (45720) pipeline runner started. > > Jul 25 22:53:20 2020 (47326) digest runner started. > > Jul 25 22:53:20 2020 (43755) lmtp runner started. > > Jul 25 22:53:20 2020 (45922) rest runner started. > > [2020-07-25 22:53:20 +0300] [45922] [INFO] Starting gunicorn 20.0.4 > > [2020-07-25 22:53:20 +0300] [45922] [INFO] Listening at: > > http://127.0.0.1:8001 (45922) > > [2020-07-25 22:53:20 +0300] [45922] [INFO] Using worker: sync > > [2020-07-25 22:53:20 +0300] [54732] [INFO] Booting worker with pid: 54732 > > [2020-07-25 22:53:20 +0300] [55467] [INFO] Booting worker with pid: 55467 > > Jul 25 22:53:21 2020 (42743) command runner started. > > > The above is all fine, but it is just Mailman core. > > > > How do I access the web UI for MM3 now? > > > The Apache config points to /opt/mailman/mm/wsgi.py, a sampole of which > is at > < > https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=At… > >. > Do you have that? > Yes, I have that file. > If you installed the Apache config literally, the location for Mailman 3 > is defined by > < > https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=At… > > > and the URL would be http(s)://your.server/mm3 - have you tried that? > > There is still some confusion on my part about the directives in the file so allow me to seek some clarifications. The following are the contents of a file I have created and placed in my Apache Includes/ directory. I was hoping that with it, I can now access http://lists.my.server/mm3 and get the UI. <CUT> # Global section WSGIDaemonProcess mailman-web display-name=mailman-web maximum-requests=1000 umask=0002 user=mailman3 \ group=mailman3 python-path=/opt/mailman/mm/venv/lib/python3.7/site-packages:/opt/mailman/mm/venv/lib/python3.7 \ python-home=/opt/mailman/mm/venv home=/opt/mailman/mm/var WSGIRestrictSignal Off <VirtualHost *:80> ServerName lists.my.server ServerAdmin odhiambo(a)gmail.com # (I'm not sure that WSGIRestrictSignal Off is required, but it was in the # provided example so I kept it. I also made changes to WSGIDaemonProcess # based on my own mod_wsgi experience elsewhere.) ErrorLog /var/log/myserver-error.log LogLevel debug # This goes in the VirtualHost block for the domain. # Mailman 3 stuff Alias /static "/var/spool/mailman-web/static" *<----- Where is this directory supposed to be and what/who creates it and with what permissions?* <Directory "/var/spool/mailman-web/static"> Require all granted </Directory> WSGIScriptAlias /mm3 /opt/mailman/mm/wsgi.py <Directory "/opt/mailman/mm/"> <Files wsgi.py> Order deny,allow Allow from all Require all granted </Files> WSGIProcessGroup mailman-web </Directory> </VirtualHost> </CUT-HERE> I end up with an "Internal server error" and from the error log I see: [Sun Jul 26 12:04:43.444127 2020] [wsgi:info] [pid 6444] [remote 197.232.81.246:53383] mod_wsgi (pid=6444, process='mailman-web', application='lists.my.server|/mm3'): Loading Python script file '/opt/mailman/mm/wsgi.py'. [Sun Jul 26 12:04:44.091922 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] mod_wsgi (pid=6444): Failed to exec Python script file '/opt/mailman/mm/wsgi.py'. [Sun Jul 26 12:04:44.092006 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] mod_wsgi (pid=6444): Exception occurred processing WSGI script '/opt/mailman/mm/wsgi.py'. [Sun Jul 26 12:04:44.092940 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] Traceback (most recent call last): [Sun Jul 26 12:04:44.093019 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/wsgi.py", line 38, in <module> [Sun Jul 26 12:04:44.093029 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] application = get_wsgi_application() [Sun Jul 26 12:04:44.093048 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/core/wsgi.py", line 12, in get_wsgi_application [Sun Jul 26 12:04:44.093057 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] django.setup(set_prefix=False) [Sun Jul 26 12:04:44.093071 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/__init__.py", line 19, in setup [Sun Jul 26 12:04:44.093090 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) [Sun Jul 26 12:04:44.093108 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/conf/__init__.py", line 76, in __getattr__ [Sun Jul 26 12:04:44.093117 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] self._setup(name) [Sun Jul 26 12:04:44.093130 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/conf/__init__.py", line 63, in _setup [Sun Jul 26 12:04:44.093138 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] self._wrapped = Settings(settings_module) [Sun Jul 26 12:04:44.093158 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/conf/__init__.py", line 142, in __init__ [Sun Jul 26 12:04:44.093166 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] mod = importlib.import_module(self.SETTINGS_MODULE) [Sun Jul 26 12:04:44.093179 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module [Sun Jul 26 12:04:44.093187 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] return _bootstrap._gcd_import(name[level:], package, level) [Sun Jul 26 12:04:44.093210 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "<frozen importlib._bootstrap>", line 994, in _gcd_import [Sun Jul 26 12:04:44.093224 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "<frozen importlib._bootstrap>", line 971, in _find_and_load [Sun Jul 26 12:04:44.093239 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked [Sun Jul 26 12:04:44.093265 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] ModuleNotFoundError: No module named 'settings' -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

4 years, 8 months

[MM3-users] Re: Apache shows No Lists

by Mark Sapiro

On 5/22/23 14:48, ceo(a)sartrack.nz wrote: > Thanks; > There has never been a 'Mailman' user. I do have a www-data user (all Apache2 data' and there is a 'list' user. So what I refer to as the Mailman user (a generic term) is `list` in your case. > I may have run the import fuction as root, but I then used chown() to set the directory tree /var/lib/mailman3/* to user 'list'. OK > So I upgraded Debian 10 to Debian 11. > This also upgraded Apache2 I assume, but I kept the original (single) config file. > > I then installed the whole Mailman3 package with 'apt'. I may also have installed additional missing bits afterwards, as I think it was not complete. I initially tried to use MariaDB but the installation failed completly because it never asked for the MariaDB root password, and as a result of that everyting else failed too. > Then I un-installed 5 times, and then I finally went for the 'default' database system. > > I then edited the file with the recommended settings for Mailman3: > Alias /mailman3/favicon.ico /var/lib/mailman3/web/static/postorius/img/favicon.ico > Alias /mailman3/static /var/lib/mailman3/web/static > > <Directory "/var/lib/mailman3/web/static"> > Require all granted > </Directory> > > <IfModule mod_proxy_uwsgi.c> > ProxyPass /mailman3/favicon.ico ! > ProxyPass /mailman3/static ! > ProxyPass /mailman3 unix:/run/mailman3-web/uwsgi.sock|uwsgi://localhost/ > </IfModule> > > So clearly Apache2 manages to access the data supplied by Postorius because it sees the lists (at least the empty ones). > -What user should the whole system running under: 'list'? of 'www-data'? > - Is there anything else I should tell Apache2? All the web based stuff is a Django application. Django is run by uWSGI which probably runs as www-data, but Django only accesses hyperkitty, postorius and django-mailman3 and other Django related tables in the database. It doesn't need access to file system files other than things like wsgi.py, urls.py, settings.py and settings_local.py and only read access to these. All the mailman3 file system stuff has to be read/write/searchable by `list`. > Now I tried to use "django-admin createsuperuser". This also fails because of this error: > ImportError: cannot import name 'force_text' from 'django.utils.encoding' (/usr/local/lib/python3.9/dist-packages/django/utils/encoding.py) > And I cannot find where this 'force_text' is called from. It must be removed as it is too old. It seems your Django installation is broken. Do cd /usr/local/lib/python3.9/dist-packages/django grep -r force_text to find where force_text is called, or maybe add def force_text(s, encoding='utf-8', strings_only=False, errors='strict'): return force_str(s, encoding, strings_only, errors) to /usr/local/lib/python3.9/dist-packages/django/utils/encoding.py -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 11 months

[MM3-users] Re: Broken web UI after upgrade

by Guillermo Hernandez (Oldno7)

El 14/12/22 a las 18:16, Odhiambo Washington escribió: > On Wed, Dec 14, 2022 at 8:02 PM Stephen J. Turnbull < > stephenjturnbull(a)gmail.com> wrote: > >> Odhiambo Washington writes: >> >> > The site is public, so I can share the link: >> https://list.rc.org/archives/ >> >> The CSS is not working. Two possibilities to try first: >> 1. (more likely) You didn't run the command that compiles the CSS. >> (probably) mailman-web compress >> 2. (much less likely) The CSS got replaced in the upgrade and the >> ownership changed because you were working as root, and the server >> process can't access the files. >> 3. other possibilities: the SASS compiler (usually sassc, possibly sass) >> >> > 1. The post-update script runs everything required. For the avoidance of > doubt, here it is: > <CUT> > #!/bin/bash > > set -e > > BINDIR="/opt/mailman/mm/bin" > CONFDIR="/opt/mailman/mm" > SETTINGS="settings" > DJANGO_CONTROL_SCRIPT="$BINDIR/django-admin" > STATIC_DIR="$CONFDIR/static" > DEBUG="False" > > set -x > > if [ "$DEBUG" == "False" ]; then > mkdir -p "$STATIC_DIR" || : > "$DJANGO_CONTROL_SCRIPT" collectstatic --clear --noinput --verbosity 0 > "$DJANGO_CONTROL_SCRIPT" compress Odhiambo, I had problems in the compress process in the last updates. I would make sure the process is running ok. I had to disable the off line compression for this matter. > "$DJANGO_CONTROL_SCRIPT" compilemessages > fi > > syncdb_cmd="$DJANGO_CONTROL_SCRIPT migrate" > if [[ ! -t 1 ]]; then > # stdout is not a terminal > syncdb_cmd="$syncdb_cmd --noinput" > fi > $syncdb_cmd > echo 'Do you want to rebuild the search index? This takes a long time.' > echo 'Type "Yes" to continue; anything else to quit.' > read -p "> " response > if [ "x$response" != "xYes" ] ; then exit; fi > "$DJANGO_CONTROL_SCRIPT" update_index > </CUT> > > The script is run within the venv. > > 2. That isn't possible. Why? Because when I do 'su -mailman' it > automatically activates the venv via a .bashrc. > I have run this: > find /opt/mailman -type f -name "*.css" -exec ls -al {} \; > > And all files are owned by mailman.mailman. > > 3. I have the following in settings_local.py: > COMPRESS_PRECOMPILERS = ( > ('text/x-scss', 'sass -t compressed {infile} {outfile}'), > ('text/x-sass', 'sass -t compressed {infile} {outfile}'), > ) > COMPRESS_OFFLINE = True > COMPRESS_ENABLED = True > > And: > root@list:/home/wash# su - mailman > (venv) mailman@list:~$ which sass > /usr/bin/sass > (venv) mailman@list:~$ `which sass` --version > Ruby Sass 3.7.4 > (venv) mailman@list:~$ > > Anything else that might be the issue? > -- ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: firma-GHP-emails.png Replaced multipart/alternative part with first alternative.

2 years, 4 months

[MM3-users] Re: add custom styles to postorius list creation menu

by mhazan＠mandint.org

Le 2022-05-03 09:44, mhazan(a)mandint.org a écrit : > Le 2022-05-02 17:32, Mark Sapiro a écrit : >> On 5/2/22 06:39, Michael Hazan wrote: >>> Hello, >>> >>> When creating a list via postorius, i can see 3 style appear >>> (announce, discussion, private). Is it possible to either create a >>> new style and add it to that list or modify one of those style ? >> >> >> See this thread >> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… >> >> > > Thanks i will look into setting up that plugin, hopefully an easier > way to do that can make it's way to the default install. working on that plugin, i have found where to inserts the various setting, however it's not clear what types and possibles value are excepted for each ? ie "advertised" is bool True/false, but what about "default_member_action" ? i've tried hold 'hold' "hold" all throw error when trying to apply the style. i cannot find the types in the documentation and all the other non bool settings. >>> Also, coming from mailman2, you could directly set the members post >>> moderation action when subscribing, now you can only do it via the >>> user menu, is there a way to automate this in mm3 ? >> >> >> I don't understand what you mean here. In MM 2.1 a member had only one >> setting - moderated yes or no and if moderated there was a list >> setting for the moderation action. There was also a list setting for >> whether new members were moderated by default. How were you setting >> the new member's moderation when subscribing? >> >> You are correct that when subscribing a new member, the member gets >> the default action for members and different action has to be set in a >> separate operation. I don't think this is significantly different from >> MM 2.1. Also, can you describe what you mean by "automate this". > > in MM2.1, if you set the list to moderated and default action to hold > posting, you can set a some member to be un-moderated (mod bit off) to > automatically allow their posting. In the web interface of MM2.1 there > is a checkbox with moderated bit on/off on the member list itself, in > the postorius interface for MM3 you have to open each members settings > page to access that. For automation, as list owner, upon registering a > member (via mass subscription) the possibility to change that ? also > the me too/dupe setting for example. > > Thanks > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

2 years, 11 months

[MM3-users] Re: Login confusion

by Stephen J. Turnbull

Markus Grandpré writes: > Dear Steve, dear list > > > That's not true on the systems I've worked with most recently -- those > > will be recognized as duplicate addresses (...) > > You are right. I am sorry for this confusion. Please let me correct my > initial posting to this list from yesterday, saying: > > >> (...) <markus.grandpre(a)uni-konstanz.de> and > <Markus.Grandpre(a)uni-konstanz.de> are being recognized as separate > identities. > > That is not the case. Meanwhile I discovered an error with logging into > Postorius when an account is associated with two semantically identical > email addresses that differ in their formatting, e.g., Yes. Apparently this is a problem of a policy difference between Mailman 3 and the Django allauth package, as Bernhard Lichtinger pointed out in <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…> allauth takes the point of view that the localpart of an email address is the "property" of the provider of the mailbox, and therefore should be taken verbatim by third parties. Mailman takes the more practical point of view that, in practice, almost all providers treat localparts as case insensitive. (Some -- Gmail, grrrrr -- go farther and ignore whole characters such as periods.) The problem is that users treat email addresses as case insensitive, and aren't very careful about using the same case every time. That's why I think the Mailman approach is appropriate. > To resolve the issue, the only solution I found was to manually delete > the second email address from the database: That will work in the short run, but I'm not sure that's a long-term solution. It's not clear to me that allauth won't do the same thing again. > Could I have achieved the same deletion using the REST interface? I don't know. I would think not. I suspect Mailman code can only find the lowercase version. Mark has a script that looks for such duplicates using psycopg, not mailmanclient, so it accesses the database via the PostgreSQL API, not Mailman's REST API. It deletes any duplicate that is not all-lowercase: https://github.com/pennersr/django-allauth/issues/3019#issuecomment-2440231… > I am not entirely sure how the duplicate email address came into the > system. allauth did it. It's explained in the issue where Mark's script is attached. It's not very easy to follow the discussion, though. And there's something weird going on because in your stack trace, it appears that allauth is doing a case insensitive database search. So I would just grab the script and run it, in case there are other users with extra addresses that compare equal when comparison is case insensitive. > I apologize again for the incorrect information in my initial message to > this list. No need to apologize. It's extremely complicated. Of course we're happy if you look around and figure out most of it for us! Steve

2 months, 1 week

[MM3-users] Re: problems installing mailman3 under gentoo and upgrading mailman2 to mailman3

by John Covici

On Sat, 13 Mar 2021 05:42:53 -0500, Guillermo Hernandez (Oldno7) via Mailman-users wrote: > > On 13/3/21 9:44, John Covici wrote: > > On Sat, 13 Mar 2021 03:25:41 -0500, > > Guillermo Hernandez (Oldno7) via Mailman-users wrote: > >> On 13/3/21 8:53, John Covici wrote: > >>>> We recommend installing Mailman and the web components in a virtualenv, > >>>> but it is not a requirement. We also suggest that postgresql is a better > >>>> choice than MariaDB. > >>>> > >>> I sort of am installing from source, gentoo installs from source. No > >>> one on the gentoo users mailing list seems to be able to help me, > >>> except to tell me to use a different mailing list mechanism called > >>> mlmmj. > >>> > >> From the perspective of someone who was in the same place (but > >> installing in a FreeBSD S.O.), I would recommend you to follow > >> the "virtualenv" guide but without executing the "env" > >> commands. It will produce a system wide mailman3 installation but > >> it will work (as it is for me since a few moths ago) > >> > >> I writed down a step by step path of my installation and it could > >> be of your interest (maybe you have to locate and install some > >> packages by different name, but I think it will be not much > >> difference). It could provide you a overall view of all the > >> processes implied. > >> > >> I'ts here: https://forums.FreeBSD.org/threads/mailman-3.61050/post-488128 > > Thanks, by the env commands you mean the creation and activation > > commands? > > I mean the part when you activate the virtualenv (Virtualenv > setup & activate virtualenv) in > https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-inst… > > The servers in which I was migrating from mailman2 were jailed > servers and I didn't want to add more virtualization layers (and > I read some articles documenting problems using virtualenv in > FreeBSD) so, I prefered to install all of it system wide as it > was before with mailman2. > > > > > I will certainly look at your thread. > > Now one thing came up while trying to create the database, I got a traceback complaining about a revision error because gentoo had a slightly different version (3.3.2) instead of 3.3.3 using pip. When I upgrade mailman, am I going to have to do something to the database as well and why should the database have some kind of reference to the mailman version? -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici wb2una covici(a)ccs.covici.com

4 years, 1 month

[MM3-users] Re: Deactivate list without deleting it?

by Colin Watson

On Wed, Jan 17, 2018 at 04:34:48PM +0900, Stephen J. Turnbull wrote: > Colin Watson writes: > > I possibly should have explained our situation in a little more > > detail. > > Well, I was pretty clear on your requirements, it turns out. I don't, > and I believe I'm in agreement with the rest of the team here, need to > know why you want something (at least not at this "let's see if we can > hash out the requirements and the design a bit" stage). After all, > even if we end up WONTFIXing it, it's open source and we're happy to > talk about how you might do something even if we won't do it for you! I appreciate that approach. At the same time, I also asked because I suspected I didn't quite understand the design principles involved. Thanks to both you and Barry for clarifying the intent of IArchiver; I'll drop that idea. > > Postorius wouldn't come into it in our case > > That's an interesting detail. Thing is, at present Postorius *is* the > "control panel" for the whole suite (all the best bands are run by > their bassists, you know), so we would bring it up if you left it out. I'm not ruling out using it in future, but right now I think we'd prefer to stick with the approach of having the "control panel" integrated tightly into LP. > However, if you don't want to use Postorius, I think the > mailman-client functionality will make it strightforward to implement > your UI within Launchpad, and it now includes a nicer CLI so if you > prefer to implement core and Hyperkitty functionality before the > Launchpad side, mailman-client will allow you to test core features at > that stage. Indeed. And I'd managed to miss the existence of mailmanclient, so thanks for bringing that up. > > I'd been thinking more of a new enum in HyperKitty for the visibility > > state, since it does seem like quite an archiver-specific thing. > > Well, there is an enum in core for it already. I'm not sure how that > works currently (specifically, how you go about telling Hyperkitty if > that changes). Keeping that in sync is an issue (at least for us), > since Postorius can query it, IIRC. I just looked into what HK does, and I notice that it syncs the archive policy from Core as part of a regular update job. So I think the answer is that we just use that: perhaps set the ArchivePolicy to private and arrange that nobody has the ability to view it other than maybe list owners, or possibly add a new item to ArchivePolicy that's something like "deactivated" or "hidden". At any rate that looks a lot more tractable since most of the mechanism already exists. Thanks, -- Colin Watson [cjwatson(a)ubuntu.com]

7 years, 1 month

[MM3-users] Re: User management page [was: Changing subscription address]

by Abhilash Raj

> On Mar 22, 2021, at 7:56 AM, Stephen J. Turnbull <turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote: > > Abhilash Raj writes: > >> Typo :( Page in Postorius is what I meant to say. > > :D No problemo, mon, I just did it too -> > >> Is that necessarily required? Every user in Django has a corresponding >> user in Core, but the converse isn’t true and I am not sure if that is even >> required. > > I meant "accessed from Postorius or HyperKitty." :-) > >> You will be able to see: >> - Does the user have a Django account? >> - All Subscriptions for the User >> - All addresses of the User >> >> You will be able to perform following actions: >> - Add or remove a user’s address >> - Verify or un-verify an address >> - Set any address as primary for the user >> - Update display name of the user >> - Update address on any of the subscriptions > > I don't think we *need* a Django account for what I'm thinking of. I > do think that creating the account on User access from Postorius might > make the implementation logic simpler. > >> What I am on fence about: >> - Provide a full blown preferences menu to the admin for that user. This >> will also make the UI extremely complex to see and implement. > > Wait to decide on that until I have a mockup or maybe a full MR. > >> But, when you have an account in Django with a different email >> (email2) and a different Mailman user (user2) and you add a second >> email (email1) to your account and are able to prove ownership by >> verifying the address, Django will ask Core to add that second address >> (email1) to your Mailman user (user2) and absorb “user1”. This >> process is already implemented. > > Good. > >> However, if two Django accounts are created for “user1” and “user2” >> then situation becomes more tricky and currently we don’t have a >> way to resolve that short of delete one of the accounts and add the >> addresses to other account. > > Delete "that" and add its attributes to "this" is what I had in mind. We could automate the absorb other account and copy all settings and subscriptions over in a user based workflow. I don’t see a lot of unknowns in implementing that. You have to manually do it today, and deleting account will delete subscriptions so when you add them to your other account you have to resubscribe. So, the path could really use some changes. I think I have a ticket open in P to allow deleting P account but not delete the Mailman User and subscriptions, which would make this manual process much better. -- thanks, Abhilash Raj (maxking)

4 years, 1 month

[MM3-users] Re: Unable to get mailman3 working on Ubuntu 20.04 LTS

by John Covici

I am interested in your docr instructions. I don't use postfix, instead I use sendmail, but I think I can figure that out, but I have been having lots of problems installing without docr, and I do have apache, so this would be great. Send off list if you think that would not be of general use. On Wed, 27 Oct 2021 19:51:17 -0400, Shawn Heisey wrote: > > On 10/26/2021 11:43 AM, Michael Heseltine wrote: > > after upgrading an older server to Ubuntu 20.04, I tried to first upgrade my old mailman 2 installation to mailman 3. I've been on it for three weeks now, but unable to get anything working. I have long forgone the idea of upgrading and thus purged everything from mailman2 on my machine, and installed and removed mailman3 several times to no avail. > > I just ran into a lot of trouble myself getting the Ubuntu > packages working. And I got the same advice you did from this > mailing list -- don't try to install it that way. > > My first attempt at installing it without using packages was the > virtualenv method. After beating my head against that for a few > hours and not making any progress at all, I gave up. > > I now have the docker install fully functional, and imported my > list configurations from data I saved from an ancient mailman2 > install. It was giving me grief too when I first installed it, > but I managed to work through each of the problems. One of the > issues was a basic networking problem -- mailman in the container > wasn't able to talk to Postfix on the host. Rebooting the host > cleared that up. Side note: There are no details in the docker > instructions about how to upgrade the containers to newer > versions. Easy upgrades was a big reason why I wanted to use the > Ubuntu packages. Upgrading other packages has always been pretty > straightforward. > > I'm using Apache, not nginx. It was already installed, and I > know more about configuring it than nginx. It wasn't too > difficult to set up Apache to proxy over to uwsgi running in the > web container. I had to work out the config myself ... the > docker instructions only have a config example for nginx. > > Email isn't very interactive, but works. You can find me on the > IRC channel for the next few days if you want something faster. > If you want to try the docker method, I can try to help. > > Thanks, > Shawn > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici wb2una covici(a)ccs.covici.com

3 years, 5 months

[MM3-users] Re: Can't post to my own list from HyperKitty

by Mark Sapiro

On 4/8/25 18:45, MegaBrutal via Mailman-users wrote: > > User-profile shows my GMail address as primary and my own-domain address as > secondary. The third address is not present, because I actually wanted to > get rid of it, and it seems I succeeded in that. This is your Django user which has the two addresses associated with it. > Mailman3/users shows me with the address I wanted to get rid of. If I click > Manage, I see all 3 addresses, with (for some reason) only the GMail one > being verified (strange because I'm sure I verified the other as well). And > here's something interesting: it shows me as a member with my GMail address > under subscriptions, but shows me with my megabrutal.com domain under > ownerships. These are your Mailman user and its addresses. They are not the same as your Django user and its addresses. I don't think you can unlink an address from your Mailman user in Postorius/HyperKitty. You can do that in `mailman shell`. ``` $ mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. >>> um = getUtility(IUserManager) >>> usr = um.get_user('user(a)example.com') # any of the three addresses >>> usr.addresses prints a list of the 3 addresses. These are indexed as 0, 1, 2 >>> usr.preferred_address prints the preferred address. If this is the one you want to delete, do >>> usr.preferred_address = usr.addresses[n] # n is the index of one you want >>> del usr.addresses[n] # n is the index of the address you don't want >>> usr.addresses.remove(n) # n is the index of the address you don't want >>> commit() ``` It may not be clear from the above, but the idea is that you are deleting the address you don't want from usr.addresses, but before that if the one you don't want is the preferred_address, you need to set the preferred address to one of the others. Also, `del usr.addresses[n]` deletes the address record, but it is still in the addresses list, thus the `usr.addresses.remove(n)`. This is untested, so before committing you may want to verify that `usr.addresses` now lists just the two addresses you want > This might be the cause? Maybe HyperKitty wants to use my owner identity, > which is not subscribed? Btw, is the owner not automatically considered as > someone who can post to the list, they also need to be subscribed? An owner is not a list member and has no special posting privileges. If only members are allowed to post, the owner address must also be a member to be allowed to post. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 weeks, 1 day

[MM3-users] Re: Unwanted E-mail to Maillists Submitted via Postorius Interface

by Stephen J. Turnbull

Brett Delmage writes: > On Thu, 26 Aug 2021, Stephen J. Turnbull wrote: > > > However, it turns out that bots are almost as good at solving captchas > > as humans, and they can retry a lot faster. Captchas are also really > > horrible from an accessibility perspective. > > I have to disagree. > > I previously put a captcha on a Joomla website which stopped 100% of all > bogus account signup attempts. > > What was it ? "How many letters are in the word 'jazz'"? > This captcha is fully accessible, too. Of course, point taken. I've done similar things, and know quite a few such anecdotes. It's a good method, as long as *only a few of us use it*. But if enough people use it, patterns will show up, and we'll get into a "proof of waste" race with the spammers as we try to come up with logic puzzles people can solve but spammers' ML can't. I don't think it really addresses the issue that attackers can solve it easily if they want to, and will eventually automate that solution, although it does address the issue of accessibility (mostly -- I am not an expert but I wouldn't be surprised if long-time screen reader users are relatively poor spellers!) And it's not a satisfactory solution for a lot of our users. We can't safely add enabling functions to Mailman, because I'm sure there are a dozen Spamming as a Service (SpAAS) groups out there with Mailman 3 clone repos. So you need a skilled admin to implement custom CAPTCHAs. > If there's a captcha problem out there, it's because too many people just > use the same old, same old - Of course they do. After all, that's what software is for -- very cheaply using others' solutions for common problems. In this case, though, the "problem" is smart -- it's human attackers and their tools. > which is worth the bots cracking because it is used everywhere. That evaluation is close, but not exact in the age of SpAAS. If they want you, they'll get you, it's trivial for an open-subscription site. And if they need a human to crack your trivial CAPTCHA, a smart spammer (or even somebody who hates you for banning them from your comments) will add it to their box of tricks. I bet you'll get tired sooner than they do. Eventually it will make it into kiddie-scripts. The other thing is that if such simple captchas become popular, you know that the pros will start scanning sites for them just to make a database of patterns, and add hacks for any popular ones (or if they're smart, for every pattern they find -- I bet on a regular basis they'll get as lucky as I did when I stopped Klez, and then Fretham too, with just "iframe.*height=1"). Steve

3 years, 7 months

[MM3-users] Re: Mailman3 not working following distribution upgrade from Debian 10 to 11.

by David Newman

On 1/25/22 2:59 PM, gordon(a)dickens.com wrote: > Thanks to David and everybody else that have replied to my problem. I have proceeded to uninstall the debian mailman3 packages and then endeavored to install everything with venv. I have implemented all of David's configuration suggestions which were very helpful. Now, mailman3 (core) appears to be working properly in that I can start/stop/restart/status the service. Also, the command "mailman info" from the venv prompt yields output which appears to be correct and the same as in the venv instructions. However, I have not been able to get mailman3-web working. One general comment: Debian packages install files with a different owner than the pip install. You want to be sure all your directories and files are owned by the Mailman user ('mailman' on Debian). > > All of my configuration files may be viewed here: > > https://dickens.com/etc-mailman3/ > > The above files are basically a mirror of the contents of my /etc/mailman3 directory except that I have obfuscated sensitive info such as keys, passwords and email addresses. > > Executing the mailman3-web service script with the command "systemctl start mailman3-web" starts everything without errors, however, the command "systemctl status mailman3-web" then generates the following output with errors: > > ● mailman3-web.service - GNU Mailman Web UI > Loaded: loaded (/etc/mailman3/mailman3-web.service; enabled; vendor preset: enabled) > Active: failed (Result: exit-code) since Tue 2022-01-25 17:10:30 EST; 6s ago > Process: 1718478 ExecStart=/opt/mailman/venv/bin/uwsgi --ini /etc/mailman3/uwsgi.ini (code=exited, status=1/FAILURE) > Main PID: 1718478 (code=exited, status=1/FAILURE) > CPU: 12ms > > Jan 25 17:10:29 slc.mailhub4u.com systemd[1]: Started GNU Mailman Web UI. > Jan 25 17:10:30 slc.mailhub4u.com uwsgi[1718478]: [uWSGI] getting INI configuration from /etc/mailman3/uwsgi.ini > Jan 25 17:10:30 slc.mailhub4u.com systemd[1]: mailman3-web.service: Main process exited, code=exited, status=1/FAILURE > Jan 25 17:10:30 slc.mailhub4u.com systemd[1]: mailman3-web.service: Failed with result 'exit-code'. > > Also, /var/log/syslog and "journalctl | grep mailman3-web" contain these two lines of errors: > > Jan 25 14:00:46 slc systemd[1]: mailman3-web.service: Main process exited, code=exited, status=1/FAILURE > Jan 25 14:00:46 slc systemd[1]: mailman3-web.service: Failed with result 'exit-code'. Do you have anything further in any of the logs in /opt/mailman/web/logs ? > > Also, other commands in the venv instructions do not work such as "mailman-web migrate" which produces volumes of error dialog starting with: > > Traceback (most recent call last): > File "/opt/mailman/venv/lib/python3.9/site-packages/django/template/utils.py", line 66, in __getitem__ > return self._engines[alias] > KeyError: 'django' This one might be due to a bug I hit as well. There's a problem with mistune, one of the components. If that's the issue here, this should address it, run as the mailman user: pip install mistune==2.0.0rc1 mailman-web migrate mailman-web compress mailman-web collectstatic mailman-web compilemessages dn

3 years, 2 months

Search results for query "dO"