Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: Is there a way of recording bounce messages?

by Mohsen Masoudfar

Thank you Mark! [EXTERNAL EMAIL] On 9/26/23 11:29, Mohsen Masoudfar wrote: > > I'm dealing with the same issue. Upon reviewing the logs, I've noticed a significant number of bounced emails for a particular mailing list. Gmail users have reported not receiving emails. > I haven't been able to locate the "munge from" information. My current setup is using Mailman 3.1.1. I read it has been introduced in 3.2. DMARC mitigations were introduced in Mailman 3.1.0. They should be available in 3.1.1. They were first exposed in Postorius 1.1.0. Where are you looking for DMARC mitigations? >> Sorry, It was my misunderstanding. DMARC mitigation is functioning well for newly created lists, even with Gmail addresses. >> I am suspicious that Gmail might be rejecting or bouncing emails from the list due to the reputation of those lists. >> It's worth noting that I am not the owner of the lists, but admin the Mailman3 infrastructure. My objective is to determine if our suspicious is accurate, whether Gmail is rejecting emails because of the bad reputation. We have asked Google, but no response so far. >> I am interested to know more about the reasons behind these bounces, they are all handled by Postfix and deleted. > The Question is what to do with Mailman 3.1.1, if the upgrade is not an option. > > I get in the logs messages like below with several Queue-ids: > > Sep 25 09:03:55 localhost postfix/smtpd[26692]: 147583E927: client=a14-93.smtp-out.amazonses.com[54.240.14.93] > Sep 25 09:03:55 localhost postfix/cleanup[26696]: 147583E927: message-id=<0100018acb94b326-fdf39193-144e-4fdc-8faf-0d3df54eb4ee-000000(a)email.amazonses.com> > Sep 25 09:03:55 localhost postfix/qmgr[1298]: 147583E927: from=<>, size=24934, nrcpt=1 (queue active) > Sep 25 09:03:55 localhost postfix/lmtp[26697]: 147583E927: to=<listname-bounces@servername>, relay=127.0.0.1[127.0.0.1]:8024, delay=0.03, delays=0.01/0/0/0.01, dsn=2.0.0, status=sent (250 Ok) > Sep 25 09:03:55 localhost postfix/qmgr[1298]: 147583E927: removed This only says that some Amazon MTA has relayed a bounce to Mailman. It says nothing about why the message bounced. >> Is there a way to send them to a mailbox? Or prevent them from being deleted, so that I can look into the message body? -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mai…<https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/> Archived at: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mai…<https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…> This message sent to mmasoudf(a)aaas.org

1 year, 9 months

[MM3-users] Re: Timeout error on anonymous subscribe

by Gerald Vogt

On 24.10.24 22:09, Mark Sapiro wrote: > On 10/23/24 23:36, vogt(a)spamcop.net wrote: >> The only difference the last times is that with curl, the server side >> sends the ACK for the request and milliseconds later a PSH,ACK with >> the response headers. When it comes internally from mailman it sends >> the ACK for the request and then 5 seconds later it sends the PSH,ACK >> with the response headers. > > I'm unable to explain what's going on. You could try one more thing in > an attempt to duplicate this outside of Mailman. Invoke the same python3 > that Mailman does - this may be in a venv or the system python3. If in a > venv, you don't need to activate the venv, just run venv/bin/python. > > Then in response to Python's prompts do > ``` > >>> import requests > >>> res = requests.get('https://lists.example.org/mailman3/api/ > templates/list/community.lists.example.com/list:user:action:subscribe', > timeout=5) > ``` > and see what that does? I think it should time out because this is > exactly what Mailman core is doing tho get the template, and if it does, > perhaps you can determine why from this simple example. I have tried it using the fqdn url as well as the localhost:8000 url but in both cases requests.get immediately returns the response. The server side pushes the response within milliseconds. But that made me looking another time at the localhost tcpdump: I have noticed two connections to localhost:8000 where gunicorn with mailman_web.wsgi:application is running. The first one is POST /mailman3/lists/community.lists.example.com/anonymous_subscribe the second one is GET /mailman3/api/templates/list/community.lists.example.com/list:user:action:subscribe Both take 5 seconds. First the first one responds then shortly after the second one. I have noticed that only two gunicorn processes are running which looked like only one is a worker. Thus I have added workers=9 (2xncore+1) to /etc/mailman3/gunicorn.conf and restarted mailmanweb.service. Now it's working. So it must have been some kind of deadlock situation because it only had a single worker running. I have just followed the docs from https://docs.mailman3.org/en/latest/install/virtualenv.html#setting-up-guni… which doesn't mention anything about workers and I seriously don't have a clue about gunicorn. Maybe that could be added to the docs? Thanks for the help! Cheers, Gerald

8 months

[MM3-users] Re: Multi-domain oddities in Hyperkitty and Postorius

by Jeremy Stanley

On 2023-02-15 18:35:35 -0800 (-0800), Mark Sapiro wrote: > On 2/15/23 07:52, Jeremy Stanley wrote: > > > > Thanks, I'm still doing my best to wrap my brain around how Django > > uses database migrations to create sites (seems like a very strange > > use of a migration, I'm only accustomed to seeing them applied for > > database schema updates). > > I don't think it does. There are a couple of migrations at > django/contrib/sites/migrations, but these are only the initial creation of > the model and an update to make the domain field unique. > > Why do you think it creates sites in this way. [...] Clearly a total misreading on my part of the Django documentation to which you referred earlier. It talks about having to feed Django multiple settings.py files with different SITE_ID values and running migrate to configure them, then links to the document on migrations which starts out explaining how to use makemigrations to create them. I took that to mean creating and configuring Django sites from the command line required supplying customized database migrations. The particularly misleading bits are where it says: "To set the correct name and domain for your project, you can use a data migration." "In order to serve different sites in production, you’d create a separate settings file with each SITE_ID" It doesn't seem to talk about using django_admin at all (or if it does, it's not mentioned by name, maybe assuming the reader knows to use it in order to perform the necessary steps). Thanks for the clear examples! If these are available somewhere in Django's documentation, I did a terrible job of searching for them. > you need to create a domain in Postorius. Look at the code behind > Postorius add new domain. Or you can create the domain in core via > REST, but you then have to modify it in Postorius to associate the > correct site. [...] Much appreciated, this was the other bit that was non-obvious to me. I guess you're saying Postorius's view of domains is independent from what's in Mailman Core, and there's no documented CLI/API for manipulating that in Postorius so I'll need to write one based on a reverse-engineering of its source code? Sounds doable now that I know where to look. Thanks again! -- Jeremy Stanley

2 years, 4 months

[MM3-users] Re: Clearing webserver cache - Nginx

by Odhiambo Washington

On Mon, Aug 5, 2024 at 9:34 PM Mark Sapiro <mark(a)msapiro.net> wrote: > On 8/5/24 10:30, Odhiambo Washington via Mailman-users wrote: > > > > Actually, what I mean is that CF is proxying requests to the host named > > m3-lists.kictanet.or.ke. > > I do not have any configuration other than DNS proxying at CF. > > ``` > dig a mm3-lists.kictanet.or.ke > ... > mm3-lists.kictanet.or.ke. 300 IN A 172.67.167.73 > mm3-lists.kictanet.or.ke. 300 IN A 104.21.11.217 > ``` > Both of those are cloudflare IPs, and why 2? > Yes, those are Cloudflare IPs and I don't know why 2! All I know is that in DNS, I created an A record for the hostname and chose the "proxy" option in the cloudflare DNS management interface. > > So when you talk to m3-lists.kictanet.or.ke via a browser, it's CF IP > who you talk to and it in turn talks to the host directly. > > And how is that configured? > You add the domain - kictanet.or.ke to CF's control panel. They give you TWO name servers. You substitute the NSes your registrar gave to your domain for those two. Subsequently you create A|CNAME|MX\TXT records on the CF's control panel. You decide whether an A or CNAME record you create is proxy-ed or not. If you proxy the record, no one will ever get to know its real IP, but if you don't proxy, then the dig tool above will tell you the real IP address. > So the IP address you see in the logs is CF's IP address, not yours, not > > mine and CF are doing some caching. That's why > > they are requesting for expired files. > > > Why is cloudfare trying to > access /opt/mailman/mm/static/CACHE/css/output.6dab123e4897.css locally if > all it is doing is proxying the request to the actual mailman server? > I don't know. However, those IPs you see in the Nginx logs, none belong to me! I have two possible IPs - one domiciled in KE and another domiciled in the US (when I use VPN). -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

10 months, 3 weeks

[MM3-users] Re: Fresh installation mailman3-suite via "pipx" - any experiences yet?

by Stephen J. Turnbull

Mark Sapiro writes: > > On Thu, Jan 25, 2024 at 4:43 PM Wikinaut <mail(a)tgries.de> wrote: > > > >> I recently found the "pipx" wrapper very useful and wanted to > >> ask you if this could be facilitate the fresh installation of > >> the mailman3-suite. Has anyone in the list already experiences > >> installing mailman and mailmanweg through "pipx"? > pipx apparently does create virtual environments. See > https://pipx.pypa.io/stable/comparisons/ > > The thing there that concerns me is "pipx replaces a subset of > pip's functionality; it lets you install cli applications but NOT > libraries that you import in your code." > > What does this say about the multiple dependencies that we install. I'm pretty sure it will install dependencies. I think (everything I say below is based on reading the README.md from pipx's pypa.io repo, so caveat installer) that the complexity that will arise is that (for example) a naive approach like $ pipx install mailman $ pipx install mailmanweb $ pipx install postorius $ pipx install hyperkitty will create 4 separate venvs with 3 copies of mailmanclient, 3 copies of django-mailman3, and two copies of Django. Besides the space waste, there's also the small problem that mailmanweb won't have copies of postorius and hyperkitty in its venv. :-þ pipx has an "inject" command that allows adding other packages to a venv, but I think this means that pipx has no great advantage over the standard way of installing mailman into a venv. pipx also installs stuff into the user's $HOME, specifically into ~/.local/share/pipx/venvs which I think will be kind of annoying if you need to dig into package resources. (I expect that you will still install configs into /etc/mailman3 and various data into /var/lib/mailman3/var or /opt/mailman3 or whatever without a problem.) This would only be an issue for heavy-duty customizers. This can probably be customized. One of the big advantages pipx touts is "look ma, no sudo". That's not really an advantage for system daemons, though. So bottom line is you can do it, but you'll need to study up on pipx arcana, because Mailman is a bunch of daemon it buys you very little, and the result won't look like any Mailman installation you've seen described on mailman-users or StackExchange. Steve

1 year, 5 months

[MM3-users] Re: Installing Mailman3 under a Python Virtual environment running on RHEL9

by Wild Coast

On 2024/05/22 18:25, Mark Sapiro wrote: > On 5/22/24 11:31 AM, Andy Macheta wrote: >> Hi Mark, >> >> Thanks; yes those 4 steps were performed. >> It looks like all the web pages are displaying is just text; so >> there's no background (apart from white) and no icons or >> graphics....almost like CCS is not being recognized perhaps. > > > If pages are not styled and you have run the collectstatic and > compress jobs, the most likely cause is your web server not finding > the static/ directory. See > https://docs.mailman3.org/en/latest/install/virtualenv.html#apache-configur… > and check your Apache logs for 404s > I had the same issue with the reverse proxy. Here's my final working Debian apache2 virtualhost file: jeff@maya:~$ cat /etc/apache2/sites-enabled/holeinthewall.conf <VirtualHost *:80> ServerName lists.holeinthewall.org.za ServerAlias mail.holeinthewall.org.za # Redirect all HTTP traffic to HTTPS RewriteEngine On RewriteCond %{HTTPS} off RewriteRule (.*) https://%{SERVER_NAME}$1 [R=301,L] </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster(a)holeinthewall.org.za ServerName lists.holeinthewall.org.za # Enable SSL/TLS SSLCertificateFile /etc/letsencrypt/live/holeinthewall.org.za/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/holeinthewall.org.za/privkey.pem Include /etc/letsencrypt/options-ssl-apache.conf # Enable SSL proxying SSLProxyEngine On SSLProxyVerify None SSLProxyCheckPeerCN Off SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire Off # Preserve the Host header ProxyPreserveHost On # Correct the domain in the Set-Cookie header from 127.0.0.1 to lists.holeinthewall.org.za ProxyPassReverseCookieDomain 127.0.0.1 lists.holeinthewall.org.za # Serve static content directly from the filesystem DocumentRoot /opt/mailman/web/static Alias /static "/opt/mailman/web/static" <Directory "/opt/mailman/web/static"> Require all granted </Directory> # Proxy configuration <Proxy *> Require all granted </Proxy> # Ensure X-Forwarded-Proto is set correctly RequestHeader set X-Forwarded-Proto expr=%{REQUEST_SCHEME} # ProxyPass configuration for specific paths ProxyPass /static ! ProxyPass / http://127.0.0.1:8000/ ProxyPassReverse / http://127.0.0.1:8000/ ProxyPass "/mailman3" "http://127.0.0.1:8000/mailman3" ProxyPass "/archives" "http://127.0.0.1:8000/archives" ProxyPass "/accounts" "http://127.0.0.1:8000/accounts" ProxyPass "/admin" "http://127.0.0.1:8000/admin" ProxyPass "/user-profile" "http://127.0.0.1:8000/user-profile" # Logging ErrorLog /var/log/apache2/holeinthewall_error.log CustomLog /var/log/apache2/holeinthewall_access.log combined </VirtualHost> HTH -- Kind regards, Jeff Brown WildCoast WebWorx m: +27 74 101 5170 <tel:+27%2074%20101%205170> v: +27 87 550 1210 <tel:+27%2087%20550%201210> a: Hole in the Wall, South Africa w: www.wildcoast.com <http://www.wildcoast.com/> e: webmaster(a)wildcoast.com ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: homcljipmaiefjea.png Replaced multipart/alternative part with first alternative.

1 year, 1 month

[MM3-users] Re: Mailman rewrite the Cc header incorrectly

by Alan So

Mark Sapiro wrote: > Yes, the issue is closed because the fix has been merged and will be in > Mailman Core 3.3.2. Thanks again. I tried to read RFC5322 again but I am not sure whether I interpret it correctly. The display-name seem allowing folding in the middle. 3.2.4 quoted-string = [CFWS] DQUOTE *([FWS] qcontent) [FWS] DQUOTE [CFWS] A quoted-string is treated as a unit. That is, quoted-string is identical to atom, semantically. Since a quoted-string is allowed to contain FWS, folding is permitted. Also note that since quoted-pair is allowed in a quoted-string, the quote and backslash characters may appear in a quoted-string so long as they appear as a quoted-pair. 3.2.5 word = atom / quoted-string phrase = 1*word / obs-phrase 3.4 display-name = phrase A quoted-string is treated as a unit. That is, quoted-string is identical to atom, semantically. Since a quoted-string is allowed to contain FWS, folding is permitted. Also note that since quoted-pair is allowed in a quoted-string, the quote and backslash characters may appear in a quoted-string so long as they appear as a quoted-pair. > Anyway, I really appreciate your diligence in following through > repeatedly on this issue. Were it not for that, I wouldn't have been > motivated to fix it. For a look at why even a trivial change is not > trivial, see the commits at > https://gitlab.com/mailman/mailman/-/merge_requests/652/commits. > The > first of these is developing good tests for the issue and then fixing > the code so the tests pass, and the second, unanticipated one was due to > the fact that Python 3.5 didn't maintain the original ordering of the > addresses in Cc: so I couldn't compare against a fixed result. This is > due to the fact that the underlying code uses dictionaries, and prior to > Python 3.6 dictionaries didn't preserve order. As our organization chose a free open-source software to provide service, I think it might be the only way that we can get the issue resolved since there is no official support service for a volunteer based project. Most importantly, without your detail explanation it will be really hard for us to further study the issue and file the issue in the bug tracking system. It is a good lesson anyway since it really gives me some solid experiences about running codes with different versions of Python and it is sometimes not so trivial when developing unit tests.

5 years, 1 month

[MM3-users] Re: Mails are not archived at all -- how to check if Hyperkitty is running or not?

by Franklin Weng

Hi, It looks like to have some clues now. Mark Sapiro <mark(a)msapiro.net> 於 2019年10月30日週三 13:07 寫道： > On 10/29/19 8:21 PM, Franklin Weng wrote: > > Yes, it is checked. > > Have you checked Mailman's logs - Mailman's var/logs/mailman.log in > particular? > I found some hyperkitty URL error: The default settings of base_url in mailman-hyperkitty.cfg is defined as base_url: http://localhost/mailman/hyperkitty/ In the mailman.log there are messages like: Oct 30 13:53:43 2019 (14259) HyperKitty failure on http://localhost/mailman3/hyperkitty/api/mailman/archive: <!DOCTYPE HTML PUBLIC "-//IETF// DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Found</h1> <p>The requested URL was not found on this server.</p> I changed it to base_url: https://localhost/mailman3/hyperkitty/ then the log became File "/usr/lib/python3/dist-packages/requests /adapters.py", line 514, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /mailman3/hyperkitty/api/mailman/archive?key=xxx (Caused by SSLError(SSLCertVerification Error("hostname 'localhost' doesn't match ' lists.slat.org'"))) which makes sense because the SSL certificate applied from Letsencrypt is for lists.slat.org. If I use base_url: https://lists.slat.org/mailman3/hyperkitty/ The log became Oct 30 15:25:50 2019 (6256) HyperKitty failure on https://lists.slat.org/mailman3/hyperkitty/api/mailman/urls: <html><title>Auth required</title><body> <h1>Authorization Required</h1> </body></html> (401) In my mailman-web.py: MAILMAN_ARCHIVER_KEY = 'my-key,same as in mailman-hyperkitty.cfg' MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1') Not sure what to change here. > Do messages received from the list have an Archived-At: header? > Yes, but it's empty List-Id: 測試論壇 <slat-list.lists.slat.org> Archived-At: <> List-Archive: <> List-Help: <mailto:slat-list-request@lists.slat.org?subject=help> List-Post: <mailto:slat-list@lists.slat.org> List-Subscribe: <mailto:slat-list-join@lists.slat.org> List-Unsubscribe: <mailto:slat-list-leave@lists.slat.org> > > What's in /etc/mailman3/mailman-hyperkitty.cfg? In particular does the > [general] setting base_url define a URL that accesses hyperkitty It seems to be the question… and > does the [general] setting api_key match the MAILMAN_ARCHIVER_KEY > setting in /etc/mailman3/mailman-web.py, but note that the > MAILMAN_ARCHIVER_KEY setting is quotes because it is a Python string > assignment and the api_key setting is not quoted > Yes, it's single quoted. Thanks for all your help! Franklin

5 years, 8 months

[MM3-users] Re: Customise Postorius templates?

by Abhilash Raj

> On Feb 1, 2022, at 11:01 AM, Mark Sapiro <mark(a)msapiro.net> wrote: > > On 2/1/22 09:47, Duane Raymond wrote: >> Hi, >> I'm looking to do some radical customisation of the Postorius (and Hyperkitty) templates over the next 6 ish months and was wondering if there is a 'best practice' way to do this that will survive updates to MM3. I've searched around and not found much and also tested copying >> venv/lib/python3.7/site-packages/postorius/templates/postorius/lists/summary.html >> to >> var/templates/lists/summary.html >> As suggested in some posts - but it didn't seem to pick up the customisation - only customisations in the venv path worked - and they get overwritten on update. > > > The var/templates directory has a specific structure and is only for custom list specific, domain specific and sitewide templates used for notifications from Mailman core. It has nothing to do with Postorius or HyperKitty templates. > > >> Is there a 'best' way for this? I'd also like to eventually share them with the community and maybe add them to the repo if people want that....but for now I'll start with my own experiments. (and if there is documentation I have missed for this, let me know!) > > > I don't know about a `best` way, but the way I do this is to keep all local mods as patches to the base and apply those patches as part of the upgrade process. The best way to do this would be utilize Django’s template loader. See here[1] for the documentation on how you can do this. You want to put the path where you are putting the templates under `DIRS` option of the `TEMPLATES` section. Do make sure that you are keeping the directory structure correct so that Django can discover them. Like, Hyperkitty’s base.html should be in `/custom/path/hyperkitty/base.html`, where `/custom/path` is what you’ve setup in DIRS setting above. One important point to note here is that we can’t promise to not break your custom template in future, because the values passed in the template’s context is not a public or stable API. So, it is technically possible for us to remove/change the variables that are being passed. In practice, I can’t remember when was the last time I removed a variable, so this would work better than patching since you are doing “radical customisation”, which Mark isn’t. [1]: https://docs.djangoproject.com/en/4.0/howto/overriding-templates/ -- thanks, Abhilash Raj (maxking)

3 years, 4 months

[MM3-users] Re: Remove old bounces in bouncedir

by Stephan Krinetzki

Mark Sapiro wrote: > On 3/9/22 00:58, Stephan Krinetzki wrote: > > Hello all, > > our Mailman3 has over 7GB of bounces in its bouncedir. I noticed that some messages there are already over 3 years old. Now my question: Is there a way to remove the old bounces? > > I don't know what you mean by bouncedir. > If you are referring to the bounceevent table in the database, it would > be OK to remove all the entries for which processed is True. Ok, so i can remove there entries which are processed. For a smaller database ;) > If you are referring to messages in Mailman's bounces queue, if these > are not processed and removed within 15 minutes of arrival, Mailman's > bounce runner is not running. Well, it is the queue.Our var directory is under /opt/mailman/var and the /opt/mailman/var/queue/bounces is over 8GB big. There are *.pck messages and some of them are very old (>3 Years). The bounce seems to be running: ``` Mar 10 09:24:53 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:00 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:06 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:12 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:18 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:24 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:30 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:36 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:43 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:49 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. ``` The process it self is there ``` mailman 376 99.4 19.1 6578312 6279000 ? R 09:17 10:52 venv/bin/python3 venv/bin/runner -C mailman.cfg --runner=bounces:0:1 ``` (Pathes are shorted) So i guess, there is everything fine. But the files are not deleted. > If you are referring to bounce messages in the message store > (var/messages/), there have been issues with those not being removed, > but Mailman core >= 3.3.5 should clean those up. SInce the messages are hashed or otherwise stored, i can't verify or deny this.

3 years, 3 months

Search results for query "sapiro"