Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: Multi-domain oddities in Hyperkitty and Postorius

by Jeremy Stanley

On 2023-02-15 18:35:35 -0800 (-0800), Mark Sapiro wrote: > On 2/15/23 07:52, Jeremy Stanley wrote: > > > > Thanks, I'm still doing my best to wrap my brain around how Django > > uses database migrations to create sites (seems like a very strange > > use of a migration, I'm only accustomed to seeing them applied for > > database schema updates). > > I don't think it does. There are a couple of migrations at > django/contrib/sites/migrations, but these are only the initial creation of > the model and an update to make the domain field unique. > > Why do you think it creates sites in this way. [...] Clearly a total misreading on my part of the Django documentation to which you referred earlier. It talks about having to feed Django multiple settings.py files with different SITE_ID values and running migrate to configure them, then links to the document on migrations which starts out explaining how to use makemigrations to create them. I took that to mean creating and configuring Django sites from the command line required supplying customized database migrations. The particularly misleading bits are where it says: "To set the correct name and domain for your project, you can use a data migration." "In order to serve different sites in production, you’d create a separate settings file with each SITE_ID" It doesn't seem to talk about using django_admin at all (or if it does, it's not mentioned by name, maybe assuming the reader knows to use it in order to perform the necessary steps). Thanks for the clear examples! If these are available somewhere in Django's documentation, I did a terrible job of searching for them. > you need to create a domain in Postorius. Look at the code behind > Postorius add new domain. Or you can create the domain in core via > REST, but you then have to modify it in Postorius to associate the > correct site. [...] Much appreciated, this was the other bit that was non-obvious to me. I guess you're saying Postorius's view of domains is independent from what's in Mailman Core, and there's no documented CLI/API for manipulating that in Postorius so I'll need to write one based on a reverse-engineering of its source code? Sounds doable now that I know where to look. Thanks again! -- Jeremy Stanley

2 years, 9 months

[MM3-users] Re: Clearing webserver cache - Nginx

by Odhiambo Washington

On Mon, Aug 5, 2024 at 9:34 PM Mark Sapiro <mark(a)msapiro.net> wrote: > On 8/5/24 10:30, Odhiambo Washington via Mailman-users wrote: > > > > Actually, what I mean is that CF is proxying requests to the host named > > m3-lists.kictanet.or.ke. > > I do not have any configuration other than DNS proxying at CF. > > ``` > dig a mm3-lists.kictanet.or.ke > ... > mm3-lists.kictanet.or.ke. 300 IN A 172.67.167.73 > mm3-lists.kictanet.or.ke. 300 IN A 104.21.11.217 > ``` > Both of those are cloudflare IPs, and why 2? > Yes, those are Cloudflare IPs and I don't know why 2! All I know is that in DNS, I created an A record for the hostname and chose the "proxy" option in the cloudflare DNS management interface. > > So when you talk to m3-lists.kictanet.or.ke via a browser, it's CF IP > who you talk to and it in turn talks to the host directly. > > And how is that configured? > You add the domain - kictanet.or.ke to CF's control panel. They give you TWO name servers. You substitute the NSes your registrar gave to your domain for those two. Subsequently you create A|CNAME|MX\TXT records on the CF's control panel. You decide whether an A or CNAME record you create is proxy-ed or not. If you proxy the record, no one will ever get to know its real IP, but if you don't proxy, then the dig tool above will tell you the real IP address. > So the IP address you see in the logs is CF's IP address, not yours, not > > mine and CF are doing some caching. That's why > > they are requesting for expired files. > > > Why is cloudfare trying to > access /opt/mailman/mm/static/CACHE/css/output.6dab123e4897.css locally if > all it is doing is proxying the request to the actual mailman server? > I don't know. However, those IPs you see in the Nginx logs, none belong to me! I have two possible IPs - one domiciled in KE and another domiciled in the US (when I use VPN). -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

1 year, 4 months

[MM3-users] Re: Fresh installation mailman3-suite via "pipx" - any experiences yet?

by Stephen J. Turnbull

Mark Sapiro writes: > > On Thu, Jan 25, 2024 at 4:43 PM Wikinaut <mail(a)tgries.de> wrote: > > > >> I recently found the "pipx" wrapper very useful and wanted to > >> ask you if this could be facilitate the fresh installation of > >> the mailman3-suite. Has anyone in the list already experiences > >> installing mailman and mailmanweg through "pipx"? > pipx apparently does create virtual environments. See > https://pipx.pypa.io/stable/comparisons/ > > The thing there that concerns me is "pipx replaces a subset of > pip's functionality; it lets you install cli applications but NOT > libraries that you import in your code." > > What does this say about the multiple dependencies that we install. I'm pretty sure it will install dependencies. I think (everything I say below is based on reading the README.md from pipx's pypa.io repo, so caveat installer) that the complexity that will arise is that (for example) a naive approach like $ pipx install mailman $ pipx install mailmanweb $ pipx install postorius $ pipx install hyperkitty will create 4 separate venvs with 3 copies of mailmanclient, 3 copies of django-mailman3, and two copies of Django. Besides the space waste, there's also the small problem that mailmanweb won't have copies of postorius and hyperkitty in its venv. :-þ pipx has an "inject" command that allows adding other packages to a venv, but I think this means that pipx has no great advantage over the standard way of installing mailman into a venv. pipx also installs stuff into the user's $HOME, specifically into ~/.local/share/pipx/venvs which I think will be kind of annoying if you need to dig into package resources. (I expect that you will still install configs into /etc/mailman3 and various data into /var/lib/mailman3/var or /opt/mailman3 or whatever without a problem.) This would only be an issue for heavy-duty customizers. This can probably be customized. One of the big advantages pipx touts is "look ma, no sudo". That's not really an advantage for system daemons, though. So bottom line is you can do it, but you'll need to study up on pipx arcana, because Mailman is a bunch of daemon it buys you very little, and the result won't look like any Mailman installation you've seen described on mailman-users or StackExchange. Steve

1 year, 10 months

[MM3-users] Re: Installing Mailman3 under a Python Virtual environment running on RHEL9

by Wild Coast

On 2024/05/22 18:25, Mark Sapiro wrote: > On 5/22/24 11:31 AM, Andy Macheta wrote: >> Hi Mark, >> >> Thanks; yes those 4 steps were performed. >> It looks like all the web pages are displaying is just text; so >> there's no background (apart from white) and no icons or >> graphics....almost like CCS is not being recognized perhaps. > > > If pages are not styled and you have run the collectstatic and > compress jobs, the most likely cause is your web server not finding > the static/ directory. See > https://docs.mailman3.org/en/latest/install/virtualenv.html#apache-configur… > and check your Apache logs for 404s > I had the same issue with the reverse proxy. Here's my final working Debian apache2 virtualhost file: jeff@maya:~$ cat /etc/apache2/sites-enabled/holeinthewall.conf <VirtualHost *:80> ServerName lists.holeinthewall.org.za ServerAlias mail.holeinthewall.org.za # Redirect all HTTP traffic to HTTPS RewriteEngine On RewriteCond %{HTTPS} off RewriteRule (.*) https://%{SERVER_NAME}$1 [R=301,L] </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster(a)holeinthewall.org.za ServerName lists.holeinthewall.org.za # Enable SSL/TLS SSLCertificateFile /etc/letsencrypt/live/holeinthewall.org.za/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/holeinthewall.org.za/privkey.pem Include /etc/letsencrypt/options-ssl-apache.conf # Enable SSL proxying SSLProxyEngine On SSLProxyVerify None SSLProxyCheckPeerCN Off SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire Off # Preserve the Host header ProxyPreserveHost On # Correct the domain in the Set-Cookie header from 127.0.0.1 to lists.holeinthewall.org.za ProxyPassReverseCookieDomain 127.0.0.1 lists.holeinthewall.org.za # Serve static content directly from the filesystem DocumentRoot /opt/mailman/web/static Alias /static "/opt/mailman/web/static" <Directory "/opt/mailman/web/static"> Require all granted </Directory> # Proxy configuration <Proxy *> Require all granted </Proxy> # Ensure X-Forwarded-Proto is set correctly RequestHeader set X-Forwarded-Proto expr=%{REQUEST_SCHEME} # ProxyPass configuration for specific paths ProxyPass /static ! ProxyPass / http://127.0.0.1:8000/ ProxyPassReverse / http://127.0.0.1:8000/ ProxyPass "/mailman3" "http://127.0.0.1:8000/mailman3" ProxyPass "/archives" "http://127.0.0.1:8000/archives" ProxyPass "/accounts" "http://127.0.0.1:8000/accounts" ProxyPass "/admin" "http://127.0.0.1:8000/admin" ProxyPass "/user-profile" "http://127.0.0.1:8000/user-profile" # Logging ErrorLog /var/log/apache2/holeinthewall_error.log CustomLog /var/log/apache2/holeinthewall_access.log combined </VirtualHost> HTH -- Kind regards, Jeff Brown WildCoast WebWorx m: +27 74 101 5170 <tel:+27%2074%20101%205170> v: +27 87 550 1210 <tel:+27%2087%20550%201210> a: Hole in the Wall, South Africa w: www.wildcoast.com <http://www.wildcoast.com/> e: webmaster(a)wildcoast.com ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: homcljipmaiefjea.png Replaced multipart/alternative part with first alternative.

1 year, 6 months

[MM3-users] Re: Mailman rewrite the Cc header incorrectly

by Alan So

Mark Sapiro wrote: > Yes, the issue is closed because the fix has been merged and will be in > Mailman Core 3.3.2. Thanks again. I tried to read RFC5322 again but I am not sure whether I interpret it correctly. The display-name seem allowing folding in the middle. 3.2.4 quoted-string = [CFWS] DQUOTE *([FWS] qcontent) [FWS] DQUOTE [CFWS] A quoted-string is treated as a unit. That is, quoted-string is identical to atom, semantically. Since a quoted-string is allowed to contain FWS, folding is permitted. Also note that since quoted-pair is allowed in a quoted-string, the quote and backslash characters may appear in a quoted-string so long as they appear as a quoted-pair. 3.2.5 word = atom / quoted-string phrase = 1*word / obs-phrase 3.4 display-name = phrase A quoted-string is treated as a unit. That is, quoted-string is identical to atom, semantically. Since a quoted-string is allowed to contain FWS, folding is permitted. Also note that since quoted-pair is allowed in a quoted-string, the quote and backslash characters may appear in a quoted-string so long as they appear as a quoted-pair. > Anyway, I really appreciate your diligence in following through > repeatedly on this issue. Were it not for that, I wouldn't have been > motivated to fix it. For a look at why even a trivial change is not > trivial, see the commits at > https://gitlab.com/mailman/mailman/-/merge_requests/652/commits. > The > first of these is developing good tests for the issue and then fixing > the code so the tests pass, and the second, unanticipated one was due to > the fact that Python 3.5 didn't maintain the original ordering of the > addresses in Cc: so I couldn't compare against a fixed result. This is > due to the fact that the underlying code uses dictionaries, and prior to > Python 3.6 dictionaries didn't preserve order. As our organization chose a free open-source software to provide service, I think it might be the only way that we can get the issue resolved since there is no official support service for a volunteer based project. Most importantly, without your detail explanation it will be really hard for us to further study the issue and file the issue in the bug tracking system. It is a good lesson anyway since it really gives me some solid experiences about running codes with different versions of Python and it is sometimes not so trivial when developing unit tests.

5 years, 6 months

[MM3-users] Re: Mails are not archived at all -- how to check if Hyperkitty is running or not?

by Franklin Weng

Hi, It looks like to have some clues now. Mark Sapiro <mark(a)msapiro.net> 於 2019年10月30日週三 13:07 寫道： > On 10/29/19 8:21 PM, Franklin Weng wrote: > > Yes, it is checked. > > Have you checked Mailman's logs - Mailman's var/logs/mailman.log in > particular? > I found some hyperkitty URL error: The default settings of base_url in mailman-hyperkitty.cfg is defined as base_url: http://localhost/mailman/hyperkitty/ In the mailman.log there are messages like: Oct 30 13:53:43 2019 (14259) HyperKitty failure on http://localhost/mailman3/hyperkitty/api/mailman/archive: <!DOCTYPE HTML PUBLIC "-//IETF// DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Found</h1> <p>The requested URL was not found on this server.</p> I changed it to base_url: https://localhost/mailman3/hyperkitty/ then the log became File "/usr/lib/python3/dist-packages/requests /adapters.py", line 514, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /mailman3/hyperkitty/api/mailman/archive?key=xxx (Caused by SSLError(SSLCertVerification Error("hostname 'localhost' doesn't match ' lists.slat.org'"))) which makes sense because the SSL certificate applied from Letsencrypt is for lists.slat.org. If I use base_url: https://lists.slat.org/mailman3/hyperkitty/ The log became Oct 30 15:25:50 2019 (6256) HyperKitty failure on https://lists.slat.org/mailman3/hyperkitty/api/mailman/urls: <html><title>Auth required</title><body> <h1>Authorization Required</h1> </body></html> (401) In my mailman-web.py: MAILMAN_ARCHIVER_KEY = 'my-key,same as in mailman-hyperkitty.cfg' MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1') Not sure what to change here. > Do messages received from the list have an Archived-At: header? > Yes, but it's empty List-Id: 測試論壇 <slat-list.lists.slat.org> Archived-At: <> List-Archive: <> List-Help: <mailto:slat-list-request@lists.slat.org?subject=help> List-Post: <mailto:slat-list@lists.slat.org> List-Subscribe: <mailto:slat-list-join@lists.slat.org> List-Unsubscribe: <mailto:slat-list-leave@lists.slat.org> > > What's in /etc/mailman3/mailman-hyperkitty.cfg? In particular does the > [general] setting base_url define a URL that accesses hyperkitty It seems to be the question… and > does the [general] setting api_key match the MAILMAN_ARCHIVER_KEY > setting in /etc/mailman3/mailman-web.py, but note that the > MAILMAN_ARCHIVER_KEY setting is quotes because it is a Python string > assignment and the api_key setting is not quoted > Yes, it's single quoted. Thanks for all your help! Franklin

6 years, 1 month

[MM3-users] Re: Customise Postorius templates?

by Abhilash Raj

> On Feb 1, 2022, at 11:01 AM, Mark Sapiro <mark(a)msapiro.net> wrote: > > On 2/1/22 09:47, Duane Raymond wrote: >> Hi, >> I'm looking to do some radical customisation of the Postorius (and Hyperkitty) templates over the next 6 ish months and was wondering if there is a 'best practice' way to do this that will survive updates to MM3. I've searched around and not found much and also tested copying >> venv/lib/python3.7/site-packages/postorius/templates/postorius/lists/summary.html >> to >> var/templates/lists/summary.html >> As suggested in some posts - but it didn't seem to pick up the customisation - only customisations in the venv path worked - and they get overwritten on update. > > > The var/templates directory has a specific structure and is only for custom list specific, domain specific and sitewide templates used for notifications from Mailman core. It has nothing to do with Postorius or HyperKitty templates. > > >> Is there a 'best' way for this? I'd also like to eventually share them with the community and maybe add them to the repo if people want that....but for now I'll start with my own experiments. (and if there is documentation I have missed for this, let me know!) > > > I don't know about a `best` way, but the way I do this is to keep all local mods as patches to the base and apply those patches as part of the upgrade process. The best way to do this would be utilize Django’s template loader. See here[1] for the documentation on how you can do this. You want to put the path where you are putting the templates under `DIRS` option of the `TEMPLATES` section. Do make sure that you are keeping the directory structure correct so that Django can discover them. Like, Hyperkitty’s base.html should be in `/custom/path/hyperkitty/base.html`, where `/custom/path` is what you’ve setup in DIRS setting above. One important point to note here is that we can’t promise to not break your custom template in future, because the values passed in the template’s context is not a public or stable API. So, it is technically possible for us to remove/change the variables that are being passed. In practice, I can’t remember when was the last time I removed a variable, so this would work better than patching since you are doing “radical customisation”, which Mark isn’t. [1]: https://docs.djangoproject.com/en/4.0/howto/overriding-templates/ -- thanks, Abhilash Raj (maxking)

3 years, 10 months

[MM3-users] Re: Remove old bounces in bouncedir

by Stephan Krinetzki

Mark Sapiro wrote: > On 3/9/22 00:58, Stephan Krinetzki wrote: > > Hello all, > > our Mailman3 has over 7GB of bounces in its bouncedir. I noticed that some messages there are already over 3 years old. Now my question: Is there a way to remove the old bounces? > > I don't know what you mean by bouncedir. > If you are referring to the bounceevent table in the database, it would > be OK to remove all the entries for which processed is True. Ok, so i can remove there entries which are processed. For a smaller database ;) > If you are referring to messages in Mailman's bounces queue, if these > are not processed and removed within 15 minutes of arrival, Mailman's > bounce runner is not running. Well, it is the queue.Our var directory is under /opt/mailman/var and the /opt/mailman/var/queue/bounces is over 8GB big. There are *.pck messages and some of them are very old (>3 Years). The bounce seems to be running: ``` Mar 10 09:24:53 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:00 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:06 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:12 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:18 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:24 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:30 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:36 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:43 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:49 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. ``` The process it self is there ``` mailman 376 99.4 19.1 6578312 6279000 ? R 09:17 10:52 venv/bin/python3 venv/bin/runner -C mailman.cfg --runner=bounces:0:1 ``` (Pathes are shorted) So i guess, there is everything fine. But the files are not deleted. > If you are referring to bounce messages in the message store > (var/messages/), there have been issues with those not being removed, > but Mailman core >= 3.3.5 should clean those up. SInce the messages are hashed or otherwise stored, i can't verify or deny this.

3 years, 9 months

[MM3-users] Re: Improve logging?

by Odhiambo Washington

On Tue, Jan 31, 2023 at 7:42 PM Mark Sapiro <mark(a)msapiro.net> wrote: > On 1/31/23 00:40, Odhiambo Washington wrote: > > Some entries in the log files can be very scary, while in actual sense > they > > are harmless. > > Such inconsequential, but scary entiess can mess up the well-being of a > > SysAdmin. > > I humbly request for the logs related to posts by non-members to a MM3 > list > > to be improved a little bit :-) > > Here is the example in mailman.log: > > > > <cut> > > Jan 31 10:03:54 2023 (92481) DISCARD: < > > 2a1c588a6ccd47abbf57ee5fc61c9946(a)phellineconsultancy.co.ke>; ['The > message > > is not from a list member'] > > [31/Jan/2023:10:04:48 +0300] "GET > > /3.1/users/1bad08f703fe4bc0a829f4bdce10fc8f HTTP/1.1" 200 253 "-" "GNU > > Mailman REST client v3.3.5" > > [31/Jan/2023:10:04:48 +0300] "GET > > /3.1/users/1bad08f703fe4bc0a829f4bdce10fc8f/addresses HTTP/1.1" 200 473 > "-" > > "GNU Mailman REST client v3.3.5" > > Jan 31 10:18:09 2023 (92343) deque: do_confirm_verify > > Traceback (most recent call last): > > File > > > "/opt/mailman/mm/venv/lib/python3.9/site-packages/mailman/app/workflow.py", > > line 69, in __next__ > > return step() > > File > > > "/opt/mailman/mm/venv/lib/python3.9/site-packages/mailman/app/subscriptions.py", > > line 476, in _step_send_confirmation > > raise StopIteration > > StopIteration > > </cut> > > > > Is there a possibility to change the logging engine to just *fold* the > last > > 7 lines into "The message is not from a list member. It was discarded", > or > > something close? > > No. > > This entry > > Jan 31 10:03:54 2023 (92481) DISCARD: > <2a1c588a6ccd47abbf57ee5fc61c9946(a)phellineconsultancy.co.ke>; ['The > message is not from a list member'] > > is a single entry about a discarded message. > > The next two are logging requests via the REST API that have nothing to > do with the discard and are logged by a different module. > > The last entry with a traceback is yet another separate event. That one > could probably be suppressed because raising StopIteration is a normal > occurrence in workflow processing. > I was thiinking that they were all related. The scary part though is the line that starts with ' Traceback'. It's what I'd love to not-see unless there is a serious problem. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)

2 years, 10 months

[MM3-users] Re: MM2 to MM3 on debian 10

by Gelpi Andrea

Il 27/04/20 18:59, Mark Sapiro ha scritto: > On 4/27/20 8:26 AM, Gelpi Andrea wrote: >> Hi, on a debian 10 with MM3 I'm trying to migrate form some old MM2 >> mailing lists. I know that not all can work. >> >> In my old lists I have email and regular expressions in >> reject_these_nonmembers and discard_these_nonmembers >> >> After importing the list all these email are in non member list. >> >> All these email have Delivery status enabled. > > > Delivery status is not relevant for non-members and only has to do with > receiving list posts in any case. > > What version of Mailman core is this? There have been changes in > `mailman import21` that may affect this, but an address in > `reject_these_nonmembers` should be imported as a non-member with a > moderation action of `reject` and similarly for `discard`. > > Note that regexps in *_these_nonmembers should be imported into the MM 3 > legacy *_these_nonmembers attributes. Some older versions of import21 > also imported non-regexp addresses here, but that was wrong. > Mailman V 3.2.1 Python V 3.7.3 Postorius V 1.3.2 Hiperkitty V 1.2.2 Yes reject_these_nonmembers has a moderation action of 'reject' and the some for discard. > >> Does this mean that all of them will receive any message sent to the list? >> The moderation rule is correct, but if I understood correct Delivery >> status should be disabled. > > > Delivery status is irrelevant for non-members. Only members are > candidates for delivery. > Ok, but will non-members receive the list posts or only members will? > >> For migration I followed this link >> https://docs.mailman3.org/en/latest/migration.html >> >> To migrate archives it states to run >> >> python manage.py hyperkitty ... >> >> I cant' find a manage.py on debian 10 > > > manage.py is a Django generic term for the Django admin command. You > probably want something called django-admin or similar. > Yes in debain 10 exists django-admin. Now I need to understand how to use it. It asks me to load a configuration. > >> A search shows the only /usr/share/mailman3-web/manage.py exists. >> >> Is there any documentation specific for debian? > > That would be up to Debian. > -- Gelpi ing. Andrea -------------------------------------------------------------- It took the computing power of three C-64s to fly to the Moon. It takes a 486 to run Windows 95. Something is wrong here. --------------------------------------------------------------

5 years, 7 months

Search results for query "sapiro"