Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: hyperkitty_import Unicode Errors

by Stephen J Turnbull

On 2023-05-07 07:39 Mark Sapiro writes: > On 5/6/23 11:27, Dave Hall via Mailman-users wrote: >> django.db.utils.OperationalError: (1366, "Incorrect string value: >> '\\xE1\\x90\\xA7\\x0A\\x0AO...' for column >> `mailman3web`.`hyperkitty_email`.`content` at row 1") >> >> Is there a known cause for this? Almost always nonconforming user clients. You write "frequently", but I'm guessing it's maybe 1% or 2% of all the messages, right? I suspect it's a particular author or institution using such a client. >> Is it fixed in some release more recent than 0+20180916? This is almost surely not something we can fix. Based on the error, it's in third party code (Django) that we import. It's possible we could provide a more explanatory message, but the only thing we can sensibly do is hold the message for an admin to edit it by hand. As long as the text in question and the Content-Type header aren't too sensitive, we can help with identifying charset (often UTF-8 as Mark suggests but why someone would be encoding FINAL MIDDLE DOT is a mystery to me). > I'm not sure what's going on here. My first thought is the database is > MySQL or MariaDB and this is a 4-byte UTF-8 encoding and the database > column charset definition is utf8 and not utf8mb4, but > '\\xE1\\x90\\xA7\\x0A\\x0A' is the 3-byte UTF-8 encoding for CANADIAN > SYLLABICS FINAL MIDDLE DOT ('\\xE1\\x90\\xA7') followed by two > newlines. so I'm not sure what the issue is. I'm guessing home-made client, possibly Asian or a spammer, which provides no content-type header. Even today I see raw 8-bit encoded text without an encoding spec from legitimate Japanese and Chinese sources, and of course with spammers all bets are off -- they might even do it deliberately to crash filters. -- University of Tsukuba　　　　　　　　　　　　 Faculty of Policy and Planning Sciences Tennodai 1-1-1, Tsukuba 305-8573 JAPAN tel/fax: +81-29-853-5091 turnbull(a)sk.tsukuba.ac.jp 　　　　　　　　　　　　　　https://turnbull.sk.tsukuba.ac.jp/

3 years, 5 months

[MM3-users] Re: Bulk changes to delivery mode and moderation action

by David Newman

On 1/7/22 2:48 PM, Mark Sapiro wrote: > On 1/7/22 1:51 PM, David Newman wrote: >> >> However, there's no evidence test messages to the list are going out. >> (Nondelivery was the reason I was mucking around in the admin panel in >> the first place.) >> >> The Postfix log shows delivery to the list address. >> >> MM3's smtp.log shows the usual handshake. >> >> However, I do not then see messages going out to subscribers in the >> Postfix log, and as one of those subscribers I do not receive a copy >> even though my preferences are set to receive my own posts. > > > Are the messages in Mailman's var/queue/shunt queue? There are 24 messages in the shunt queue, the most recent of which is a digest from 0000 hours today. The test messages I sent later today are not there. There are also 13 messages in the out queue. I can't read these as the filenames appear to change on every invocation of ls. > Are there errors > and tracebacks in var/logs/mailman.log? Not in response to today's test messages. Here is an entry of a test message I sent going to Hyperkitty. Jan 07 12:49:03 2022 (252309) HyperKitty archived message <6e88ec49-1326-92f9-aa17-1d4c1d6828be(a)domain.tld> to https://lists.domain.tld/archives/list/listname@lists.domain.tld/message/FH… But the Postfix log doesn't show it being distributed to me or anyone else. Oddly, neither of the two list owner emails (me and someone else) show this message in the archive via the admin panel. However if I log in as the list owner directly to the archives: https://lists.domain.tld/archives/ I can see today's test posts if logged in as the list member, but do not see the posts if logged directly into the archive as the Django superuser. The nondelivery of list mail is more serious than the archive problem, though. Thanks in advance for further troubleshooting clues. dn > >

4 years, 9 months

[MM3-users] Re: Counter-intuitive header filter processing

by Jonas Meurer

Hi Mark, Am 18.09.2018 um 20:48 schrieb Mark Sapiro: > On 09/18/2018 10:26 AM, Jonas Meurer wrote: >> >> Apparently, that's not the case. If the default message acceptance >> setting for members is set to "reject", then mails from >> *(a)freesources.org get rejected even though the header filter says >> "accept". In other words, the default message acceptance action seems to >> overrule any header filters. >> >> Is this expected behaviour? > > > Yes. Rules are normally checked in a sequence defined in > mailman/chains/builtin.py and that sequence checks several things > including member moderation before checking header matches, so if > "Default action to take when a member posts to the list" is anything > other than "Default processing", that action will take precedence over > header matches. Thanks for the clarification. Maybe then the description for chain "accept" in postorius needs to be adjusted. Currently it says "Accept immediately (bypass other rules)". This gives the impression, that it will overrule the default action. A related followup question: what configuration is recommended for an announcement list where only specific senders (e.g. sender(a)example.org) should be allowed to post and mail from all (other) subscribers should be rejected? The only way I see to achieve this currently is: * Set default action for member posts to "Default processing" * Add a header filter to accept "sender(a)example.org" * Add a header filter to reject ".*" In my eyes, that's very counter-intuitive and cumbersome to configure. But maybe I just missed a more obvious and elegant solution to achieve the same? And one more comment: if I make the list an "anonymous list" (i.e. "hide the sender of a message, replacing it with the list address"), then header filters for the "from" header don't work any longer. That's kind of unexpected as well. I would have expected mailman to process the header filters at reading the incoming mail, not after rewriting headers of the outgoing mail? Cheers jonas

8 years, 1 month

[MM3-users] Re: Mailman runners missing - nothing in logs

by Torge Riedel

Am 18.01.20 um 23:14 schrieb Mark Sapiro: > On 1/18/20 6:13 AM, Torge Riedel wrote: >> So now back to good. I checked all the logs prior to restarting the >> service in directory /opt/mailman/var/log, but I do not see any errors. >> And the master was still running, I would expect that the master will >> start / restart a runner in case it died? > > What's in /opt/mailman/var/log/mailman.log? There should be entries like > > Jan 18 13:51:11 2020 (11700) xxxx runner started. > > for each runner every time it starts and like > > Jan 18 13:47:31 2020 (10096) xxxx runner caught SIGTERM. Stopping. > Jan 18 13:47:31 2020 (10096) xxxx runner exiting. > > for each runner every time it stops, but with perhaps a reason other > than `caught SIGTERM` > > That should at least tell you when they stopped. Also, if they stopped > for some reason. The master will only restart runners that exit if they > exit because of SIGUSR1 or some internal error and even then, only the > configured max_restarts number of times. > > Do you have a logrotate script for the logs in /opt/mailman/var/log, and > if so, does it have a postrotate script that signals Mailman to reopen > logs other than by a `mailman reopen` command, perhaps by signaling the > master with other than SIGHUP? > Hi Mark, I checked the logs for such entries and I do only see them when the logrotation runs and from yesterday where I restarted the service by myself. So - yes, I have logrotation configured. The logrotation has a postrotate script configured which executes `systemctl reload mailman` and I see in the logs, that it handles SIGUSR1. This happens daily in the morning and runs - for what I see - without any problems. If I understand you right I should change it to "reopen" instead of "reload". I think this is something I have to pass to mailman executable itself instead to systemctl, right? Best regards Torge

6 years, 9 months

[MM3-users] Re: Error while mass operation, no progress display while mass operation is running

by Stephen J. Turnbull

Mark Sapiro writes: > On 2/6/25 04:53, Markus Grandpré wrote: > > Unfortunately, I have not found any error message in the log > > files. > > If you are using uWSGI, a harakiri timeout should be logged in uWSGI's log. I think gunicorn logs to access.log and error.log in the Mailman logs directory. I would think it would log 500 level events, that indicates server trouble. > > Regardless of whether a mass operation process runs without errors or > > continues to run despite an error, I find it hard that Postorius does > > not display the progress of such a mass operation in order to monitor > > the process, as mass operations with more than 1000 members are very > > time-consuming. Perhaps this could be changed in a future version of > > Postorius? > > This is a reasonable request. The best place for such a request is > https://gitlab.com/mailman/postorius/-/issues/new It's reasonable, but for unsubscribe Postorius invokes the mass remove REST API. So to report progress there it would require new code in Mailman and mailmanclient to report progress, or Postorius could use an explicit loop over addresses as mass subscribe does -- which would be noticably slower, I suspect. Periodic heavy use of mass operations is a common use case at universities. It's not uncommon for each term to see thousands of list averaging scores of subscriptions or unsubscriptions, with both number of lists and average roster size spike at academic year transitions when departmental and university-wide lists turn over. We should look at speeding up mass operations. I suspect internally Mailman is looping over individual addresses, probably creating a separate RDBMS transaction for each one. It should be possible to batch these, allowing the RDBMS to optimize its own table locking. Perhaps for the university use case it would make sense to even have Mailman multiplex these transactions across multiple lists being accessed at the same time. Steve

1 year, 8 months

[MM3-users] Re: Twitter OAuth2

by Odhiambo Washington

On Tue, Dec 27, 2022 at 8:08 PM Mark Sapiro <mark(a)msapiro.net> wrote: > On 12/27/22 01:35, Odhiambo Washington wrote: > > 3. Lastly, I needed to add the relevant options under > > the SOCIALACCOUNT_PROVIDERS = in settings.py. > > > > Now I have this: > ... > > And those are working but was wondering what to add for Twitter. > > > I see from another post that you've decided against using Twitter, but > to answer the question, Twitter does not require adding anything to > SOCIALACCOUNT_PROVIDERS. > I tried it and ended up with: ERROR 2022-12-28 08:40:18,895 97440 django.request Internal Server Error: /accounts/twitter/login/ Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner response = get_response(request) File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/providers/oauth/views.py", line 43, in view return self.dispatch(request, *args, **kwargs) File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/providers/base/mixins.py", line 18, in dispatch return self.login(request, *args, **kwargs) File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/providers/oauth/views.py", line 75, in login client = self._get_client(request, callback_url) File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/providers/oauth/views.py", line 49, in _get_client app = provider.get_app(request) File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/providers/base/provider.py", line 30, in get_app return adapter.get_app(request, self.id) File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/adapter.py", line 204, in get_app app = SocialApp.objects.get_current(provider, request) File "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/models.py", line 31, in get_current app = self.get(sites__id=site.id, provider=provider) File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/query.py", line 650, in get raise self.model.DoesNotExist( allauth.socialaccount.models.SocialApp.DoesNotExist: SocialApp matching query does not exist. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)

3 years, 10 months

[MM3-users] Re: only show mail domains from web host on web-host-url

by Jens Günther

Thank you for your answer. I should have seen that in the Docs, but somehow missed it: https://docs.mailman3.org/en/new/prodsetup.html I set it in the corresponding file, but unfortunately it doesn't work :-( I tried other setting to prove, that the file is used (ALLOWED_HOSTS works, changed here from '*' to 'localhost' and my example-domains, this works). It seems to be the same as this user describes (a while ago): https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… How could I check, that the setting is used!? But maybe some additional information: - I use the Debian installation on a stable buster, so mailman is version 3.2.1 - I use apache2 as my web-server and use the given alias-conf-file - Python is version 3.7.3 - Postorius is Version 1.2.4 - HyperKitty is version 1.2.2 -> In Django admin-Panel (/mailman3/admin/) I added 2 Websites test1.xyz.de and test2.xyz.de -> In Postorius (/mailman3/postorius/domains/) I added 2 Domains as Mail Hosts test1.xyz.de and test2.xyz.de with the corresponding Web hosts as stated above -> I added 1 list (/mailman3/postorius/lists/) for each Mail Domain as follows: test1(a)test1.xyz.de and test2(a)test2.xyz.de -> then created a vhost in apache2 with SSL-cert and added the Domains in ALLOWED_HOSTS: test1.xyz.de and test2.xyz.de -> When I go to the 2 Testdomains (corresponding web hosts), I see on each both lists for the different Mail domains What do I miss? Or where to investigate? Thank you in advance, best regards. By the way: what'S the difference in the 2 docs? https://docs.mailman3.org/en/latest/index.html and https://mailman.readthedocs.io/en/latest/README.html Am 18.03.21 um 21:52 schrieb Mark Sapiro: > On 3/18/21 9:20 AM, jens.guenther(a)posteo.de wrote: >> Is it possible, to only show publicly (if advertised) these lists on a certain url, that are corresponding to the web host the url I accessed mailman through? > > Set FILTER_VHOST = True in your settings_local.py or where ever you put > your site Django settings. > >

5 years, 7 months

[MM3-users] Re: Apache+mod_wsgi issue

by Odhiambo Washington

On Fri, Dec 23, 2022 at 4:29 AM Mark Sapiro <mark(a)msapiro.net> wrote: > On 12/22/22 16:03, Odhiambo Washington wrote: > > > > There is just one more thing - I have this in settings.py: > > LOGGING['handlers']['file']['filename'] = > > '/opt/mailman/mm/var/logs/mailmanweb.log' > > > And what are LOGGING['handlers']['file']['level'], > LOGGING['handlers']['file']['class'] and > LOGGING['handlers']['file']['formatter'] > Those are left as defaults in my setup.py. > > However, the referenced file is ever empty. At what point is it supposed > to > > record anything? > > What is logged depends on `level` but even with `DEBUG` not much routine > stuff is logged. See https://docs.djangoproject.com/en/3.2/topics/logging/ I will read that and see if I get to pick up something. And now I am having a problem with mod_wsgi. First, the URL displayed when I load the page looks bogus to me. While I expect the URL to be https://mm3-lists.kictanet.or.ke/mailman3/domains/, mod_wsgi give me https://mm3-lists.kictanet.or.ke/mailman3/mailman3/domains/ - there is an extra /mailman3! So I have; 1. https://mm3-lists.kictanet.or.ke/mailman3/mailman3/lists/ - working, or appears to. 2. https://mm3-lists.kictanet.or.ke/mailman3/mailman3/domains/ - working, or appears to. 3. https://mm3-lists.kictanet.or.ke/mailman3/mailman3/bans/ - working, or appears to 4. https://mm3-lists.kictanet.or.ke/mailman3/mailman3/users - working, or appears to 5. https://mm3-lists.kictanet.or.ke/mailman3/mailman3/lists/kictanet.lists.kic… - This throws an exception error when I click "Manage Subscription". -> https://pastebin.ubuntu.com/p/RkKH793Jgw/ And while I do not quite understand the error itself, there is also one particular line that I also saw that has left me baffled: [Fri Dec 23 08:53:18.290411 2022] [core:info] [pid 91236] [client 197.232.81.246:13865] AH00128: File does not exist: /usr/local/www/apache24/data/archives/list/ kictanet(a)lists.kictanet.or.ke/thread/VIHCC6MSXZSNHY7YPEJ3D2US4N7MHJEC/, refe rer: https://mm3-lists.kictanet.or.ke/archives/list/kictanet@lists.kictanet.or.ke -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)

3 years, 10 months

[MM3-users] Re: About to Install MM3 -- Seeking clarification

by Onyeibo Oku

On Thu, 13 Oct 2022 12:12:32 -0700 Mark Sapiro <mark(a)msapiro.net> wrote: > On 10/12/22 22:03, Onyeibo Oku wrote: > > Subsequent parts of the doc show the path "/opt/mailman/mm/var/*" > > which should correspond to "/usr/lib/mailman/mm/var/*" on my > > instance. However, I don't see any instructions on how to get the > > "/mm" folder and the associated sub-folders. I just have > > "/usr/lib/mailman" > > I think if you have `var_dir: /usr/lib/mailman/mm/var` in the > /etc/mailman3/mailman.cfg file you create per > https://docs.mailman3.org/en/latest/install/virtualenv.html#installing-mail…, > then starting Mailman core `/usr/lib/mailman/venv/bin/mailman start` > will create it. > Thanks Mark, that was very helpful. I now have "mm" folder. I have reached the section "Starting Mailman automatically". After creating the cfg file. Here is what I get: # systemctl status mailman3 mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2022-10-19 19:52:03 UTC; 4min 51s ago Process: 782 ExecStart=/opt/mailman/venv/bin/mailman start (code=exited, status=1/FAILURE) CPU: 156ms Oct 19 19:52:02 mail.server.tld systemd[1]: Starting mailman3.service - GNU Mailing List Manager... Oct 19 19:52:03 mail.server.tld mailman[782]: Traceback (most recent call last): Oct 19 19:52:03 mail.server.tld mailman[782]: File "/opt/mailman/venv/bin/mailman", line 33, in <module> Oct 19 19:52:03 mail.server.tld mailman[782]: sys.exit(load_entry_point('mailman==3.3.5', 'console_scripts', 'mailman')()) Oct 19 19:52:03 mail.server.tld mailman[782]: File "/opt/mailman/venv/bin/mailman", line 25, in importlib_load_entry_point Oct 19 19:52:03 mail.server.tld mailman[782]: return next(matches).load() Oct 19 19:52:03 mail.server.tld mailman[782]: StopIteration Oct 19 19:52:03 mail.server.tld systemd[1]: mailman3.service: Control process exited, code=exited, status=1/FAILURE Oct 19 19:52:03 mail.server.tld systemd[1]: mailman3.service: Failed with result 'exit-code'. Oct 19 19:52:03 mail.server.tld systemd[1]: Failed to start mailman3.service - GNU Mailing List Manager. Mailman starts well from virtual environment though, ... but systemd doesn't replicate that success. Regards Onyeibo

4 years

[MM3-users] Re: Confirm on mass subscription

by tlhackque

On 05-Jul-17 22:09, Mark Sapiro wrote: > On 7/5/17 1:48 PM, David Krantz wrote: > > >> Is there a way of requiring all mass subscribed users to confirm their >> email adress? I have an use case where we have a current mail list but >> when moving it to mailman I for different reasons would like to >> confirm all subscribers. Some very shallow testing suggests that if I >> just add them using mass subscribe, they will not have to confirm at >> all. >> >> I have yet to upgrade to 3.1 but I will do so in due time. > > In MM 2.1, you just send the users invitations rather than subscribing > them. In MM3/Postorius, this does not seem to be an option at present. > Actually, 2.1 lets you do it either way. The first radio button on Mass Subscriptions is: "Subscribe these users now or invite them? " MM 2.1 also allows control of whether the new subscribers and/or list owner are notified. And it allows custom text to be added to the invitation/notification (when one is sent). MM3 has none of these controls. Postorious issue #203 and mailman #362 refer to the MM 2.1 regressions. However, I read this somewhat differently: given a list with subscribers (however they were subscribed), it is sometimes useful to be able to ask the members to re-confirm their address/interest. For example, when there's a change of policy (terms of use), or when local opt-in regulations require that that opt-in be reconfirmed periodically. It's handled out-of-band now, but this would seem to be something that Mailman could manage - change users' state to "unconfirmed" and issue the re-invite/re-subscribe notice. But there are some details: re-confirmation probably doesn't want to require moderator re-approval (at least not always). How this interacts with "don't deliver mail from this list" needs definition (probably overrides), but some members (notably moderators/owners/robots) need to be exempt. And like 2.1, the user notification needs to be different to explain that it's a re-confirmation.

9 years, 3 months

Search results for query "sapiro"