Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: Lost posting in archives

by Axel Rau

> Am 22.01.2022 um 02:39 schrieb Mark Sapiro <mark(a)msapiro.net <mailto:mark@msapiro.net>>: > > On 1/21/22 3:06 PM, Axel Rau wrote: >> I created a list, did migration from 2.1 and run update_index_one_list. >> Then I posted a test mail to the list and it got distributed and was seen in the archive. >> Now 2 days later it does no longer show up in the archive. >> What is going on here? >> May this have to do with >> HYPERKITTY_RENDER_INLINE_IMAGE >> which I have turned on meanwhile. > > It may. Is there anything in Django's log about this? Yes: - - - Jan 19 16:46:59 2022 (1997) HyperKitty failure on https://lists.framail.de/hyperkitty/api/mailman/urls: <https://lists.framail.de/hyperkitty/api/mailman/urls:> <html><title>Forbidden</title><body> <h1>Access is forbidden</h1><p>Please check the IP addresses assigned to MAILMAN_ARCHIVER_FROM in the settings file. </p></body></html> (403) - - - I have: - - - MAILMAN_ARCHIVER_FROM = ('{{ fqdn }}') - - - Should I put localhost here? > If you have the original message from the list, can you go to the URL in the Archived-At: header? Is empty: - - - Archived-At: <> - - - > is the message in the hyperkitty_email table in the database? E.g., > > SELECT * FROM hyperkitty_email WHERE message_id_hash ='xxx'; > > (xxx is the value of the Message-ID-Hash: header from the message) No. > > If the prototype archiver is enabled for the list, the message should be in Mailman's var/archives/prototype/list(a)example.com <mailto:var/archives/prototype/list@example.com>/new directory and that message will have the Archived-At: and Message-ID-Hash: headers. No. How can it be enabled? That’s all I have: - - - (mailman_33) [root@lists5 /usr/local/mailman3]# ls -lR var/archives total 1 drwxrwx--- 3 mailman mailman 3 18 Jan. 18:05 hyperkitty var/archives/hyperkitty: total 25 drwxrwx--- 2 mailman mailman 6 19 Jan. 16:46 spool var/archives/hyperkitty/spool: total 1310 -rw-rw---- 1 mailman mailman 838274 19 Jan. 16:46 1642610819.7180374+05663686ecac2e85272bac93299d8469aa299a6d.pck -rw-rw---- 1 mailman mailman 8888 19 Jan. 16:46 1642610819.7836225+51d3aa9170bfb408821d5d05bd50acb687f23c19.pck -rw-rw---- 1 mailman mailman 13215 19 Jan. 16:46 1642610819.846788+bdfe7825a5a718d081683cf9cfa53cc1f36e64f2.pck -rw-rw---- 1 mailman mailman 469150 19 Jan. 16:46 1642610819.9367082+6c8adeae7d43ec9f721959a57bc1d1415c0b4857.pck - - - Thanks for your time, Axel --- PGP-Key: CDE74120 ☀ computing @ chaos claudius

3 years, 3 months

[MM3-users] Re: Uncaught runner exception: 'str' object has no attribute 'decode'

by Abhilash Raj

On Tue, Oct 3, 2017, at 12:28 AM, Henrik Rasmussen wrote: > > Fra: Mark Sapiro [mailto:mark@msapiro.net] > > Sendt: 2. oktober 2017 17:26 > > Emne: [MM3-users] Re: Uncaught runner exception: 'str' object has no attribute 'decode' > > > >> Is there some kind of workaround, I can give my users or something that the list admins can do as a workaround? > > > > There is no user or list admin workaround, but the defense is a fairly > > simple patch. See the MR at > > <https://gitlab.com/mailman/mailman/merge_requests/322> and the commit > > at > > <https://gitlab.com/mailman/mailman/commit/37b3f62afdf225a8e29b66fcefcdb56b9…>. > > The commit changes 7 files, but only two are needed. The rests are tests > > or reverting <https://gitlab.com/mailman/mailman/issues/256> which isn't > >necessary. > > > > The only changes you need are those to mailman/email/message.py and > > mailman/mta/connection.py. > > I am using Docker-mailman (not yet very familiar Docker and version > control). I would expect I have to wait for a Docker-mailman release > including this change, right? > > I have at least five+ lists which almost haven't been working since my > switch from Mailman 2.1 to Mailman 3.1 (Docker-mailman 0.1.1), and I am > desperate trying to figure out whether it is best to fall back to the > Mailman 2 server, for the time being, where the lists came from, or it is > better to wait for an update to the Docker-image. > > Will https://github.com/maxking/docker-mailman/milestone/2 fix my > problems regarding failed mail handling and will it become available soon > (when?) (https://github.com/maxking/docker-mailman/issues/147)? The container images have stable releases of Mailman components, so those changes will make in to the container when a new Core release happens. The next image release is pending a version upgrade of Hyperkitty, to 1.1.3, but other changes are just meant to make the process of setting up Mailman better. They do not include changes from git repo after the release. -- Abhilash Raj maxking(a)asynchronous.in

7 years, 7 months

[MM3-users] Re: mail stuck in "out" queue

by Dmitry Makovey

So our mailman got stuck again, but I did follow your advise. Here's what I've found: On 3/29/19 2:58 PM, Mark Sapiro wrote: > On 3/29/19 2:28 PM, Dmitry Makovey wrote: >> I've had encountered this situation now twice: mail is being stuck in >> "out" queue and nothing in the logs indicates the issue. Is there a way >> to ask mailman-core politely to attempt delivery (perhaps with increased >> debugging in the logs) *without* restarting the core (which does solve >> *this* problem but creates other problems, like availability of the >> mailman as a whole) > > > Is the "out" runner running? yes: # docker top mailman-core | grep -F out: 100 21681 13215 0 Mar28 ? 01:14:16 /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg > ps -fwwA |grep runner=out > > If not, you should be able to start it as the Mailman user with a > command like > > /path/to/python3 /path/to/mailman/bin/runner -C /path/to/mailman.cfg > --runner=out:0:1 > > See 'ps -fwwA|grep runner' for examples > > If the out runner is running, is there a file named *.bak along with all > the *.pck files in queue/out/ there is one, yes: # ls -l /srv/mail/mailman/core/var/queue/out/*.bak -rw-rw---- 1 100 65533 21512 Apr 14 19:45 /srv/mail/mailman/core/var/queue/out/1555296331.8386078+2c905825b82708e5a682b872b0aff320c9dd0b2e.bak what does it indicate and what should my steps be to resolve this (other than restarting queues)? > And if it's not, is there anything in Mailman's logs to indicate why it > died? > > Also, 'mailman restart' should only take a matter of seconds, but if the > out runner is running but not processing its queue, I think sending it a > SIGINT should restart it. SIGINT killed the process allright, but it didn't auto-respawn. I ended up spawning it with: /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg but soon realized it doesn't go to background nicely so I've nohup'ed it in the end. but it did solve my problem. -- Sr System and DevOps Engineer SoM IRT

6 years, 1 month

[MM3-users] [Suspected Spam]Re: [MM3-users]Exim, Mailman3 and missing Message-ID:

by Stephen J. Turnbull

Mark Sapiro writes: > Rob's reply in this thread was garbled due to > <https://gitlab.com/mailman/mailman/issues/616>. The text of the reply > is quoted below. > > On 7/27/19 12:54 AM, Rob Lister wrote: > > > > One thing that does occur to me is that my exim header fudge will > > probably cause the Message-ID header to move around from its original > > position, and if someone is doing hashing or actually validating (gpg > > signing?) the headers haven't been tampered with, then it might > > complain, since although the message-id hasn't changed, it might be in a > > different place in the headers. I don't think this is a problem. GPG or S/MIME signing with any software I know of only signs the message body (or a MIME part). DKIM message signing isn't going to care about the order of fields because DKIM canonicalizes the order (at least for the unique fields) according to the specification in the signature field, and that will be verified on the way in. Of course adding a Message-ID will likely break DKIM for the outgoing post, but Mailman as normally used breaks DKIM anyway. This can be partly mitigated by using ARC (authenticated received chain), which is supported by Google already, and possibly some of the other usual DMARC suspects (Yahoo!, AOL). > > Probably the more correct way to do it is only add a new header if it's > > missing, and if it's there, leave it alone. True, and I think this does it: headers_add = ${if def:h_message-id:{fail}{Message-ID: ... id generator code ...}} If you like that and test it, I would really appreciate feedback so we can update the documentation. One thing *I* would like to add to this recipe is logging. I'd be curious to see how frequently it happens outside of spam and junior high school hackers using netcat as their MUA. I don't see how to do this in Exim, though. Another is using a prefix other than "E" to indicate this isn't standard Exim behavior. Steve

5 years, 9 months

[MM3-users] Re: UnicodeEncodeError: 'ascii' codec can't encode character

by Marvin Gülker

Hi, Am 10. Dezember 2019 um 10:25 Uhr -0800 schrieb Mark Sapiro: >The underlying issue here is probably ><https://bugs.python.org/issue32330>, but that knowledge doesn't help >us. We need to get rid of the offending message in the message store. >The first step is in mailman shell. Note the paths /opt/mailman/mm/bin/ >and /opt/mailman/mm/var/ will need to be adjusted. Thanks for this information. Ubuntu's mailman stores the messages below /var/lib/mailman3/messages. The "mailman" command is in the $PATH. >This should print the file name and Message-ID-Hash (two lines the same) >for each message in the messages directory until it prints only one line >followed by the UnicodeEncodeError exception. That list line is the file >name/Message-ID-Hash of the offending message. I've execucted that code, but it does not cause an exception at all. It quietly runs through. I've attached the transcript of the shell session to this mail. With a little crude use of grep(1), however I found my held message in WQ/ZS/WQZSFFJLVMWQHMG4VYKGXJEKQVQXVK67. Does the messages directory contain only held messages? I've looked at some messages and all of these I looked at were spam. The directory structure contains 40 messages in total, accumulating to 480 KiB disk space. If required, I can zip that directory up and e-mail it to you for inspection. >DELETE FROM message WHERE message_id_hash = xxx; As the exception you expected did not occur, I didn't do this yet. I can make a database and directory backup and then run it, though. Should I try? I assume after this I should also delete the respective files from the messages/ directory? >As of Mailman core 3.3.0 there is a 'mailman notify' command designed to >be run periodically by cron to notify owners/moderators of prnding requests. I did receive an e-mail to the list owner address that told me that the e-mail from my incorrect e-mail address "requires approval". That was the first time I received such an e-mail. In parallel, on the incorrect e-mail address, I received the notification for moderator approval. Marvin -- Blog: https://mg.guelker.eu

5 years, 5 months

[MM3-users] Re: possible backscatter attack using Mailman3 servers

by David Newman

On 1/3/25 5:44 PM, Mark Sapiro wrote: >> Greetings. On a system running Mailman 3.3.9 and Postfix, I'm seeing >> about 20-30 entries per day in the Postfix queue where it appears a >> Gmail user signs up for a mailing list that requires confirmation, and >> Gmail responds that the user is too busy to handle requests. >> >> There are no publicly advertised email lists on this server, and I >> don't ever see anything in the Mailman logs indicating the user ever >> tried signing up. > > > This is an attack mail bombing the user. The requests that result in the > can come via web or email. Mailman's logging of subscribes has been > missing most events through Mailman 3.3.10. See https://gitlab.com/ > mailman/mailman/-/issues/1143 which will be fixed in 3.3.11, but > subscribes waiting user confirmation still won't be logged. > > However, the message with subject "Please Confirm Your Email Address" > comes from Django allauth so it isn't actually Mailman sending it but > rather Django allauth as a result of a request to sign up for a Django > account at https://mail.example3.com/accounts/signup/. You can probably > find that request in your web server logs, and you may find the user > and/or email in the Django admin UI. Thanks VERY much for this. No such users in the Django UI, but the web server logs have 252 attempts from 132 unique IPv4 addresses registered to different ISPs throughout Europe. So, even though Mailman support for more detailed logging of sub and unsub requests would be useful, it likely would not have helped with attacks from many source IP addresses. > > Since django-mailman3 1.3.6, you can disable these signups by putting > > ACCOUNT_ADAPTER = 'django_mailman3.views.user_adapter.DisableSignupAdapter' > > in your Django settings, but then your users won't be able to sign up > for web accounts. I have made this change. As for not having web accounts, this just means new users cannot sign up to manage their Mailman settings, correct? I presume existing web accounts will continue to work. Thanks again! dn

4 months, 1 week

[MM3-users] Re: Disabled by bounce

by Allan Hansen

Thank you, Mark. By "However, all of the above can be avoided without any DMARC changes by enabling bounce probes." Do you mean that I can get rid of the 'From' mangling, which has been a frequent source of embarrassment because the list address is captured and reused by type-ahead when sending non-list messages and those messages instead going to 1000 subscribers. __ Allan On 2/1/23, 09:59, "Mark Sapiro" <mark(a)msapiro.net> wrote: On 1/31/23 22:32, Allan Hansen wrote: > Hi all, > > Apparently Google just started to get very aggressive about DKIM settings. Some hundreds of my subscribers have had their subscriptions disabled because of Google's bounces. > > I tried to 'Enable' some of them, expecting that their bounce scores would be reset, but when I list the users, the bounce scores are still 5. > > Please advise. Will enabling them without the bounce score being reset just cause the server to disable them again. Yes, Google has started enforcing a policy which is effectively DMARC p=reject even for senders whose domains don't publish DMARC p=reject or quarantine. To avoid this issue, you may need to set DMARC Mitigate unconditionally to Yes as well as DMARC mitigation action to Replace From: with list address if it isn't already. As far as enabling without resetting the bounce score is concerned, I think you are correct that they will just be disabled again on the next bounce. This is a bug and should be fixed. However, all of the above can be avoided without any DMARC changes by enabling bounce probes. in the [mta] section of mailman.cfg, set ``` verp_probes: yes ``` Since the probe is sent From: the list's domain, it should pass googles SPF and DKIM checks and not bounce. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: mailman-users(a)mailman3.org <https://lists.mailman3.org/archives/list/<a href=>/message/56WVO6NIOGOBAOOS2B4YCIDJN7VCIXTF/">https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… This message sent to hansen(a)rc.org

2 years, 3 months

[MM3-users] Re: Fwd: [Django] ERROR (EXTERNAL IP): Internal Server Error: /archives/search

by Dave Hall

Hello, Please pardon a slightly off-topic comment, but I would like to highlight a part of Odhiambo's post. Heavily nested inline replies also reflect the issue I wish to comment on, so I'm not going to. Granted, I am a slow learner, and perhaps already so mixed up after reading > the myriad HOWTOs for > installing MM3 that are available online. I am tempted to say there are > many like me who are likely to > miss "where to obtain" other settings from. and > Is it possible to modify the documentation to somehow make it obvious (in a > language that a layman can easily understand, I have seen this with several open source projects lately. There's no big picture, just 'hundreds' of cross-linked little bits of disjointed information that almost seem like documentation for individual modules. I once joined a large commercial project for a major US computer company (sorry, I won't share details). They were quite proud of their extensive documentation. The docs for their equivalent of the kernel syscall interface, when printed (two-sided) and stacked, would easily approach 2 meters tall. Yet, every bit of it was incomprehensible unless and until you called the developer and had them explain it to you. The problem that we all have, us humans, is that when we are very familiar with something, like the code we've been working on, it is very hard to describe its purpose to somebody who knows nothing about it. ("It's obvious to me, so it should be obvious to you."). So, a challenge for us all: When writing and reviewing your documentation, imagine that you're explaining your whole project face-to-face to your grandmother, or to anybody who never learned to program. From experience I promise that it's easy once you get the hang of it. Thank you and sorry for the speech. -Dave -- Dave Hall Binghamton University On Sat, Jan 28, 2023 at 2:54 AM Odhiambo Washington <odhiambo(a)gmail.com> wrote: > On Fri, Jan 27, 2023 at 9:34 PM Mark Sapiro <mark(a)msapiro.net> wrote: > > > On 1/27/23 05:53, Odhiambo Washington wrote: > > > On Mon, Jan 23, 2023 at 11:48 PM Mark Sapiro <mark(a)msapiro.net> wrote: > > >> > > >> Note that when following > > >> https://docs.mailman3.org/en/latest/install/virtualenv.html you > create > > >> /etc/mailman3/settings.py and it contains > > >> ``` > > >> from mailman_web.settings.base import * > > >> from mailman_web.settings.mailman import * > > >> ``` > > >> to establish a base set of settings that can be augmented or > overridden > > >> in /etc/mailman3/settings.py. mailman_web.settings.mailman in turn > > contains > > >> ``` > > >> HAYSTACK_CONNECTIONS = { > > >> 'default': { > > >> 'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', > > >> 'PATH': "fulltext_index", > > >> ``` > > >> which is a relative path for fulltext_index and may need to be set to > an > > >> absolute path. > > >> > > > > > > Mi Mark, > > > > > > I have read this page - > > > https://docs.mailman3.org/en/latest/install/virtualenv.html and > further > > - > > > > > > https://docs.mailman3.org/en/latest/install/virtualenv.html#initial-config > > > - without clicking any external links. > > > I still don't see the HAYSTACK_CONNECTIONS in the settings.py included > > on > > > that page. > > > I am missing something obvious, no? > > > > > > Please reread what I wrote above. HAYSTACK_CONNECTIONS is not in the > > suggested settings.py but it is in mailman_web.settings.mailman and is > > imported by the suggested settings.py. > > > I have reread what you wrote now and I'm beginning to get the hang of this > method. > My observation is that it can only be _obvious_ to someone very familiar > with Python config > to decipher that *additional* settings *that can be augmented or > overridden > in /etc/mailman3/settings.py* can be obtained from: > /PATH-TO/mailman_web/settings/{base|mailman}.py after they have noticed > these two lines in > the sample settings given in /etc/mailman3/settings.py: > > from mailman_web.settings.base import *from > mailman_web.settings.mailman import * > > [At first instance, it is not obvious that mailman_web.settings.base and > mailman_web.settings.mailman are > actually paths in Python - which magically translates to > mailman_web/settings/base.py and mailman_web/settings/mailman.py] > Granted, I am a slow learner, and perhaps already so mixed up after reading > the myriad HOWTOs for > installing MM3 that are available online. I am tempted to say there are > many like me who are likely to > miss "where to obtain" other settings from. > For example, my mind all along has been tuned to having a settings_local.py > for overriding the settings.py. > However, with this particular documentation, it would appear that > everything is intended to just go into /etc/mailman3/settings.py. > > Is it possible to modify the documentation to somehow make it obvious (in a > language that a layman can easily understand, > that those two "import *" _DO establish a base set of settings that can be > augmented or overridden in /etc/mailman3/settings.py_ > and that those settings are in what file(s) ? > > I am not saying @Johannes Rohr <johannes(a)rohr.org> is a layman like me, > but > I think that is what hit him. > > > -- > Best regards, > Odhiambo WASHINGTON, > Nairobi,KE > +254 7 3200 0004/+254 7 2274 3223 > "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to kdhall(a)binghamton.edu >

2 years, 3 months

[MM3-users] Re: Held messages: Long time waiating for response of the Mailman API

by Stephan Krinetzki

Mark Sapiro wrote: > On 1/26/22 00:11, Stephan Krinetzki wrote: > > After we enabled the "slow query log", we got in the postgresql database the following output: > > 2022-01-25 12:49:41.538 CET [16126] LOG: unexpected EOF on client connection with an open transaction > > 2022-01-25 15:34:36.047 CET [16864] LOG: duration: 30760.028 ms statement: DELETE FROM pendedkeyvalue WHERE pendedkeyvalue.id = XXX > > 2022-01-25 15:34:36.047 CET [3900] LOG: duration: 9372.471 ms statement: DELETE FROM pendedkeyvalue WHERE pendedkeyvalue.id = XXX > > 2022-01-25 16:42:07.243 CET [3900] LOG: unexpected EOF on client connection with an open transaction > > > > > > Pretty sounds like a database problem? > > The unexpected EOF is because the gunicorn worker timed out and was killed. > There are two DELETE FROM pendedkeyvalue WHERE pendedkeyvalue.id = XXX > transactions because the first one timed out. These came from > https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/app/moderator.p... > The accept action is processed and ultimately we get to > https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/app/moderator.p... > and that in turn is doing the DELETE at > https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/model/pending.p... > There must be some kind of database locking causing this, but I have no > clue as to what it might be. If you haven't done so, I would stop all > the mailman services and the database server and then start them again. > I don't know if that would help, but it might. We did this and nothing changed. I even tried to lookup up potential DB locks, but they are no locks in the database. Side note: Wie have a 37GB big Database with ~4.000 Mailinglists and ~70.000 addresses > > sometimes the log (mailmansuite.log) show us this error > > That too is from the gunicorn worker timing out and being killed. Okay. We already set the timout of the gunicorn process of the mailmancore higher (6 Min) I think there is a communication problem in the chain: apache -> uwsgi -> mailmancore/API ->database but i can't dig deeper there. Let me know if i should provide more details or logs

3 years, 3 months

[MM3-users] Re: mm3 installation question

by Stephen J. Turnbull

Odhiambo Washington writes: > Mark Sapiro writes: > > Some of the `path` settings are probably not in the same order as > > in that file. The > > > > ``` > > path('mailman3/', include('postorius.urls')), > > path('archives/', include('hyperkitty.urls')), > > ``` > > settings should come after the > > ``` > > path('postorius/', include('postorius.urls')), > > path('hyperkitty/', include('hyperkitty.urls')), > > ``` > > settings (unfortunately, mailman-web 0.0.8 has these in the wrong > > order). Or you can just delete the 'postorius/' and 'hyperkitty/' > > ones or you can add > > ``` > > ProxyPass "/postorius" "http://127.0.0.1:8000/postorius" > > ProxyPass "/hyperkitty" "http://127.0.0.1:8000/hyperkitty" > > ``` > > to your Apache config. > The odd thing is that the HOWTO that he followed never involves the > manipulation of any urls.py at all - it's not even mentioned at any > point. So obviously the issue is caused by something else and I > suspect something within Apache needs to be re-evaluated in the > guide. I believe this is the Apache configuration the OP is using: <IfModule mod_proxy.c> ProxyPreserveHost On ProxyPass "/mailman3" "http://127.0.0.1:8000/mailman3" ProxyPass "/archives" "http://127.0.0.1:8000/archives" ProxyPass "/accounts" "http://127.0.0.1:8000/accounts" ProxyPass "/admin" "http://127.0.0.1:8000/admin" ProxyPass "/user-profile" "http://127.0.0.1:8000/user-profile" </IfModule> Note that Apache doesn't know about the "/postorius" and "/hyperkitty" URI paths. It is quite unclear to me how this works at all with either the path('mailman3/', include('postorius.urls')), path('archives/', include('hyperkitty.urls')), path('postorius/', include('postorius.urls')), path('hyperkitty/', include('hyperkitty.urls')), version of urls.py or the reverse order version path('postorius/', include('postorius.urls')), path('hyperkitty/', include('hyperkitty.urls')), path('mailman3/', include('postorius.urls')), path('archives/', include('hyperkitty.urls')), I think Mark must have a configuration with ProxyPass "/" "http://127.0.0.1:8000/" in the Apache configuration in mind? Or an alias or rewrite rule. Otherwise I don't see how "/postorius" and "/hyperkitty" paths ever get transmitted to the WSGI server. Steve

1 year, 2 months

Search results for query "sapiro"