Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: Hyperkitty on Debian 12

by Michael Oberrauch

On Tue, 2024-03-26 at 08:06 +0100, Markus Grandpré wrote: > Dear Mr Sapiro, > > thank you very much for your help. The required command is: > > # mailman-web hyperkitty_import \ > -l mailman21-migration-test-04(a)mailman.uni-konstanz.de \ > /var/mailman21/testlist.mbox > > Due to the following warnings, I still have to customize Hyperkitty: > > django_mailman3.MailDomain: (models.W042) Auto-created primary key > used > when not defining a primary key type, by default > 'django.db.models.AutoField'. > HINT: Configure the DEFAULT_AUTO_FIELD setting or the > DjangoMailman3Config.default_auto_field attribute to point to a > subclass > of AutoField, e.g. 'django.db.models.BigAutoField'. > ... > > Can you give me another hint? As Linux admin I installed the packages > mailman and mailman-web that were offered to me under Debian 12. > Although during the installation of the packages user "list" was > created, I cannot use it: > > # su - list > This account is currently not available > > As with Mailman 2.1, I will probably have to continue working with > Mailman3 as root. Too bad that there is no documentation for Debian > admins like me, who don't want to dive too deep into the python > framework. Or have I overlooked something? > > Best regards, > Markus > > Hi Markus, we are also running MM3 on Debian 12 from Package Repositories. First of all, you should not try to directly switch user (su) to the lists user, it's generally intended to executes commands as that user, if necessary; either using `runuser`, `sudo`, or `su -c`. However, the prepackaged scripts `mailman-wrapper` and `mailman-web` already do this and therefore you ca run those as root without permission problems. On Debian there are two relevant users for MM3: the list user runs mailman core, whereas the www-data user runs Postorius and Hyperkitty. So you have to make shure, that the www-data user can access the .mbox files while importing. Hope that helps. Cheers, Michael -- Michael Oberrauch (er/ihm) Computerreferent Fachschaft Mathematik/Physik/Informatik/Chemie Studentische Vertretung Technische Universität München Boltzmannstraße 3 85748 Garching eMail: oberrauch(a)fs.tum.de Tel: (+49) 089 289 18545 Fax: (+49) 089 289 18546 https://mpic.fs.tum.de

1 year, 3 months

[MM3-users] Re: Timeout error on anonymous subscribe

by vogt＠spamcop.net

Mark Sapiro wrote: > On 10/23/24 13:23, Gerald Vogt wrote: > > I don't quite understand the relevance of the user accessing the > > localhost tcp port. Any local user can do that. > I am trying to duplicate what Mailman does in an attempt to find the issue. > > I can access the template content with > > curl > > http://localhost:8000/mailman3/api/templates/list/community.lists.example.co... > > as well as > > curl > > http://lists.example.org/mailman3/api/templates/list/community.lists.example... > > as any user on the mailman server. > > How about > curl -A 'python-requests/2.32.3' > http://lists.example.org/mailman3/api/templates/list/community.lists.exampl… > > as the Mailman user? I have tried anything I could think of but I cannot reproduce the issue using curl as user mailman (or any other for that matter) Going directly for localhost:8000, using some or all of the headers I can capture with tcpdump: ``` $ curl -v -H 'Accept-Encoding: gzip, deflate' -H 'Connection: Keep-Alive' -H 'Host: lists.example.org' -A 'python-requests/2.32.3' http://localhost:8000/mailman3/api/templates/list/community.lists.example.c… ``` as well as going through the local httpd: ``` $ curl -v --http1.1 -H 'Accept-Encoding: gzip, deflate' -H 'Connection: Keep-Alive' -H 'Host: lists.example.org' -A 'python-requests/2.32.3' https://lists.example.org/mailman3/api/templates/list/community.lists.examp… ``` Running tcpdump during those runs shows me it's sending the same headers as mailman does. Thus on http protocol level it's the same, but still the server returns immediately. What I have just noticed is that the Date header in the delayed response is the "late" time. I have even compared tcp headers from either capture. They look identical down to the frame sizes. The only difference the last times is that with curl, the server side sends the ACK for the request and milliseconds later a PSH,ACK with the response headers. When it comes internally from mailman it sends the ACK for the request and then 5 seconds later it sends the PSH,ACK with the response headers. Thanks, Gerald

8 months, 1 week

[MM3-users] Re: search this list searches more then just this list

by Marco van Tol

Op 5 feb 2024, om 15:45 heeft Marco van Tol <mvantol(a)ripe.net> het volgende geschreven: > Op 29 jan 2024, om 18:29 heeft Mark Sapiro <mark(a)msapiro.net <mailto:mark@msapiro.net>> het volgende geschreven: >> On 1/29/24 03:09, Marco van Tol wrote: >>> Right now I’m back at the original issue as I see no other solution than to go back to whoosh. >>> Thanks! >>> Marco van Tol >>> Paste: >>> ---- >>> Indexing 194620 emails >>> [ERROR/MainProcess] Failed indexing 156001 - 157000 (retry 5/5): Term too long (> 245): XSUBJECThttp://www.google.com/url?q=%68%74%74%70%73%3a%2f%2f%68%64%72%65%64… (pid 32): Term too long (> 245): XSUBJECThttp://www.google.com/url?q=%68%74%74%70%73%3a%2f%2f%68%64%72%65%64… >> >> >> I'm not sure why the patch you are using doesn't avoid this, but you could try the other patch at <https://github.com/alexsilva/xapian-haystack/commit/a53523d2d0d13929a0729d4…> instead. If that fails, you could always find the offending message in the database maybe with a query like >> ``` >> SELECT * FROM hyperkitty_email Where subject LIKE 'http://www.google.com/url?q=\%68\%74\%74%'; >> ``` >> and modify or delete it - it's probably spam anyway. > > I tried the other patch, which looked very promising until something that had come in over SMTP some day threw a spanner in the wheels: > ``` > [ERROR/MainProcess] Failed indexing 287001 - 288000 (retry 5/5): Term too long (> 245): XTEXTº@[åeèúp¢i*h{õimô;]>ò&žôyþiýã#dzç8"¹ë= > ;æmyš€vqe.âés:æä>üzúõœ'âú·ž[]kzñ-µ€3æfdñù£8çô<bœkkñ/ãžæjïw¿òþp-ùšã7/'ûvksqé (pid 404): Term too long (> 245): XTEXTº@[ > åeèúp¢i*h{õimô;]>ò&žôyþiýã#dzç8"¹ë=;æmyš€vqe.âés:æä>üzúõœ'âú·ž[]kzñ-µ€3æfdñù£8çô<bœkkñ/ãžæjïw¿òþp-ùšã7/' > ûvksqé > ``` > > I agree that this is very likely the result of some spam, but the main point is that something that comes in over SMTP shouldn’t cause manual work on the listserver side. > > I’ll try it with a maximum length of 122 which should never explode into more than 244 bytes, unless I overlook something. This ended up working. :-) Summary: Patch from https://github.com/alexsilva/xapian-haystack/commit/a53523d2d0d13929a0729d4… But with `TERM_LENGTH_LIMIT = 122` And str() instead of force_str() Thanks so much for all your efforts! Marco van Tol

1 year, 4 months

[MM3-users] Re: Mailman, etc. upgrade woes and persistent bugs

by Allan Hansen

Mark, Thank you for this reminder, but it’s not really sufficient for most. The ‘Mailman Settings’ dialog does, indeed show the other preferences tags, and yes, as you say, the dialog for ‘List-based preferences’ only shows values that are not inherited and when you set a value there, it shows. There a two problems with this: a. Once a value is overridden on the ‘List-based preferences’ dialog, you cannot go back to it being inherited by removing the override. b. You cannot see, on this dialog, what the inherited value is. This is not user-friendly. May I suggest the following: That each option shows the following: o Value 1 o Value 2 (or the equivalent for pull-down menus) A box showing inheritance status: [ ] Use inherited value from address-based preference: Value2 If the ‘Use inherited value’ is chosen, then the inherited value shows. but the value radio buttons become non-selectable. If not, then whatever value the value radio box has become the value for the list, From what I see, the ‘Use inherited value’ is the default for new lists/addresses. To change a value, the user would uncheck this box, making the radio buttons selectable, and then set the desired value. Yours, Allan > On Feb 12, 2021, at 11:52 , Mark Sapiro <mark(a)msapiro.net> wrote: > >> I then looked at several of the disabled accounts, but the subscription info page in Postorius had no information about whether an account was enabled or disabled. Why is this not displayed???. The members can't see if their account is enabled. Is this another example of the disconnect between Mailman and Postorius? > > > If the user has a Django account, she can see all that info at (e.g. for > this list) <https://lists.mailman3.org/mailman3/accounts/subscriptions/ <https://lists.mailman3.org/mailman3/accounts/subscriptions/>> > She gets there from `Mailman settings` in the dropdown under her user > name. She can also get there via the `Manage Subscription` button on the > list's Info page. That takes her to `List-based preferences` for the > list. Any setting not selected there is inherited from the Address-based > preferences or Global Mailman preferences

4 years, 4 months

[MM3-users] Re: Lost posting in archives

by Axel Rau

> Am 22.01.2022 um 02:39 schrieb Mark Sapiro <mark(a)msapiro.net <mailto:mark@msapiro.net>>: > > On 1/21/22 3:06 PM, Axel Rau wrote: >> I created a list, did migration from 2.1 and run update_index_one_list. >> Then I posted a test mail to the list and it got distributed and was seen in the archive. >> Now 2 days later it does no longer show up in the archive. >> What is going on here? >> May this have to do with >> HYPERKITTY_RENDER_INLINE_IMAGE >> which I have turned on meanwhile. > > It may. Is there anything in Django's log about this? Yes: - - - Jan 19 16:46:59 2022 (1997) HyperKitty failure on https://lists.framail.de/hyperkitty/api/mailman/urls: <https://lists.framail.de/hyperkitty/api/mailman/urls:> <html><title>Forbidden</title><body> <h1>Access is forbidden</h1><p>Please check the IP addresses assigned to MAILMAN_ARCHIVER_FROM in the settings file. </p></body></html> (403) - - - I have: - - - MAILMAN_ARCHIVER_FROM = ('{{ fqdn }}') - - - Should I put localhost here? > If you have the original message from the list, can you go to the URL in the Archived-At: header? Is empty: - - - Archived-At: <> - - - > is the message in the hyperkitty_email table in the database? E.g., > > SELECT * FROM hyperkitty_email WHERE message_id_hash ='xxx'; > > (xxx is the value of the Message-ID-Hash: header from the message) No. > > If the prototype archiver is enabled for the list, the message should be in Mailman's var/archives/prototype/list(a)example.com <mailto:var/archives/prototype/list@example.com>/new directory and that message will have the Archived-At: and Message-ID-Hash: headers. No. How can it be enabled? That’s all I have: - - - (mailman_33) [root@lists5 /usr/local/mailman3]# ls -lR var/archives total 1 drwxrwx--- 3 mailman mailman 3 18 Jan. 18:05 hyperkitty var/archives/hyperkitty: total 25 drwxrwx--- 2 mailman mailman 6 19 Jan. 16:46 spool var/archives/hyperkitty/spool: total 1310 -rw-rw---- 1 mailman mailman 838274 19 Jan. 16:46 1642610819.7180374+05663686ecac2e85272bac93299d8469aa299a6d.pck -rw-rw---- 1 mailman mailman 8888 19 Jan. 16:46 1642610819.7836225+51d3aa9170bfb408821d5d05bd50acb687f23c19.pck -rw-rw---- 1 mailman mailman 13215 19 Jan. 16:46 1642610819.846788+bdfe7825a5a718d081683cf9cfa53cc1f36e64f2.pck -rw-rw---- 1 mailman mailman 469150 19 Jan. 16:46 1642610819.9367082+6c8adeae7d43ec9f721959a57bc1d1415c0b4857.pck - - - Thanks for your time, Axel --- PGP-Key: CDE74120 ☀ computing @ chaos claudius

3 years, 5 months

[MM3-users] Re: Uncaught runner exception: 'str' object has no attribute 'decode'

by Abhilash Raj

On Tue, Oct 3, 2017, at 12:28 AM, Henrik Rasmussen wrote: > > Fra: Mark Sapiro [mailto:mark@msapiro.net] > > Sendt: 2. oktober 2017 17:26 > > Emne: [MM3-users] Re: Uncaught runner exception: 'str' object has no attribute 'decode' > > > >> Is there some kind of workaround, I can give my users or something that the list admins can do as a workaround? > > > > There is no user or list admin workaround, but the defense is a fairly > > simple patch. See the MR at > > <https://gitlab.com/mailman/mailman/merge_requests/322> and the commit > > at > > <https://gitlab.com/mailman/mailman/commit/37b3f62afdf225a8e29b66fcefcdb56b9…>. > > The commit changes 7 files, but only two are needed. The rests are tests > > or reverting <https://gitlab.com/mailman/mailman/issues/256> which isn't > >necessary. > > > > The only changes you need are those to mailman/email/message.py and > > mailman/mta/connection.py. > > I am using Docker-mailman (not yet very familiar Docker and version > control). I would expect I have to wait for a Docker-mailman release > including this change, right? > > I have at least five+ lists which almost haven't been working since my > switch from Mailman 2.1 to Mailman 3.1 (Docker-mailman 0.1.1), and I am > desperate trying to figure out whether it is best to fall back to the > Mailman 2 server, for the time being, where the lists came from, or it is > better to wait for an update to the Docker-image. > > Will https://github.com/maxking/docker-mailman/milestone/2 fix my > problems regarding failed mail handling and will it become available soon > (when?) (https://github.com/maxking/docker-mailman/issues/147)? The container images have stable releases of Mailman components, so those changes will make in to the container when a new Core release happens. The next image release is pending a version upgrade of Hyperkitty, to 1.1.3, but other changes are just meant to make the process of setting up Mailman better. They do not include changes from git repo after the release. -- Abhilash Raj maxking(a)asynchronous.in

7 years, 8 months

[MM3-users] Re: mail stuck in "out" queue

by Dmitry Makovey

So our mailman got stuck again, but I did follow your advise. Here's what I've found: On 3/29/19 2:58 PM, Mark Sapiro wrote: > On 3/29/19 2:28 PM, Dmitry Makovey wrote: >> I've had encountered this situation now twice: mail is being stuck in >> "out" queue and nothing in the logs indicates the issue. Is there a way >> to ask mailman-core politely to attempt delivery (perhaps with increased >> debugging in the logs) *without* restarting the core (which does solve >> *this* problem but creates other problems, like availability of the >> mailman as a whole) > > > Is the "out" runner running? yes: # docker top mailman-core | grep -F out: 100 21681 13215 0 Mar28 ? 01:14:16 /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg > ps -fwwA |grep runner=out > > If not, you should be able to start it as the Mailman user with a > command like > > /path/to/python3 /path/to/mailman/bin/runner -C /path/to/mailman.cfg > --runner=out:0:1 > > See 'ps -fwwA|grep runner' for examples > > If the out runner is running, is there a file named *.bak along with all > the *.pck files in queue/out/ there is one, yes: # ls -l /srv/mail/mailman/core/var/queue/out/*.bak -rw-rw---- 1 100 65533 21512 Apr 14 19:45 /srv/mail/mailman/core/var/queue/out/1555296331.8386078+2c905825b82708e5a682b872b0aff320c9dd0b2e.bak what does it indicate and what should my steps be to resolve this (other than restarting queues)? > And if it's not, is there anything in Mailman's logs to indicate why it > died? > > Also, 'mailman restart' should only take a matter of seconds, but if the > out runner is running but not processing its queue, I think sending it a > SIGINT should restart it. SIGINT killed the process allright, but it didn't auto-respawn. I ended up spawning it with: /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg but soon realized it doesn't go to background nicely so I've nohup'ed it in the end. but it did solve my problem. -- Sr System and DevOps Engineer SoM IRT

6 years, 2 months

[MM3-users] [Suspected Spam]Re: [MM3-users]Exim, Mailman3 and missing Message-ID:

by Stephen J. Turnbull

Mark Sapiro writes: > Rob's reply in this thread was garbled due to > <https://gitlab.com/mailman/mailman/issues/616>. The text of the reply > is quoted below. > > On 7/27/19 12:54 AM, Rob Lister wrote: > > > > One thing that does occur to me is that my exim header fudge will > > probably cause the Message-ID header to move around from its original > > position, and if someone is doing hashing or actually validating (gpg > > signing?) the headers haven't been tampered with, then it might > > complain, since although the message-id hasn't changed, it might be in a > > different place in the headers. I don't think this is a problem. GPG or S/MIME signing with any software I know of only signs the message body (or a MIME part). DKIM message signing isn't going to care about the order of fields because DKIM canonicalizes the order (at least for the unique fields) according to the specification in the signature field, and that will be verified on the way in. Of course adding a Message-ID will likely break DKIM for the outgoing post, but Mailman as normally used breaks DKIM anyway. This can be partly mitigated by using ARC (authenticated received chain), which is supported by Google already, and possibly some of the other usual DMARC suspects (Yahoo!, AOL). > > Probably the more correct way to do it is only add a new header if it's > > missing, and if it's there, leave it alone. True, and I think this does it: headers_add = ${if def:h_message-id:{fail}{Message-ID: ... id generator code ...}} If you like that and test it, I would really appreciate feedback so we can update the documentation. One thing *I* would like to add to this recipe is logging. I'd be curious to see how frequently it happens outside of spam and junior high school hackers using netcat as their MUA. I don't see how to do this in Exim, though. Another is using a prefix other than "E" to indicate this isn't standard Exim behavior. Steve

5 years, 11 months

[MM3-users] Re: UnicodeEncodeError: 'ascii' codec can't encode character

by Marvin Gülker

Hi, Am 10. Dezember 2019 um 10:25 Uhr -0800 schrieb Mark Sapiro: >The underlying issue here is probably ><https://bugs.python.org/issue32330>, but that knowledge doesn't help >us. We need to get rid of the offending message in the message store. >The first step is in mailman shell. Note the paths /opt/mailman/mm/bin/ >and /opt/mailman/mm/var/ will need to be adjusted. Thanks for this information. Ubuntu's mailman stores the messages below /var/lib/mailman3/messages. The "mailman" command is in the $PATH. >This should print the file name and Message-ID-Hash (two lines the same) >for each message in the messages directory until it prints only one line >followed by the UnicodeEncodeError exception. That list line is the file >name/Message-ID-Hash of the offending message. I've execucted that code, but it does not cause an exception at all. It quietly runs through. I've attached the transcript of the shell session to this mail. With a little crude use of grep(1), however I found my held message in WQ/ZS/WQZSFFJLVMWQHMG4VYKGXJEKQVQXVK67. Does the messages directory contain only held messages? I've looked at some messages and all of these I looked at were spam. The directory structure contains 40 messages in total, accumulating to 480 KiB disk space. If required, I can zip that directory up and e-mail it to you for inspection. >DELETE FROM message WHERE message_id_hash = xxx; As the exception you expected did not occur, I didn't do this yet. I can make a database and directory backup and then run it, though. Should I try? I assume after this I should also delete the respective files from the messages/ directory? >As of Mailman core 3.3.0 there is a 'mailman notify' command designed to >be run periodically by cron to notify owners/moderators of prnding requests. I did receive an e-mail to the list owner address that told me that the e-mail from my incorrect e-mail address "requires approval". That was the first time I received such an e-mail. In parallel, on the incorrect e-mail address, I received the notification for moderator approval. Marvin -- Blog: https://mg.guelker.eu

5 years, 6 months

[MM3-users] Re: possible backscatter attack using Mailman3 servers

by David Newman

On 1/3/25 5:44 PM, Mark Sapiro wrote: >> Greetings. On a system running Mailman 3.3.9 and Postfix, I'm seeing >> about 20-30 entries per day in the Postfix queue where it appears a >> Gmail user signs up for a mailing list that requires confirmation, and >> Gmail responds that the user is too busy to handle requests. >> >> There are no publicly advertised email lists on this server, and I >> don't ever see anything in the Mailman logs indicating the user ever >> tried signing up. > > > This is an attack mail bombing the user. The requests that result in the > can come via web or email. Mailman's logging of subscribes has been > missing most events through Mailman 3.3.10. See https://gitlab.com/ > mailman/mailman/-/issues/1143 which will be fixed in 3.3.11, but > subscribes waiting user confirmation still won't be logged. > > However, the message with subject "Please Confirm Your Email Address" > comes from Django allauth so it isn't actually Mailman sending it but > rather Django allauth as a result of a request to sign up for a Django > account at https://mail.example3.com/accounts/signup/. You can probably > find that request in your web server logs, and you may find the user > and/or email in the Django admin UI. Thanks VERY much for this. No such users in the Django UI, but the web server logs have 252 attempts from 132 unique IPv4 addresses registered to different ISPs throughout Europe. So, even though Mailman support for more detailed logging of sub and unsub requests would be useful, it likely would not have helped with attacks from many source IP addresses. > > Since django-mailman3 1.3.6, you can disable these signups by putting > > ACCOUNT_ADAPTER = 'django_mailman3.views.user_adapter.DisableSignupAdapter' > > in your Django settings, but then your users won't be able to sign up > for web accounts. I have made this change. As for not having web accounts, this just means new users cannot sign up to manage their Mailman settings, correct? I presume existing web accounts will continue to work. Thanks again! dn

5 months, 3 weeks

Search results for query "sapiro"