Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: How to fix lists with characters errors?

by Helio Loureiro

Hi, it is python 3.10. Problem is... some mailing lists were created long ago, with python2 something. Most probably 2.5. And the tables on mysql were set to latin1. That I know. Question now is how to fix this. Any suggestion? What is the table? I probably can run some script for these specific lists that have the failure, import data as latim-1, and update back to the table as utf-8. Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro On Fri, 22 Mar 2024 at 20:53, Mark Sapiro <mark(a)msapiro.net> wrote: > On 3/22/24 05:37, Helio Loureiro wrote: > > Hi, > > > > I just found several lists have the issue like this regarding charset: > > > > (venv) mailman@new-mailman3 ~ (v3.1.1)> mailman members > ... > > Traceback (most recent call last): > ... > > line 109, in display_members > > print(f'{address.display_name} <{address.original_email}>', > > UnicodeEncodeError: 'latin-1' codec can't encode character '\u0142' in > > position 9: ordinal not in range(256) > > > There is something in your python that has set the encoding for stdout > to latin1 instead of the default utf-8. E.g. > ``` > $ python3.10 > Python 3.10.9 (main, Dec 11 2022, 12:59:24) [GCC 9.4.0] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> name = 'abc\u0142def' > >>> print (f'{name}') > abcłdef > >>> import sys > >>> sys.stdout.reconfigure(encoding='latin1') > >>> print (f'{name}') > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > UnicodeEncodeError: 'latin-1' codec can't encode character '\u0142' in > position 3: ordinal not in range(256) > >>> > ``` > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to helio(a)loureiro.eng.br >

1 year, 2 months

[MM3-users] Re: Speed/progress of update_index_one_list?

by Philip Colmer

We've now completed the indexing of our lists, some of which are quite large (100K+ messages per list). I wanted to share our findings of using Xapian to do this. Firstly, using "-v 2", e.g. "mailman-web update_index_one_list -v 2 <email address>", causes the command to print out the progress being made in batches of 1,000 messages. Secondly, Xapian is single-threaded. It only allows one writer at a time. Therefore, we focussed on speed of the hard drives and, on the advice received from the Xapian list, memory. Since we were using AWS EC2 with EBS storage, I elected to move the EBS storage from gp2 to gp3 - which has a higher base I/O figure - and I selected a r6i.8xlarge as this guaranteed 10Gb throughput to the EBS storage. It also delivered 32 vCPU and 256GB RAM. It was possibly overkill but it did the job (see below) and we've now switched back to a t3a.xlarge. Speed-wise, with the above configuration, the system indexed a list of nearly 85K messages in 46 minutes. I hope that is helpful information to anyone else who finds themselves migrating large Mailman 2 archives. Regards Philip On Fri, 14 Jan 2022 at 17:33, Mark Sapiro <mark(a)msapiro.net> wrote: > On 1/14/22 12:38 AM, Philip Colmer wrote: > > I have now switched to Xapian but, in a way, my original questions still > > stand. Is there a way of monitoring the progress of > > "update_index_one_list"? What can I do to the specification of the server > > to make that process go (much) faster? > > > > HyperKitty does the indexing by calling the Haystack update_index > command. See > > https://django-haystack.readthedocs.io/en/master/management_commands.html#u… > > This command has options such as --verbosity and --workers. I think > verbosity is set from the option provided to update_index_one_list and > if set to 2 will give some progress info. > > workers could increase parallelism but is unconditionally passed by > hyperkitty as 0 > > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

3 years, 4 months

[MM3-users] Re: Another interesting error...

by Darren Smith

OK chalk another one up to a python newbie... commit() was the magic sauce that we needed. The error 500 is no longer happening on that mailing list. We have three different lists so far that have gotten into this state, and we are going to work on the other two to make sure that this all works. FYI if this is fixed somehow in a later release (later than 3.1.0) we are working on migrating to the latest mailman code. -Darren On Mon, Apr 23, 2018 at 4:20 PM, Mark Sapiro <mark(a)msapiro.net> wrote: > On 04/23/2018 02:59 PM, Darren Smith wrote: > > Well, using the mailman shell we were able to find the request that was > > causing the problem - it was apparently a spam message that had some > > chinese characters in it. > > > I am trying to duplicate this issue in a test environment. I can create > a message with an invalid RFC 2047 encoded subject, i.e., > > Subject: =?gb2312?Q?=ff=fe=de=21=a2?= > > and send it to a list where it gets held, but I don't get a 500 in > postorius and I see the held message with a raw (not decoded) header. > > I think there were some recent changes that affect this. I'll look some > more. > > > > We used the shell to delete the held request... > > > > And postorius is still giving me an error 500. > > > > If I'm not mistaken the held messages are kept in .pck files - do you > know > > if when postorius asks for held messages through the API, does mailman go > > to the database or to the .pck? I would actually suspect the latter, or > > removing the request should have fixed the problem. > > > They were held in .pck files in Mailman 2.1, but not in Mailman 3. They > should just be in the database and deleting them via mailman shell > should delete them as long as you quit the shell or commit(). > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

7 years, 1 month

[MM3-users] Re: hyperkitty.lib.utils Failed to update the fulltext index

by Abhilash Raj

On Wed, Apr 17, 2019, at 8:18 AM, Mark Sapiro wrote: > On 4/17/19 8:10 AM, Abhilash Raj wrote: > > On Wed, Apr 17, 2019, at 8:06 AM, Ben Harris wrote: > >> April 17, 2019 11:03 AM, "Abhilash Raj" <maxking(a)asynchronous.in> wrote: > >>> > >>> Which version of Hyperkitty are you using? > >>> > >>> Did you upgrade the version of Hyperkitty when you migrated to the > >>> new host? > >> > >> I'm using the mailman3-full package in ubuntu (i'm aware it's behind a couple > >> releases), so it's hyperkitty 1.1.4. > >> > >> I didn't upgrade anything with the move. > > > > I am not sure where does Debian packaged version store it's fulltext index, but > > I am assuming the move somehow makes Hyperkitty not be able to read the > > index. > > > I think this is a known incompatibility between the Python 2.7 and > Python 3 versions of Whoosh. I *think* 1.1.x releases were based on Python 2.7, so that part looks right to me. > > From the tracebacks, the current HyperKitty is running under Python 2.7. > If the prior server was running HyperKitty with Python 3, that would > explain this. Given that 1.1.x supports only 2.7 and Ubuntu doesn't have 1.2 release, it is unlikely Python 3 was used. > > > Can you try removing and re-building the entire index? I don't know again > > the right command for Django there, but the hyperkitty command you want > > to run is called `rebuild_index`. For source package, something like: > > > > $ python manage.py rebuild_index > > > > Do note that this will take a lot of time if you have a big archive. > > > If this is the incompatibility, rebuilding the index will fix it. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

6 years, 1 month

[MM3-users] Re: including a template failed

by Wolfgang Bock

Dear Mark, Q: Can you get https://webmail.domain-02.de/mailman3/postorius/api/templates/list/mylist.li sts.domain-03.de/list:member:regular:footer on the Mailman machine? A: No Q: What is your setting for POSTORIUS_TEMPLATE_BASE_URL? A: was: POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost/mailman3/' Changed it to: POSTORIUS_TEMPLATE_BASE_URL = 'https://localhost/mailman3/' Now another problem occurs: Sep 23 22:51:34 2020 (13492) Cannot connect to SMTP server localhost on port 25 HTTPSConnectionPool ... 'localhost' doesn't match either of 'blog.domain-04.de', 'lists.domain-04.de', 'mail.domain-04.de', 'domain-04.de', 'www.domain-04.de'"))) Port 25 entry results from the manual regarding the postfix configuration: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.ht ml e.g.: [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver lmtp_host: mail.example.com lmtp_port: 8024 smtp_host: mail.example.com smtp_port: 25 Regards Wolfgang -----Ursprüngliche Nachricht----- Von: Mark Sapiro <mark(a)msapiro.net> Gesendet: Sonntag, 20. September 2020 01:05 An: mailman-users(a)mailman3.org Betreff: [MM3-users] Re: including a template failed On 9/19/20 2:47 PM, Wolfgang Bock via Mailman-users wrote: > Hi, > > I created a new mailinglist and got during the test-process the > following errormessage in my mailman.log after: > ... > Sep 19 22:55:08 2020 (1433) ACCEPT: > <000001d68ec7$2553e6e0$6ffbb4a0$(a)domain-01.de> > > Errors: > > Sep 19 22:55:09 2020 (1436) Cannot connect to SMTP server localhost on > port > 25 > HTTPSConnectionPool(host='webmail.domain-02.de', port=443): Max > retries exceeded with url: > /mailman3/postorius/api/templates/list/mylist.lists.domain-03.de/list: > member :regular:footer (Caused by NewConnectionError('<urllib3.connec > tion.VerifiedHTTPSConnection object at 0x7fc8380f1748>: Failed to > establish a new connection: [Errno -2] Name or service not known')) > ... This is wierd. It looks like we are trying to establish a connection to host='webmail.domain-02.de', port=443 in order to GET /mailman3/postorius/api/templates/list/mylist.lists.domain-03.de/list:member :regular:footer, but the error says "Cannot connect to SMTP server localhost on port 25". I have no idea how port 25 is coming into it. Can you get https://webmail.domain-02.de/mailman3/postorius/api/templates/list/mylist.li sts.domain-03.de/list:member:regular:footer on the Mailman machine? What is your setting for POSTORIUS_TEMPLATE_BASE_URL? -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

4 years, 8 months

[MM3-users] Re: can't get mail into mailman3 in docker

by matthew＠alberti.us

I use the docker-mailman setup. Below is the partial contents of my mailman-extra.cfg. I think by default, you would use the Docker assigned private network IP addresses for communicating with the containers. In my setup, 172.19.199.1 is the host (where postfix is) and 172.19.199.2 is mailman-core... # EMAIL SERVER / MTA CONFIGURATION [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver lmtp_host: 172.19.199.2 lmtp_port: 8024 smtp_host: 172.19.199.1 smtp_port: 25 configuration: /etc/postfix-mailman.cfg This is all really going to depend on your particular setup. I'm late to the thread; but let me know if that improves the situation. - Matt Alberti -----Original Message----- From: Mark Sapiro <mark(a)msapiro.net> Sent: Saturday, August 7, 2021 2:04 AM To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: can't get mail into mailman3 in docker On 8/6/21 11:28 AM, bob B wrote: > I see nothing for today when the error happens in > /opt/mailman/var/logs/smtp.log > > But the log has a bunch of these, but these stopped around the 4th > > Aug 04 19:46:04 2021 (28) Available AUTH mechanisms: LOGIN(builtin) > PLAIN(builtin) Aug 04 19:46:04 2021 (28) Peer: ('127.0.0.1', 33912) > Aug 04 19:46:04 2021 (28) ('127.0.0.1', 33912) handling connection Aug > 04 19:46:04 2021 (28) ('127.0.0.1', 33912) EOF received Aug 04 > 19:46:04 2021 (28) ('127.0.0.1', 33912) Connection lost during > _handle_client() Aug 04 19:46:04 2021 (28) ('127.0.0.1', 33912) > connection lost These are normal messages from the startup of Mailman's lmtp runner. If you correlate these with mailman.log, you'll probably see the server starting at these times. The fact that there is nothing in this log correlating with the Postfix message 2021-08-06T09:08:41.974225-04:00 dockerhost postfix/lmtp[109783]: D630B3083ED0: to=<test(a)mailman-docker.###.####>, relay=localhost[127.0.0.1]:8024, delay=0.11, delays=0.05/0.02/0.04/0, dsn=4.4.2, status=deferred (lost connection with localhost[127.0.0.1] while receiving the initial server greeting) seems to say that Postfix isn't really connecting to Mailman's lmtp and this is consistent with netstat -l not showing something listening on port 8024. I note you have lmtp_host: localhost in the [mta] section in mailman-extra.cfg. Try changing that to lmtp_host: 127.0.0.1 -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

3 years, 10 months

[MM3-users] Re: update_index

by Darren Smith

OK - next issue...here's the warning that I'm receiving when I run that: /__init__.py:1451: RuntimeWarning: DateTimeField Email.archived_date received a naive datetime (2018-04-03 07:51:25) while time zone support is active. RuntimeWarning) I'm not sure that anything else occurred. The indexing took 2 seconds, and there have been tens of thousands of emails imported since the last index. Thoughts? -Darren On Tue, Apr 3, 2018 at 10:55 AM, Darren Smith <silas.crutherton(a)gmail.com> wrote: > Holy cow I think that's exactly what the problem is. I will give that a > try and let you know what happens! > > On Tue, Apr 3, 2018 at 10:50 AM, Mark Sapiro <mark(a)msapiro.net> wrote: > >> On 04/03/2018 09:02 AM, Darren Smith wrote: >> > Hello, >> > >> > If I'm reading the docs correctly, when we run manage.py update_index, >> it >> > should be an incremental index of new emails that have been added since >> the >> > last time update_index was run - can anyone confirm that this is the >> > expected behavior? Or is there another command line option that I am >> > missing? >> >> >> There are two ways to run update_index. >> >> Hyperkitty's update_index is incremental and is run by >> >> manage.py runjob update_index >> >> Running >> >> manage.py update_index >> >> runs haystack's update_index which with no options does a full update. Run >> >> manage.py update_index --help >> >> to see the options. The incremental 'manage.py runjob update_index' >> should be run every minute by 'manage.py runjobs minutely' assuming cron >> is running the periodic Django jobs, so running either update_index job >> manually shouldn't be necessary >> >> >> ... >> > So...can anyone help with how to get the update_index to do only emails >> > added since the last time the update_index was run? >> >> >> manage.py runjob update_index >> >> -- >> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, >> San Francisco Bay Area, California better use your sense - B. Dylan >> _______________________________________________ >> Mailman-users mailing list >> mailman-users(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> > >

7 years, 2 months

[MM3-users] Re: Emergency Moderation

by Allan Hansen

> On May 24, 2020, at 11:34 , Mark Sapiro <mark(a)msapiro.net> wrote: > > On 5/24/20 10:59 AM, Allan Hansen wrote: >> In the MM3 settings there is an option to ‘Hold for moderation’ when members or non-members post to the list. >> I use that for the same cases where in MM2 I switched a list to ‘Emergency Moderation.’ > > > If you imported your MM 2.1 lists with `mailman import21`, `emergency` > was set appropriately for the imported list and should be working to > hold posts regardless of other settings. Not sure what is the difference between ‘emergency moderation’ and ‘Hold for moderation.’ It appears redundant. > > >> My own current top-of-the-list issues: >> >> A bug: Moderators can’t see/edit the list of members even when allowed to by list setting. > > The visibility issue is <https://gitlab.com/mailman/postorius/-/issues/369>. > > Editing membership beyond what could be done via the admindb interface > (i.e. clearing a moderated poster's mod flag) was never allowed for a > moderator in MM 2.1. > > In MM 3, a moderator can set a member's moderation when handling a held > post. Do you think a non-owner moderator should be able to make other > changes to a member? I can only speak for my own application of MM3: Desiring consistency between the lists, I don’t want to turn all the moderators into owners who can change list settings. Many of my moderators keep asking me to see the list of members and to subscribe and remove members. I end up servicing those requests. So... a. Moderators should be able to unsubscribe and subscribe members. b. Moderators should be able to set the member’s moderation status. c. Moderators should not be able to see/modify any other member settings. > > >> A feature request: Member changing email address on all lists he/she belongs to, without involving moderators. > > > This is <https://gitlab.com/mailman/postorius/-/issues/399> and we're > working on it. Awesome!!!! Thank you, Allan > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

5 years

[MM3-users] Re: Apple Mail encoding again

by David Newman

On 4/14/22 9:50 AM, Mark Sapiro wrote: > On 4/14/22 08:21, Stephen J. Turnbull wrote: >> Mark Sapiro writes: >> >> > That combination at least produces multipart/signed messages with >> > multipart/* parts containing a Content-Transfer-Encoding: header. >> > >> > The Python 3 email library considers this to be a >> > InvalidMultipartContentTransferEncodingDefect() message defect. >> >> Merely containing C-T-E is not an error per RFC 2045, although it >> seems to consider it a stupid thing to do. If present, it may be >> "binary", "7-bit", or "8-bit" (below summarized as "cleartext"). >> >> Did Apple Mail apply a non-cleartext C-T-E? Or has an RFC tightening >> that restriction been published since? (I don't know why it would, >> though.) > > > Yes. The C-T-E on all the multipart parts was quoted-printable, not > allowed under RFC 2045. > > >> Even with a non-cleartext C-T-E, it seems reasonable for Mailman to >> treat it as application/octet-stream and pass it through, unless the >> list is configured to do something like strip application/octet-stream >> or that requires decoding like spam-filtering (Postel principle, >> although one could argue it shouldn't be sent on on the same principle >> -- ah, the joys of being an e-mail Mediator). That would be >> especially true it if were quoted-printable, since any relevant MIME >> headers would (presumably) be pure ASCII and therefore unencoded. > > > The Python email library says the message is defective. Mailman's lmtp > runner in turn rejects the defective message with a 501 status > https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/runners/lmtp.p… > Yes. To be clear, this is an Apple Mail issue. Mailman 3 accepted PGP-signed messages from Apple Mail on versions prior to 15.0 running on MacOS 12.0.1. Something changed on Apple's part. > > It could accept a defective message or one with specific defects, but > this is problematic for further processing. For example, at least one > MUA, Thunderbird, renders such a message with an empty body. That's not my experience. Mailman 3 accepted without complaint a PGP-signed message I sent from Thunderbird 91.8.0 and forwarded it to list subscribers with the message body intact. dn

3 years, 1 month

[MM3-users] Re: Mails sent by mailman-web runjobs hourly

by Abhilash Raj

On Fri, Jun 17, 2022, at 2:38 AM, Mark Sapiro wrote: > On 6/16/22 03:44, Eggert Ehmke wrote: >> I get lots of mails originating from the mailman-web runjobs hourly. They are >> sent to mailman@<mytld>. The content looks like >> >> 07:00:03 [Q] INFO Enqueued 12571 >> 07:00:03 [Q] INFO Enqueued 12572 >> 07:00:03 [Q] INFO Enqueued 12573 >> >> How can I disable these emails? > > These message come from Django Q, https://django-q.readthedocs.io/en/latest/ > > I don't think there is a way to disable those specific messages which > occur at times when tasks are queued instead of started immediately. > They are sent to `mailman` because `mailman` owns the crontab and the > crontab has no MAILTO=. You can't stop those specific messages, but you can set up the log level of django-q to WARNING from INFO so that it doesn't send out these logs. You want to add something like this in your settings.py in LOGGING config: https://github.com/Koed00/django-q/blob/15155c7a99ea85a092b1d11da730ba0c4fa… "loggers": { "django_q": { "handlers": ["file"], "level": "WARNING", }, }, Please make sure to *add* "django_q" settings as there might be other existing loggers defined there. Also, I used "file" as the handler for example, so it won't even write to stdout for warning and/or send emails. Make sure that you change that to your liking to include "console" if you want emails for warning logs. > > You can redirect these and all other messages from hourly jobs by > surrounding the hourly jobs command in the crontab with MAILTO= as in > ``` > MAILTO=other(a)example.com > @hourly /opt/mailman/mm/bin/django-admin runjobs hourly > MAILTO=mailman > ``` > or possibly discard them with something like > ``` > @hourly /opt/mailman/mm/bin/django-admin runjobs hourly > /dev/null > ``` > but either of these could redirect or discard other messages from hourly > jobs which you might want. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ -- thanks, Abhilash Raj (maxking)

2 years, 11 months

Search results for query "sapiro"