Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: Issues with importing archives from a particular list

by Philip Colmer

On Thu, 13 Jan 2022 at 18:01, Mark Sapiro <mark(a)msapiro.net> wrote: > On 1/13/22 4:33 AM, Prasanth Nair wrote: > > Hello, > > > > We are trying to import archives from Mailman2 to Mailman3. For a > > particular list, it is not importing all of the messages. We can see that > > Mailman2 has archives from 2015 to 2022. But, it only imports messages > from > > 2018 to 2022 to the Mailman3 server. It was not giving any errors while > > importing. > > > The problem list probably already had an archived message from 2018. If > you don't specify a --since= date for hyperkitty_import, it only imports > messages newer than the newest existing message in the archive. > > You can run it again with, e.g., --since=2000-01-01. It won't reimport > the messages already imported because of duplicate message ids. > Thank you, Mark, for your reply. It turned out that this wasn't an issue with any part of the Mailman 3 suite. Unbeknownst to Prasanth, the affected list had previously been migrated from a different server in 2018 but, for some reason, the mbox file was not copied over - just the pipermail archive directory was. That explains the split between the pipermail archives and the mbox file. Thankfully, the original server still had the older mbox file so we've now been able to import all of the messages into HyperKitty. Apologies for the noise. Regards Philip

3 years, 11 months

[MM3-users] Re: REST API authorization failed

by Axel Rau

Hi Mark, thanks for your patience. > Am 10.01.2022 um 04:56 schrieb Mark Sapiro <mark(a)msapiro.net>: > > Anyway, the above is saying that restadmin:??? doesn't match what's configured in Mailman core and reported by `mailman info`. Please check carefully that these are the same in both mailman.cfg admin_pass and settings MAILMAN_REST_API_PASS. Yes, again with password disclosed: - - - (mailman_33) [root@lists5 /usr/local/mailman3]# service mailman restart Restarting the Mailman runners (mailman_33) [root@lists5 /usr/local/mailman3]# su -m www -c "curl -v -u restadmin:restpass http://localhost:8001/3.1/lists" * Trying 127.0.0.1:8001... * Connected to localhost (91.216.35.42) port 8001 (#0) * Server auth using Basic with user 'restadmin' > GET /3.1/lists HTTP/1.1 > Host: localhost:8001 > Authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz > User-Agent: curl/7.80.0 > Accept: */* > * Mark bundle as not supporting multiuse < HTTP/1.1 401 Unauthorized < Server: gunicorn < Date: Mon, 10 Jan 2022 09:39:43 GMT < Connection: close * Authentication problem. Ignoring this. < www-authenticate: Basic realm="mailman3-rest",charset="utf-8" < content-type: application/json < vary: Accept < content-length: 77 < * Closing connection 0 {"title": "401 Unauthorized", "description": "REST API authorization failed“} (mailman_33) [root@lists5 /usr/local/mailman3]# (mailman_33) [root@lists5 /usr/local/mailman3]# grep REST settings.py MAILMAN_REST_API_URL = 'http://localhost:8001' MAILMAN_REST_API_USER = 'restadmin' MAILMAN_REST_API_PASS = 'restpass' (mailman_33) [root@lists5 /usr/local/mailman3]# grep admin var/etc/mailman.cfg admin_user: restadmin admin_pass: restpass (mailman_33) [root@lists5 /usr/local/mailman3]# - - - What am I doing wrong? Is something wrong with my installed python packages? pip freeze attached. clueless, Axel --- PGP-Key: CDE74120 ☀ computing @ chaos claudius

3 years, 11 months

[MM3-users] Re: Help with fresh debian install

by Mark Dadgar

On Oct 5, 2019, at 12:22 PM, Mark Sapiro <mark(a)msapiro.net> wrote: > You could just run, e.g., `sudo -u list bin/mailman import21 ...` which > will work even if 'list' can't log in. That’s what I get for being old school and just su’ing to the user. Thanks. > If you run hyperkitty_import --since=1990-01-01, that should do it. > Awesome - rerunning now. Thank you. > As far as "the archives for many months consist of just a single > identical thread", were there error messages when you ran the initial > import? Were there issues in the imported mbox that you have corrected? > Mailman 2.1's bin/cleanarch can help identify unescaped 'From ' lines > that cause problems. Yes, I ran the archive mbox through cleanarch before importing it. It’s a 24GB mbox file with 17 years of archives for a very active mailing list. > If unescaped 'From ' lines is the issue and you have fixed them, > rerunning the import should import the messages correctly, but it won't > delete the bogus threads. You can delete these threads manually in > HyperKitty if you are logged in as a Django superuser. Great - thank you. I see the occasional “table is locked” error (I am running against sqlite because I only run a couple of lists and I was having issues getting the postgres config working) so it skips the occasional message. We’ll see what happens this time. Your help is much appreciated. - Mark ----- mark(a)pdc-racing.net | 408-348-2878

6 years, 2 months

[MM3-users] Re: Monthly Password Reminders

by Stephen J. Turnbull

Odhiambo Washington writes: > On Thu, Feb 2, 2023 at 8:21 PM Mark Sapiro <mark(a)msapiro.net> wrote: > > On 2/2/23 06:29, Odhiambo Washington wrote: > > > Does MM3 send these monthly subscription reminders the way MM2 used to > > do? > > > I am looking for the setting, but can't find that in the list settings. > > > > No, it does not. > > The rationale behind the decision to toss away that feature? Security?? For the passwords, yes. It was frequently reported as a bug, in fact. Mostly the reminders generate requests to know how to shut them off. You're the first in probably a decade who's asked how to turn them on! While in theory we could protect in transit with SMTP over TLS, we can't guarantee end-to-end encryption (eg, if there's a .forward at the subscribed address), and while TLS is widely implemented, it's still not universal. And there's no way to protect unencrypted email from admins at the receiving host. Receiving encrypted email is pretty much not implemented at all, except in some specialized communities. Also, now that passwords (if they're used) are attributes of Users who may have several Addresses and worse, they may have admin privileges. So Mailman 3 passwords are generally higher value than in Mailman 2. I can imagine a use for the list of subscriptions part that a few users might like, but most folks would prefer not to get another regular mailing. Steve

2 years, 10 months

[MM3-users] Re: autoresponse not sent

by Peter Sørensen

Thanks Mark, I found the mysql.py as you suggested and a few places more. /usr/lib/python3/dist-packages/mailman/database/mysql.py /usr/lib/python3/dist-packages/alembic/ddl/mysql.py /usr/lib/python3/dist-packages/passlib/handlers/mysql.py /usr/lib/python2.7/dist-packages/django_extensions/management/mysql.py Could this be the problem ? Best regards Peter Sørenen University of Southern Denmark Den 19/10/18 18:47 skrev "Mark Sapiro" <mark(a)msapiro.net>: >On 10/19/18 8:21 AM, shadiakiki1986(a)gmail.com wrote: >> Hello. Newbie to mailman here. >> I tried out mailman3 today. My setup is >> - mailman3 suite using the docker-compose file from asynchronous.io >> - exim4 on the local machine and configuring it as smarthost using >>mailgun smtp server >> >> The emails to the mailing list are received by the subscribers, >> but the autoresponse is not being received by the sender (who is not >>from the subscribers list). >> I'm not sure if the problem is from mailman-core, settings from >>mailman-web, or from exim. > > >This is most likely <https://gitlab.com/mailman/mailman/issues/504>. The >fix has been in the master branch for a couple of months but is not in >3.2.0. It may be in the asynchronous.io "rolling release". > >-- >Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, >San Francisco Bay Area, California better use your sense - B. Dylan >_______________________________________________ >Mailman-users mailing list -- mailman-users(a)mailman3.org >To unsubscribe send an email to mailman-users-leave(a)mailman3.org >https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

7 years, 1 month

[MM3-users] Re: no access to Postorius web pages after upgrade

by David Newman

On 10/22/23 2:16 PM, Mark Sapiro wrote: > Here is the nginx error, with obfuscation of client and server info: >> >> 2023/10/22 10:43:24 [error] 2264140#2264140: *135 >> "/opt/www/roundcubemail/postorius/domains/index.php" is not found (2: >> No such file or directory), client: 1.2.3.4, server: >> lists.example.com, request: "GET /postorius/domains/ HTTP/2.0", host: >> "lists.example.com", referrer: >> "https://lists.example.com/mailman3/listinfo" > > > Here you are trying to get > https://lists.example.com/postorius/domains/, but (see below) > # begin mailman3 stuff >> >> location /static/ { >> alias /opt/mailman/web/static/; >> } >> >> location /mailman3/ { >> proxy_pass http://127.0.0.1:8000; >> proxy_set_header Host $host; >> proxy_set_header X-Forwarded-For $remote_addr; >> } > Above you have defined proxying for the paths beginning with > /mailman3/, /archives/, /mailman/ and others, but not /postorius/ or > /hyperkitty/. Bingo. Adding location statements for hyperkitty and postorius and restarting Nginx cleared the issue. Unclear why this ever worked before the upgrade. I don't remember monkeying with a urls.py file. Anyway, it works now. Thank you Mark! >> (venv) mailman@mail10:~$ mailman-web migrate >> System check identified some issues: >> >> WARNINGS: >> account.EmailAddress: (models.W036) MariaDB does not support unique >> constraints with conditions. > > This is an issue with MariaDB and MySQL. > Thanks again. No issues here, but glad to see you previously raised this as a known issue. dn

2 years, 1 month

[MM3-users] Re: Welcome message template errors

by Mark

On 2024-04-24 10:38, Mark Sapiro wrote: > Try the following in `mailman shell` running as user mailman > ``` > (venv) $ mailman shell > Welcome to the GNU Mailman shell > Use commit() to commit changes. > Use abort() to discard changes since the last commit. > Exit with ctrl+D does an implicit commit() but exit() does not. > >>>> import requests >>>> r = requests.get( > ... > 'https://LISTS.MYDOMAIN.COM/mailman3/api/templates/list/MYTESTLIST.LISTS.MYD…', > ... timeout=5) >>>> t.text > ``` > Note the URL above is all on one line. > This is exactly what Mailman core is doing to get the template > assuming the `uri` in the template` table entry is the argument of the > requests.get(). > > If this times out, try increasing the timeout= value. ************************************************ I crossed emails with you on my last post. I reset the POSTORIUS_TEMPLATE_BASE_URL (in settings.py) back to the domain name, restarted everything, created a new Welcome template (using Postorius) and ... >>> import requests >>> r = >>> requests.get('https://MYDOMAIN.COM/mailman3/api/templates/list/MYTESTLIST.MYDOMAIN.COM/li…', >>> timeout=5) >>> r.text 'Another test Welcome message for new members.' >>> ... so all well there. And mailman is running mailman core ... # cd /opt/mailman/venv/bin/ # ls -la ... -rwxr-xr-x 1 mailman mailman 230 Jan 26 04:00 gunicorn -rwxr-xr-x 1 mailman mailman 231 Jan 26 04:00 mailman -rwxr-xr-x 1 mailman mailman 230 Mar 25 22:18 mailman-web ...

1 year, 7 months

[MM3-users] Re: Problems with New MM3 Installation

by Jeff

> On May 11, 2021, at 1:18 PM, Mark Sapiro <mark(a)msapiro.net> wrote: > > > Set base_url: in /usr/local/mailman3/mailman-hyperkitty.cfg to a url > that will reliably access HyperKitty, probably the same base URL that a > user with a web browser would use. > Thanks Mark, I really appreciate your assistance. New messages are now being archived as expected, but I still have a few small remaining issues to work out with my MM3 installation: 1) some emails that arrived before I got the archive working are still sitting in: /usr/local/mailman3/mm/var/archives/prototype/[listname]@[domain]/new Strangely this includes at least one email that was archived. Should this directly be cleared as messages are archived? Is there a way to force it to be cleared? 2) Some messages that arrived while I was having other problems with the setup have been stored in: /usr/local/mailman3/mm/var/queue/shunt What should I do to get these messages processed? 3) Similar to above, some messages are in: /usr/local/mailman3/mm/var/queue/bad How can I get these messages processed? 4 Modified the template for “list:admin:action:post” through Postorious and emails that arrived after that were put in the “shunt” queue. I traced this problem to POSTORIUS_TEMPLATE_BASE_URL. By default, it is set to "http://localhost:8000 <http://localhost:8000/>” which is where uwsgi is listening. However, uwsigi is listening for uwsgi:/ not http://. Should I change POSTORIUS_TEMPLATE_BASE_URL to “uwsgi://localhost:8000 <uwsgi://localhost:8000>” or keep http but make it something else? Thanks again for any help anyone can provide. Jeff

4 years, 7 months

[MM3-users] Re: High memory usage

by Stephen Daniel

I'm not an expert, and you might be better served by posting this question in one of the communities that serves sysadmins; however, my reading says that the oom-killer is invoked when the system cannot allocate more virtual memory, not more physical memory. The graph in your screenshot is physical memory usage. The first thing I would try is increasing the size of your swap partition. That adds to the amount of available virtual memory, without the need for actually buying more RAM. If you really need more RAM the result will be that the system stays up, oom-killer stays quiet, but performance is bad and the "si" and "so" columns in vmstat get into the triple digits or higher. On Wed, Feb 16, 2022 at 3:18 AM Philip Colmer <philip.colmer(a)linaro.org> wrote: > Thank you for the replies from Mark and Stephen. > > One of the reasons why Prasanth & I are concerned with this specific system > is because the first time memory usage crept up, it got to the point where > oomkiller stepped in and, as is usual for that particular mechanism, one of > the processes it killed was sshd, thus removing access to the system :(. I > know that oomkiller can be tuned to stop it killing processes like sshd but > it will, nevertheless, start killing *some* processes if it sees an out of > memory situation. > > So, as Prasanth said, we doubled the memory available and stuck an alarm on > it to trigger when usage exceeded 85% for 5 minutes. I've saved a > screenshot of the monitored memory usage over the last couple of weeks: > > https://1drv.ms/u/s!Aq7dn8GDGLdCodsU-pybJBdckxWU1A?e=BJY1UA > > (It is stored on OneDrive in case you don't recognise the URL format) > > We changed the server on 4th February to the larger memory size. On 11th > Feb, it triggered the alarm and we rebooted the server. Memory usage is > starting to climb again, albeit not at the pace that it was previously. > > What is particularly curious for us is that we have four Mailman 3 servers. > Two of them are single domain and two are hosting two domains. This server > is one of the servers hosting two domains, but it is also the ONLY one of > the four servers exhibiting this memory pattern. By comparison, the memory > usage for the other server hosting two domains looks like this: > > https://1drv.ms/u/s!Aq7dn8GDGLdCodsVyEufkeQqit_cGA?e=ifrJHs > > The biggest difference we are aware of with the "problematic" server is > that some of the list archives are quite large by comparison with our other > servers. The largest is 15K messages, then 2K then 1K. The other three > servers have lists with relatively small archives by comparison (double or > triple digits at worst). > > Finally, and just FYI, we're using PostgreSQL for the database and Xapian > for the indexing engine. > > Regards > > Philip > > On Tue, 15 Feb 2022 at 16:36, Mark Sapiro <mark(a)msapiro.net> wrote: > > > On 2/15/22 08:11, Stephen Daniel wrote: > > > TL;DR: quite likely your system has plenty of memory and is operating > > > normally. > > ... > > > On Tue, Feb 15, 2022 at 5:28 AM Prasanth Nair < > prasanth.nair(a)linaro.org> > > > wrote: > > > > > >> Hi, > > >> > > >> We are having an issue with high memory usage on our Mailman3 server. > > >> Initially, our server had 16GB RAM and the memory usage went up from > 5% > > to > > >> 92% within a week. So, we increased the memory capacity to 32GB RAM > and > > we > > >> can see that the memory usage is going up in the same pattern. > > >> > > >> Any idea why it is happening? or any suggestions? > > > > > > Just for reference, mail.python.org supports 231 MM 2 lists and 182 MM 3 > > lists with 16 GB of ram and no swap. > > > > The server that supports this list and also the www.list.org web site > > has 4 GB of ram and no swap. > > > > -- > > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > > San Francisco Bay Area, California better use your sense - B. Dylan > > _______________________________________________ > > Mailman-users mailing list -- mailman-users(a)mailman3.org > > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

3 years, 9 months

[MM3-users] Re: Welcome message template errors

by Mark

On 2024-04-28 01:32, Mark Sapiro wrote: >> I also note that this timeout only occurs when it is a subscription >> via REST. I.e. subscription via email or `mailman addmembers` sends >> the proper welcome message. >> >> So it has something to do with the retrieval being done by a rest >> runner, but I still don't know what the underlying issue is. I created a text file as a custom welcome message and chowned it to mailman:mailman, then edited the database record (template.uri) from http://localhost:8000/mailman3/api/templates/list/mytestlist.mydomain.com/l… TO file:///home/mark/welcome.txt This delivered the custom welcome message to the user when subscribed via Mass Subscribe. I'm not sure what this tells me other than a call can be made to a file but not the database. > More strangeness. I've now done a test on one of the production > servers I manage, and there is no error. Both the production server > and the development server are running the identical Mailman code > installed in a venv, they both have the same version of urllib3 and > similar Python versions, 3.9.16 on the development server and 3.9.10 > on the production server. Both servers are running under Ubuntu > 20.04.6. The only other difference I see is on the production server > everything runs as user mailman and on the development server > everything runs as user mark. Mine is Debian 12 with Python 3.11.2. Could a newer Python version really cause this amount of grief?

1 year, 7 months

Search results for query "sapiro"