Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: Mailman not able to process "big" requests

by Darren Smith

Our installation at https://lists.rootsweb.ancestry.com has around 33,000 lists and somewhere around 1.5 million members. We have lists with 5,000+ members. Note that I haven't tried any mass subscriptions. When we upgraded from mailman 2 to mailman 3, we used the old .pck files and ran the import on each of the lists. It did take about 6 weeks to do the full import all 33,000 lists. I'm not joking. As for sending messages - there isn't any problem sending that many messages out, but then we are also not trying to subscribe list members at the time. We are, though, in the process of importing the old archives from our old mailman 2 system. This doesn't seem to affect the rest of the system. We have seen some inefficiencies in the new system, such as the front page of postorius - loading all of the information for 200 lists (if the user set the "lists per page" to 200) was bringing down the system. We have our own metadata system to be able to search for lists by state/region/surname (ours is a genealogy mailing list system), so we had to replace the front page with our own "find a list" functionality. The other main problem was the page where a user can see what lists they subscribe to. That page was also a denial of service if the user had several hundred lists they are subscribed to. And for a genealogy mailing list system, this is common. We have actually ended up creating our own API layer (only accessible on the local machine) and make certain calls to it for the most painful of the issues. One of these days (when we get a moment to breathe) we are going to post some of our changes to the project so that we can ditch our own API and just use the internal REST API. To be fair, the inefficiencies are not necessarily with mailman or the rest API itself - but with Postorius making hundreds of calls to load a single page. Though if the API supported loading multiple lists at once instead of having to load them serially, that might help as well. So while we have not seen any timeout issues on the call to mass subscribe users, we HAVE found several other calls that are effectively a denial of service and have gotten around them ourselves, with plans to contribute to the project in the near future. -Darren On Tue, Apr 24, 2018 at 8:47 AM, Mark Sapiro <mark(a)msapiro.net> wrote: > On 04/24/2018 12:36 AM, Abhilash Raj wrote: > > On Mon, Apr 23, 2018, at 10:41 PM, Jonas Felber wrote: > > >> We also have a problem with sending out emails to "large" (>50) groups, > >> as they seem to not getting sent out at all. Do you have any idea what > >> the problem here could be? > > > > Which MTA are you using? What do you see in mailman's logs and MTA's > logs? > > > > I haven't seen this kind of problem before with Mailman 3. This very > list is hosted using Mailman 3 and I believe has more than 50 subscribers. > > > Nor have I seen this issue. This list currently has 158 non-digest > members. There are several @python.org Mailman 3 lists with over 100 > non-digest members including the active yt-users(a)python.org list with > 301 members, and I have gotten no reports of an issue like this. > > Also, there are over 600 lists at <https://lists.fedoraproject.org/>, > and I don't think this is an issue there either. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

7 years, 6 months

[MM3-users] Re: Uncaught runner exception: 'str' object has no attribute 'decode'

by Henrik Rasmussen

Thanks, Mark. @Abhilash, any chance for an update to Docker-mailman to fix these known issues soon? https://gitlab.com/mailman/mailman/issues/256 https://gitlab.com/mailman/mailman/issues/359 @Mark, Mark Sapiro wrote: >>> In addition, the log contains other trace backs that might or might not associate to this problem: >>> >>> Sep 24 21:11:27 2017 (34) Uncaught runner exception: 'ascii' codec can't encode character '\udcea' in position 3093: ordinal not in range(128) >> >> This may be a bug. The msgtext argument in >> 'self._connection.sendmail(envsender, recipients, msgtext)' is supposed >> to be a string containing only ascii characters. I'm interested in >> what's in Mailman's >> var/queue/shunt/1506287488.0529234+cbf8a31cad265a40a18e8955be0c9ad8b48a1b66.pck. >> You may be able to print the contents with Mailman's >> >> bin/mailman queue >> var/queue/shunt/1506287488.0529234+cbf8a31cad265a40a18e8955be0c9ad8b48a1b66.pck Isn't this a Mailman 2 command? What is the Mailman 3 equivalent? -- Henrik Rasmussen <her(a)adm.ku.dk> University of Copenhagen, Danmark

8 years

[MM3-users] Re: How to fix: [Django] ERROR (EXTERNAL IP)

by Jigme Datse

On Fri, 1 Nov 2024 20:56:42 -0700 Mark Sapiro <mark(a)msapiro.net> wrote: > On 11/1/24 20:14, Jigme Datse wrote: > > > > python venv/lib/python3.12/site-packages/mailman_web/manage.py > > migrate > > > > which *might* be the wrong manage.py but other than the change with > > the migrate command, without it looks to throw the same output. > > > > The Django log was at that time empty it's now attached (larger than > > I'd like). > > It is not necessary to post the entire log which repeats the same > ERROR and traceback over 300 times. The errors all seem to be > disallowed hosts. 321 from IP 45.79.102.219, 5 from > mail01.datsemultimedia.com and a few others. The PTR from IP > 45.79.102.219 is mail01.datsemultimedia.com and its A is > 45.79.102.219. Is this your server, if so add both 45.79.102.219 and > mail01.datsemultimedia.com to ALLOWED_HOSTS. > Thank you... I while helpful, I think I'll abandon this project... Appreciate the help.

11 months, 3 weeks

[MM3-users] Re: Mailman3-Web lots of database queries

by Stefan Tatschner

On Sun, 2018-11-18 at 14:38 -0800, Mark Sapiro wrote: > No. This is not normal. This is apparently coming from Django qcluster. > At least it is qcluster that uses the django_q_ormq table, but these > repeated queries are not normal, at least I don't see them in the sites > I administer. Thank you for the pointer to qcluster. I don't know so much about these web components and their relationships. According to the documentation [1] it seems that qcluster is not needed as long as cronjobs are configured. Since debian ships the entries in /etc/cron.d [2, 3] I disabled the qcluster components. Also there is a debian package issue; qcluster is started twice when using the provided service files: * the uwsgi script ini starts qcluster * the mailman3-qcluster service starts it as well I know the debian thing is out of scope here, I just want it documented somewhere. Stefan [1]: https://hyperkitty.readthedocs.io/en/latest/install.html#asynchronous-tasks [2]: https://gitlab.com/mailman/hyperkitty/blob/master/example_project/crontab [3]: https://salsa.debian.org/mailman-team/mailman-suite/blob/master/debian/mail…

6 years, 11 months

[MM3-users] Re: import21: ModuleNotFoundError: No module named 'Mailman'

by Christian Schwarz

On Sun, 2019-07-28 at 17:07 -0700, Mark Sapiro wrote: > As to how to proceed, there are a couple of ways. One is to fix import21 > so it ignores this issue. I just reported > <https://gitlab.com/mailman/mailman/issues/618> on this, but I'm not > sure how to fix it. I don't think it will work to try the pickle.load > and pass the ModuleNotFoundError exception because I think the rest of > the pickled data won't be returned. > > It looks like you still have the MM 2.1 installation, so you could use > either MM 2.1's bin/withlist or bin/config_list to set the list's > bounce_info to an empty dictionary. and then use the updated config.pck > for the import. > > This assumes that bounce_info is the issue. You can verify that there is > a non-empty bounce_info dictionary with either bin/withlist or bin/dumpdb. Indeed, bounce_info is not empty! I think I need more precise instructions on how to empty bounce_info using config_list, though.

6 years, 2 months

[MM3-users] Re: An archive/read-only list on a full setup?

by Andreas Schamanek

Thanks a lot, Mark! After some tests with a test setup I came to the conclusion that what you suggested was better than what I had so far (i.e. the empty Mailman list). So, I deleted the empty list, leaving only the Hyperkitty list. On Tue, 11 Apr 2023, at 11:22, Mark Sapiro wrote: > Since there is no Mailman list, attempts to post or reply directly from > HyperKitty will fail, ... Yep, I get a 404. Doesn't look nice but should be OK. Not many people will try, and the description of the list makes clear that it is only an archive. I have also added a Redirect on the webserver so that people who click [Manage subscription/this list] are redirected to the actual remote list. That helps, too. So, no fake list, and the link to subscribing still works. The particular list is low volume. So, it will take some time until I can report how well it worked out. But after my tests I am confident. Thanks again, -- -- Andreas :-)

2 years, 6 months

[MM3-users] Re: mailman3-web and the django middleware problem

by Thomas Stein

Using the virtenv installation method seems to be reasonable, yes. Adding the suggested line i did not try yet. Thanks! Am 26. April 2024 10:29:18 MESZ schrieb Odhiambo Washington <odhiambo(a)gmail.com>: >On Fri, Apr 26, 2024 at 8:30 AM Thomas Stein <himbeere(a)meine-oma.de> wrote: > >> Am 2024-04-26 00:52, schrieb Mark Sapiro: >> > On 4/25/24 14:01, Thomas Stein wrote: >> >> Am 2024-04-25 22:03, schrieb Mark Sapiro: >> >>> >> >>> I don't know how things are arranged in the Debian/Ubuntu package, >> >>> but you may have a settings.py which at the end does something like >> >>> ``` >> >>> try: >> >>> from settings_local import * >> >>> except ImportError: >> >>> pass >> >>> ``` >> >>> If that is the case, adding the above snippet to the end of >> >>> settings_local.py should work. >> >> >> >> Yes, that's the case. I added the snippet in question to >> >> settings_local.py but this leads to another error: >> >> >> >> File "/usr/share/mailman3-web/settings.py", line 503, in <module> >> >> from settings_local import * >> >> File "/usr/share/mailman3-web/settings_local.py", line 194, in >> >> <module> >> >> x = list(MIDDLEWARE) >> >> ^^^^^^^^^^ >> >> NameError: name 'MIDDLEWARE' is not defined >> >> dpkg: error processing package mailman3-web (--configure): >> > >> > Yes. I understand. In your case, MIDDLEWARE is only defined in >> > settings.py and not in settings_local.py. As I understand it originally >> > you added >> > ``` >> > x = list(MIDDLEWARE) >> > x.insert(0, 'allauth.account.middleware.AccountMiddleware') >> > MIDDLEWARE = x >> > ``` >> > to settings.py and you say it didn't help. I don't know why that >> > wouldn't work as long as there is no setting for MIDDLEWARE in >> > settings_local.py and you added that after the MIDDLEWARE setting in >> > settings.py. >> > When you say it didn't help, do you mean you still get the >> > >> > django.core.exceptions.ImproperlyConfigured: >> > allauth.account.middleware.AccountMiddleware must be added to >> > settings.MIDDLEWARE >> > >> > message or something else? I note that the original post in this thread >> > refers to /etc/mailman3/mailman-web.py which is presumably where the >> > package ultimately puts the Django settings, and I don't know what >> > `dpkg --configure` is doing. >> > >> > One thing you could try is in settings_local.py, put >> > ``` >> > from settings import MIDDLEWARE >> > x = list(MIDDLEWARE) >> > x.insert(0, 'allauth.account.middleware.AccountMiddleware') >> > MIDDLEWARE = x >> > ``` >> > to work around the NameError. >> >> After that I get: >> >> File "/usr/share/mailman3-web/urls.py", line 20, in <module> >> from django.conf.urls import include, url >> ImportError: cannot import name 'url' from 'django.conf.urls' >> (/usr/lib/python3/dist-packages/django/conf/urls/__init__.py) >> dpkg: error processing package mailman3-web (--configure): >> installed mailman3-web package post-installation script subprocess >> returned error exit status 1 >> Errors were encountered while processing: >> >> I guess I'll wait for the new package for now. >> > > >How about adding: > >MIDDLEWARE += ('allauth.account.middleware.AccountMiddleware',) > >in settings_local.py > >And how about you use the virtualenv method ><https://docs.mailman3.org/en/latest/install/virtualenv.html> for more >peace of mind in the future? > >

1 year, 5 months

[MM3-users] Re: mailman.cfg (use_https: yes) fails

by Wolfgang Bock

Hi again, in this case I'll check out either and report. Regards Wolfgang Am 18.11.2019 um 05:06 schrieb Mark Sapiro: > On 11/16/19 4:54 AM, Wolfgang Bock via Mailman-users wrote: > I will look at the https issue further when I get time. > > >> In this context I faced a strange behavior: >> >> In the database mailman3web in table django_site are listed several >> domains, either in the django-domain-management site >> https://[mydomain]/mailman3/admin/sites/site/ >> >> id domain name >> 1 example.com Example.com >> 2 [mydomain-1] [mydomain_name_1] >> 3 [mydomain-2] [mydomain_name_2] >> 4 [mydomain-3] [mydomain_name_3] >> >> In case I delete line 1 with the id 1, in the database or in the >> django-web-site, postorius fails to work. After re-inserting this line, >> postorius works again. > > In your settings.py, you probably have SITE_ID = 1. Thus, when you > delete the site with id = 1, Postorius fails. You need to set SITE_ID to > an existing site in settings_local.py. >

5 years, 11 months

[MM3-users] Re: ValueError: ...:EVP_DigestInit_ex:disabled for fips

by Ralf Wiegand

Hello Mark, Can I substitute Django with something that will be FIPS complaint? Can I do without it? Thank You, Ralf On 5/19/20 1:02 PM, Mark Sapiro wrote: > On 5/19/20 5:57 AM, Ralf Wiegand wrote: >> (venv-3.6) [root@maildev mailman-suite_project]# python manage.py migrate >> Operations to perform: >> Apply all migrations: account, admin, auth, contenttypes, >> django_mailman3, django_q, hyperkitty, openid, postorius, sessions, >> sites, socialaccount >> Running migrations: >> Applying contenttypes.0001_initial...Traceback (most recent call last): > ... >> File >> "/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/backends/utils.py", >> line 221, in names_digest >> h = hashlib.md5() >> *ValueError: error:060800A3:digital envelope >> routines:EVP_DigestInit_ex:disabled for fips* >> >> >> >> Hello group, I am running into this error. any idea what is going >> wrong here? > This is a Django issue, not Mailman per se. The migration that's failing > is django/contrib/contenttypes/migrations/0001_initial.py. A brief > search seems to indicate that this won't work with NSS FIPS. >

5 years, 5 months

[MM3-users] Re: certain msgs not archived

by Sandor Dibuz

On 2/17/22 19:16, Mark Sapiro wrote: > > If you rerun hyperkitty_import with the since= option set to a date > earlier that the oldest message to import, it should just work. Messages > which are already in the archive (by Message-ID:) will be ignored by the > import process. Great! So valuable info. Thank you, Mark. "earlier that the oldest message" you sure mean earlier then the oldest of already archived. Or since= any date of the mbox messages to be imported. Will check the hyperkitty_import for since= option format. Yet another question. Is it supposed to stop minutely/hourly/etc. mailman3-web cron jobs during import to avoid any collisions? Or even mailman3-web completely but probably it needs to be running. I only ask because my last import attempt on a test maillist failed with [ERROR/MainProcess] Failed indexing 1 - 2 (retry 5/5): (1712, 'Index hyperkitty_attachment is corrupted') (pid 5841): (1712, 'Index hyperkitty_attachment is corrupted') Optimizing cured the hyperkitty_attachment InnoDB index corruption problem but scared the sh.. out of me for a moment... Thank you for your help, Mark. Sanyi

3 years, 8 months

Search results for query "sapiro"