Search results for "Postorius" - Mailman-users

Re: There are currently no mailing lists!?

by Odhiambo Washington

On Mon, Jul 22, 2024 at 9:06 PM Christian via Mailman-users < mailman-users(a)mailman3.org> wrote: > I recently upgraded Debian 10 'buster' and when I logged in to Postorius in > the browser I got the ' Mailman REST API not available. Please start > Mailman > core' message. So I started it up from the command line doing 'sudo -u > mailman /opt/mailman/mm/venv/bin/mailman start -f' which yielded > You upgraded Debian from Buster to what??? Bullseye or Bookworm? Why not just 'sudo systemctl start mailman'? > Starting Mailman's master runner > Generating MTA alias maps > mailman@lists: > > Now when I do ' https://lists.ccalternatives.org/mailman3/lists/?all-lists > ' > in my browser I get the message ' There are currently no mailing lists.' > Meanwhile their archives exist. Are the lists actually gone or ? > The lists are there since your DB is intact and your venv is intact as well. > > What am I doing wrong/what's going on here please? Thank you in advance. > I suspect your Python was upgraded to a version different than the one you use to create your virtualenv in the 1st place. If that is the case, you'll need to create a new virtualenv and migrate the data. > One more thing...I am running mailman core version GNU Mailman 3.3.5, Core > API Version 3.1, and Core Python Version 3.7.3. > Is there a reason you did not upgrade MM3 core and MM3 Web all this time? Anyway, check the version of Python used to build your virtualenv against the system version after you upgraded: # cat /opt/mailman/mm/venv/pyvenv.cfg home = /usr/local/bin include-system-site-packages = false version = 3.11.2 executable = /usr/local/bin/python3.11 command = /usr/local/bin/python -m venv /opt/mailman/mm/venv If older than the system version, you just need to build a new virtualenv and migrate the data. HTH. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

1 year, 4 months

Re: Subscription request cannot be moderated: 404 Not Found

by Eric Broens

Hi Mark, I can't find any log entry which indicate a problem with this subscription request. The _request table does not contain a request for the specific mailinglist, but I am not sure if subscription requests are included in this table. The token is found in the pended table: +-----+------------------------------------------+---------------------+ | id | token | expiration_date | +-----+------------------------------------------+---------------------+ | 115 | 4d6b4b11266c7332684c5a492eb3f92c531f761b | 2031-01-04 16:17:38 | +-----+------------------------------------------+---------------------+ These are the relevant entries in the pendedkeyvalue table: +------+--------------+-------------------------------------+-----------+ | id | key | value | pended_id | +------+--------------+-------------------------------------+-----------+ | 1130 | type | subscription | 115 | | 1131 | list_id | "********************" | 115 | | 1132 | email | "***********************@*********" | 115 | | 1133 | display_name | "" | 115 | | 1134 | when | "2021-01-06T16:17:38" | 115 | | 1135 | token_owner | "moderator" | 115 | +------+--------------+-------------------------------------+-----------+ Can I workaround this problem for this specific subscriber by adding entries in the _request table, and if so what do I need to add? Thanks in advance, Eric On Friday, January 15, 2021, 11:42:47 PM GMT+1, Mark Sapiro <mark(a)msapiro.net> wrote: On 1/14/21 3:43 PM, Eric Broens via Mailman-users wrote: > Hi Mark, > The link that shows the subscription request pending approval is as follows: 'https://<host_url>/postorius/lists/<listname>.<domain>/subscription_requests' > > The link behind the accept button is as follows. 'https://<host_url>/postorius/lists/<listname>.<domain>/handle_subscription_request/4d6b4b11266c7332684c5a492eb3f92c531f761b/accept' > > I have checked if there is already a subscription with that email address, but that's not the case. I don't know why that would 404 unless the token 4d6b4b11266c7332684c5a492eb3f92c531f761b didn't exist, but since it came from a "current" view, it should be good. I have seen occasional issues where two moderators attempt to handle the same request in close succession - i.e. both get the pending request and try to handle it and the second fails because the request is gone, but that appears not to be the case here. You might check the web server logs, the Django log and Mailman's logs to see if there is any more of a clue. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

4 years, 10 months

Re: Confirmation emails to Users has wrong domain name (example.com!)

by Odhiambo Washington

On Thu, Sep 30, 2021 at 4:09 PM Abhilash Raj <maxking(a)asynchronous.in> wrote: > > > On Thu, Sep 30, 2021, at 2:20 AM, Odhiambo Washington wrote: > > On Thu, Sep 30, 2021 at 1:59 AM Abhilash Raj <maxking(a)asynchronous.in> > > wrote: > > > >> > >> > >> > On Sep 29, 2021, at 2:34 PM, Odhiambo Washington <odhiambo(a)gmail.com> > >> wrote: > >> > > >> > On Wed, Sep 29, 2021 at 8:31 PM Mark Sapiro <mark(a)msapiro.net> wrote: > >> > > >> >> On 9/29/21 9:50 AM, Odhiambo Washington wrote: > >> >>> 1. Confirmation emails to Users has wrong domain name (example.com > !) > >> >>> <https://docs.mailman3.org/en/latest/faq.html#id1> > >> >>> < > >> >> > >> > https://docs.mailman3.org/en/latest/faq.html#confirmation-emails-to-users-h… > >> >>> > >> >>> > >> >>> This happens when your reverse (SSL) proxy isn’t setting up the > correct > >> >>> headers when proxying requests. Fix this by setting the right > >> >>> proxy_set_header directives: > >> >> ... > >> >>> How is this supposed to be mitigated in Apache when using WSGI? > >> >>> > >> >>> My config: > >> >>> > >> >>> WSGIDaemonProcess hyperkitty threads=25 > python-path=/usr/local/mailman > >> >>> user=mailman group=mailman > >> >>> WSGIPythonHome "/usr/local" > >> >>> WSGIProcessGroup hyperkitty > >> >> > >> >> > >> >> You are using mod_wsgi and not proxying at all, so this is not > relevant > >> >> in your case. > >> >> > >> >> Are you actually seeing this issue? If so, it might be related to > >> >> > >> >> > >> > https://docs.mailman3.org/en/latest/faq.html#the-domain-name-displayed-in-h… > >> >> > >> > > >> > My issue is related to this, but the documentation referred to is not > for > >> > the faint-hearted! > >> > I can't make head or tails of it. > >> > >> Click on “Domain” in Postorius from the top bar, which should take you > to > >> the Domains page. > >> > >> For your domain (in the “Mail Host” column), see the corresponding “Web > >> Host” column, it should look something like: > >> > >> lists.mailman3.org (lists.mailman3.org) > >> (Edit) > >> SITE_ID = 1 > >> > > > > Mine looks different, slightly. But there is no example.com at all. > > > > [image: Abhilash.png] > > > > > > > >> > >> on a new-ish version of Postorius. > >> > > > > I have the newest versions of everything, having installed only > yesterday. > > > > > >> > >> If it doesn’t show the right values and instead shows “example.com” for > >> you, click on the “Edit” link, which will take > >> you to a page that will allow you to edit both the values. > >> > > > > It shows the right values, but with "SITE_ID = 2". In my > settings_local.py > > I have SITE_ID = 1. > > I suppose the example.com is the one tied to SITE_ID = 1 and that is > what I > > have in my settings_local.py. > > That is most likely what is happening. > > > Should I edit my settings_local.py? > > Yes, please update it to have SITE_ID = 2 to correspond to the site you > want and restart. > Thank you very much. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' :-)

4 years, 2 months

Re: sqlite3 Database "read only" error on mailman_sync job

by Ugnius

Hi. I have fixed my RO database issue. Thank You for help, Abhilash. It was 100% my stupid fault. I had issues with "django-admin". If I ran manage.py it works correct, if I run django-admin - it not: only basic commands. I did not find way how to give correct environment. Then I found on some notes, that manage.py does exactly the same what django-admin does. So I made work around : I had created /usr/local/bin/django-admin to my project file manage.py and it started to work. Later I installed mailman from 0 under another user, and I forgot about that sym link. So why because database was Read Only. It had simply leaking write privileges to database file. Understandable, because owner was another user... > On 6 Sep 2019, at 00:20, Abhilash Raj <maxking(a)asynchronous.in> wrote: > >> On Thu, Sep 5, 2019, at 1:49 PM, Ugnius wrote: >> >> Thank You. This is very valuable information. According documentation I understood that "runserver" should run on some internal port and uwsgi just works like proxy connects to the same as runserver port and serves content to apachi. Because when I was playing with the setup without "runserver" Web interface didn't work. >> Yes, I have stopped "runserver" and page still works fine. :) This didn't solve me database RO issue, but this information was important. Thank You. I am thinking what to do now... Maybe to recreate database (make initiation once again). It could take 2-3 hours. I did not noticed option to export/import lists setting. Or to try find solution how to make sqlite3 multiple >> writable. > > I think your problem about locked database might be solved. Like I mentioned, it is possible that those two running server could be stepping over each other toes. > > After removing the "runserver" process, I think you shouldn't fall into anymore of those locking issues. > > FWIW, it is also highly recommended to not use sqlite for any production use cases. Either postgres or mysql is preferred. > >> SQLite version 3.16.2 2017-01-06 16:32:41 >> Enter ".help" for usage hints. >> >> BR >> Ugnius >> >>> On 5 Sep 2019, at 23:31, Abhilash Raj <maxking(a)asynchronous.in> wrote: >>>> On Thu, Sep 5, 2019, at 1:15 PM, Ugnius wrote: >>>> >>>> I do not know... In the setup example settings was something like process number = 2. But I understand at least 1 uwsgi and 1 run server should run? >>> >>> No, only uwsgi needs to run. It is fine if there are two of those, since it probably forks to have 2 worker process. There shouldn't be any "python manage.py runserver" processes running at all, it is meant only for testing purposes. >>> >>> The doc mentions that it is "development" server[1][2]. >>> >>> [1]: # Run the Django's "development" server at localhost:8000 >>> [2]: http://docs.mailman3.org/en/latest/prodsetup.html#setting-up-django-project >>> >>> Perhaps it should be more descriptive about what does it mean. >>> >>>> >>>>> On 5 Sep 2019, at 23:08, Abhilash Raj <maxking(a)asynchronous.in> wrote: >>>>> 6644 ? S 0:12 /usr/bin/uwsgi --master --ini /opt/home/uwsgi.ini >>>>> 6645 ? S 0:00 /usr/bin/python3 /opt/home/manage.py runserver 0:8015 >>>>> 6653 ? Sl 636:28 /usr/bin/python3 /opt/home/manage.py runserver 0:8015 >>>>> 6656 ? Sl 0:27 /usr/bin/uwsgi --master --ini /opt/home/uwsgi.ini >>>>> >>>>> This fishy, why do you have both uwsgi and "python manage.py runserver" running? >>>>> >>>>> You only need one of those, since they both do the exact same thing. Please remove >>>>> whatever is running the "python manage.py runserver" part and only use uwsgi. >>>>> >>>>>> On Thu, Sep 5, 2019, at 1:02 PM, Ugnius S wrote: >>>>>> Thank You, Abhilash, to helping. I have checked the processes, but also I have read, that sqlite3 can run in multi writing mode. >>>>>> May mailman processes are next: >>>>>> >>>>>> 6644 ? S 0:12 /usr/bin/uwsgi --master --ini /opt/home/uwsgi.ini >>>>>> 6645 ? S 0:00 /usr/bin/python3 /opt/home/manage.py runserver 0:8015 >>>>>> 6653 ? Sl 636:28 /usr/bin/python3 /opt/home/manage.py runserver 0:8015 >>>>>> 6656 ? Sl 0:27 /usr/bin/uwsgi --master --ini /opt/home/uwsgi.ini >>>>>> 6643 ? S 0:00 /usr/bin/python3 /usr/bin/master -C /opt/home/mailman.cfg >>>>>> 659 ? S 0:46 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=in:0:1 >>>>>> 6660 ? S 0:42 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=nntp:0:1 >>>>>> 6661 ? S 0:42 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=digest:0:1 >>>>>> 6662 ? S 0:48 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=pipeline:0:1 >>>>>> 6663 ? S 0:44 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=archive:0:1 >>>>>> 6664 ? S 0:41 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=command:0:1 >>>>>> 6665 ? S 0:01 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=retry:0:1 >>>>>> 6666 ? S 0:45 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=virgin:0:1 >>>>>> 6667 ? Sl 0:16 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=lmtp:0:1 >>>>>> 6668 ? Sl 1:30 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=rest:0:1 >>>>>> 6669 ? S 0:42 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=bounces:0:1 >>>>>> 6670 ? S 0:49 /usr/bin/python3 /usr/bin/runner -C /opt/home/mailman.cfg --runner=out:0:1 >>>>>> >>>>>> Looks like there are no double. Nothing more is running. >>>>>> >>>>>> 2019-09-05, kt, 03:16 Abhilash Raj <maxking(a)asynchronous.in> rašė: >>>>>> >>>>>>> On Wed, Sep 4, 2019, at 2:05 PM, Ugnius S wrote: >>>>>>> >>>>>>> I have installed using documentation. My linux distribution version has no full package available. I have installed step by step: >>>>>>> $ sudo pip3 install mailman >>>>>>> $ sudo pip3 install postorius >>>>>>> $ sudo pip3 install hyperkitty >>>>>>> $ sudo pip3 install mailman-hyperkitty >>>>>>> And of course all mandatory components... Everything without errors, I can drop db files and I can download mailman-suite again and start from empty page again. But it will not guarantee that DB will not become READ ONLY again. Here should be logic. No developing support here? >>>>>> >>>>>> Could you be running two separate processes such that the database is locked by one and so other finds it as READ Only? When you find a read only database, can you check if it is open in some other process? >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> 2019-09-03, an, 19:36 Abhilash Raj <maxking(a)asynchronous.in> rašė: >>>>>>> >>>>>>> >>>>>>> On Sun, Sep 1, 2019, at 3:22 AM,ugniusviln(a)gmail.com wrote: >>>>>>> > Hi. All the project installation is under the same user / path and >>>>>>> > it works. Archiving working as well, (just one remark: no matter what >>>>>>> > to set in postorius, hyperkitty shows list as public) >>>>>>> > I have found work around - it is possible to change list as private in >>>>>>> > "admin" page. >>>>>>> > I started to get issues with runjobs after I have created couple of >>>>>>> > new lists with "no archive" option. >>>>>>> >>>>>>> How did you install all these components. It seems there the installation is broken in weird ways. >>>>>>> >>>>>>> > >>>>>>> > If as owner user from project home I run: >>>>>>> > >>>>>>> > django-admin mailman_sync >>>>>>> > >>>>>>> > Traceback (most recent call last): >>>>>>> > File >>>>>>> > "/home/user/.local/lib/python3.5/site-packages/django/db/backends/utils.py", >>>>>>> > line 84, in _execute >>>>>>> > return self.cursor.execute(sql, params) >>>>>>> > File >>>>>>> > "/home/user/.local/lib/python3.5/site-packages/django/db/backends/sqlite3/base.py", >>>>>>> > line 383, in execute >>>>>>> > return Database.Cursor.execute(self, query, params) >>>>>>> > sqlite3.OperationalError: attempt to write a readonly database >>>>>>> > >>>>>>> > "django-admin runjob sync_mailman " - makes read only errors as well. >>>>>>> > "django-admin runjob new_lists_from_mailman" >>>>>>> > django.db.utils.OperationalError: attempt to write a readonly database >>>>>>> > END TRACEBACK >>>>>>> > >>>>>>> > I did not find information how to fix this. It was worked without errors >>>>>>> > before. >>>>>>> > I noticed, that if I stop mailman, "django-admin mailman_sync" starts >>>>>>> > to work. >>>>>>> > If I start "mailman start", then "django-admin mailman_sync" failing >>>>>>> > again. >>>>>>> > I did not check other jobs, but I think what is related to database, >>>>>>> > will get errors. >>>>>>> > Seems mailman locking the database. It is not a file permissions issue. >>>>>>> > Mailman db location is: /project_directory/var/data/mailman.db >>>>>>> > >>>>>>> > if to rename /project_directory/var/data/mailman.db to something, >>>>>>> > mailman creates database file size 0, django-admin runjobs start >>>>>>> > working, but of course postorius stops working. I did it for testing. >>>>>>> > I have returned this database file back, postorius started to work >>>>>>> > again, but django-admin runjobs does not work, error: "database read >>>>>>> > only". >>>>>>> > Please help me how to fix this issue. >>>>>>> > _______________________________________________ >>>>>>> > Mailman-users mailing list -- mailman-users(a)mailman3.org >>>>>>> > To unsubscribe send an email to mailman-users-leave(a)mailman3.org >>>>>>> > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >>>>>>> > >>>>>>> >>>>>>> -- >>>>>>> thanks, >>>>>>> Abhilash Raj (maxking) >>>>>> >>>>>> -- >>>>>> thanks, >>>>>> Abhilash Raj (maxking) >>>>>> >>>>>> >>>>> >>>>> -- >>>>> thanks, >>>>> Abhilash Raj (maxking) >>>>> >>>>> >>> >>> -- >>> thanks, >>> Abhilash Raj (maxking) >>> >>> > > -- > thanks, > Abhilash Raj (maxking) > >

6 years, 2 months

Re: moving list from mm3 -> mm3

by Jens Günther

Thanks Mark for our answers, I already got a bit further. Am 09.12.23 um 00:27 schrieb Mark Sapiro: >> But it returns {"title": "405 Method Not Allowed"} and I don't know >> where to look in the docs, since it shows almost always a way through >> mailman shell. > You need a PUT command, not POST. Try the above with `-X PUT`. With the `-X PUT`-command, there is the following error: {"title": "400 Bad Request", "description": "Unexpected parameters: bounces_address, created_at, digest_last_sent_at, fqdn_listname, http_etag, join_address, last_post_at, leave_address, list_name, mail_host, next_digest_number, no_reply_address, owner_address, post_id, posting_address, request_address, usenet_watermark, volume"} -> I don't know why, since I tried it with the exact same example-list from where I got all these parameters (ex- and then import) ... do I still miss something? Or does it only work on "fresh" lists? > You can use Postorius -> Mass operations -> Mass subscribe to add the > members, but you can't do that for owners and moderators. Also you lose > information about the members such as Delivery status, Delivery mode and > other user settings. Is there another way to preserve these information about the members? (in general I don't think a lot of people put in effort to customize there memeberships ... and furthermore they would need to register, right?) >> ## registered users > > That is not the answer. Those are Mailman core users, not Django (web > login) users. You need to get the relevant entries from the auth_users > and account_emailaddress tables and add them to the new DB. Ok, that would be fine. Are there any more relations to other tables/columns that need to be set? (I just realised, that there are 2 different id in each table ... the one in auth_users is the user_id in account_emailaddress :-) scary). Do you have any suggestions on how to add them correctly? Thank you in advance for now and another big one in general to all of you on this list that try to help continouisly with all these problems. It's a great pleasure to be able to ask for help on a list like that. Thank you! Have a good rest of the year. Regards, Jens.

1 year, 11 months

Re: [SOLVED] Re: how to unhold a hold message from a nonmmeber in hold list.

by Odhiambo Washington

On Fri, Apr 21, 2023 at 1:27 PM Guillermo Hernandez (Oldno7) via Mailman-users <mailman-users(a)mailman3.org> wrote: > > El 21/4/23 a las 11:03, Guillermo Hernandez (Oldno7) via Mailman-users > escribió: > > > > El 21/4/23 a las 10:37, Odhiambo Washington escribió: > >> > >> > >> On Fri, Apr 21, 2023 at 11:20 AM Guillermo Hernandez (Oldno7) via > >> Mailman-users <mailman-users(a)mailman3.org> wrote: > >> > >> I'm having troubles with Django and cannot access to web admin. > >> > >> I need to unhold a message while figures out what is happening. > >> > >> Searching the mailman cli help commands didn't help me. > >> > >> Anyone who knows how to unhold a message, please? > >> > >> > >> TBH, I find it a lot easier to fix the web UI than mess with the > >> python shell :-) > > > Well... I found you where right. Messing with the CLI seems a nightmare > to me. > > The problem with the UI seems solved. What I finally have done: > > Detected that the module "asgiref" was in a different version that the > server (with all the same config) that is working. pip upgraded it. This > didn't work, but it raised new error messages (about hyperkitty parsing > messages) > > Installed pip-review and upgraded ALL python modules with pip-review > --local --interactive > > (Re)upgraded all mailman3 packages with > > pip install --upgrade --force-reinstall --no-deps --no-cache-dir > django-mailman3 mailman mailman-hyperkitty mailmanclient postorius > django hyperkitty uwsgi mod-wsgi > > Restarted apache24 > > And then all begun to work. > > Thanks a lot for your advising and help. > Do you now see why a virtualenv style of setup will save you lots of grief? :-) -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

2 years, 7 months

Re: MM3 Docker

by Odhiambo Washington

On Sun, 6 Jan 2019 at 23:39, Mark Sapiro <mark(a)msapiro.net> wrote: > On 1/6/19 8:47 AM, Odhiambo Washington wrote: > > > > If you could include the missing steps that you left out then I could > > possibly be able to follow the steps :-) > > > Please help me understand what steps are missing and I will try to > provide the information. Okay. At the installation stage, you say the following: "I already had some things set up in /opt/mailman including a git subdirectory containing clones of the GitLab mailman, mailmanclient, mailman-hyperkitty, hyperkitty, django-mailman3 and postorius projects. On the third server I have added mailman-suite because I use the settings.py from that project as the basis for mine." Please assume that the reader is a newbie to python3 virtualenv and give the details of the commands they need to execute to (a) clone all the projects needed and (b) create the virtualenv. The next steps are: mkdir /opt/mailman/mm Then you say "I then created a /opt/mailman/mm directory and within that a Python 3.6 virtualenv" What happens here is something that needs clarification, because you are specific about "python3.6". I have googled and realized that one has to be specific about the python version they want to use in their virtualenv as shown here -> https://stackoverflow.com/questions/45293436/how-to-specify-python-version-… exact command to execute to create the /opt/mailman/mm/venv exact command to execute to activate virtualenv From there, I think the newbie can follow and fumble till they come up with a working MM3 site. Come to think of it, your steps kind of look clearer than most that I have encountered. I do understand that there are only a few people working on the MM3 development, which I think is the reason there still isn't a document similar to the ones included with MM2.1x source under doc/ However, if your steps here are polished a little, they can possibly become one such doc. I think MM3 docs would come with three options - virtualenv option, docker option and package option. Installing from source seems like something not easily doable. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

6 years, 10 months

Re: moving lists to another server

by Mark Sapiro

On 11/21/23 04:32, Odhiambo Washington wrote: > > I tested the above out of curiosity. It failed. There are 9 lists in the > setup, but I am only interested in 1. > > ``` > (venv) mailman@debian12:~$ mailman-web shell > Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] > Type 'copyright', 'credits' or 'license' for more information > IPython 8.17.2 -- An enhanced Interactive Python. Type '?' for help. > > In [1]: from hyperkitty.lib.compat import get_list_by_name > > In [2]: mlist = get_list_by_name('list_name_only', 'list_domain') You can't use literally ('list_name_only', 'list_domain'). If the list of interest is for example alist(a)example.com, the above should be mlist = get_list_by_name('alist', 'example.com') ... > Was I supposed to substitute something for the real listname of the list I > wanted? Yes. > Another thing - totally unrelated - that surprised me: > While using the commented-out block below, I could not click and get > anything beyond http://http://list.name/mailman3 as anything to do with > /postorius/xxxxx gave "Page not found" error. > However, with the second config block that uses the UNIX socket, I am able > to access everything > > ``` > #<IfModule mod_proxy.c> > # ProxyPreserveHost On > # ProxyPass "/mailman3" "http://127.0.0.1:8010/mailman3" > # ProxyPass "/archives" "http://127.0.0.1:8010/archives" > # ProxyPass "/accounts" "http://127.0.0.1:8010/accounts" > # ProxyPass "/admin" "http://127.0.0.1:8010/admin" > # ProxyPass "/user-profile" "http://127.0.0.1:8010/user-profile" > # </IfModule> > > ProxyPassMatch ^/static/ ! > ProxyPassMatch ^/favicon.ico ! > ProxyPass / unix:/opt/mailman/mm/var/uwsgi.sock|uwsgi://localhost/ > ProxyPassReverse / unix:/opt/mailman/mm/var/uwsgi.sock|uwsgi://localhost/ > ``` > > Why is it that the UNIX socket option worked flawlessly while the other > option failed??? The first option does not have ProxyPass for urls matching /postorius or /hyperkitty so such urls don't work. The second option passes everything (ProxyPass / ) to uwsgi via the socket so all urls that uwsgi can handle will work. You could do something similar with the first option with simply ProxyPass "/" "http://127.0.0.1:8010/" assuming uwsgi is listening on port 8010. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years

how to migrate mailing lists to a new domain using mailman 3

by C. Souza

In a few days my current domain will be disabled (it is out of my control) and I would like to know how to migrate the mailing lists to the new domain name. As a solution I added a new domain in mailman 3 and created a script that replicates the lists in this new domain. To forward the emails received by the old lists, the script removes the members and adds the email of the new list. Is there a fashionable way to do that? Also, one problem that I don't have a solution is the archive of the old lists. How to bring them to the new ones? Thanks for your help. My mailing list server is running on a CenOS-7x with postgreSQL and the following mailman packages: Package Version ------------------- ---------- 1 aiosmtpd 1.2.4 2 alembic 1.5.3 3 arrow 0.17.0 4 asgiref 3.3.1 5 atpublic 2.1.2 6 authheaders 0.13.0 7 authres 1.2.0 8 bleach 3.3.0 9 blessed 1.17.12 10 certifi 2020.12.5 11 cffi 1.14.4 12 chardet 4.0.0 13 click 7.1.2 14 cmarkgfm 0.5.2 15 cryptography 3.3.1 16 decorator 4.4.2 17 defusedxml 0.6.0 18 Django 3.0.12 19 django-allauth 0.44.0 20 django-appconf 1.0.4 21 django-compressor 2.4 22 django-extensions 3.1.0 23 django-gravatar2 1.4.4 24 django-haystack 3.0 25 django-mailman3 1.3.5 26 django-picklefield 3.0.1 27 django-q 1.3.4 28 djangorestframework 3.12.2 29 dkimpy 1.0.5 30 dnspython 2.1.0 31 docutils 0.16 32 falcon 2.0.0 33 flufl.bounce 3.0.1 34 flufl.i18n 2.0.2 35 flufl.lock 3.2 36 gunicorn 20.0.4 37 HyperKitty 1.3.3 38 idna 2.10 39 importlib-resources 5.1.0 40 lazr.config 2.2.3 41 lazr.delegates 2.0.4 42 mailman 3.3.2 43 mailman-hyperkitty 1.1.0 44 mailman-web 0.0.3 45 mailmanclient 3.3.2 46 Mako 1.1.4 47 MarkupSafe 1.1.1 48 networkx 2.5 49 nose 1.3.7 50 oauthlib 3.1.0 51 packaging 20.9 52 passlib 1.7.4 53 pip 21.0.1 54 postorius 1.3.3 55 psycopg2-binary 2.8.6 56 publicsuffix2 2.20191221 57 pycparser 2.20 58 Pygments 2.7.4 59 PyJWT 2.0.1 60 pyparsing 2.4.7 61 python-dateutil 2.8.1 62 python-editor 1.0.4 63 python3-openid 3.2.0 64 pytz 2021.1 65 rcssmin 1.0.6 66 readme-renderer 28.0 67 requests 2.25.1 68 requests-oauthlib 1.3.0 69 rjsmin 1.1.0 70 robot-detection 0.4 71 setuptools 39.2.0 72 six 1.15.0 73 SQLAlchemy 1.3.23 74 sqlparse 0.4.1 75 typing-extensions 3.7.4.3 76 urllib3 1.26.3 77 uWSGI 2.0.19.1 78 wcwidth 0.2.5 79 webencodings 0.5.1 80 wheel 0.36.2 81 Whoosh 2.7.4 82 zipp 3.4.0 83 zope.component 4.6.2 84 zope.configuration 4.4.0 85 zope.deferredimport 4.3.1 86 zope.deprecation 4.4.0 87 zope.event 4.5.0 88 zope.hookable 5.0.1 89 zope.i18nmessageid 5.0.1 90 zope.interface 5.2.0 91 zope.proxy 4.3.5 92 zope.schema 6.0.1 -- Carlos Souza

3 years, 4 months

Re: Mailman 3 deployment: ok to use same database instance for core, web, and hyperkitty tables?

by Andrew Hodgson

Stephen J. Turnbull wrote: >s.dobrau(a)ucl.ac.uk writes: > > We are currently working on a migration plan for Mailman -> Mailman > 3. We have around over 3000 mailing lists with a total of 50K > members >Medium-size. I guess a 2-CPU/4GB Linode or similar should handle the Mailman load (including Postorius and HyperKitty), you might want more memory and maybe more CPUs if you're running the RDBMS on the same host. In the 23k lists/50k users/100k inbound messages/day migration described below, >we ended up reconfiguring the VMs 3 times before declaring the system stable. IMO the 2 x 8CPU/16GB final setup was way overpowered for the need, but we didn't try to calibrate finer than that. Yep, I have helped a migration with around 150 lists but not as much archived data, and ended up with a 2 CPU with 8GB machine. I think we could have gone lower, but found we needed the larger VM to help us import the archives. I used a managed PostGreSQL instance from Amazon for the database. > > Our plan is to have core+web (instance 1) and Hyperkitty (instance > 2). With the archives on the FS local to the Hyperkitty. >The archives are in the database as BLOBs. There's no reason to have a separate instance for HyperKitty. It's not obvious to me that you need more than that 2x4 Linode for both Mailman and RDBMS, but if you're going to split it's Mailman vs RDMBS, not Mailman vs HyperKitty. I would agree. I think the issue here is there are several ways of splitting out Mailman people get bogged down in whether they should run separate parts on different instances. In my view I have always ran Mailman, Postorius and Hyperkitty on the same host and I think it works better that way. For example, if you run Mailman and the web components on different hosts, you need to make sure the Mailman REST interface is secured, whereas if its on the same host then the REST interface can just listen on localhost. The Dockerised instances do use separate containers for Mailman Core and the web components, but that is the only real time I have seen the components split. > > And all 3 tables (core, web, Hyperkitty) pointing to the same > database on a remote SQL (mysql-enterprise) instance. Using class > mailman.database.mysql.MySQLDatabase (and whatever necessary on the > Django/Hyperkitty instance). >Make sure your MySQL database(s) for Mailman are configured with the utf8mb4 option for 4 byte UTF-8 support, or it will choke on emojis and the like. I don't think there are any other gotchas for MySQL. Are there any performance benchmarks for running MySQL vs PostGreSQL? I tend to fall back to PostGreSQL because it seems to be what is used in a lot of places, I have never recommended going with MySQL for a Mailman instance. >Note there are a lot more than 3 tables in Mailman's databases. The "traditional" configuration is a "mailman" database for core and a "mailmanweb" database for Django and the archives, but Mark says that as the tables are disjoint across Mailman, Django, and the archives there's no reason not to use a >single "mailman" database for all of them. I believe that's how this list's host is configured. Yep, the original installation guide created one database for Mailman and the tables for all components are in the same database, this was how my larger install was done. Later revisions of the guide have us split the core vs web components to different databases, I don't think there is any performance hit either way by doing that. >Importing list configurations, users, and subscriptions into Mailman 3 is pretty fast, as long as you're not using the stock Postfix support.[2] The problem is that generating the Postfix alias files for the lists seems to be noticably linear in number of lists, which means it's quadratic for the mass import. IIRC with >5000 lists you'd be looking at >1m/list just to keep regenerating Postfix's alias database, ie 5000 minutes. I found two solutions, both of which required patching Mailman. I hope to get both into the next release. >I think we got it down to <5s/list, for 5000 lists that's < 1hr. That is very interesting I had a similar issue with the Postfix alias generation in my larger setup but didn't have time to identify the root cause. In the end I used Exim sending all mails for the list domain to Mailman which is the setup I use elsewhere and it doesn't rely on the alias file generation. >Importing archives is ... yeeech. The client decided they didn't want a ton of archives anyway (you know corporations, if it's not required by law, shred it after 6 months). I don't recall the size estimate accurately but we kept 6 months X 4500 lists, maybe 100GB out of 2.3TB of mboxes. That took 24 hours to >import into HyperKitty, without doing the full-text indexing. I do know the original conservative estimate was 20 days to import the whole 2,3 TB. The full-text indexing took more than a week if I remember correctly. Yes, I had the same problem with the archives. I did import all the archives, in our case it took us several months to get the archives in so as not to stress load on the box. Full text indexing really caused an issue for me, I ended up disabling it for the time we were importing archives then generated a clean index once the archives were imported. This wasn't a good user experience and I would probably plan this out better next time. >It's also quite possible to migrate incrementally, a few lists at a time. Mailman 2 and Mailman 3 can coexist happily on the same host. >Mark can advise on that, I think. Its worth noting this is more difficult to do now on Debian/Ubuntu installs as Python2 has been removed. This was something I ran into myself when trying to co-exist both installs on the same box. Andrew.

2 years, 1 month

Mailman-users search results for query "Postorius"