Just mostly out of curiousity: why don't you install as venv or docker? Then it would be somewhat more independent of underlying OS and somewhat more up to date...

I also did the installation first with the Debian-packages when moving in from mailman2, but fortunately realised quite fast, that there might be a bunch of problems. Also the last years I read on this list gave me the impression, that there are quite often problems with the Debian-packaged-installation that arise while installing ... or later when upgrading the OS. I'm quite happy with docker right now (and hope that it's kept up to date for some time longer :-D ... thanks in any cases!).

Regards, Jens.

Am 26.08.23 um 14:54 schrieb Franklin Weng:

The problem was reported here:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1041681

AND here:

https://gitlab.com/mailman/django-mailman3/-/issues/68

Not sure which it should belong to.  There are screenshots in the bug report, but some users said they couldn't open our Nextcloud instance. If you have problems opening the screenshots too you can give me a private mail I'll send pictures to you.

And - honestly I ran a mm3 instance on Debian Buster (10) a few years ago, and when I tried to upgrade my Buster to Bullseye the mm3 was totally broken.  Fortunately I made a VM snapshot before upgrading so I reverted it back.  However that instance was unstable too - it would run out of memory about every two weeks, so I had to setup a cron to reboot the VM every week.  I have no idea what happened.  It was pretty frustrated.

Then we moved all our services to another server hosts and I decided to rebuild the mm3 instances.  I tried to build-remove-rebuild several times, including building it on Bullseye but got a lot of database locked errors, then tried it on Bookworm and got a broken django admin page, and finally set up a pretty stable instance on Bullseye which has run for about 6 weeks without any problem.  So I decided to share the progress and what I found.

Hope it helps.

Franklin

Franklin Weng writes:

Thanks for your comments!

Thank you for following up on my queries.

...

As far as I know, Debian packages use the system Python and the system Django. Is that correct?

Correct.

This means that you're stuck with the Debian version of the Mailman 3 suite until they update which can be slow. Mailman 3 is now mature enough that if you're still happy with the Debian version a month after installing, you'll probably still be happy with it when Debian updates. It's a good tradeoff for the majority.

But for example mixing a "testing" Mailman with a "stable" Python is likely to end badly because we frequently upgrade required levels of dependencies. So you either have to get a Debian upgrade of Mailman of your current Debian version, or you need to move to a whole not-stable Debian system. Not a pleasant choice if you think you need new features or a bugfix *now*.

This is why we recommend the venv or other isolated installations. They allow you to upgrade the dependencies as well as the core packages without interfering with the rest of your system.

IIRC I got some errors about handling characters encoding if I don't set up charset in language.master and language.en section. But the memory is vague and I don't want to go back retesting that...

I suspect that you set up charset in .master and realized that it doesn't work because .en overrides it with ASCII, and then fixed .en. That's what happened to me.

Honestly I was totally confused by all these, and as I have said, the MM3/Django/Hyperkitty/... documents didn't help much.

Upstream documents are rarely very helpful with whole sets of large packages from distros in my experience. You install the packages, you go through a bunch of configuration screens, and you finish knowing *nothing about how anything works*. Then you need a little tweak that the configuration screens don't deal with, and you don't even have the vocabulary to grep or Google effectively. :-( Then again, I'm the kind of person who prepares to sleep by reading software manuals. ;-) But I do think you lose quite a bit of knowledge by using packages, and it's painful to find obscure details without it.

Django doesn't actually show its face much in Mailman, so Django docs are not much use even to the site admin, unless you actually hack on Mailman. The only exceptions are authentication and the template language. Authentication is just hard, with more details then the Lego set in the Lego store. The templating docs are pretty good, IMO.

The Mailman docs are in somewhat sad state. The content is actually quite comprehensive and detailed as these things go, but much of it is hard to discover. The original idea was to keep the docs in the sources, as doctests for examples and in the docs directory for exposition. These docs are more developer-oriented, obviously. Then there were several attempts to create a suite package which would require core, Postorius, and HyperKitty and streamline the setup process. The "mailman-suite" version came with a companion "mailman-suite-docs" package, which I think is now the basis for mailman.readthedocs.io aka docs.mailman3.org. However mailman-suite turned out to more trouble than it was worth, and mailman-web was developed to address the more useful approach of unifying some mailman-to-Django APIs for Postorius and HyperKitty (the django_mailman3 package) and configuration (a unified settings.py, as well as standardized top-level urls.py and views.py). Of course each of these contributed more documentation, some of it redundant, older versions left to slowly become obsolete (ie, incorrect) ....

"Someday" I'd like get the documents reorganized, eliminate as much redundancy as possible and make things more discoverable. But it's a mammoth task.

Steve

On Wed, Aug 30, 2023 at 4:06 PM Andy Smith <andy@strugglers.net> wrote:

Hi Franklin,

On Sat, Aug 26, 2023 at 04:59:04PM +0800, Franklin Weng wrote:

...

I had a blog article describing how I did it. Maybe there were something you find it ridiculous, but they are all what I found important and may not be shown in the current documents.

https://blogs.slat.org/node/1012

Having been through this process with Debian 11 last year, in addition to several of your discoveries, I found that the default Whoosh full text indexer was just not usable. It is ridiculously slow and memory-hungry, and used 6+ gigabytes of RAM and days of CPU time to try to index some of my Mailman 2 mboxes.

I should think almost everyone will want to switch to Xapian instead. That is accomplished on Debian with:

# apt install libxapian30 python3-xapian python3-xapian-haystack

and then in mailman-web.py:

HAYSTACK_CONNECTIONS = { 'default': { 'ENGINE': 'xapian_backend.XapianEngine', 'PATH': '/var/lib/mailman3/web/fulltext_xapian_index', }, }

As far as I understand Xapian is not the default because although Xapian has an open source license, it is one that's not compatible with Mailman's GPL3 license.

At the time I installed, Debian 12 had not yet been released, and I won't upgrade that host until necessary, but I personally would not do a new install on Debian 11 so I would want to resolve the problems you had with Mailman 3 on Debian 12.

Cheers, Andy

And some unlucky admins might want to look at https://gitlab.com/mailman/hyperkitty/-/issues/408

-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]