Re: Hyperkitty on Debian 12
by Markus Grandpre
Dear Michael,
thank you very much for your clarification.
Best regards,
Markus
Am 3/26/24 um 17:38 schrieb Michael Oberrauch:
> On Tue, 2024-03-26 at 08:06 +0100, Markus Grandpré wrote:
>> Dear Mr Sapiro,
>>
>> thank you very much for your help. The required command is:
>>
>> # mailman-web hyperkitty_import \
>> -l mailman21-migration-test-04(a)mailman.uni-konstanz.de \
>> /var/mailman21/testlist.mbox
>>
>> Due to the following warnings, I still have to customize Hyperkitty:
>>
>> django_mailman3.MailDomain: (models.W042) Auto-created primary key
>> used
>> when not defining a primary key type, by default
>> 'django.db.models.AutoField'.
>> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
>> DjangoMailman3Config.default_auto_field attribute to point to a
>> subclass
>> of AutoField, e.g. 'django.db.models.BigAutoField'.
>> ...
>>
>> Can you give me another hint? As Linux admin I installed the packages
>> mailman and mailman-web that were offered to me under Debian 12.
>> Although during the installation of the packages user "list" was
>> created, I cannot use it:
>>
>> # su - list
>> This account is currently not available
>>
>> As with Mailman 2.1, I will probably have to continue working with
>> Mailman3 as root. Too bad that there is no documentation for Debian
>> admins like me, who don't want to dive too deep into the python
>> framework. Or have I overlooked something?
>>
>> Best regards,
>> Markus
>>
>>
>
> Hi Markus,
>
> we are also running MM3 on Debian 12 from Package Repositories.
>
> First of all, you should not try to directly switch user (su) to the
> lists user, it's generally intended to executes commands as that user,
> if necessary; either using `runuser`, `sudo`, or `su -c`.
>
> However, the prepackaged scripts `mailman-wrapper` and `mailman-web`
> already do this and therefore you ca run those as root without
> permission problems.
>
> On Debian there are two relevant users for MM3: the list user runs
> mailman core, whereas the www-data user runs Postorius and Hyperkitty.
> So you have to make shure, that the www-data user can access the .mbox
> files while importing.
>
> Hope that helps.
>
> Cheers,
> Michael
>
>
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>
> This message sent to markus.grandpre(a)uni-konstanz.de
--
Markus Grandpré
Universität Konstanz
Kommunikations-, Informations-, Medienzentrum (KIM)
Abteilung IT-Dienste Forschung, Lehre und Infrastruktur,
Tel: ++49 7531 88 4342
2 years, 5 months
Re: Guidance Request: Migrating Existing Mailman 3 Installation to Docker-Based Deployment
by Washington Odhiambo
On Tue, Apr 14, 2026 at 1:08 PM Prasanth Nair <prasanth.nair(a)linaro.org>
wrote:
> Hi Odhiambo and Stephen,
>
> Thank you both for your helpful responses and guidance.
>
> Given that this is a critical production system, I am now considering
> migrating to a new server using a host-based Mailman installation first,
> before evaluating a Docker-based setup at a later stage.
>
> In that case, could you please advise on the recommended approach for
> migrating from my current host-based Mailman 3 installation to a new
> host-based server? In particular, I would appreciate any guidance on:
>
> -
>
> Safely migrating the PostgreSQL databases (e.g., dump and restore
> approach)
> -
>
> Preserving mailing lists, memberships, and configurations
> -
>
> Migrating HyperKitty archives and Postorius data
> -
>
> Any best practices to ensure a smooth cutover with minimal downtime
>
> Any documentation or pointers you can share would be greatly appreciated.
>
For a host-based installation, the recommended method is documented here:
https://docs.mailman3.org/en/latest/install/virtualenv.html
If your current installation is based on the above, then (off the top of my
head):
1. Ensure that the database backend on the new host meets the
specified requirements in the above documentation.
2. Ensure that the Python version is the same on both hosts (so that
nothing bites you immediately to migrate). @Stephen J. Turnbull
<steve(a)turnbull.jp> call them software gremlins :-)
3. Install MM3 the normal way on the destination host and ensure you run
all the tests. 'mailman info' should show similar details on both hosts.
4. Postfix should be configured the same way on both hosts.
The migration will involve the following:
(a) dumping the DB on the source and restoring on the destination
(b) zip up /etc/mailman3 on the source and extract it on the destination
(c) zip up /opt/mailman, move it to the destination server, extract it and
copy the files/directories (except the venv directory) from this archive to
/opt/mailman/
You then fix the permissions so that /opt/mailman is owned by the
mailman user. The commands are on the link liven above
(d) You need your Nginx vhost configs to be migrated to the new server.
(e) DNS. Oh DNS! You will flip the records to point to the new server
That should preserve everything as you want.
These details are off the top of my head. I might have forgotten something
so I beg your pardon in advance.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
In an Internet failure case, the #1 suspect is a constant: DNS.
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
5 months, 1 week
Re: Confusing mailman user model
by Gilles Filippini
Gilles Filippini a écrit le 19/07/2020 à 18:00 :
> Mark Sapiro a écrit le 19/07/2020 à 17:30 :
>> On 7/19/20 8:10 AM, Gilles Filippini wrote:
>>> Hi,
>>>
>>> I'm trying to understand the Mailman user model, and I'm confused about
>>> the respective roles of tables 'auth_user' and 'user'.
>>
>>
>> auth_user is a Django auth table. These are the users listed in the
>> Django admin web UI.
>>
>> user is Mailman core's users.
>>
>>
>>> Both have an 'id' column referred to as 'user_id' when used as a foreign
>>> key. But there seems to be no relation at all between them as they use
>>> different ids for the same user.
>>
>>
>> Right, they are unrelated.
>>
>>
>>> Our mailman server was migrated from mailman2 to mailman3 about 1.5 year
>>> ago, and there is an inconsistency I'd like to fix:
>>> 2 of our users have a registered Mailman account with their respective
>>> email addresses in table 'auth_user', but these very same addresses in
>>> the 'address' table are linked to another old admin account in the
>>> 'user' table.
>>
>> Their Mailman account is the 'old' one. the auth_user account is a
>> Django account
>>>
>>> Then if add one of these addresses as a domain owner, it it the old
>>> admin address which is selected instead. How could I fix that?
>>
>>
>> That's because domain owner in a Mailman thing and involvs the Mailman
>> user. You could delete the Django users via the Django admin UI and then
>> re-register them via Postorius. That *might* work.
>
> When login into postorius, it is the Django account that is used, right?
> Then how is the mailman account retrieved from there? It seems UUIDs are
> used in the process, but I fail understanding how, so far. Wouldn't it
> be safe to retrieve the new Mailman account associated to these Django
> users, and link there emails from 'address' table' to them?
Reading the mailman-web source code I understand now that the mailman
account is retrieved from the Django account email address. Then both
users are tied to the same old mailman admin account, and deleting then
registering them again won't change anything on this aspect.
Would this work?
1- For both addresses, update their record in table 'address' to set
'user_id' to null
2- On their next login to Mailman-web, a new mailman account would be
created and associated with their own email address.
Is my understanding correct?
_g.
6 years, 2 months
broken route to accounts
by Thomas Krichel
I am using apache to run the web interface, and my Postel port, and
I have the proxy routs
ProxyPass "/mailman3" "http://127.0.0.1:4386/mailman3" works
ProxyPass "/archives" "http://127.0.0.1:4386/archives" works
ProxyPass "/accounts" "http://127.0.0.1:4386/accounts" is broken
ProxyPass "/admin" "http://127.0.0.1:4386/admin" works
ProxyPass "/user-profile" "http://127.0.0.1:4386/user-profile" works
What does "is broken" mean here? Well, since I have debug on,
GET "http://127.0.0.1:4386/accounts"
yields something like (shortened to cut to the salient matter)
| <body>
| <p>
| Using the URLconf defined in <code>mailman_web.urls</code>,
| Django tried these URL patterns, in this order:
| </p>
| <ol>
| <li>postorius/</li>
| <li>hyperkitty/</li>
| <li>mailman3/</li>
| <li>archives/</li>
| <li>^user-profile/delete$
| [name='mm_user_account_delete']</li>
| <li>^user-profile/$
| [name='mm_user_profile']</li>
| <li>accounts/</li>
| <li>admin/</li>
| </ol>
| <p>
| The current path, <code>accounts</code>, didn’t match any of these.
| </p>
| </body>
Sure I have accounts/ rather than accounts, but neither are working routes
endowed with a trailing slash. Digging for mailman_web.urls, I am lead
to believe this is related to my settings, and as I inspect them, I am
reminded that I appended
del ACCOUNT_AUTHENTICATION_METHOD
ACCOUNT_LOGIN_METHODS = {'email', 'username'}
del ACCOUNT_EMAIL_REQUIRED
ACCOUNT_SIGNUP_FIELDS = ['email*', 'username*', 'password1*', 'password2*']
to my settings file. Commenting these lines out, restarting mailmanweb, still the
same error.
Ideally an error like this should be shown to the user. Instead it leads
to a endlessly repeated requests for users to verify their email address, as seen
in my live system at say
https://lists.repec.info/accounts/confirm-email/
These have no CSS, pointing to an apache missconfiguration
For what it's worth, here is my current apache configuration
| <VirtualHost *:443>
| ServerName lists.repec.info
| ServerAlias www.lists.repec.info
| ServerAlias list.repec.info
| ServerAlias www.list.repec.info
| ServerAdmin webmaster@localhost
|
| RewriteEngine On
| RewriteRule "^/?$" "https://lists.repec.info/mailman3/lists/" [R]
|
| Alias /static "/usr/local/mailman/web/static"
| <Directory "/usr/local/mailman/web/static">
| Require all granted
| </Directory>
|
| Alias /favicon.ico "/usr/local/mailman/web/static/postorius/img/favicon.ico"
|
| <IfModule mod_headers.c>
| RequestHeader unset X-Forwarded-Proto
| <If "%{HTTPS} =~ /on/">
| RequestHeader set X-Forwarded-Proto "https"
| </If>
| </IfModule>
|
| <IfModule mod_proxy.c>
| ProxyPreserveHost On
| ProxyPassMatch ^/static/ !
| ProxyPass "/mailman3" "http://127.0.0.1:4386/mailman3"
| ProxyPass "/archives" "http://127.0.0.1:4386/archives"
| ProxyPass "/accounts" "http://127.0.0.1:4386/accounts"
| ProxyPass "/admin" "http://127.0.0.1:4386/admin"
| ProxyPass "/user-profile" "http://127.0.0.1:4386/user-profile"
| </IfModule>
|
| LogLevel debug
| ErrorLog /var/log/apache2/lists.repec.info_error.log
|
| CustomLog /var/log/apache2/lists.repec.info_access.log combined
| ServerSignature On
|
| Include /etc/letsencrypt/options-ssl-apache.conf
| SSLCertificateFile /etc/letsencrypt/opt/live/repec.info/fullchain.pem
| SSLCertificateKeyFile /etc/letsencrypt/opt/live/repec.info/privkey.pem
|
| </VirtualHost>
--
Written by Thomas Krichel http://openlib.org/home/krichel on his 22209th day.
6 months
No emails distributed after mail received for a new list following https://docs.list.org/en/latest/install/virtualenv.html on new mailman3 Debian 12 installation
by Philip Bondi
Hello to all:
I require a Debian 12 VM that supports:
- Subversion
- Let's Encrypt and required challenges
- IMAP server
- mailman3
- DKIM server
I'm currently running same on a CentOS 7.x VM. So that needs to be retired/migrated sooner rather than later because CentOS is out-of-support. On the Debian 12 VM other services appear to be setup and working. mailman3 is the 2nd last service that I'm standing up.
In this moment, I've been able to follow https://docs.list.org/en/latest/install/virtualenv.html and https://docs.list.org/en/latest/config-web.html
I have mailman3 and mailmanweb services running successfully. I have not yet been able to integrate mailmanweb into Apache (probably because of the Subversion and Let's Encrypt requirements). But I'm able to access local Postorius instance and configure a superuser and a list with web browser running locally on Debian 12 VM.
My list is named ebill. An email was received for the DL, but not distributed. Can you advise how to troubleshoot from here?
Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: connect from smtprelay0015.hostedemail.com[216.40.44.15]
Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: Anonymous TLS connection established from smtprelay0015.hostedemail.com[216.40.44.15]: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256
Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: 3CD47189B: client=smtprelay0015.hostedemail.com[216.40.44.15]
Mar 25 15:23:04 shackleton12 postfix/cleanup[247722]: 3CD47189B: message-id=<a7aee014-f200-4bd4-bbc4-bc9f37abb6df(a)systemdatabase.com>
Mar 25 15:23:04 shackleton12 postfix/qmgr[241822]: 3CD47189B: from=<pjbondi(a)systemdatabase.com>, size=3238, nrcpt=1 (queue active)
Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: disconnect from smtprelay0015.hostedemail.com[216.40.44.15] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Mar 25 15:23:04 shackleton12 postfix/lmtp[247723]: 3CD47189B: to=<ebill(a)systemdatabase.ca>, relay=127.0.0.1[127.0.0.1]:8024, delay=0.29, delays=0.06/0.03/0.02/0.19, dsn=2.0.0, status=sent (250 Ok)
Mar 25 15:23:04 shackleton12 postfix/qmgr[241822]: 3CD47189B: removed
root@shackleton12:~# postconf -n
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
compatibility_level = 3.6
inet_protocols = ipv4
local_recipient_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp
mailbox_size_limit = 0
mailbox_transport = lmtp:unix:private/dovecot-lmtp
masquerade_domains = systemdatabase.ca
mydestination = $myhostname, systemdatabase.ca, localhost.systemdatabase.ca, localhost, systemdatabase.homelinux.com
mydomain = systemdatabase.ca
myhostname = systemdatabase.ca
mynetworks = 127.0.0.0/8
myorigin = systemdatabase.ca
owner_request_special = no
proxy_interfaces = 192.168.1.8
readme_directory = no
recipient_delimiter = +
relay_domains = hash:/opt/mailman/mm/var/data/postfix_domains
relayhost = [smtp.bellnet.ca]:25
smtp_tls_loglevel = 1
smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtp_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination permit_inet_interfaces
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination permit_inet_interfaces
smtpd_tls_cert_file = /etc/letsencrypt/live/systemdatabase.ca-0001/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/systemdatabase.ca-0001/privkey.pem
smtpd_tls_loglevel = 1
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtputf8_enable = no
transport_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp
unknown_local_recipient_reject_code = 550
1 year, 6 months
Re: Advice on migrating to Mailman 3
by Daniel Armengod
Hello Christoph,
> If we setup such a system, can we assume that it will silently operate
> without causing much administration work most of the time (like my
> nginx server)? Or is maintaining a Mailman 3 server a significant burden?
As with all things, it is easy once you've mastered it, and easy for
someone who /has/ mastered it to say it's easy :P
It's an interesting technical challenge, but please bear in mind that
you'll need to gain a reasonable level of expertise not only in
configuring, deploying and maintaining Mailman3, but also the associated
MTA (usually postfix) with (very probably) a SPAM filter hooked up to
it. And, of course, both Django-based WebUI projects for administering
(Postorius) and archiving (Hyperkitty), with associated HTTP servers and
DBMSs.
The (almost-)complete list of software you'll need be familiar with to
achieve any reasonable degree of success is:
* An HTTP server with WSGI backend (*Apache2 / Nginx, mod_wsgi / uwsgi*)
* An MTA for reception and sending of messages (*Postfix*)
* (Optionally) an anti-spam & DKIM-signing solution (*rspamd / amavis*)
* The *Django* Python web framework - deploying *Postorius* +
*Hyperkitty* does require some TLC on this front. Familiarity
deploying Django-based projects helps /immensely/.
* *MySQL / PostgreSQL* / any other supported DBMS - for the
aforementioned Web UIs.
* Assorted knowledge of how email servers interoperate, in particular
setting up FCrDNS, SPF records for better deliverability, possibly
an understanding of DMARC/DKIM in case things ever go wrong
(spoiler: yahoo does /not/ play nice with mailing lists ever since
they published their DMARC policies...)
* And, of course, general SysAdmin knowledge to install the OS, set up
SysV/Systemd units so everything starts up smoothly after a reboot,
probably a monitoring system to get alerts & insights in case
something goes wrong, etc.
> The only problem is that our list runs on Mailman 2. We would like to
> move to Mailman 3
Why? Is there any feature you're missing in Mailman2 that Mailman3 has?
If your Mailman2 installation "just works" and serves all of your needs,
my advice would be to /keep it/. It's an old warhorse, but works
reliably, and is maintained (so long as your hosting provider actually
bothers to maintain it up-to-date, of course). E-mail technology is
ancient; one hardly needs the latest version of /anything/ that is based
around e-mail.
If your current ISP offers a decent service and you're happy with them,
I'd definitely stay. Especially if you don't have a dedicated sysadmin
profile in your project's team.
> are you aware of organizations that propose Mailman 3 hosting for open
> source projects?
If you're based in Europe, I can offer a few recommendations. I'm not
comfortable with posting specific names on a public mailing list (that's
tantamount to self-advertising), but feel free to ask by private reply.
Regards,
Daniel
6 years, 7 months
Re: Weird response to postings after subscriptions and accounts
by Paul Arenson
And the to clarify my problems with the signup procedure and how I might deal
with it. (Mostly a very small change on your part and an explanation on my website for my users)
"I understand it's not totally explicit, but it says 'To subscribe or unsubscribe from this list, please log in first. If you have not previously logged in, you may need to set up an account with the appropriate email address.’ To me the implication is once you have created an account, you have to come back, log in and subscribe. What do you want it to say?"
I had to test to find out that there is not a lot I would change. Let me just preface it by sharing my experience. As a person with learning disabilities (following instructions is the hardest), I found the procedures dating when using a few pretend signups.
This brings me back to a job I had in the 80s and 90s. When I was working on user manual for Seiko watches, the engineers wrote the manuals in Japanese-Engllish based on what the watches could do. But I put myself in the place of the user, who often did not know why they needed that function. And so I had to translate into an English explanation started from the point of view of what they might want to do with the watch.
In the case of Mailman 3 much depends on the individual list and its settings, so my conclusion is it is mostly on me. With just a suggestion that you have a SIGN UP blue button to make things less confusing and not necessitating multiple clicking.
The main obstacle I found was I had my list settings as NO to "Choose whether to include this list on the list of all lists". Changing this to "Yes" made it a little easier.
Here are my two tests and conclusion.
2) TEST 1
________
Sign up first...
https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres…
...you will find that if I have said NO on "Choose whether to include this list on the list of all lists” you will find it hard to go back to the subscribe page. Unless you backspace. Or paste the address again. Not ideal
QUESTIONS: Am I missing something? By not having my discussion listed, people cannot find it to subscribe.
Or is NOT LISTED better where I add people manually to subscribe and then tell them to sign up?
3) TEST 2
________
One solution is I could put the list back on the host’s list of lists back and tell people to click on Discuss.
That is what I did for now. Then it is rather smooth. I could tell my users (especially ones from previous company’s mailing list who want to interact with the hyperkitty site)
to
CLICK LOGIN
CLICK SIGNIN
then follow all these steps:
Verify
Confirm
Get taken to sign in page
Click Discuss
Subscribe
Click Archives
Or I could leave it as it is and tell people to backspace until they find the signup page and then subscribe.
Or tell them to copy and paste https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres…
or https://list.tokyoprogressive.org/accounts/signup/ in the same browser window and then subscribe.
CONCLUSION and SUGGESTION—> WHY NOT HAVE A SEPARATE BLUE BUTTON TO SIGN UP TO AVOID THE CONFUSING clicking twice?
IS THIS THE ANSWER?—
Tell my users as follows on a WORDPRESS PAGE as follows:
- - - - - - - - - - - -
Greetings. If you have never signed up before, follow instructions A. If you are coming from the previous dadamail list, follow instructions B
INSTRUCTIONS A
Steps 1-3 to get on the mailing list and be able to post from your email account.
Continue to steps 4 and 5 if you want to see the archives and post from that page. (Note to Mark: I have turned the archives off by default to make it more of a no brainer to want to sign up)
INSTRUCTIONS B
You are already registered, you will continue to get messages and be able to send to the list. If you want to keep using the same email address you are all set and do not need to do anything.
If you want to keep using the same address, but also be able to access the archives, follow steps 4 and 5 only.
If you want to register a new address, follow steps 1-5 below.
1) Go here
https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres…
Add your email address at the BOTTOM of the page. Name is optional.
2) Then wait for email to reply to come.
3) That’s it. You can stop here. However, you can sign up to the archives which makes this list more of a forum to see at a glance old conversations. And you can also post from it. To do so, continue to steps 4 and 5,
4) Right on the same page, you can sign up and sign in by clicking
log in and then sign up and then follow the same procedure as above. (Mark, SEE SUGGESTION FOR A BLUE SIGN UP BUTTON).
5) Once you have signed up, you can log in to access the archives and web posting.
- - - - - - - - - - - -
Hope the above long winded post was clear. Feel like I am sseing the lite.
7 years
Re: Unable to download attachments from HyperKitty
by Derek Lambert
This happens with all attachments, regardless of file type.
I've rebuilt my containers numerous times and have tried running them on different hosts.
uwsgi does log it's only generating 26 bytes for attachment requests. Example:
uwsgi: [pid: 15|app: 0|req: 3/20] 192.168.253.1 () {48 vars in 2110 bytes} [Wed Feb 20 20:18:35 2019] GET /hyperkitty/list/test-list@domain.com/message/XWTEPEKLG5DNHW4M223RH3ULQ3ZMNOKK/attachment/3/attachment.htm => generated 26 bytes in 36 msecs (HTTP/1.1 200) 13 headers in 649 bytes (1 switches on core 1)
It doesn't appear it's a header issue, they're identical (except for the dates) with and without the patch. I can provide them if you'd like.
I assume it's not an issue with HyperKitty since no one else has reported it. Was hoping someone had encountered this when running with uwsgi. Google was no help.
Derek Lambert
On 2/19/19, 5:21 PM, "Abhilash Raj" <maxking(a)asynchronous.in> wrote:
On Tue, Feb 19, 2019, at 7:32 AM, Derek Lambert wrote:
> I’m having an issue downloading attachments from HyperKitty. The
> download begins as expected, but never progresses. Eventually it times
> out. I imagine the browser is expecting data matching the
> content-length header, but never receives it.
>
> Inspecting the contents of the download file reveals (address varies):
>
> <memory at 0x7f148fcb9588>
>
> I’ve been able to work around the issue with the following patch:
>
> ---
> env/lib/python3.7/site-packages/hyperkitty/models/email.py.orig 2018-12-06 21:38:39.544448735 +0000
> +++
> env/lib/python3.7/site-packages/hyperkitty/models/email.py 2018-12-06
> 23:01:39.923239421 +0000
> @@ -309,7 +309,7 @@
> def get_content(self):
> folder = self._get_folder()
> if folder is None:
> - return self.content
> + return self.content.tobytes()
> filepath = os.path.join(folder, str(self.counter))
> if not os.path.exists(filepath):
> logger.error("Could not find local attachment %s for email
> %s",
>
> I haven't been able to replicate the issue with the HyperKitty tests,
> presumably the issue is somewhere in the Django <-> uwsgi <-> nginx
> stack.
>
> My configs are based on mailman-suite and maxking/docker-mailman. I've
> tried stripping out all the performance tweaks from my uwsgi and nginx
> configs, but no change.
>
> Versions:
> HyperKitty 1.2.1
> Postorius 1.2.3
> Django 2.1.7
> nginx 1.14.1
> uwsgi 2.0.18
> Python 3.7.2
>
> Has anyone come across this before?
I am not sure if this is exactly a bug in Hyperkitty. I am not able to replicate this with a extremely complicated test: Downloading a signature file in an email on lists.mailman3.org. It runs on gunicorn + Django 2.1.5 ( I think) and latest Git heads of respective projects.
Is this a recurring or one time thing that you observed?
It would be great if we could you could capture the HTTP Headers with and without the patch you sent above to help debug why does `.tobytes()` lets you download.
--
thanks,
Abhilash Raj (maxking)
7 years, 7 months
venv and upgrade from python 3.11 to 3.13 because of OS-upgrade
by Stephen J. Turnbull
Andreas Vetter writes:
> I have a venv installation (mailman postorius django-mailman3
> hyperkitty mailman-web) with python 3.11 from the Linux
> distribution. Since distribution EOL nears, I checked python
> version in next distribution and it is python
> 3.13.
>
> So my guess is, after distro upgrade, this will stop working.
>
> How should I upgrade?
I haven't done this, so take it with a grain of salt.
1. Stop mailman and mailman-web.
2. Upgrade the OS.
3. Check that the RDBMS is still available, using the command line
utility. In Debian I seem to recall that PostgreSQL tries to
automatically upgrade the database to the new version, you may not
need to do anything further.
- If not you may need to deal with the database upgrade if it is
a major version upgrade.
4. Check that the old venv is in fact broken (I think there's usually
versioned symlinks rather than an actual copy of the python install,
so it "should" break). If it's not, you're done.
5. You may be able to use "--upgrade":
/usr/bin/python3.13 -m venv --upgrade
However I seem to recall that --upgrade doesn't work across major
upgrades. If it does work, you're done.
6. If that fails, create a new venv in the same directory that holds
the existing venv. (I typically name my venvs like ".v3.13".)
* Reinstall mailman etc there:
python -m venv --upgrade-deps # ensure most recent pip etc
python -m pip install mailman mailman-web # or your list may be better
* Update paths in external configuration to point to new venv:
systemd (maybe /usr/local/lib/systemd/system/mailman*)
cron (probably /etc/cron.d/mailman*)
* Alternatively, you can delete the old venv and replace it with a
symlink to the new one. I don't think you can rename the new
venv to the old one's name because I think there are absolute
paths in the venv's internal configuration, but you could try,
and switch back to the symlink method if it doesn't work.
7. I suppose you will get upgrades of a number of packages. Django
or one of its dependencies may start issuing warnings about
deprecated settings for authentication, and you may have issues
with a missing nntp module (it was deprecated several versions ago
and finally removed from Python 3.13, the fix is to install the
nntp module from PyPI (it has "standard" or "stdlib" in the name).
Both of these issues have been discussed recently on this list.
Note that I *have* used alternate Python installs via the symlink
method and switched back and forth.
Hope this helps. Maybe there's a better way, but that's how I would
handle it.
--
GNU Mailman consultant (installation, migration, customization)
Sirius Open Source https://www.siriusopensource.com/
Software systems consulting in Europe, North America, and Japan
6 months, 1 week
Re: Reg Archive Inactive Status
by Nirmal J
Hereby I am sharing the full settings.py file.
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
import os
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'change-this-on-your-production-server'
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = False
ADMINS = (
('Mailman Suite Admin', 'root@localhost'),
)
SITE_ID = 1
# Hosts/domain names that are valid for this site; required if DEBUG is False
# See https://docs.djangoproject.com/en/1.8/ref/settings/#allowed-hosts
ALLOWED_HOSTS = [
"localhost", # Archiving API from Mailman, keep it.
# "lists.your-domain.org",
# Add here all production URLs you may have.
]
# Mailman API credentials
MAILMAN_REST_API_URL = 'http://localhost:8001'
MAILMAN_REST_API_USER = 'restadmin'
MAILMAN_REST_API_PASS = 'restpass'
MAILMAN_ARCHIVER_KEY = 'SecretArchiverAPIKey'
MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1')# Application definition
INSTALLED_APPS = (
'hyperkitty',
'postorius',
'django_mailman3',
# Uncomment the next line to enable the admin:
'django.contrib.admin',
# Uncomment the next line to enable admin documentation:
# 'django.contrib.admindocs',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount',
'django_mailman3.lib.auth.fedora',
# 'allauth.socialaccount.providers.openid',
# 'allauth.socialaccount.providers.github',
# 'allauth.socialaccount.providers.gitlab',
# 'allauth.socialaccount.providers.google',
# 'allauth.socialaccount.providers.facebook',
# 'allauth.socialaccount.providers.twitter',
# 'allauth.socialaccount.providers.stackexchange',
)
MIDDLEWARE = (
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware',
)
ROOT_URLCONF = 'urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.i18n',
'django.template.context_processors.media',
'django.template.context_processors.static',
'django.template.context_processors.tz',
'django.template.context_processors.csrf',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
'django_mailman3.context_processors.common',
'hyperkitty.context_processors.common',
'postorius.context_processors.postorius',
],
},
},
]
WSGI_APPLICATION = 'wsgi.application'
# Database
# https://docs.djangoproject.com/en/1.8/ref/settings/#databases
DATABASES = {
'default': {
# Use 'sqlite3', 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
'ENGINE': 'django.db.backends.postgresql_psycopg2',
# DB name or path to database file if using sqlite3.
'NAME': 'mailman3web',
# The following settings are not used with sqlite3:
'USER': 'mailmansuite',
'PASSWORD': 'mmpass',
# HOST: empty for localhost through domain sockets or '127.0.0.1' for
# localhost through TCP.
'HOST': '',
# PORT: set to empty string for default.
'PORT': '',
# OPTIONS: for mysql engine only, do not use with other engines.
# 'OPTIONS': {'charset': 'utf8mb4'} # Enable utf8 4-byte encodings.
}
# Example for PostgreSQL (recommanded for production):
#'default': {
# 'ENGINE': 'django.db.backends.postgresql_psycopg2',
# 'NAME': 'database_name',
# 'USER': 'database_user',
# 'PASSWORD': 'database_password',
# 'HOST': 'localhost',
#}
}
# If you're behind a proxy, use the X-Forwarded-Host header
# See https://docs.djangoproject.com/en/1.8/ref/settings/#use-x-forwarded-host
USE_X_FORWARDED_HOST = True
# And if your proxy does your SSL encoding for you, set SECURE_PROXY_SSL_HEADER
# https://docs.djangoproject.com/en/1.8/ref/settings/#secure-proxy-ssl-header
# SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
# SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_SCHEME', 'https')
# Other security settings
# SECURE_SSL_REDIRECT = True
# If you set SECURE_SSL_REDIRECT to True, make sure the SECURE_REDIRECT_EXEMPT
# contains at least this line:
# SECURE_REDIRECT_EXEMPT = [
# "archives/api/mailman/.*", # Request from Mailman.
# ]# SESSION_COOKIE_SECURE = True
# SECURE_CONTENT_TYPE_NOSNIFF = True
# SECURE_BROWSER_XSS_FILTER = True
# CSRF_COOKIE_SECURE = True
# CSRF_COOKIE_HTTPONLY = True
# X_FRAME_OPTIONS = 'DENY'
# Password validation
# https://docs.djangoproject.com/en/1.9/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
{
'NAME':
'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME':
'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME':
'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME':
'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]# Internationalization
# https://docs.djangoproject.com/en/1.8/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.8/howto/static-files/
# Absolute path to the directory static files should be collected to.
# Don't put anything in this directory yourself; store your static files
# in apps' "static/" subdirectories and in STATICFILES_DIRS.
# Example: "/var/www/example.com/static/"
STATIC_ROOT = '/var/lib/mailman3/web/static'
# URL prefix for static files.
# Example: "http://example.com/static/", "http://static.example.com/"
STATIC_URL = '/mailman3/static/'
# Additional locations of static files
STATICFILES_DIRS = (
# Put strings here, like "/home/html/static" or "C:/www/django/static".
# Always use forward slashes, even on Windows.
# Don't forget to use absolute paths, not relative paths.
# BASE_DIR + '/static/',
)
# List of finder classes that know how to find static files in
# various locations.STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
# 'django.contrib.staticfiles.finders.DefaultStorageFinder',
'compressor.finders.CompressorFinder',
)
# Django 1.6+ defaults to a JSON serializer, but it won't work with
# django-openid, see
# https://bugs.launchpad.net/django-openid-auth/+bug/1252826
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer'
LOGIN_URL = 'account_login'
LOGIN_REDIRECT_URL = 'list_index'
LOGOUT_URL = 'account_logout'
HOSTNAME = 'localhost.local'
# If you enable internal authentication, this is the address that the emails
# will appear to be coming from. Make sure you set a valid domain name,
# otherwise the emails may get rejected.
# https://docs.djangoproject.com/en/1.8/ref/settings/#default-from-email
# DEFAULT_FROM_EMAIL = "mailing-lists(a)you-domain.org"
DEFAULT_FROM_EMAIL = 'postorius@{}'.format(HOSTNAME)
# If you enable email reporting for error messages, this is where those emails
# will appear to be coming from. Make sure you set a valid domain name,
# otherwise the emails may get rejected.
# https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SERVER_EMAIL
# SERVER_EMAIL = 'root(a)your-domain.org'
SERVER_EMAIL = 'root@{}'.format(HOSTNAME)
# Change this when you have a real email backend
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
# Compatibility with Bootstrap 3
from django.contrib.messages import constants as messages # flake8: noqa
MESSAGE_TAGS = {
messages.ERROR: 'danger'
}
#
# Social auth
#
AUTHENTICATION_BACKENDS = (
'django.contrib.auth.backends.ModelBackend',
'allauth.account.auth_backends.AuthenticationBackend',
)
# Django Allauth
ACCOUNT_AUTHENTICATION_METHOD = "username_email"
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = "mandatory"
# You probably want https in production, but this is a dev setup file
ACCOUNT_DEFAULT_HTTP_PROTOCOL = "https"
ACCOUNT_UNIQUE_EMAIL = True
SOCIALACCOUNT_PROVIDERS = {
#'openid': {
# 'SERVERS': [
# dict(id='yahoo',
# name='Yahoo',
# openid_url='http://me.yahoo.com'),
# ],
#},
#'google': {
# 'SCOPE': ['profile', 'email'],
# 'AUTH_PARAMS': {'access_type': 'online'},
#},
#'facebook': {
# 'METHOD': 'oauth2',
# 'SCOPE': ['email'],
# 'FIELDS': [
# 'email',
# 'name',
# 'first_name',
# 'last_name',
# 'locale',
# 'timezone',
# ],
# 'VERSION': 'v2.4',
#},
}
#
# Gravatar
# https://github.com/twaddington/django-gravatar
#
# Gravatar base url.
# GRAVATAR_URL = 'http://cdn.libravatar.org/'
# Gravatar base secure https url.
# GRAVATAR_SECURE_URL = 'https://seccdn.libravatar.org/'
# Gravatar size in pixels.
# GRAVATAR_DEFAULT_SIZE = '80'
# An image url or one of the following: 'mm', 'identicon', 'monsterid',
# 'wavatar', 'retro'.
# GRAVATAR_DEFAULT_IMAGE = 'mm'
# One of the following: 'g', 'pg', 'r', 'x'.
# GRAVATAR_DEFAULT_RATING = 'g'
# True to use https by default, False for plain http.
# GRAVATAR_DEFAULT_SECURE = True
#
# django-compressor
# https://pypi.python.org/pypi/django_compressor
#
#COMPRESS_PRECOMPILERS = (
# ('text/less', 'lessc {infile} {outfile}'),
# ('text/x-scss', 'sassc -t compressed {infile} {outfile}'),
# ('text/x-sass', 'sassc -t compressed {infile} {outfile}'),
#)
# On a production setup, setting COMPRESS_OFFLINE to True will bring a
# significant performance improvement, as CSS files will not need to be
# recompiled on each requests. It means running an additional "compress"
# management command after each code upgrade.
# http://django-compressor.readthedocs.io/en/latest/usage/#offline-compression
# COMPRESS_OFFLINE = True
# Needed for debug mode
# INTERNAL_IPS = ('127.0.0.1',)
#
# Full-text search engine
#HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine',
'PATH': '/var/lib/mailman3/web/fulltext_index',
# You can also use the Xapian engine, it's faster and more accurate,
# but requires another library.
# http://django-haystack.readthedocs.io/en/v2.4.1/installing_search_engines.h…
# Example configuration for Xapian:
#'ENGINE': 'xapian_backend.XapianEngine'
},
}
#
# Asynchronous tasks
#
Q_CLUSTER = {
'timeout': 300,
'save_limit': 100,
'orm': 'default',
'poll': 5,
}# A sample logging configuration. The only tangible logging
# performed by this configuration is to send an email to
# the site admins on every HTTP 500 error when DEBUG=False.
# See http://docs.djangoproject.com/en/dev/topics/logging for
# more details on how to customize your logging configuration.
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
},
'file':{
'level': 'INFO',
'class': 'logging.handlers.RotatingFileHandler',
#'class': 'logging.handlers.WatchedFileHandler',
'filename': '/var/log/mailman3/web/mailman-web.log',
'formatter': 'verbose',
},
'console': {
'class': 'logging.StreamHandler',
'formatter': 'simple',
},
},
'loggers': {
'django.request': {
'handlers': ['mail_admins', 'file'],
'level': 'INFO',
'propagate': True,
},
'django': {
'handlers': ['file'],
'level': 'INFO',
'propagate': True, },
'hyperkitty': {
'handlers': ['file'],
'level': 'INFO',
'propagate': True,
},
'postorius': {
'handlers': ['file'],
'level': 'INFO',
'propagate': True,
},
},
'formatters': {
'verbose': {
'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'
},
'simple': {
'format': '%(levelname)s %(message)s'
},
},
#'root': {
# 'handlers': ['file'],
# 'level': 'INFO',
#},
}
if DEBUG == True:
EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend'
EMAIL_FILE_PATH = os.path.join(BASE_DIR, 'emails')
#
# HyperKitty-specific
#
# Only display mailing-lists from the same virtual host as the webserver
FILTER_VHOST = False
POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost:8000'
try:
from settings_local import *
except ImportError:
pass
-----Original Message-----
From: Stephen <steve(a)turnbull.jp>
To: nirmal <nirmal(a)iitm.ac.in>
Cc: mailman-users <mailman-users(a)mailman3.org>
Date: Friday, 26 September 2025 1:15 PM IST
Subject: [MM3-users] Reg Archive Inactive Status
nirmal--- via Mailman-users writes:
> > Sep 24 10:45:43 2025 (187698) HyperKitty failure on
> > http://list1.iitm.ac.in/hyperkitty/api/mailman/urls:
> > <html><title>Forbidden</title><body>
> > <h1>Access is forbidden</h1><p>Please check the IP addresses
> > assigned to MAILMAN_ARCHIVER_FROM in the settings file.
> > </p></body></html>
> > (403)
> You need to ensure the IP address of list1.iitm.ac.in is in the
> MAILMAN_ARCHIVER_FROM list in your Django settings.
>
> I have also ensured that the IP Address of list1.iitm.ac.in is in
> the Mailman_Archiver_From List in settings. But still not
> working. Please help.
Please be more specific than "not working", as much as possible. In
this case, copy the setting from your settings.py file and paste it in
the email. Always describe the error message. If it's identical to
the previous one except for the timestamp and server process, just
saying "error message is the same" is OK. If it's not, any small
difference could be a clue, and you should copy and paste it.
Check the syntax of the assignment to MAILMAN_ARCHIVER_FROM in
settings.py. The variable name must be all uppercase. The assignment
must use '-', and the value should be a Python sequence[1] of strings,
like this:
MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1')
Even if there is only one IP, I suspect the parentheses are required,
and the single entry *must* be followed by a comma, like this:
MAILMAN_ARCHIVER_FROM = ('127.0.0.1',)
If your setting is correct, check your webserver's (Apache, nginx,
etc) configuration for access restrictions. Many Mailman installation
guides assume localhost and provide examples restricting access to
localhost.
Finally, if no configured access restriction applies, there should be
a corresponding entry in the webserver's access log. Check that the
source IP is what you expect.
Footnotes:
[1] If you are an experienced Python programmer, I believe you can
use any sequence syntax you are comfortable with. Tuples will
definitely work because that's the test configuration we use. :-)
--
GNU Mailman consultant (installation, migration, customization)
Sirius Open Source https://www.siriusopensource.com/
Software systems consulting in Europe, North America, and Japan
11 months, 4 weeks