Re: Upgrade path
by David Krantz
Hi.
Getting back to this...
Some log of what I'm doing.
* First git fetch; git pull on all parts.
Started new envs to simplify rollback etc.
* activate the new 3.5 venv.
( I run the commands one at a time, but this is more compact in text form:)
> cd git/mailman; python setup.py build; python setup.py install
> cd ../mailman-hyperkitty; python setup.py build; python setup.py install
> deactivate
activate 2.7 venv, then
> cd ../mailmanclient; python setup.py build; python setup.py install
> cd ../django-mailman3; python setup.py build; python setup.py install
> cd ../postorius; python setup.py build; python setup.py install
> cd ../hyperkitty; python setup.py build; python setup.py install
So far no errors. Just warnings.
Backups:
* deployment directory (as user with rw in directory)
> tar -cvzf backup.tgz deployment_directory
* postgres databases (as user with postgres-priv)
> pg_dump maiman | gzip -9 > mailman.bak.gz
> pg_dump mailmanweb | gzip -9 > mailmanweb.bak.gz
Create new .wsgi-file based on mailman-suite (I used the kludge in the
script to get the python path correct)
Change apache2.conf to point to new venv and new wsgi-version
Updated the mailman3wrapper.sh I use for init to start the service.
I could not get hyperkitty interface to run so I deactivated it and
set mailman as index instead. It complains that a column is missing,
but I cannot get the migrate script to run so I decided that I do not
need the archiver as I currently don't use it. So I will have to get
back to this and either remove the archiver completely or get it to
run.
So the line with ^$ in urls.py now is
url(r'^$', RedirectView.as_view(url=reverse_lazy('list_index'))),
instead, and the line "hyperkitty," is commented in the django application list.
cheers.
// David
On Tue, Aug 8, 2017 at 6:48 PM, Mark Sapiro <mark(a)msapiro.net> wrote:
> On 08/08/2017 04:42 AM, David Krantz wrote:
>>
>> What is the rekommended way to upgrade to the newest version? (newest
>> _usable_ version, of course)? Is it docker-only with a migration from
>> the current solution to a docker based? If so, any ideas or pointers
>> on how to go about this?
>
>
> You may find this wiki article helpful <https://wiki.list.org/x/17891998>.
>
> I started with bundler on both this (lists.mailman3.org) and the
> mail.python.org installs, but have been updating from the heads of the
> various gitlab branches since very early on.
>
> At the moment, these installations are somewhat ahead of the 3.1
> release, but I am not updating as the latest core has dropped Python 3.4
> support, and I haven't yet installed Python 3.5 or 3.6 on these (Ubuntu
> 14.04) servers. (I will probably install 3.6 from source at some point.)
>
> Please feel free to ask here if you have more specific questions.
>
> --
> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
> San Francisco Bay Area, California better use your sense - B. Dylan
> _______________________________________________
> Mailman-users mailing list
> mailman-users(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
8 years, 2 months
mailman-web migrate error
by pashdown＠xmission.com
I'm migrating from Ubuntu 20.04, mailman 3.2.2, postgresql 12 to Debian 13, mailman 3.3.10, postgresql 17.
After migrating the postgresql databases and assuring the perms and passwords work, this is what I get with a "mailman-web migrate":
(user(a)domain.com) redacted from the original
# mailman-web migrate
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, \
sessions, sites, socialaccount
Running migrations:
Applying account.0003_alter_emailaddress_create_unique_verified_email... OK
Applying account.0004_alter_emailaddress_drop_unique_email... OK
Applying account.0005_emailaddress_idx_upper_email... OK
Applying account.0006_emailaddress_lower...Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "account_emailaddress_user_id_\
email_987c8728_uniq"
DETAIL: Key (user_id, email)=(24, user(a)domain.com) already exists.
The above exception was the direct cause of the following exception:
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/share/mailman3-web/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
utility.execute()
~~~~~~~~~~~~~~~^^
File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 412, in run_from_argv
self.execute(*args, **cmd_options)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 458, in execute
output = self.handle(*args, **options)
File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 106, in wrapper
res = handle_func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/django/core/management/commands/migrate.py", line 356, in handle
post_migrate_state = executor.migrate(
targets,
...<3 lines>...
fake_initial=fake_initial,
)
File "/usr/lib/python3/dist-packages/django/db/migrations/executor.py", line 135, in migrate
state = self._migrate_all_forwards(
state, plan, full_plan, fake=fake, fake_initial=fake_initial
)
File "/usr/lib/python3/dist-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
state = self.apply_migration(
state, migration, fake=fake, fake_initial=fake_initial
)
File "/usr/lib/python3/dist-packages/django/db/migrations/executor.py", line 252, in apply_migration
state = migration.apply(state, schema_editor)
File "/usr/lib/python3/dist-packages/django/db/migrations/migration.py", line 132, in apply
operation.database_forwards(
~~~~~~~~~~~~~~~~~~~~~~~~~~~^
self.app_label, schema_editor, old_state, project_state
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/lib/python3/dist-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
self.code(from_state.apps, schema_editor)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/allauth/account/migrations/0006_emailaddress_lower.py", line 11, in forwards
EmailAddress.objects.all().exclude(email=Lower("email")).update(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
email=Lower("email")
^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1206, in update
rows = query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1984, in execute_sql
cursor = super().execute_sql(result_type)
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
cursor.execute(sql, params)
~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
~~~~~~~~~~~~~~~~~~~~~~~~~~~^
sql, params, many=False, executor=self._execute
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute
with self.db.wrap_database_errors:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "account_emailaddress_user_id_email_987c8728_uniq"
DETAIL: Key (user_id, email)=(24, user(a)domain.com) already exists.
2 weeks, 2 days
Re: Held messages not delivered after approval
by Krinetzki, Stephan
Hi Mark,
>That's a misleading message. It's been improved by https://gitlab.com/mailman/mailman/-/merge_requests/1362, however usually the underlying issue is inability to access a header or footer template when decorating the message. This in turn is most likely >caused by some template(s) in Mailman's var/templates/ directory being owned by `root` or some other non-mailman user and not readable by the Mailman user which in turn most likely results from running `mailman import21` as root.
You are my hero! After "patching" the outgoing.py file with the error messages I got
Aug 06 09:52:51 2025 (443278) Cannot connect to SMTP server 127.0.0.1 on port 25: HTTPSConnectionPool(host='lists.example.com', port=443): Max retries exceeded with url: /postorius/api/templates/list/hpc-admintreffen-info.lists.example.com/list:member:regular:footer (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)')))
And was shocked. Why is there an SSL Certificate Error? Why can't python requests verify the certificate?
So I googled a bit and found, that we can fix this be editing the cacert.pem of certifi. I did, but the error was still there. So I checked my https VHost (Apache btw) and there was a wrong chain in it. Oof. An error, that a browser does not see, but any ssl client. I fixed it and boom - everything works again.
Thank you again and thank you Stephen for trying to resolve the problem 😊
--
Stephan Krinetzki
IT Center
Gruppe: Anwendungsbetrieb und Cloud
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24866
Fax: +49 241 80-22134
krinetzki(a)itc.rwth-aachen.de
www.itc.rwth-aachen.de
Social Media Kanäle des IT Centers:
https://blog.rwth-aachen.de/itc/
https://www.facebook.com/itcenterrwth
https://www.linkedin.com/company/itcenterrwth
https://twitter.com/ITCenterRWTH
https://www.youtube.com/channel/UCKKDJJukeRwO0LP-ac8x8rQ
-----Original Message-----
From: Mark Sapiro <mark(a)msapiro.net>
Sent: Tuesday, August 5, 2025 6:19 PM
To: mailman-users(a)mailman3.org
Subject: [MM3-users] Re: Held messages not delivered after approval
On 8/5/25 02:06, Krinetzki, Stephan wrote:
> Aug 05 10:21:05 2025 (217986) Cannot connect to SMTP server localhost
> on port 25
>
> The last error is a very often in the mailman.log:
That's a misleading message. It's been improved by https://gitlab.com/mailman/mailman/-/merge_requests/1362, however usually the underlying issue is inability to access a header or footer template when decorating the message. This in turn is most likely caused by some template(s) in Mailman's var/templates/ directory being owned by `root` or some other non-mailman user and not readable by the Mailman user which in turn most likely results from running `mailman import21` as root.
Bottom line - check that all sub-directories and files in Mailman's var/templates/ directory are searchable and readable by the Mailman user.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
_______________________________________________
Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
This message sent to krinetzki(a)itc.rwth-aachen.de
3 months, 3 weeks
Re: Mailman, etc. upgrade woes and persistent bugs
by Abhilash Raj
On Wed, Feb 17, 2021, at 10:24 PM, Stephen J. Turnbull wrote:
> Abhilash Raj writes:
>
> > So it seems like we have some sort of agreement that we need all the
> > roles to be able to see all the 4 states.
>
> I think it's more like there's disagreement about which roles need to
> see which states, but for each role and each state, somebody thinks
> that role needs to see that state. I think there's agreement that
> it's harmless to allow everybody to see all states.
>
> > Roles here are basically Admin( i.e. list owner/moderator/site
> > admin) and Subscriber.
>
> In this discussion, yes. However, if we're going to take the
> discussion seriously, we should be more general than just this case.
> I think we need to consider five roles. Starting with Brian's list of
> List Owner, Moderator, List Member, and adding Anonymous (so that a
> pre-login connection has a named role) and Site Owner for completeness.
Yep, those are all the roles that even we recognize in Postorius. We also
have the concept of Domain Owner, though that isn't really implemented yet.
So, from this list, List Owner and above all will use the "by_admin" state
to disable if they need to and they are the only ones who can set that state.
>
> > There is no access control to transition "from" any state but each
> > value of non-enabled states is sort of a role's way to disable the
> > delivery. Not all states are settable by all roles.
>
> Off-list, Ruth Ivey-Cook informs me that in "who sets what" both "who"
> and "what" need to be considered is what she had in mind. So I think
> that's exactly right and there's agreement.
Nice.
>
> > 1. A subscriber can only transition from any to "enabled" or "by_user"
> > state but can see if delivery was disabled "by_admin" or "by_bounces"
> > as separate state in the Options page.
> > 2. An admin can transition to "enabled", "by_user" and "by_moderator".
>
> I assume you mean "by_admin" instead of "by_moderator"?
Ok, I've made a mess of it in my responses, the Core actually calls this value
"by_moderator" but I have used them interchangeably in my emails. Sorry
about that.
https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/interfaces/mem…
>
> > They should use "by_user" when acting on behalf of the user and other
> > when there are any administrative reason to disable delivery, whatever
> > that may be. I expect "by_admin" to be used sparingly, and I don't see
> > any use cases for it.
> > 3. Mailman itself will use the "by_bounces" to disable delivery due to
> > bounces.
> >
> > Is that an accurate description?
>
> +1 for 1 -- 3 as written. Others' mileage may vary.
Cool, thanks!
>
> Steve
>
>
--
thanks,
Abhilash Raj (maxking)
4 years, 9 months
Re: Post-installation documentation?
by David Newman
On 11/25/21 1:16 PM, Mark Sapiro wrote:
> On 11/25/21 10:56 AM, David Newman wrote:
>>
>> Thanks for this. After setting these variables and restarting mailman3
>> and mailmanweb, the mailman user still cannot find a hyperkitty_import
>> command (neither can root, btw). Again, I installed MM3 and
>> Django/Postorius/Hyperkitty using the venv and web docs previously
>> referenced.
>
>
> Don't run mailman or django admin commands as root. This causes issues
> when files are created by root and can't be read by mailman.
Understood, thanks. I'd only tried root to determine if there was a
permissions issue, where hyperkitty_import really did exist but wasn't
available to the mailman user.
>
> What django admin command are you running?
As the mailman user, with venv activated:
(venv) mailman@lists:~$ hyperkitty_import -l testlist(a)lists.domain.com
/var/lib/mailman/archives/private/testlist.mbox/testlist.mbox
'testlist' and 'lists.domain.com' are redactions; they are correct in
the given command, but the command fails because the system can't find
hyperkitty_import.
(BTW
Also, because I'm looking to migrate from Mailman 2.1 installed from
Debian packages, all the 2.1 stuff is owned by the 'list' user. I'd
changed ownership of the above private archive to mailman:mailman for
the command not to complain about permissions.
> If you don't set
> DJANGO_SETTINGS_MODULE in the environment and you have activated your
> virtualenv and run `mailman-web` as the django admin command it will set
> /etc/mailman3/settings.py as the settings.
I'd run these two environment commands as the mailman user with
virtualenv activated:
export DJANGO_SETTINGS_MODULE=settings
export PYTHONPATH=/etc/mailman3/settings.py
Am I being overly literal here with 'settings' in that first command?
Asking because 'mailman-web check' runs clean before running these
commands and returns this error afterward:
ModuleNotFoundError: No module named 'settings'
Either way, the hyperkitty_import command still isn't found.
>> On the plus side, all lists now appear in the admin web page. However,
>> lists created from the admin page do not appear in a site list page,
>> even though all lists are in the same site/domain.
>
>
> By default, only lists that you are owner, moderator or subscriber for
> are shown. What if you click Filter by Role and select All?
Same result. In this case, the ID of the admin and list owner are the same.
>
>
>> Finally, how to make these exports permanent? I did not see this
>> covered in the migration doc above. This is on a Debian system where
>> the mailman user's shell is bash. Is this simply a matter of adding
>> environment variables in the mailman user's .bashrc file?
>
> You can do that but as I note above, if you run `mailman-web` you
> shouldn't need to. You can add a `source /opt/mailman/venv/bin/activate`
> command as mentioned at
> https://docs.mailman3.org/en/latest/install/virtualenv.html#activate-virtua….
Yes, this is what I have in mailman's .bashrc file. I'd followed that
venv doc for initial installation, and it activates the virtualenv
whenever launching a shell for mailman.
dn
4 years
Re: KeyError: 'subject_prefix'
by Abhilash Raj
On Sun, Nov 15, 2020, at 1:20 AM, Torge Riedel wrote:
> Hi,
>
> today I had the following error in my inbox:
>
> KeyError: 'subject_prefix'
> AttributeError: 'MailingList' object has no attribute 'subject_prefix'
>
> I checked the database, there is a column "subject_prefix", so I feel
> it is something in the code causing the error.
>
> It's mailman 3.3.1 and I don't see a fix for such an error in the
> Changes of 3.3.2
>
Is the value of subject prefix for that list set to *something* in Mailman? You can check via API or if you have Postorius, you can go to List's settings page and under the "List Identity" tab there is a field.
I am suspecting Hyperkitty wants to read this attribute but since it is not set in database, Core is skipping the field in API response for /list/<listid>/config which typically returns all the list settings.
Abhilash
> Kind regards
> Torge
>
> Traceback (most recent call last):
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 119, in __getattr__
> return self._get(name)
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 86, in _get
> raise KeyError(key)
> KeyError: 'subject_prefix'
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/hyperkitty/models/mailinglist.py", line 197, in update_from_mailman
> value = getattr(mm_list, propname)
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 124, in __getattr__
> self.__class__.__name__, name))
> AttributeError: 'MailingList' object has no attribute 'subject_prefix'
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/django_extensions/management/commands/runjobs.py", line 36, in runjobs
> job().execute()
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/hyperkitty/jobs/sync_mailman.py", line 36, in execute
> sync_with_mailman()
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/hyperkitty/lib/mailman.py", line 132, in sync_with_mailman
> mlist.update_from_mailman()
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/hyperkitty/models/mailinglist.py", line 199, in update_from_mailman
> value = mm_list.settings[propname]
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 146, in __getitem__
> return self._get(key)
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 88, in _get
> return self.rest_data[key]
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 74, in rest_data
> response, content = self._connection.call(self._url)
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 112, in call
> error_msg, response, None)
> urllib.error.HTTPError: HTTP Error 500: <html>
> <head>
> <title>Internal Server Error</title>
> </head>
> <body>
> <h1><p>Internal Server Error</p></h1>
>
> </body>
> </html>
>
> ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty)
> START TRACEBACK:
> END TRACEBACK
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>
--
thanks,
Abhilash Raj (maxking)
5 years
Re: Question about adding addresses using the REST API
by Stephen Daniel
TL;DR: Listening to the advice, I will create and manage user/address pairs.
I am in the process of porting a web site that is used to manage a small
organization. The web site maintains a directory of users of that
organization, and also allows them to subscribe/unsubscribe to a small set
of interest-group mailing lists for that organization. The users are not
tech savvy, and are not expecting or expected to manage their subscriptions
through Postorius, nor do they have access (normally) to the archives.
Indeed, the URL to Postorious will normally be turned off at the reverse
proxy server that sits in front of both Postorious and the organization's
website.
The one-source-of-truth is the website's database. The website code has
callouts for "subscribe <address> to <list>" and "unsubscribe <address>
from <list>".
I am trying to minimize the amount of code I write to implement the web
site's callouts, to initialize the mailing lists when I port the
organization to this new implementation, and make it relatively easy to
keep the website's database and mailman3's database in sync.
The impression I had from the docs was that unlinked addresses were a first
class object. Since my website callouts only deal with addresses, this
seemed a good mapping. However, I understand from this email thread that
unlinked addresses are not the way to go, so I'll map <address> in the
website database to <user>/<address> in mailman3.
Thanks for the advice and feedback!
-- Stephen Daniel
On Sat, Jan 15, 2022 at 2:27 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 1/15/22 10:03 AM, Stephen Daniel wrote:
> >
> > The only way I've found to create a standalone address (not linked to any
> > user) is to create a user with that address, unlink the address, then
> > delete the user.
> >
> > Is there a more straightforward way to just create an address with no
> user?
>
>
> The more straightforward way is via the IUserManager create_address
> method. This method is not exposed directly in REST, so you can't do it
> via REST.
>
> My question is why do you want to?
>
> Granted when an unknown nonmember address posts to a list we create a
> standalone address record, but maybe we should be creating a user too.
> We don't because it's tricky. Suppose User A with address A posts to a
> list from nonmember address B. We create the standalone address B record
> as a nonmember, but now user A can add address B as another address. If
> we also created User B, this would not be possible.
>
> But again, what is your use case for creating standalone addresses?
>
>
> --
> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
> San Francisco Bay Area, California better use your sense - B. Dylan
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>
3 years, 10 months
Innstall and run Mailman 3 with Postfix in a Docker enviroment
by kwp.klein＠gmail.com
## Install Mailman 3
The idea is to install Mailman 3 in ad docker container. Also I want to install
Postfix in a container. All the mail which will send and receive should handled
in a docker container. The reason is, that on the server only applications in a
docker environment working. There is no MTA installed. On the server are
Discourse as forum, wikijs, portainer, watchtower, homepage, and the NGINX Proxy
Manager installed. At next I need Mailman 3.
I don't know how to install and run Postfix for Mailman in a docker environment.
Here is my first try. I can visit the site lists.example.com, but I can not set
the Admin password. I think the reason is, that the MTA not working.
I have a gitlab repo. If you like please look here:
https://gitlab.com/joklein/mailman
### Configure the docker-compose.yml file
```yml
version: '2'
services:
postfix:
image: mailu/postfix
restart: unless-stopped
container_name: postfix
volumes:
- /opt/mailman/postfix:/etc/postfix/main.cf.d
- /opt/mailman/core:/opt/mailman
ports:
- "25:25"
networks:
mailman:
ipv4_address: 172.19.199.6
services:
mailman-core:
image: maxking/mailman-core:0.3
restart: unless-stopped
stop_grace_period: 30s
container_name: mailman-core
hostname: mailman-core
volumes:
- /opt/mailman/core:/opt/mailman
links:
- database:database
depends_on:
- database
environment:
- DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb
- DATABASE_TYPE=postgres
- DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase
- SMTP_HOST=172.19.199.1
- HYPERKITTY_API_KEY=ZawB3NGBUgbmQcMGhZ0kYIW0WRNw8P9osEudsGiFZWv
networks:
mailman:
ipv4_address: 172.19.199.2
mailman-web:
image: maxking/mailman-web:0.3
container_name: mailman-web
restart: unless-stopped
hostname: mailman-web
ports:
- "8000:8000"
depends_on:
- database
links:
- mailman-core:mailman-core
- database:database
volumes:
- /opt/mailman/web:/opt/mailman-web-data
environment:
- DATABASE_TYPE=postgres
- DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb
- HYPERKITTY_API_KEY=ZawB3NGBUgbmQcMGhZ0kYIW0WRNw8P9osEudsGiFZWv
- UWSGI_STATIC_MAP=/static=/opt/mailman-web-data/static
- SERVE_FROM_DOMAIN=lists.example.com
- MAILMAN_ADMIN_USER=admin
- MAILMAN_ADMIN_EMAIL=lists(a)example.com
- SECRET_KEY=mailmangeheim
networks:
mailman:
ipv4_address: 172.19.199.3
database:
image: postgres:9.6-alpine
restart: unless-stopped
environment:
POSTGRES_DB: mailmandb
POSTGRES_USER: mailman
POSTGRES_PASSWORD: mailmanpass
volumes:
- /opt/mailman/database:/var/lib/postgresql/data
networks:
mailman:
ipv4_address: 172.19.199.4
networks:
mailman:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.19.199.0/24
```
#### Build the API Key für Hyperkitty and SECRET_KEY
You can build a key with `pwgen -s 128`
#### Make some directorys
```sh
mkdir -p /opt/mailman/core
mkdir -p /opt/mailman/web
mkdir -p /opt/mailman/mailman-web-data/static
```
#### Make a file in /opt/mailman/web/settings_local.py
```sh
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = '172.19.199.1'
EMAIL_PORT = 25
USE_SSL = False
DEFAULT_FROM_EMAIL = "<E-MAIL>"
SERVER_EMAIL = "<E-MAIL>"
```
Change the `<E-MAIL>` with your server address. For example
`mailman(a)lists.example.com`
So that Postfix also sends your e-mails, you create the file
`/opt/mailman/core/mailman-extra.cfg` with the following content:
```sh
[mta]
incoming: mailman.mta.postfix.LMTP
outgoing: mailman.mta.deliver.deliver
lmtp_host: 172.19.199.2
lmtp_port: 8024
smtp_host: 172.19.199.1
smtp_port: 25
configuration: /etc/postfix-mailman.cfg
```
Now you are ready to start your container:
```sh
sudo su -
cd /root/mailman-docker
docker-compose up -d
```
#### Check the mailman images
```sh
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f6990a2f808f maxking/mailman-web:0.3 "docker-entrypoint.s…" 55 seconds ago Up 52 seconds 8000/tcp, 8080/tcp mailman-web
a7fa6334b2b8 maxking/mailman-core:0.3 "docker-entrypoint.s…" 58 seconds ago Up 54 seconds 8001/tcp, 8024/tcp mailman-core
e8812dc42663 postgres:9.6-alpine "docker-entrypoint.s…" About a minute ago Up 57 seconds 5432/tcp docker-mailman_database_1
```
#### Check the conection
```sh
curl http://172.19.199.3:8000/postorius/lists/
```
4 years, 9 months
Re: Hyperkitty "All Threads" results in error
by Mark Dadgar
Aaaand I spoke too soon. I am now seeing the same error when I click the All Threads button in hyperkitty. Not every time, because that would be too easy, but often.
I really only have one high-volume list with archives, so I can’t tell if it’s list-specific.
I ran the mailman-post-update script after the last update, which I believe executes all the commands suggested below.
Here’s the crash trace.
Any thoughts?
- Mark
Internal Server Error: /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
VariableDoesNotExist at /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
Failed lookup for key [sender] in None
Request Method: GET
Request URL: https://mail.pdc-racing.net/archives/list/trackjunkies@pdc-racing.net/latest <https://mail.pdc-racing.net/archives/list/trackjunkies@pdc-racing.net/latest>
Django Version: 3.0.12
Python Executable: /opt/mailman/mm/venv/bin/python
Python Version: 3.8.5
Python Path: ['/opt/mailman/mm', '/opt/mailman/mm/', '/opt/mailman/mm/bin', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/opt/mailman/mm/venv/lib/python3.8/site-packages']
Server time: Mon, 1 Mar 2021 23:25:02 -0800
Installed Applications:
('hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Traceback (most recent call last):
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 828, in _resolve_lookup
current = current[bit]
During handling of the above exception ('NoneType' object is not subscriptable), another exception occurred:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 836, in _resolve_lookup
current = getattr(current, bit)
During handling of the above exception ('NoneType' object has no attribute 'sender'), another exception occurred:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 842, in _resolve_lookup
current = current[int(bit)]
During handling of the above exception (invalid literal for int() with base 10: 'sender'), another exception occurred:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/lib/view_helpers.py", line 134, in inner
return func(request, *args, **kwargs)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/views/mlist.py", line 115, in archives
return _thread_list(request, mlist, threads, extra_context=extra_context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/views/mlist.py", line 144, in _thread_list
return render(request, template_name, context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/shortcuts.py", line 19, in render
content = loader.render_to_string(template_name, context, request, using=using)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader.py", line 62, in render_to_string
return template.render(context, request)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/backends/django.py", line 61, in render
return self.template.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 171, in render
return self._render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py", line 150, in render
return compiled_parent._render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py", line 62, in render
result = block.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/defaulttags.py", line 209, in render
nodelist.append(node.render_annotated(context))
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py", line 188, in render
return template.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 173, in render
return self._render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/defaulttags.py", line 513, in render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/templatetags/i18n.py", line 132, in render
context.update({var: val.resolve(context) for var, val in self.extra_context.items()})
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/templatetags/i18n.py", line 132, in <dictcomp>
context.update({var: val.resolve(context) for var, val in self.extra_context.items()})
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 691, in resolve
arg_vals.append(arg.resolve(context))
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 795, in resolve
value = self._resolve_lookup(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 847, in _resolve_lookup
raise VariableDoesNotExist("Failed lookup for key "
Exception Type: VariableDoesNotExist at /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
Exception Value: Failed lookup for key [sender] in None
Request information:
USER: mark
GET: No GET data
POST: No POST data
FILES: No FILES data
On Feb 17, 2021, at 5:34 PM, Mark Dadgar <mark(a)pdc-racing.net> wrote:
>
> On Oct 14, 2020, at 4:37 PM, Mark Sapiro <mark(a)msapiro.net> wrote:
>> On 10/14/20 11:12 AM, Mark Dadgar wrote:
>>>> On Oct 4, 2020, at 10:34 AM, Mark Dadgar <mark(a)pdc-racing.net> wrote:
>>>>
>>>> So I have been wrestling with this problem for a while now. Most times, but not every time, when someone clicks on the All Threads link in hyperkitty, a 500 error is generated.
>>
>>
>> Does this happen on all lists, only some lists or only one list?
>>
>>
>>>> I googled it, obviously, and at first I thought it was related to this issue, but that has not proven to be the case:
>>>>
>>>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…
>>
>> What did you google? This query gets a lot of stuff with some hints, but
>> nothing definitive.
>>
>> django render raises VariableDoesNotExist("Failed lookup for key "
>>
>> One thing I noted at
>> <https://askbot.org/en/question/3095/django-error-variabledoesnotexist-faile…>
>> was
>>
>> models out of date - run python manage.py syncdb and python manage.py
>> migrate
>>
>> You could try that. Note that your django admin command might be
>> something other than `python manage.py`. Also note syncdb has been
>> removed from Django >=1.9. Currently, the above becomes `makemigrations`
>> followed by `migrate`.
>>
>> I also suggest running the django admin commands
>>
>> `collectstatic --clear --noinput`
>> and
>> `compress`
>
>
> My re-install seems to have fixed this issue as well. BONUS!
>
> - Mark
> -----
> mark(a)pdc-racing.net | 408-348-2878
>
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
- Mark
-----
mark(a)pdc-racing.net | 408-348-2878
4 years, 8 months
Re: how to integrate subscription into static jekyll website
by Stephen J. Turnbull
Hagen Bauer writes:
> Has anybody tried to hide the direct api access behind a nginx reverse
> proxy url?
Sounds straightforward.
I doubt anybody's done it, though, because you still face the problem
that anybody with that access is a site manager, with unclear limits
on their power outside of Mailman. (Shouldn't be much they can do
outside of Mailman, but I wouldn't bet my host on it.) In many cases,
the real site manager is also the host admin, so they have sufficient
access to a shell account. There's no need for her to have a proxy;
why make it available to people who are supposed to have less power?
Note I'm *not* saying this *never* makes sense. I'm just saying that
in a lot of cases it won't buy you very much because Mailman isn't set
up to provide granular authorization of REST endpoints.
> on the Webseite I would have a url like this
>
> mailmanserver.tld/subcsribe/user(a)mail.com
>
> And within the nginx proxy configuration on the server this would
> translate to
>
> localhost:80xx/"mailman-restapi-with-credentials"user(a)mail.com
First of all, the address to subscribe is a variable, so at a bare
minimum your nice static site has to provide a form, which isn't very
static. I guess you could put the URL to the nginx proxy as the
action URL in the form, but then the proxy needs to deal with a
form-encoded query string to translate to the appropriate REST
endpoint.
I suppose you could also have the user type the URL including the
email address into the browser address bar, but Abhilash's idea of
providing a mailto:subscribe@mailmanserver.tld URL seems more
convenient. The mailto URL also solves the problem of confirming
ownership of the mailbox, since that logic is built in to the
subscribe-by-mail code.
Second, if your proxy is configured very carefully, you might be able
to restrict this to only the exact operations you want to expose, here
subscription. However, although the email interface implements
verification by providing a one-time token that must be read from the
mailbox of the address (and so proves ownership of the mailbox), I
don't think the REST API implements operations like that. IIRC that
is implemented in Postorius. (Abhilash?) If my memory is correct,
you would also need to implement that, and that's definitely not static.
> So the public does not see the credentials. And if my nginx host
> config is hacked I have a problem anyway.
I can't speak for Abhilash, but what would worry me is not that nginx
is already hacked, but rather that if not properly configured, it
might be possible for someone to access REST endpoints you don't want
exposed through this interface, and maybe even to other parts of the
webserver. To give a simple example, consider a variant on the
ancient Apache traversal bug:
mailmanserver.tld/subscribe/../unsubscribe/user(a)mail.com
Probably nginx and/or Mailman and/or your proposed proxy config are
already capable of stopping this particular well-known attack, but
undoubtedly there are more sophisticated attacks that I don't know
about.
7 years, 1 month