Re: Upgrade path
by David Krantz
Hi.
Getting back to this...
Some log of what I'm doing.
* First git fetch; git pull on all parts.
Started new envs to simplify rollback etc.
* activate the new 3.5 venv.
( I run the commands one at a time, but this is more compact in text form:)
> cd git/mailman; python setup.py build; python setup.py install
> cd ../mailman-hyperkitty; python setup.py build; python setup.py install
> deactivate
activate 2.7 venv, then
> cd ../mailmanclient; python setup.py build; python setup.py install
> cd ../django-mailman3; python setup.py build; python setup.py install
> cd ../postorius; python setup.py build; python setup.py install
> cd ../hyperkitty; python setup.py build; python setup.py install
So far no errors. Just warnings.
Backups:
* deployment directory (as user with rw in directory)
> tar -cvzf backup.tgz deployment_directory
* postgres databases (as user with postgres-priv)
> pg_dump maiman | gzip -9 > mailman.bak.gz
> pg_dump mailmanweb | gzip -9 > mailmanweb.bak.gz
Create new .wsgi-file based on mailman-suite (I used the kludge in the
script to get the python path correct)
Change apache2.conf to point to new venv and new wsgi-version
Updated the mailman3wrapper.sh I use for init to start the service.
I could not get hyperkitty interface to run so I deactivated it and
set mailman as index instead. It complains that a column is missing,
but I cannot get the migrate script to run so I decided that I do not
need the archiver as I currently don't use it. So I will have to get
back to this and either remove the archiver completely or get it to
run.
So the line with ^$ in urls.py now is
url(r'^$', RedirectView.as_view(url=reverse_lazy('list_index'))),
instead, and the line "hyperkitty," is commented in the django application list.
cheers.
// David
On Tue, Aug 8, 2017 at 6:48 PM, Mark Sapiro <mark(a)msapiro.net> wrote:
> On 08/08/2017 04:42 AM, David Krantz wrote:
>>
>> What is the rekommended way to upgrade to the newest version? (newest
>> _usable_ version, of course)? Is it docker-only with a migration from
>> the current solution to a docker based? If so, any ideas or pointers
>> on how to go about this?
>
>
> You may find this wiki article helpful <https://wiki.list.org/x/17891998>.
>
> I started with bundler on both this (lists.mailman3.org) and the
> mail.python.org installs, but have been updating from the heads of the
> various gitlab branches since very early on.
>
> At the moment, these installations are somewhat ahead of the 3.1
> release, but I am not updating as the latest core has dropped Python 3.4
> support, and I haven't yet installed Python 3.5 or 3.6 on these (Ubuntu
> 14.04) servers. (I will probably install 3.6 from source at some point.)
>
> Please feel free to ask here if you have more specific questions.
>
> --
> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
> San Francisco Bay Area, California better use your sense - B. Dylan
> _______________________________________________
> Mailman-users mailing list
> mailman-users(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
7 years, 8 months
Re: Mailman, etc. upgrade woes and persistent bugs
by Abhilash Raj
On Wed, Feb 17, 2021, at 10:24 PM, Stephen J. Turnbull wrote:
> Abhilash Raj writes:
>
> > So it seems like we have some sort of agreement that we need all the
> > roles to be able to see all the 4 states.
>
> I think it's more like there's disagreement about which roles need to
> see which states, but for each role and each state, somebody thinks
> that role needs to see that state. I think there's agreement that
> it's harmless to allow everybody to see all states.
>
> > Roles here are basically Admin( i.e. list owner/moderator/site
> > admin) and Subscriber.
>
> In this discussion, yes. However, if we're going to take the
> discussion seriously, we should be more general than just this case.
> I think we need to consider five roles. Starting with Brian's list of
> List Owner, Moderator, List Member, and adding Anonymous (so that a
> pre-login connection has a named role) and Site Owner for completeness.
Yep, those are all the roles that even we recognize in Postorius. We also
have the concept of Domain Owner, though that isn't really implemented yet.
So, from this list, List Owner and above all will use the "by_admin" state
to disable if they need to and they are the only ones who can set that state.
>
> > There is no access control to transition "from" any state but each
> > value of non-enabled states is sort of a role's way to disable the
> > delivery. Not all states are settable by all roles.
>
> Off-list, Ruth Ivey-Cook informs me that in "who sets what" both "who"
> and "what" need to be considered is what she had in mind. So I think
> that's exactly right and there's agreement.
Nice.
>
> > 1. A subscriber can only transition from any to "enabled" or "by_user"
> > state but can see if delivery was disabled "by_admin" or "by_bounces"
> > as separate state in the Options page.
> > 2. An admin can transition to "enabled", "by_user" and "by_moderator".
>
> I assume you mean "by_admin" instead of "by_moderator"?
Ok, I've made a mess of it in my responses, the Core actually calls this value
"by_moderator" but I have used them interchangeably in my emails. Sorry
about that.
https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/interfaces/mem…
>
> > They should use "by_user" when acting on behalf of the user and other
> > when there are any administrative reason to disable delivery, whatever
> > that may be. I expect "by_admin" to be used sparingly, and I don't see
> > any use cases for it.
> > 3. Mailman itself will use the "by_bounces" to disable delivery due to
> > bounces.
> >
> > Is that an accurate description?
>
> +1 for 1 -- 3 as written. Others' mileage may vary.
Cool, thanks!
>
> Steve
>
>
--
thanks,
Abhilash Raj (maxking)
4 years, 3 months
Re: Post-installation documentation?
by David Newman
On 11/25/21 1:16 PM, Mark Sapiro wrote:
> On 11/25/21 10:56 AM, David Newman wrote:
>>
>> Thanks for this. After setting these variables and restarting mailman3
>> and mailmanweb, the mailman user still cannot find a hyperkitty_import
>> command (neither can root, btw). Again, I installed MM3 and
>> Django/Postorius/Hyperkitty using the venv and web docs previously
>> referenced.
>
>
> Don't run mailman or django admin commands as root. This causes issues
> when files are created by root and can't be read by mailman.
Understood, thanks. I'd only tried root to determine if there was a
permissions issue, where hyperkitty_import really did exist but wasn't
available to the mailman user.
>
> What django admin command are you running?
As the mailman user, with venv activated:
(venv) mailman@lists:~$ hyperkitty_import -l testlist(a)lists.domain.com
/var/lib/mailman/archives/private/testlist.mbox/testlist.mbox
'testlist' and 'lists.domain.com' are redactions; they are correct in
the given command, but the command fails because the system can't find
hyperkitty_import.
(BTW
Also, because I'm looking to migrate from Mailman 2.1 installed from
Debian packages, all the 2.1 stuff is owned by the 'list' user. I'd
changed ownership of the above private archive to mailman:mailman for
the command not to complain about permissions.
> If you don't set
> DJANGO_SETTINGS_MODULE in the environment and you have activated your
> virtualenv and run `mailman-web` as the django admin command it will set
> /etc/mailman3/settings.py as the settings.
I'd run these two environment commands as the mailman user with
virtualenv activated:
export DJANGO_SETTINGS_MODULE=settings
export PYTHONPATH=/etc/mailman3/settings.py
Am I being overly literal here with 'settings' in that first command?
Asking because 'mailman-web check' runs clean before running these
commands and returns this error afterward:
ModuleNotFoundError: No module named 'settings'
Either way, the hyperkitty_import command still isn't found.
>> On the plus side, all lists now appear in the admin web page. However,
>> lists created from the admin page do not appear in a site list page,
>> even though all lists are in the same site/domain.
>
>
> By default, only lists that you are owner, moderator or subscriber for
> are shown. What if you click Filter by Role and select All?
Same result. In this case, the ID of the admin and list owner are the same.
>
>
>> Finally, how to make these exports permanent? I did not see this
>> covered in the migration doc above. This is on a Debian system where
>> the mailman user's shell is bash. Is this simply a matter of adding
>> environment variables in the mailman user's .bashrc file?
>
> You can do that but as I note above, if you run `mailman-web` you
> shouldn't need to. You can add a `source /opt/mailman/venv/bin/activate`
> command as mentioned at
> https://docs.mailman3.org/en/latest/install/virtualenv.html#activate-virtua….
Yes, this is what I have in mailman's .bashrc file. I'd followed that
venv doc for initial installation, and it activates the virtualenv
whenever launching a shell for mailman.
dn
3 years, 6 months
Re: KeyError: 'subject_prefix'
by Abhilash Raj
On Sun, Nov 15, 2020, at 1:20 AM, Torge Riedel wrote:
> Hi,
>
> today I had the following error in my inbox:
>
> KeyError: 'subject_prefix'
> AttributeError: 'MailingList' object has no attribute 'subject_prefix'
>
> I checked the database, there is a column "subject_prefix", so I feel
> it is something in the code causing the error.
>
> It's mailman 3.3.1 and I don't see a fix for such an error in the
> Changes of 3.3.2
>
Is the value of subject prefix for that list set to *something* in Mailman? You can check via API or if you have Postorius, you can go to List's settings page and under the "List Identity" tab there is a field.
I am suspecting Hyperkitty wants to read this attribute but since it is not set in database, Core is skipping the field in API response for /list/<listid>/config which typically returns all the list settings.
Abhilash
> Kind regards
> Torge
>
> Traceback (most recent call last):
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 119, in __getattr__
> return self._get(name)
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 86, in _get
> raise KeyError(key)
> KeyError: 'subject_prefix'
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/hyperkitty/models/mailinglist.py", line 197, in update_from_mailman
> value = getattr(mm_list, propname)
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 124, in __getattr__
> self.__class__.__name__, name))
> AttributeError: 'MailingList' object has no attribute 'subject_prefix'
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/django_extensions/management/commands/runjobs.py", line 36, in runjobs
> job().execute()
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/hyperkitty/jobs/sync_mailman.py", line 36, in execute
> sync_with_mailman()
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/hyperkitty/lib/mailman.py", line 132, in sync_with_mailman
> mlist.update_from_mailman()
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/hyperkitty/models/mailinglist.py", line 199, in update_from_mailman
> value = mm_list.settings[propname]
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 146, in __getitem__
> return self._get(key)
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 88, in _get
> return self.rest_data[key]
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 74, in rest_data
> response, content = self._connection.call(self._url)
> File
> "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 112, in call
> error_msg, response, None)
> urllib.error.HTTPError: HTTP Error 500: <html>
> <head>
> <title>Internal Server Error</title>
> </head>
> <body>
> <h1><p>Internal Server Error</p></h1>
>
> </body>
> </html>
>
> ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty)
> START TRACEBACK:
> END TRACEBACK
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>
--
thanks,
Abhilash Raj (maxking)
4 years, 6 months
Re: Question about adding addresses using the REST API
by Stephen Daniel
TL;DR: Listening to the advice, I will create and manage user/address pairs.
I am in the process of porting a web site that is used to manage a small
organization. The web site maintains a directory of users of that
organization, and also allows them to subscribe/unsubscribe to a small set
of interest-group mailing lists for that organization. The users are not
tech savvy, and are not expecting or expected to manage their subscriptions
through Postorius, nor do they have access (normally) to the archives.
Indeed, the URL to Postorious will normally be turned off at the reverse
proxy server that sits in front of both Postorious and the organization's
website.
The one-source-of-truth is the website's database. The website code has
callouts for "subscribe <address> to <list>" and "unsubscribe <address>
from <list>".
I am trying to minimize the amount of code I write to implement the web
site's callouts, to initialize the mailing lists when I port the
organization to this new implementation, and make it relatively easy to
keep the website's database and mailman3's database in sync.
The impression I had from the docs was that unlinked addresses were a first
class object. Since my website callouts only deal with addresses, this
seemed a good mapping. However, I understand from this email thread that
unlinked addresses are not the way to go, so I'll map <address> in the
website database to <user>/<address> in mailman3.
Thanks for the advice and feedback!
-- Stephen Daniel
On Sat, Jan 15, 2022 at 2:27 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 1/15/22 10:03 AM, Stephen Daniel wrote:
> >
> > The only way I've found to create a standalone address (not linked to any
> > user) is to create a user with that address, unlink the address, then
> > delete the user.
> >
> > Is there a more straightforward way to just create an address with no
> user?
>
>
> The more straightforward way is via the IUserManager create_address
> method. This method is not exposed directly in REST, so you can't do it
> via REST.
>
> My question is why do you want to?
>
> Granted when an unknown nonmember address posts to a list we create a
> standalone address record, but maybe we should be creating a user too.
> We don't because it's tricky. Suppose User A with address A posts to a
> list from nonmember address B. We create the standalone address B record
> as a nonmember, but now user A can add address B as another address. If
> we also created User B, this would not be possible.
>
> But again, what is your use case for creating standalone addresses?
>
>
> --
> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
> San Francisco Bay Area, California better use your sense - B. Dylan
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>
3 years, 4 months
Innstall and run Mailman 3 with Postfix in a Docker enviroment
by kwp.klein＠gmail.com
## Install Mailman 3
The idea is to install Mailman 3 in ad docker container. Also I want to install
Postfix in a container. All the mail which will send and receive should handled
in a docker container. The reason is, that on the server only applications in a
docker environment working. There is no MTA installed. On the server are
Discourse as forum, wikijs, portainer, watchtower, homepage, and the NGINX Proxy
Manager installed. At next I need Mailman 3.
I don't know how to install and run Postfix for Mailman in a docker environment.
Here is my first try. I can visit the site lists.example.com, but I can not set
the Admin password. I think the reason is, that the MTA not working.
I have a gitlab repo. If you like please look here:
https://gitlab.com/joklein/mailman
### Configure the docker-compose.yml file
```yml
version: '2'
services:
postfix:
image: mailu/postfix
restart: unless-stopped
container_name: postfix
volumes:
- /opt/mailman/postfix:/etc/postfix/main.cf.d
- /opt/mailman/core:/opt/mailman
ports:
- "25:25"
networks:
mailman:
ipv4_address: 172.19.199.6
services:
mailman-core:
image: maxking/mailman-core:0.3
restart: unless-stopped
stop_grace_period: 30s
container_name: mailman-core
hostname: mailman-core
volumes:
- /opt/mailman/core:/opt/mailman
links:
- database:database
depends_on:
- database
environment:
- DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb
- DATABASE_TYPE=postgres
- DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase
- SMTP_HOST=172.19.199.1
- HYPERKITTY_API_KEY=ZawB3NGBUgbmQcMGhZ0kYIW0WRNw8P9osEudsGiFZWv
networks:
mailman:
ipv4_address: 172.19.199.2
mailman-web:
image: maxking/mailman-web:0.3
container_name: mailman-web
restart: unless-stopped
hostname: mailman-web
ports:
- "8000:8000"
depends_on:
- database
links:
- mailman-core:mailman-core
- database:database
volumes:
- /opt/mailman/web:/opt/mailman-web-data
environment:
- DATABASE_TYPE=postgres
- DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb
- HYPERKITTY_API_KEY=ZawB3NGBUgbmQcMGhZ0kYIW0WRNw8P9osEudsGiFZWv
- UWSGI_STATIC_MAP=/static=/opt/mailman-web-data/static
- SERVE_FROM_DOMAIN=lists.example.com
- MAILMAN_ADMIN_USER=admin
- MAILMAN_ADMIN_EMAIL=lists(a)example.com
- SECRET_KEY=mailmangeheim
networks:
mailman:
ipv4_address: 172.19.199.3
database:
image: postgres:9.6-alpine
restart: unless-stopped
environment:
POSTGRES_DB: mailmandb
POSTGRES_USER: mailman
POSTGRES_PASSWORD: mailmanpass
volumes:
- /opt/mailman/database:/var/lib/postgresql/data
networks:
mailman:
ipv4_address: 172.19.199.4
networks:
mailman:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.19.199.0/24
```
#### Build the API Key für Hyperkitty and SECRET_KEY
You can build a key with `pwgen -s 128`
#### Make some directorys
```sh
mkdir -p /opt/mailman/core
mkdir -p /opt/mailman/web
mkdir -p /opt/mailman/mailman-web-data/static
```
#### Make a file in /opt/mailman/web/settings_local.py
```sh
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = '172.19.199.1'
EMAIL_PORT = 25
USE_SSL = False
DEFAULT_FROM_EMAIL = "<E-MAIL>"
SERVER_EMAIL = "<E-MAIL>"
```
Change the `<E-MAIL>` with your server address. For example
`mailman(a)lists.example.com`
So that Postfix also sends your e-mails, you create the file
`/opt/mailman/core/mailman-extra.cfg` with the following content:
```sh
[mta]
incoming: mailman.mta.postfix.LMTP
outgoing: mailman.mta.deliver.deliver
lmtp_host: 172.19.199.2
lmtp_port: 8024
smtp_host: 172.19.199.1
smtp_port: 25
configuration: /etc/postfix-mailman.cfg
```
Now you are ready to start your container:
```sh
sudo su -
cd /root/mailman-docker
docker-compose up -d
```
#### Check the mailman images
```sh
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f6990a2f808f maxking/mailman-web:0.3 "docker-entrypoint.s…" 55 seconds ago Up 52 seconds 8000/tcp, 8080/tcp mailman-web
a7fa6334b2b8 maxking/mailman-core:0.3 "docker-entrypoint.s…" 58 seconds ago Up 54 seconds 8001/tcp, 8024/tcp mailman-core
e8812dc42663 postgres:9.6-alpine "docker-entrypoint.s…" About a minute ago Up 57 seconds 5432/tcp docker-mailman_database_1
```
#### Check the conection
```sh
curl http://172.19.199.3:8000/postorius/lists/
```
4 years, 3 months
Re: Hyperkitty "All Threads" results in error
by Mark Dadgar
Aaaand I spoke too soon. I am now seeing the same error when I click the All Threads button in hyperkitty. Not every time, because that would be too easy, but often.
I really only have one high-volume list with archives, so I can’t tell if it’s list-specific.
I ran the mailman-post-update script after the last update, which I believe executes all the commands suggested below.
Here’s the crash trace.
Any thoughts?
- Mark
Internal Server Error: /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
VariableDoesNotExist at /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
Failed lookup for key [sender] in None
Request Method: GET
Request URL: https://mail.pdc-racing.net/archives/list/trackjunkies@pdc-racing.net/latest <https://mail.pdc-racing.net/archives/list/trackjunkies@pdc-racing.net/latest>
Django Version: 3.0.12
Python Executable: /opt/mailman/mm/venv/bin/python
Python Version: 3.8.5
Python Path: ['/opt/mailman/mm', '/opt/mailman/mm/', '/opt/mailman/mm/bin', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/opt/mailman/mm/venv/lib/python3.8/site-packages']
Server time: Mon, 1 Mar 2021 23:25:02 -0800
Installed Applications:
('hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Traceback (most recent call last):
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 828, in _resolve_lookup
current = current[bit]
During handling of the above exception ('NoneType' object is not subscriptable), another exception occurred:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 836, in _resolve_lookup
current = getattr(current, bit)
During handling of the above exception ('NoneType' object has no attribute 'sender'), another exception occurred:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 842, in _resolve_lookup
current = current[int(bit)]
During handling of the above exception (invalid literal for int() with base 10: 'sender'), another exception occurred:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/lib/view_helpers.py", line 134, in inner
return func(request, *args, **kwargs)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/views/mlist.py", line 115, in archives
return _thread_list(request, mlist, threads, extra_context=extra_context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/views/mlist.py", line 144, in _thread_list
return render(request, template_name, context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/shortcuts.py", line 19, in render
content = loader.render_to_string(template_name, context, request, using=using)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader.py", line 62, in render_to_string
return template.render(context, request)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/backends/django.py", line 61, in render
return self.template.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 171, in render
return self._render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py", line 150, in render
return compiled_parent._render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py", line 62, in render
result = block.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/defaulttags.py", line 209, in render
nodelist.append(node.render_annotated(context))
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py", line 188, in render
return template.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 173, in render
return self._render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/defaulttags.py", line 513, in render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/templatetags/i18n.py", line 132, in render
context.update({var: val.resolve(context) for var, val in self.extra_context.items()})
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/templatetags/i18n.py", line 132, in <dictcomp>
context.update({var: val.resolve(context) for var, val in self.extra_context.items()})
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 691, in resolve
arg_vals.append(arg.resolve(context))
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 795, in resolve
value = self._resolve_lookup(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 847, in _resolve_lookup
raise VariableDoesNotExist("Failed lookup for key "
Exception Type: VariableDoesNotExist at /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
Exception Value: Failed lookup for key [sender] in None
Request information:
USER: mark
GET: No GET data
POST: No POST data
FILES: No FILES data
On Feb 17, 2021, at 5:34 PM, Mark Dadgar <mark(a)pdc-racing.net> wrote:
>
> On Oct 14, 2020, at 4:37 PM, Mark Sapiro <mark(a)msapiro.net> wrote:
>> On 10/14/20 11:12 AM, Mark Dadgar wrote:
>>>> On Oct 4, 2020, at 10:34 AM, Mark Dadgar <mark(a)pdc-racing.net> wrote:
>>>>
>>>> So I have been wrestling with this problem for a while now. Most times, but not every time, when someone clicks on the All Threads link in hyperkitty, a 500 error is generated.
>>
>>
>> Does this happen on all lists, only some lists or only one list?
>>
>>
>>>> I googled it, obviously, and at first I thought it was related to this issue, but that has not proven to be the case:
>>>>
>>>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…
>>
>> What did you google? This query gets a lot of stuff with some hints, but
>> nothing definitive.
>>
>> django render raises VariableDoesNotExist("Failed lookup for key "
>>
>> One thing I noted at
>> <https://askbot.org/en/question/3095/django-error-variabledoesnotexist-faile…>
>> was
>>
>> models out of date - run python manage.py syncdb and python manage.py
>> migrate
>>
>> You could try that. Note that your django admin command might be
>> something other than `python manage.py`. Also note syncdb has been
>> removed from Django >=1.9. Currently, the above becomes `makemigrations`
>> followed by `migrate`.
>>
>> I also suggest running the django admin commands
>>
>> `collectstatic --clear --noinput`
>> and
>> `compress`
>
>
> My re-install seems to have fixed this issue as well. BONUS!
>
> - Mark
> -----
> mark(a)pdc-racing.net | 408-348-2878
>
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
- Mark
-----
mark(a)pdc-racing.net | 408-348-2878
4 years, 3 months
Re: how to integrate subscription into static jekyll website
by Stephen J. Turnbull
Hagen Bauer writes:
> Has anybody tried to hide the direct api access behind a nginx reverse
> proxy url?
Sounds straightforward.
I doubt anybody's done it, though, because you still face the problem
that anybody with that access is a site manager, with unclear limits
on their power outside of Mailman. (Shouldn't be much they can do
outside of Mailman, but I wouldn't bet my host on it.) In many cases,
the real site manager is also the host admin, so they have sufficient
access to a shell account. There's no need for her to have a proxy;
why make it available to people who are supposed to have less power?
Note I'm *not* saying this *never* makes sense. I'm just saying that
in a lot of cases it won't buy you very much because Mailman isn't set
up to provide granular authorization of REST endpoints.
> on the Webseite I would have a url like this
>
> mailmanserver.tld/subcsribe/user(a)mail.com
>
> And within the nginx proxy configuration on the server this would
> translate to
>
> localhost:80xx/"mailman-restapi-with-credentials"user(a)mail.com
First of all, the address to subscribe is a variable, so at a bare
minimum your nice static site has to provide a form, which isn't very
static. I guess you could put the URL to the nginx proxy as the
action URL in the form, but then the proxy needs to deal with a
form-encoded query string to translate to the appropriate REST
endpoint.
I suppose you could also have the user type the URL including the
email address into the browser address bar, but Abhilash's idea of
providing a mailto:subscribe@mailmanserver.tld URL seems more
convenient. The mailto URL also solves the problem of confirming
ownership of the mailbox, since that logic is built in to the
subscribe-by-mail code.
Second, if your proxy is configured very carefully, you might be able
to restrict this to only the exact operations you want to expose, here
subscription. However, although the email interface implements
verification by providing a one-time token that must be read from the
mailbox of the address (and so proves ownership of the mailbox), I
don't think the REST API implements operations like that. IIRC that
is implemented in Postorius. (Abhilash?) If my memory is correct,
you would also need to implement that, and that's definitely not static.
> So the public does not see the credentials. And if my nginx host
> config is hacked I have a problem anyway.
I can't speak for Abhilash, but what would worry me is not that nginx
is already hacked, but rather that if not properly configured, it
might be possible for someone to access REST endpoints you don't want
exposed through this interface, and maybe even to other parts of the
webserver. To give a simple example, consider a variant on the
ancient Apache traversal bug:
mailmanserver.tld/subscribe/../unsubscribe/user(a)mail.com
Probably nginx and/or Mailman and/or your proposed proxy config are
already capable of stopping this particular well-known attack, but
undoubtedly there are more sophisticated attacks that I don't know
about.
6 years, 7 months
Re: Problems after update to mailman 3.3.7
by Guillermo Hernandez (Oldno7)
El 6/12/22 a las 15:59, Danil Smirnov escribió:
> Hi Guillermo,
>
> We've faced the same issue and already reported it:
> https://gitlab.com/mailman/mailman/-/issues/1044
>
I see. Thanks, Danil, for the confirmation that I'm not alone in this..
> I would not recommend anyone using MySQL/MariaDB database engine upgrade to
> the latest version yet.
Too late for that... Doy you know what components could I downgrade to
avoid this? SQLAlchemy? Which versions of which packages could I
downgrade to work prpoerly?
Again, thanks a lot
> With my best regards,
> Danil Smirnov
> Mailman3.com
>
> On Tue, Dec 6, 2022 at 4:32 PM Guillermo Hernandez (Oldno7) via
> Mailman-users<mailman-users(a)mailman3.org> wrote:
>
>> I've getting the most strange behaviour after update to last releases.
>> In one of my mailman3 servers, the messages doesnt reach for some lists
>> (the five last ones)
>>
>> the update have not showed any error but in the var/data for the lmtp
>> and vmap postfix files mailman does not create aliases for that last 5
>> lists. Any message addressed to them is returned with a "no such user"
>> error.
>>
>> If I create a new list the system or restart mailman runners it shows
>> this error:
>>
>> Generating MTA alias maps
>> /usr/local/lib/python3.9/site-packages/pymysql/connections.py:799:
>> UserWarning: Previous unbuffered result was left incomplete
>> warnings.warn("Previous unbuffered result was left incomplete")
>>
>> I did a CHECK TABLE on mailinglists table but all seems ok.
>>
>> I tried reinstalling PyMySQL, but has no effect. (well, I tried
>> reinstalling all)
>>
>> It seems to me that something is preventing creation of that aliases, an
>> then the that last five lists are unreachable.
>>
>> I have no clue of what to do now and I'm driving crazy.
>>
>> Some direction to look for will be much apreciated.
>>
>> Thanks in advance.
>>
>> P.S.: I can access via postorius/hyperkitty to the archives of the lists
>> that are unreacheable.
>>
>>
>> --
>>
>> ___________________________________________
>> Mailman's content filtering has removed the
>> following MIME parts from this message.
>>
>> Content-Type: image/png
>> Name: firma-GHP-emails.png
>>
>> Replaced multipart/alternative part with first alternative.
>> _______________________________________________
>> Mailman-users mailing list --mailman-users(a)mailman3.org
>> To unsubscribe send an email tomailman-users-leave(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>> Archived at:
>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>>
>> This message sent todanil(a)smirnov.la
>>
> _______________________________________________
> Mailman-users mailing list --mailman-users(a)mailman3.org
> To unsubscribe send an email tomailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> Archived at:https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/mess…
>
> This message sent toguillermo(a)querysoft.es
>
--
___________________________________________
Mailman's content filtering has removed the
following MIME parts from this message.
Content-Type: image/png
Name: firma-GHP-emails.png
Replaced multipart/alternative part with first alternative.
2 years, 6 months
Re: Mailman, etc. upgrade woes and persistent bugs
by Stephen J. Turnbull
No answer required, I think I understand the situation pretty well
now. But if I'm missing something, I would very much appreciate
criticism.
Andreas Barth writes:
> Well, currently these are exclusive-states, I'd rather see them as
> or-able-states.
Right, I understand that. I don't understand *why*.
There are real costs to making them or-able. There are constraints:
'enabled' shouldn't (mustn't?) be or-able with any 'disabled-by-foo'
state. Such invariants are finicky to code and tricky to maintain,
compared to a simple "mutually exclusive".
I guess you want to allow the or of any subset of disabled states, but
presumably then they need to individually resettable in order for the
admin to clear their disable without overriding a subscriber's
preference or a bounce record. That set of controls is more complex,
and requires additional mental effort on the part of anyone using
them. If there are restrictions on which states can be or-ed, that's
more coding and maintenance cost.
Then there's the question whether the subscriber can override the admin's
setting. If they can, why does the admin have this power in the first
place?
> > > - "disabled by member" (could be set by an admin as well of course)
> >
> > Currently this is determined by whether the logged in user is the
> > subscriber or not. It's an interesting idea to allow it to be set by
> > an admin. Then we could have the semantics that the only transition
> > allowed from this state would be to enabled, so that admins would know
> > the user had intentionally disabled and know not to enable without
> > permission.
>
> Exactly.
But if the admin should respect the DBS (disabled by subscriber)
setting, DBS|DBA (disabled by admin) is logically equivalent to DBS.
Are there use cases where the admin does not want to repect the
subscriber's choice, and enables it anyway? What are they?
> Or while importing from another server, and there are people who
> have receiving mails disabled, this is the state that could be used
> for it.
Why wouldn't they get the exact state from the original server? Or do
you mean from a non-Mailman server?
> I expect that state to be the most-often used when setting a state
> manually.
>
> For "who set that", I would consider having some kind of history as
> useful (and displaying "last set by user / $person" to the admin but
> not the user).
Why not to the subscriber? As I explain below, the subscriber at
least needs to be able to distinguish between DBA and DBB (disabled by
bounce).
Keeping history of who is additional complexity, consumes a minor
amount of resources, and is insufficient to the purpose. Assuming
that there are reasons for admins to disable/enable other than as a
courtesy to a subscriber, that reason is presumably important. Not only
will the subscriber not know, but one admin may forget, and there may be
multiple admins.[1]
I still don't see any reason for an admin to disable delivery other
than as a courtesy to the subscriber or to the bounce processor.
> > > - "disabled for administrative reasons" (only by admin but user
> > > should be able to see it)
> >
> > I don't see why a user would need to see the difference. Unless
> > you're suggesting that the user would not be able to reenable without
> > asking the admin? What is the use case for this?
>
> I'm suggesting exactly that, yes. Why? Well, worked with user for too
> long. ;)
>
> E.g. consider that seeing archives works only for subscribers. And the
> subscriber in question complains about too many mails but still
> re-enables getting mails. This is basically "an admin has blocked you
> from more mails but not unsubscribed you". Might be an corner case,
> but still. Might also be too unimportant.
This is possible. If it occurs less than once in a million years,
it's "too unimportant" to even think about.[2] Have you actually done
this, or know anybody who has?
Is it really different from "at request of subscriber"? Practically,
the only difference I can see is if such a subscriber should be
prevented from overriding DBA. Are you suggesting that?
> > > - "bounced" (user and admin could reset it but not set it)
> >
> > Reenable but not set "bounced" is the current situation.
>
> Except if someone sets the state to "disabled" and then undo this, the
> state is automatically cleared.
This is possible, but it's not clear what harm would be done. In most
cases, bounces are intermittent and clear themselves up, in which case
this is the right thing. But if the subscriber bounces again, they'll
get disabled again, only cost is a slight waste of resources.
Here's how I would assess the issues. In the descriptions below, I am
assuming that each state is the only "active" state.
1. DBS is probably a "vacation" or "post-only" setting. For that
reason it should be re-enabled only by the subscriber (or by an
admin at the request of the subscriber).
2. DBB is a "warning" and a minor convenience to the email system
(slightly reducing traffic to mailboxes that are expected to fail,
and perhaps avoiding some reputational damage to the list).
Re-enabling much of the time causes no damage, as bounce disables
are typically due to things like out of disk space, and are likely
fixed by the time the subscription is reenabled. Most of the rest
will be zombie mailboxes, which will rarely be reenabled, will
quickly be redisabled by bounce, and at worst cause mild
inconvenience. Finally, events like a Mailman upgrade and the
April 2014 DMARC fiasco will be pretty evident, and the damage
there has already been done, reenabling actually fixes it
(although it may not be effective if the systemic problem hasn't
been resolved).
3. DBA is interesting mostly to *subscribers*, who need to
distinguish DBA (which the subscriber can reenable freely) from
DBB (which warns the subscriber to get in touch with their
postmaster). It's comforting to subscribers who may be sure they
never disabled their subscription; otherwise, it's semantically
identical to DBS as far as I can see.
Am I missing something?
It seems to me that given 1-3, assuming a 4-state design:
0. It's never unreasonable to re-enable, except DBS.
1. In the DBA state, changing to either DBS or DBB causes no
problems. (Changing to DBB is unlikely, since no mail is being
sent. However, it's possible, due to the race between Postorius
and mail in transit, and due to spam and other spoofing of the
list domain.)
2. In either of the other two states, we should never change to DBA.
3. In either of the other two states, if the other event occurs, you
would want the state to be DBS|DBB. I contend that if we
substitute DBS, little harm is done. If the subscriber reenables, it's
on them to fix the bounce problem if it reoccurs in any case.
Since the subscriber has deliberately set DBS, the admin should not
reenable without permission of the subscriber (I think we're mostly in
agreement on that).
I think this 4-state design is much easier to design, validate, and
maintain than the 16-state design you propose. If you're really
worried about DBS|DBB, then we could have a 5-state design where
enabling from DBS|DBB generates a warning that there may still be a
bounce problem on the subscription. Instead, it could leave the
subscription in state DBB, but that seems to me to be obnoxious, since
there's little likelihood of harm from reenabling, and in many cases
the bounce problem will have resolved itself in the meantime.
Regards,
Steve
Footnotes:
[1] There's also the question of GDPR obligations if personal reasons
(such as "violated code of conduct") are in the history.
[2] I'm not sure where we'd actually draw the line, probably on the
order of "once a year per thousand subscribers", that being every week
for say a campus-wide list at Ohio State University with 55,000 students.
4 years, 3 months