Search results for "Postorius" - Mailman-users

Today no question, but my toolbox: set "global" user preferences; list or delete held messages

by Wikinaut

After some (sorry for bothering you) questions here, I was able to set up a mailman3 system which I regard as perfect for my use. Perhaps, you had similar problems with your installation, especially when coming from mailman2 and, like me, from majordomo...... I collected a lot of information which was presented in this mailling list archive here and also answers of my questions (thanks Mark and others) News: https://github.com/Wikinaut/mailman3-tools has all of these tools. README says: Scripts for maintenance or setting-up of modern mailman3 instances set preferences for all users list held messages delete held messages older n days My setting is strictly as described in the documentation: mailman3 + postorius/hyperkitty are installed via pip in an (venv) /opt/mailman/ My user scripts and a huge mailing list generator (not published here) are in a directory apart from that: user maillistgen home directory /home/maillistgen user maillistgen has some sudo rights set uo in /etc/sudoers a wrapper script is /home/maillistgen/run_mailman.sh which sets the correct path for Python, mailman3 and the (venv). I hope, you like my tiny part of contribution and attribution back to Mark.

1 year

Today no question, but my toolbox: set "global" user preferences; list or delete held messages

by Wikinaut

After some (sorry for bothering you) questions here, I was able to set up a mailman3 system which I regard as perfect for my use. Perhaps, you had similar problems with your installation, especially when coming from mailman2 and, like me, from majordomo...... I collected a lot of information which was presented in this mailling list archive here and also answers of my questions (thanks Mark and others) News: https://github.com/Wikinaut/mailman3-tools has all of these tools. README says: Scripts for maintenance or setting-up of modern mailman3 instances set preferences for all users list held messages delete held messages older n days My setting is strictly as described in the documentation: mailman3 + postorius/hyperkitty are installed via pip in an (venv) /opt/mailman/ My user scripts and a huge mailing list generator (not published here) are in a directory apart from that: user maillistgen home directory /home/maillistgen user maillistgen has some sudo rights set uo in /etc/sudoers a wrapper script is /home/maillistgen/run_mailman.sh which sets the correct path for Python, mailman3 and the (venv). I hope, you like my tiny part of contribution and attribution back to Mark.

1 year

Re: Hyperkitty fulltext search performances

by Mark Dadgar

On Oct 4, 2020, at 8:13 AM, Mark Sapiro <mark(a)msapiro.net> wrote: > On 10/4/20 8:07 AM, Brian Carpenter wrote: >> >> On 10/4/20 8:59 AM, Loïc Dachary wrote: >>> After a little digging it turns out the mailman3-web Debian GNU/Linux >>> package I'm using is configured to use Woosh as a haystack backend. >>> And it is not fit for volumes greater than a few hundred mega bytes. I >>> should switch to something else from the list of supported >>> backendshttps://django-haystack.readthedocs.io/en/master/backend_support.html >>> >> For our servers running Postorius/Hyperkitty, we use Xapian. I believe >> the mailman-users (MM3) list also uses Xapian. It is a lot quicker. For >> our Affinity/Empathy servers, we use Elasticsearch which we are very >> happy with. Elasticsearch does have a higher memory requirement I >> believe than Xapian. > > > Brian is correct about this list. We started with Whoosh, then went to > Elasticsearch and ultimately to Xapian. Each step gave better > performance and a smaller footprint than the one before. +1 I have ~700K posts in my archive and Xapian is FAST. - Mark ----- mark(a)pdc-racing.net | 408-348-2878

5 years, 10 months

Internal Server Error: /mailman3/hyperkitty/api/mailman/archive

by Mail Person

Hello everyone. I am trying to sort out an error happening when messages are sent to the mailman server. The emails are being sent to the list but I get this error each time: Internal Server Error: /mailman3/hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1") Request Method: POST Request URL: http://localhost/mailman3/hyperkitty/api/mailman/archive?key=7OF5IogiQItSUK… Django Version: 2.2.12 Python Executable: /usr/bin/uwsgi-core Python Version: 3.8.10 Python Path: ['.', '', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Thu, 17 Aug 2023 17:13:09 +0000 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Traceback: File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 71. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 209. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 315. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 226. _mysql.connection.query(self, query) The above exception ((1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1")) was the direct cause of the following exception: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 115. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 113. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py" in inner 40. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in _decorator 71. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py" in wrapped_view 54. return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in archive 117. add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py" in add_to_list 153. email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save 740. self.save_base(using=using, force_insert=force_insert, File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save_base 777. updated = self._save_table( File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _save_table 870. result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _do_insert 907. return manager._insert([self], fields=fields, return_id=update_pk, File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_method 82. return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py" in _insert 1186. return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py" in execute_sql 1375. cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 67. return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute_with_wrappers 76. return executor(sql, params, many, context) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py" in __exit__ 89. raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 71. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 209. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 315. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 226. _mysql.connection.query(self, query) Exception Type: OperationalError at /hyperkitty/api/mailman/archive Exception Value: (1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1") Request information: USER: AnonymousUser

3 years

Re: Can't deliver mails in queue

by Mark Sapiro

On 5/20/21 4:01 AM, Nathanael Schneider wrote: > > It would have been nice though to have a more refined error to figure > this out. But its solved now and hopefully this will help someone else > as well. I'm glad you resolved it, and I'm sorry I didn't consider the template issue myself. You actually had two separate template issues. If you created templates in Postorius, they were inaccessible because of the POSTORIUS_TEMPLATE_BASE_URL issue, but your initial problem with imported lists is different. It probably resulted from running the `mailman import21` command as root which you should never do. You should always run `mailman` commands as the Mailman user. This is mentioned at https://docs.mailman3.org/en/latest/migration.html#id1 The `mailman import21` command can create list specific templates in Mailman's var/templates/lists/LIST_ID/LC/ directory and if run as root these will be user:group root:root and not world readable and therefore can't be read by Mailman. Also, deleting and recreating the list with the same LIST_ID doesn't remove those templates, so the list still doesn't work. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years, 3 months

Re: modify subscription page by mailinglist?

by Stephen J. Turnbull

Lars Schimmer writes: > And I think I could fiddle out how it works. But in general, e.g. a > custom page per mailinglist for the "list information page" is a bit > more work than expected and I will not follow this idea. > (e.g. adding a logo per list, ...) Just adding a list-specific logo is not hard. Choose a naming convention (probably something like f"logo-{list_id}.png"), put the logo PNG in one of the static directories and add <IMG SRC="logo-{{list_id}}.png" ALT="Logo for {{list_id}} list." /> in the right place in each template (maybe even "base.html", but more likely an intermediate template specifically to allow putting such customizations in one place for list-specific pages). Adding the ability for a list admin to easily upload a file and set the logo to it would require surgery to Postorius's Python code, though. I will admit that though I think this is a cool idea, I am not interested in doing it myself. But if your interest is at the "I'd rather wait a few months than do myself", GSoC is coming up again, and you could lobby for it when the students start applying. ;-) Steve

4 years, 7 months

Re: urls.py location on venv install

by Torsten

Am 2026-08-10 01:41, schrieb Mark Sapiro: > On 8/9/26 09:08, Torsten wrote: >> >> In fact, my settings.py does contain these import lines: >> >> import os > > This should not be needed unless something further down references os. > I have removed this line (s. other mail where Odhiambo suggested to take the config from the documentation and adjust it) >> from mailman_web.settings.base import * >> from mailman_web.settings.mailman import * >> >> BASE_DIR = ('/var/lib/mailman3/web') >> >> SECRET_KEY = '<MYSECRETKEY>' >> DEBUG = False >> >> #: Default list of admins who receive the emails from error logging. >> ADMINS = ( ('Mailman Suite Admin', 'torsten(a)bbadmin.de'),) >> >> ALLOWED_HOSTS = ['localhost', '127.0.0.1',] > > You may also want your public facing web domain here. > They are there, I have just omitted them here. >> INSTALLED_APPS = [ >> 'django_mailman3', >> 'postorius', >> 'hyperkitty', >> # Uncomment the next line to enable the admin: >> 'django.contrib.admin', > > > I'm hoping there's more to this file, but in any case unless you're > adding or deleting something from the definition in > mailman-web/settings/base.py, this is redundant and if you are adding, > it's better to do > ``` > INSTALLED_APPS.append('app.to.add') > ``` > or to remove something, for example > ``` > del INSTALLED_APPS[INSTALLED_APPS.index(django.contrib.admin')] > ``` > Yes, there's more in this config and also more in INSTALLED_APPS. >> And I have the mailman_web/urls.py in the venv where mailman was >> installed. > > If you are using gunicorn as your wsgi server, I think there is an > issue in the documentation. > <https://docs.mailman3.org/en/latest/install/virtualenv.html#setting-up-guni…> > says etc/mailman3/gunicorn.conf should contain `chdir = > "/opt/mailman/mm"`. I think this is wrong. It should be `chdir = > "/etc/mailman3"`. I am using uwsgi. Here's the complete settings.py, I have in /etc/mailman3 now: # Mailman Web configuration file. # /etc/mailman3/settings.py # Get the default settings. from mailman_web.settings.base import * from mailman_web.settings.mailman import * # Settings below supplement or override the defaults. #: Default list of admins who receive the emails from error logging. ADMINS = ( ('Mailman Suite Admin', 'torsten(a)bbadmin.de'), ) DATABASES = { 'default': { 'ENGINE': 'django.db.backends.sqlite3', 'NAME': '/var/lib/mailman3/web/mailman3web.db', 'USER': '', 'PASSWORD': '', 'HOST': '', 'PORT': '', } } # 'collectstatic' command will copy all the static files here. # Alias this location from your webserver to `/static` STATIC_ROOT = '/var/lib/mailman3/web/static' # enable the 'compress' command. COMPRESS_ENABLED = True # Make sure that this directory is created or Django will fail on start. LOGGING['handlers']['file']['filename'] = '/var/log/mailman3/web/mailmanweb.log' #: See https://docs.djangoproject.com/en/dev/ref/settings/#allowed-hosts ALLOWED_HOSTS = [ "localhost", # Archiving API from Mailman, keep it. "127.0.0.1", "mail.<MYDOMAIN1>", "<MYDOMAIN1>", "<MYDOMAIN2>", # "lists.your-domain.org", # Add here all production domains you have. ] #: See https://docs.djangoproject.com/en/dev/ref/settings/#csrf-trusted-origins #: For Django <4.0 these are of the form 'lists.example.com' or #: '.example.com' to include subdomains and for Django >=4.0 they include #: the scheme as in 'https://lists.example.com' or 'https://*.example.com'. CSRF_TRUSTED_ORIGINS = [ # "lists.your-domain.org", "https://*.<MYDOMAIN1>", "https://*.<MYDOMAIN2>", # Add here all production domains you have. ] #: Current Django Site being served. This is used to customize the web host #: being used to serve the current website. For more details about Django #: site, see: https://docs.djangoproject.com/en/dev/ref/contrib/sites/ SITE_ID = 1 # Set this to a new secret value. SECRET_KEY = '<MYSECRETKEY>' # Set this to match the api_key setting in # /opt/mailman/mm/mailman-hyperkitty.cfg (quoted here, not there). MAILMAN_ARCHIVER_KEY = '<ANOTHERSECRETKEY>' # The sender of emails from Django such as address confirmation requests. # Set this to a valid email address. EMAILNAME = '<MYDOMAIN1>' DEFAULT_FROM_EMAIL = 'postorius@{}'.format(EMAILNAME) # The sender of error messages from Django. Set this to a valid email # address. SERVER_EMAIL = 'root@{}'.format(EMAILNAME)

2 weeks, 3 days

Re: Exception with processing subscription confirmation email

by Eric Broens

Thanks Mark, We are indeed running mailman core 3.3.2 and using mysql. The data appears indeed to be truncated judging from the missing closing curly bracket (see below). Would it be OK to already extend the data column to VARCHAR(510) before upgrading to mailman core 3.3.3, or will this cause problems with upgrading to a newer version? +--------------------------------------------------------------------------- ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- ------------------------------+ | data | +--------------------------------------------------------------------------- ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- ------------------------------+ | {"pre_approved": false, "pre_confirmed": false, "pre_verified": false, "invitation": false, "address_key": "******************@************", "subscriber_key": 1, "user_key": "a44e8b9a34c64775b712f383d438ce1d", "token_owner_key": 1, "send_welcome_message" | +--------------------------------------------------------------------------- ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- ------------------------------+ Best Regards, Eric -----Original Message----- From: Mark Sapiro [mailto:mark@msapiro.net] Sent: donderdag 29 april 2021 18:43 To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: Exception with processing subscription confirmation email On 4/29/21 7:56 AM, Eric Broens wrote: > Hi, > > A new user tried to subscribe to a mailinglist, received the confirmation > mail, and replied to it. > > The subscriber is however not subscribed and remains listed in the "pending > confirmation" list. > > The following exception appears in the error.log. > > Does anyone knows how to resolve this? In the past we didn't experience any > issues and no changes but security updates have been installed. > > Apr 29 16:15:55 2021 (1581) deque: do_confirm_verify > > Traceback (most recent call last): > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/workflow.py", > line 69, in __next__ > > return step() > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/subscriptions.p > y", line 356, in _step_send_confirmation > > raise StopIteration > > StopIteration Tracebacks like the above are normal and probably shouldn't be logged at all. > Apr 29 16:16:28 2021 (1490) Uncaught runner exception: Expecting ':' > delimiter: line 1 column 256 (char 255) > > Apr 29 16:16:28 2021 (1490) Traceback (most recent call last): > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py", > line 173, in _one_iteration > > self._process_one_file(msg, msgdata) > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py", > line 266, in _process_one_file > > keepqueued = self._dispose(mlist, msg, msgdata) > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/runners/command.py" > , line 208, in _dispose > > mlist, msg, msgdata, parts, results) > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/commands/eml_confir > m.py", line 54, in process > > mlist).confirm(token) > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/subscriptions.p > y", line 578, in confirm > > workflow.restore() > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/workflow.py", > line 146, in restore > > data = json.loads(state.data) > > File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads > > return _default_decoder.decode(s) > > File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode > > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > > File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode > > obj, end = self.scan_once(s, idx) > > json.decoder.JSONDecodeError: Expecting ':' delimiter: line 1 column 256 > (char 255) > > Apr 29 16:16:28 2021 (1490) SHUNTING: > 1619705788.5856323+57a1a759275c59b732cfa8e2985f76cfcc27dda5 This one is the issue. There was some problem with the user's confirmation reply. View the shunted message with the command ``` mailman qfile path/to/var/queue/shunt/1619705788.5856323+57a1a759275c59b732cfa8e2985f76cfc c27dda5.pck ``` although that probably won't show anything wrong as it appears that the issue is with the saved workflow. You can get the token from the shunted message and then look at the database with something like ``` SELECT data FROM workflowstate WHERE token = '<the token>' ``` but I suspect the issue is because the database is MySQL or MariaDB, your Mailman core is older than 3.3.3 and your database server mode is such that data too long for column results in truncation, and the data in this case is truncated to 255 bytes. In Mailman 3.3.3, this column was changed from VARCHAR(255) to VARCHAR(510) for MySQL. As far as subscribing the user is concerned, you can in Postorius view the subscriptions waiting user confirmation and delete this one and then `mass subscribe` the user. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

5 years, 3 months

Re: Database Schema - v3.2.1 to v3.3.3

by Mark Sapiro

On 4/4/23 21:28, Dave Hall via Mailman-users wrote: > Hello. > > Have 4 Debian servers running Mailman3 v3.2.1 and sharing the same external > MySql databases for Mailman3 and Mailman3-web. I am in the process of > migrating to v3.3.3. It would be easiest to perform this upgrade one > server at a time, but this would mean that for a short time I would have > v3.2.1 and v3.3.3 using the same databases. > > I am wondering if there are any MySql schema differences between the two > versions. If there are, having the two different versions 'cohabitating' > won't work.. As far as the Mailman3 database is concerned there are at least 3 migrations between 3.2.1 and 3.3.3 in mailman/database/alembic/versions. Namely: 2d2d0ef0828f_add_tag_column.py, 83339e4039da_add_usenet_watermark.py and d151c0b8d6f7_bounce_parameters.py. As far as Mailman3-web is concerned, this is presumably a Debian/Ubuntu Mailman3-web package upgrade. There may be migrations in django_mailman3/migrations/, hyperkitty/migrations/ and postorius/migrations/ that affect this depending on what the specific version updates are in that upgrade. In any case, there are schema differences for Mailman core between 3.2.1 and 3.3.3. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years, 4 months

Re: using SSH/TLS with external MTA

by Roland Giesler

On 2024/07/28 17:06, Mark Sapiro wrote: > On 7/28/24 06:26, Roland Giesler via Mailman-users wrote: >> On 2024/07/28 00:39, Mark Sapiro wrote: >>> >>> What do you see in Mailman's smtp.log and in the mail.log on the >>> mail server? >> >> The smtp log is in syslog... > > The smtp.log I refer to is Mailman's smtp.log which is in Mailman's > var/logs/ directory. There are no entries in the /var/log/mailman3/smtp.log that get created when I try to log in (and authenticate) in Postorius. Jul 28 15:30:18 2024 (516) Available AUTH mechanisms: LOGIN(builtin) PLAIN(builtin) Jul 28 15:30:18 2024 (516) Peer: ('192.168.161.103', 54464) Jul 28 15:30:18 2024 (516) ('192.168.161.103', 54464) handling connection Jul 28 15:30:18 2024 (516) ('192.168.161.103', 54464) EOF received Jul 28 15:30:18 2024 (516) ('192.168.161.103', 54464) Connection lost during _handle_client() Jul 28 15:30:18 2024 (516) ('192.168.161.103', 54464) connection lost Jul 28 15:50:22 2024 (1059) Available AUTH mechanisms: LOGIN(builtin) PLAIN(builtin) Jul 28 15:50:22 2024 (1059) Peer: ('192.168.161.103', 34814) Jul 28 15:50:22 2024 (1059) ('192.168.161.103', 34814) handling connection Jul 28 15:50:22 2024 (1059) ('192.168.161.103', 34814) EOF received Jul 28 15:50:22 2024 (1059) ('192.168.161.103', 34814) Connection lost during _handle_client() The recent attempts I made where after 18:00.

2 years

Mailman-users search results for query "Postorius"