Search results for "Postorius" - Mailman-users

Re: Held for notification - runner exception: 'ascii' codec can't encode character

by Seth Seeger

> On Jan 2, 2022, at 3:47 PM, Mark Sapiro <mark(a)msapiro.net> wrote: > > Thanks for sending me the shunted .pck. It turns out I didn't need it. Had I looked more closely at the Traceback, I would have realized that the issue is that the list's list:admin:action:post template contains non-ascii. In particular, the character in position 5 is ’ which is a unicode right_single_quote, not an ascii apostrophe. > > There may be other unicode fancy quotes ad well. > > You can fix the template to contain only ascii or you can set the character set for English to utf-8 by adding > ``` > [language.en] > charset: utf-8 > ``` > to mailman.cfg. The downside of the latter is that outgoing message bodies will always be base64 encoded which generally precludes grepping mailboxes to find things. Ah hah! Right you are. I removed the offending fancy apostrophe. What about that traceback pointed to the template? (Is it something I could have figured out?) Is it possible to filter out UTF-8 chars from the templates when they are saved in Postorius? Thanks again, Seth

4 years, 3 months

Re: MM3 Docker

by Odhiambo Washington

On Tue, Jan 8, 2019, 20:34 Abhilash Raj <maxking(a)asynchronous.in wrote: > On Tue, Jan 8, 2019, at 5:52 AM, Odhiambo Washington wrote: > > On Tue, 8 Jan 2019 at 03:29, Mark Sapiro <mark(a)msapiro.net> wrote: > > > > > On 1/7/19 1:18 AM, Odhiambo Washington wrote: > > > > On Sun, 6 Jan 2019 at 23:39, Mark Sapiro <mark(a)msapiro.net> wrote: > > > >> > > > >> Please help me understand what steps are missing and I will try to > > > >> provide the information. > > > > > > > > > > > > Okay. > > > > > > > > At the installation stage, you say the following: > > > > "I already had some things set up in /opt/mailman including a git > > > > subdirectory containing clones of the GitLab mailman, mailmanclient, > > > > mailman-hyperkitty, hyperkitty, django-mailman3 and > > > > postorius projects. On the third server I have added mailman-suite > > > > because I use the settings.py from that project as the basis for > mine." > > > > > > > > Please assume that the reader is a newbie to python3 virtualenv and > give > > > the > > > > details of the commands they need to execute to (a) clone all the > > > projects > > > > needed > > > > and (b) create the virtualenv. > > > > > > > > The next steps are: > > > > mkdir /opt/mailman/mm > > > > > > > > Then you say "I then created a /opt/mailman/mm directory and within > that > > > a > > > > Python 3.6 virtualenv" > > > > What happens here is something that needs clarification, because you > are > > > > specific about "python3.6". > > > > I have googled and realized that one has to be specific about the > python > > > > version they want to use in their virtualenv > > > > as shown here -> > > > > > > > > https://stackoverflow.com/questions/45293436/how-to-specify-python-version-… > > > > > > > > exact command to execute to create the /opt/mailman/mm/venv > > > > exact command to execute to activate virtualenv > > > > > > > > > I have added a comment to the page which I hope gives you what you > want. > > > > > > I also added a section on templates instead of referring to a prior > > > version of the page. > > > > > > > That makes it better. > > However, as a newbie, something is not yet clear: > > > > "When active in a particular shell, it can be deactivated with the > command > > > > deactivate > > or by exiting that shell." > > > > HI am trying to understand the effect of detaching from a venv. I am > > assuming that a virtualenv is a container that is supposed to be always > > active for the services within to be accessible. Is that right? So, how > do > > I ensure it's always running/up, or that does not matter as the scripts > > which run the services will always refer to the files? > > virtualenv is not actually a container that is supposed to be active. It > is merely a set of settings that are applied to your SHELL when you are > in the virtual environment. > > It is not supposed to be always active, only when you actually want to > run commands in that virtual environment. The services will keep on > running after you deactivate your shell and would function like other > normal services on your machine. > > -- > thanks, > Abhilash Raj (maxking) > Thank you for clarifying. So once I have a venv created, the services will just run as long as the startup scripts are running even when I'm logged out of the system? >

7 years, 3 months

Today no question, but my toolbox: set "global" user preferences; list or delete held messages

by Wikinaut

After some (sorry for bothering you) questions here, I was able to set up a mailman3 system which I regard as perfect for my use. Perhaps, you had similar problems with your installation, especially when coming from mailman2 and, like me, from majordomo...... I collected a lot of information which was presented in this mailling list archive here and also answers of my questions (thanks Mark and others) News: https://github.com/Wikinaut/mailman3-tools has all of these tools. README says: Scripts for maintenance or setting-up of modern mailman3 instances set preferences for all users list held messages delete held messages older n days My setting is strictly as described in the documentation: mailman3 + postorius/hyperkitty are installed via pip in an (venv) /opt/mailman/ My user scripts and a huge mailing list generator (not published here) are in a directory apart from that: user maillistgen home directory /home/maillistgen user maillistgen has some sudo rights set uo in /etc/sudoers a wrapper script is /home/maillistgen/run_mailman.sh which sets the correct path for Python, mailman3 and the (venv). I hope, you like my tiny part of contribution and attribution back to Mark.

8 months

Today no question, but my toolbox: set "global" user preferences; list or delete held messages

by Wikinaut

After some (sorry for bothering you) questions here, I was able to set up a mailman3 system which I regard as perfect for my use. Perhaps, you had similar problems with your installation, especially when coming from mailman2 and, like me, from majordomo...... I collected a lot of information which was presented in this mailling list archive here and also answers of my questions (thanks Mark and others) News: https://github.com/Wikinaut/mailman3-tools has all of these tools. README says: Scripts for maintenance or setting-up of modern mailman3 instances set preferences for all users list held messages delete held messages older n days My setting is strictly as described in the documentation: mailman3 + postorius/hyperkitty are installed via pip in an (venv) /opt/mailman/ My user scripts and a huge mailing list generator (not published here) are in a directory apart from that: user maillistgen home directory /home/maillistgen user maillistgen has some sudo rights set uo in /etc/sudoers a wrapper script is /home/maillistgen/run_mailman.sh which sets the correct path for Python, mailman3 and the (venv). I hope, you like my tiny part of contribution and attribution back to Mark.

8 months

Re: Hyperkitty fulltext search performances

by Mark Dadgar

On Oct 4, 2020, at 8:13 AM, Mark Sapiro <mark(a)msapiro.net> wrote: > On 10/4/20 8:07 AM, Brian Carpenter wrote: >> >> On 10/4/20 8:59 AM, Loïc Dachary wrote: >>> After a little digging it turns out the mailman3-web Debian GNU/Linux >>> package I'm using is configured to use Woosh as a haystack backend. >>> And it is not fit for volumes greater than a few hundred mega bytes. I >>> should switch to something else from the list of supported >>> backendshttps://django-haystack.readthedocs.io/en/master/backend_support.html >>> >> For our servers running Postorius/Hyperkitty, we use Xapian. I believe >> the mailman-users (MM3) list also uses Xapian. It is a lot quicker. For >> our Affinity/Empathy servers, we use Elasticsearch which we are very >> happy with. Elasticsearch does have a higher memory requirement I >> believe than Xapian. > > > Brian is correct about this list. We started with Whoosh, then went to > Elasticsearch and ultimately to Xapian. Each step gave better > performance and a smaller footprint than the one before. +1 I have ~700K posts in my archive and Xapian is FAST. - Mark ----- mark(a)pdc-racing.net | 408-348-2878

5 years, 6 months

Internal Server Error: /mailman3/hyperkitty/api/mailman/archive

by Mail Person

Hello everyone. I am trying to sort out an error happening when messages are sent to the mailman server. The emails are being sent to the list but I get this error each time: Internal Server Error: /mailman3/hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1") Request Method: POST Request URL: http://localhost/mailman3/hyperkitty/api/mailman/archive?key=7OF5IogiQItSUK… Django Version: 2.2.12 Python Executable: /usr/bin/uwsgi-core Python Version: 3.8.10 Python Path: ['.', '', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Thu, 17 Aug 2023 17:13:09 +0000 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Traceback: File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 71. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 209. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 315. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 226. _mysql.connection.query(self, query) The above exception ((1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1")) was the direct cause of the following exception: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 115. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 113. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py" in inner 40. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in _decorator 71. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py" in wrapped_view 54. return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in archive 117. add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py" in add_to_list 153. email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save 740. self.save_base(using=using, force_insert=force_insert, File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save_base 777. updated = self._save_table( File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _save_table 870. result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _do_insert 907. return manager._insert([self], fields=fields, return_id=update_pk, File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_method 82. return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py" in _insert 1186. return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py" in execute_sql 1375. cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 67. return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute_with_wrappers 76. return executor(sql, params, many, context) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py" in __exit__ 89. raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute 84. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 71. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 209. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 315. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 226. _mysql.connection.query(self, query) Exception Type: OperationalError at /hyperkitty/api/mailman/archive Exception Value: (1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1") Request information: USER: AnonymousUser

2 years, 8 months

Re: Can't deliver mails in queue

by Mark Sapiro

On 5/20/21 4:01 AM, Nathanael Schneider wrote: > > It would have been nice though to have a more refined error to figure > this out. But its solved now and hopefully this will help someone else > as well. I'm glad you resolved it, and I'm sorry I didn't consider the template issue myself. You actually had two separate template issues. If you created templates in Postorius, they were inaccessible because of the POSTORIUS_TEMPLATE_BASE_URL issue, but your initial problem with imported lists is different. It probably resulted from running the `mailman import21` command as root which you should never do. You should always run `mailman` commands as the Mailman user. This is mentioned at https://docs.mailman3.org/en/latest/migration.html#id1 The `mailman import21` command can create list specific templates in Mailman's var/templates/lists/LIST_ID/LC/ directory and if run as root these will be user:group root:root and not world readable and therefore can't be read by Mailman. Also, deleting and recreating the list with the same LIST_ID doesn't remove those templates, so the list still doesn't work. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 11 months

Re: modify subscription page by mailinglist?

by Stephen J. Turnbull

Lars Schimmer writes: > And I think I could fiddle out how it works. But in general, e.g. a > custom page per mailinglist for the "list information page" is a bit > more work than expected and I will not follow this idea. > (e.g. adding a logo per list, ...) Just adding a list-specific logo is not hard. Choose a naming convention (probably something like f"logo-{list_id}.png"), put the logo PNG in one of the static directories and add <IMG SRC="logo-{{list_id}}.png" ALT="Logo for {{list_id}} list." /> in the right place in each template (maybe even "base.html", but more likely an intermediate template specifically to allow putting such customizations in one place for list-specific pages). Adding the ability for a list admin to easily upload a file and set the logo to it would require surgery to Postorius's Python code, though. I will admit that though I think this is a cool idea, I am not interested in doing it myself. But if your interest is at the "I'd rather wait a few months than do myself", GSoC is coming up again, and you could lobby for it when the students start applying. ;-) Steve

4 years, 3 months

Re: Exception with processing subscription confirmation email

by Eric Broens

Thanks Mark, We are indeed running mailman core 3.3.2 and using mysql. The data appears indeed to be truncated judging from the missing closing curly bracket (see below). Would it be OK to already extend the data column to VARCHAR(510) before upgrading to mailman core 3.3.3, or will this cause problems with upgrading to a newer version? +--------------------------------------------------------------------------- ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- ------------------------------+ | data | +--------------------------------------------------------------------------- ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- ------------------------------+ | {"pre_approved": false, "pre_confirmed": false, "pre_verified": false, "invitation": false, "address_key": "******************@************", "subscriber_key": 1, "user_key": "a44e8b9a34c64775b712f383d438ce1d", "token_owner_key": 1, "send_welcome_message" | +--------------------------------------------------------------------------- ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- ------------------------------+ Best Regards, Eric -----Original Message----- From: Mark Sapiro [mailto:mark@msapiro.net] Sent: donderdag 29 april 2021 18:43 To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: Exception with processing subscription confirmation email On 4/29/21 7:56 AM, Eric Broens wrote: > Hi, > > A new user tried to subscribe to a mailinglist, received the confirmation > mail, and replied to it. > > The subscriber is however not subscribed and remains listed in the "pending > confirmation" list. > > The following exception appears in the error.log. > > Does anyone knows how to resolve this? In the past we didn't experience any > issues and no changes but security updates have been installed. > > Apr 29 16:15:55 2021 (1581) deque: do_confirm_verify > > Traceback (most recent call last): > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/workflow.py", > line 69, in __next__ > > return step() > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/subscriptions.p > y", line 356, in _step_send_confirmation > > raise StopIteration > > StopIteration Tracebacks like the above are normal and probably shouldn't be logged at all. > Apr 29 16:16:28 2021 (1490) Uncaught runner exception: Expecting ':' > delimiter: line 1 column 256 (char 255) > > Apr 29 16:16:28 2021 (1490) Traceback (most recent call last): > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py", > line 173, in _one_iteration > > self._process_one_file(msg, msgdata) > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py", > line 266, in _process_one_file > > keepqueued = self._dispose(mlist, msg, msgdata) > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/runners/command.py" > , line 208, in _dispose > > mlist, msg, msgdata, parts, results) > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/commands/eml_confir > m.py", line 54, in process > > mlist).confirm(token) > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/subscriptions.p > y", line 578, in confirm > > workflow.restore() > > File > "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/workflow.py", > line 146, in restore > > data = json.loads(state.data) > > File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads > > return _default_decoder.decode(s) > > File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode > > obj, end = self.raw_decode(s, idx=_w(s, 0).end()) > > File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode > > obj, end = self.scan_once(s, idx) > > json.decoder.JSONDecodeError: Expecting ':' delimiter: line 1 column 256 > (char 255) > > Apr 29 16:16:28 2021 (1490) SHUNTING: > 1619705788.5856323+57a1a759275c59b732cfa8e2985f76cfcc27dda5 This one is the issue. There was some problem with the user's confirmation reply. View the shunted message with the command ``` mailman qfile path/to/var/queue/shunt/1619705788.5856323+57a1a759275c59b732cfa8e2985f76cfc c27dda5.pck ``` although that probably won't show anything wrong as it appears that the issue is with the saved workflow. You can get the token from the shunted message and then look at the database with something like ``` SELECT data FROM workflowstate WHERE token = '<the token>' ``` but I suspect the issue is because the database is MySQL or MariaDB, your Mailman core is older than 3.3.3 and your database server mode is such that data too long for column results in truncation, and the data in this case is truncated to 255 bytes. In Mailman 3.3.3, this column was changed from VARCHAR(255) to VARCHAR(510) for MySQL. As far as subscribing the user is concerned, you can in Postorius view the subscriptions waiting user confirmation and delete this one and then `mass subscribe` the user. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

4 years, 11 months

Re: Database Schema - v3.2.1 to v3.3.3

by Mark Sapiro

On 4/4/23 21:28, Dave Hall via Mailman-users wrote: > Hello. > > Have 4 Debian servers running Mailman3 v3.2.1 and sharing the same external > MySql databases for Mailman3 and Mailman3-web. I am in the process of > migrating to v3.3.3. It would be easiest to perform this upgrade one > server at a time, but this would mean that for a short time I would have > v3.2.1 and v3.3.3 using the same databases. > > I am wondering if there are any MySql schema differences between the two > versions. If there are, having the two different versions 'cohabitating' > won't work.. As far as the Mailman3 database is concerned there are at least 3 migrations between 3.2.1 and 3.3.3 in mailman/database/alembic/versions. Namely: 2d2d0ef0828f_add_tag_column.py, 83339e4039da_add_usenet_watermark.py and d151c0b8d6f7_bounce_parameters.py. As far as Mailman3-web is concerned, this is presumably a Debian/Ubuntu Mailman3-web package upgrade. There may be migrations in django_mailman3/migrations/, hyperkitty/migrations/ and postorius/migrations/ that affect this depending on what the specific version updates are in that upgrade. In any case, there are schema differences for Mailman core between 3.2.1 and 3.3.3. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years

Mailman-users search results for query "Postorius"