Search results for "Postorius" - Mailman-users

Re: Postorius claims 2 messages held when there are none - out of sync

by Mark Sapiro

On 6/26/23 10:27 PM, Ken Alker wrote: > > So is the only thing stored in var/messages/ the held messages, and > nothing more? Two kinds of messages are in the message store. They are messages which are held for moderator action and bounce DSN messages. Held messages should be removed from the message store when they are handled by a moderator. If there are no held messages in Postorius for any list, any in the message store can be safely removed. Bounce DSNs are only needed for a very short time. They are saved when a bounce is received and removed after a few days by the task runner, but they are only needed until they are attached to the admin notice which happens within minutes, and this was only implemented in Mailman 3.3.5. Prior to that bounce DSNs weren't saved. Probably most if not all of the messages in your message store are not needed and could be deleted, but it's more complicated than that because there are also entries in various database tables that should be removed as well. > Is there an outline as to how to clean up the var/messages hierarchy? > (I assume one has to be careful not to remove current held messages and > maybe old ones that have not been processed; or maybe "old ones" can all > be assumed orphaned, and thus, removed w/o concern?) You could try something like this in mailman shell ``` >>> pendings = getUtility(IPendings) >>> messages = getUtility(IMessageStore) >>> mids = [] >>> for mlist in getUtility(IListManager): ... requests = IListRequests(mlist) ... for request in requests.held_requests: ... key, data = requests.get_request(request.id) ... if data.get('_mod_message_id'): ... mids.append(data.get('_mod_message_id')) ... >>> for key, data in pendings: ... mid = data.get('_mod_message_id') ... if data['type'] != 'probe' and mid and mid not in mids: ... pendings.confirm(key, expunge=True) ... messages.delete_message(mid) ... >>> commit() This will find all the message ids from all held message requests, and then find those pendings for message ids not DSNs and not found in requests and delete the pending an the message store message. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years, 2 months

Re: Splitting hosts: mta&core vs postorius

by Ruth Ivimey-Cook

>> My reason for thinking, in my original post, that the wrong host >> address is being used is that the ConnectionError above contains >> 0.0.0.0 in its url for /3.1/domains/... but 0.0.0.0 is only present as >> the listen address on the core web service and does not appear in the >> config for postorius. > The ConnectionError above comes from requests.request (docs at > <https://requests.readthedocs.io/en/master/>). Client only provides a > URL in the call to request and the URL is the baseurl + the path and the > baseurl is MAILMAN_REST_API_URL which you have as > http://greyarea-post:8870. Entries above have > http://greyarea-post.cam.ivimey.org:8870/. Is that the correct FQDN? > > I suspect a DNS lookup issue. How does greyarea-web1 resolve > greyarea-post? You could try a FQDN that greyarea-web1 can lookup in DNS > rather than relying on /etc/resolv.conf and/or /etc/hosts. No, ignore that... I edited the FQDN down to the hostname in the email for simplicity, but even if it was in the log the hostname does map properly (i.e. host greyarea-post and host greyarea.cam.ivimey.org return the same thing). In referring to the ConnectionError from: 'ConnectionError(MaxRetryError("HTTPConnectionPool(host=\'0.0.0.0\', port=8870): Max retries exceeded with url: /3.1/domains/ch-bc.org.uk (Caused by NewConnectionError(\'<urllib3.connection.HTTPConnection object at 0x7f5416db8518>: Failed to establish a new connection: [Errno 111] Connection refused\',))",),)') if you reconstruct the url passed in it was presumably: "http://0.0.0.0:8870/3.1/domains/ch-bc.org.uk" ... and if that is correct I am not surprised to see an error message. But perhaps that isn't what it was... Ruth

6 years, 3 months

Re: Postorius not HTML-decoding < to "<" characters

by Ron / BCLUG

Mark Sapiro wrote on 2024-11-09 13:05: > This is a HyperKitty issue. Ah, I see, I thought it'd be on the display process. Does that mean the DB has data with mismatched tags in it? If so, will that require a query to fix? Interestingly, the quoted part of my message has < where my message has proper less-than and greater-than for the Apache tags. https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… It's inconsistent with the archive rendering mode at lists.mailman3.org. I haven't currently got an instance that I can test on but will keep this in mind when the time comes to migrate. Should the rendering at lists.mailman3.org get modified, please share the procedure. Testing: less-than and greater-than around a tag: <Location /mailman/accounts/signup> Testing: less-than and greater-than around a tag in a quoted block: > <Location /mailman/accounts/signup>

1 year, 10 months

Announcing Mailman Suite 3.2 release

by Abhilash Raj

I am very pleased to announce the release of Mailman Suite 3.2. This includes Mailman Core 3.2.0, MailmanClient 3.2.0, Postorius 1.2.0, Django-Mailman3 1.2.0, Hyperkitty 1.2.0. This release includes tons of new features and bug fixes. We had support from tons of new contributors and I, on behalf of the whole Mailman Core team, would like to thank each one of you. Mailman Core includes support for plugins, more attributes exposed in REST API and improved support for migrating Mailman 2.x lists. You can now migrate settings and memberships from Mailman 2.x lists to Mailman 3.1. Full change log is available here: https://mailman.readthedocs.io/en/latest/src/mailman/docs/NEWS.html#la-vill… Postorius, The New Mailman Web UI, now has more settings exposed. The most noteworthy of the features include Python 3.5+ support. Also, you can now set templates for headers, footers and automatic responses inside of Postorius. Full change log is available here: https://postorius.readthedocs.io/en/latest/news.html#id1 Hyperkitty, the next-generation mailing-list archiver, has also jumped on the Python 3.5+ wagon and supports all the latest and stable versions of Django. Importing archives from Mailman 2.x is now even more improved. Full change log is available here: https://hyperkitty.readthedocs.io/en/latest/news.html#id1 All the components can be downloaded from https://pypi.org. Mailman is free software for managing email mailing lists and e-newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites. The Full documentation for the Suite is available at: http://docs.mailman3.org/en/latest/ For more information, please see our web site at one of: http://www.list.org https://www.gnu.org/software/mailman http://mailman.sourceforge.net/ https://mirror.list.org/ -- thanks, Abhilash Raj (maxking)

8 years, 2 months

Re: Mailman REST API not available. Please start Mailman core.

by Mark Sapiro

On 7/28/20 1:07 PM, joseverde(a)gmail.com wrote: > [Does your systemd script set the user to the Mailman user, and does the > Mailman user have read/write permission on Mailman's var/locks/?] > > How do I check for that? ls -la /path/to/mailman/var/locks > Also, what's the difference between the mailman command and the systemctl mailman3? Are they the same processes? Presumably your mailman3.service actually uses the `mailman start` command to start Mailman. The `mailman` command is a shell with many subcommands for various purposes including starting and stopping Mailman. > Oh and does the /3.1 matter at the end of the Rest url for Django? Do not put the 3.1 at the end of your MAILMAN_REST_API_URL. Postorius adds it. See <https://gitlab.com/mailman/postorius/-/blob/master/src/postorius/utils.py#L…>. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years, 1 month

Re: Things I miss about mailman2

by Quanah Gibson-Mount

--On Wednesday, March 18, 2020 2:58 AM -0700 Mark Sapiro <mark(a)msapiro.net> wrote: > This was implemented by > <https://gitlab.com/mailman/postorius/-/merge_requests/397> and released > in Postorius 1.3.0. Ok, that would be the issue... Debian 10 only has 1.2.4-1 bundled. I'll see if I can find a backport for 1.3. >>> This was not "removed". It just hasn't been implemented in MM 3 yet. >> >> So in terms of available functionality that was present in MM2, it was >> removed. It may need to be re-implemented, but this was a core feature >> that was present in the prior version that is no longer available. > > > Yes, we are aware of many things that are not yet exposed/implemented in > Postorius and elsewhere. There are currently 580 open issues at > <https://gitlab.com/groups/mailman/-/issues>. We are working on these > things as we can. We would love to have your help in fixing these. In > the mean time, there are really only 2 or 3 core developers working on > this and we are all volunteers, so it takes time. I fully understand and would love to contribute. The problem being that I already am doing this for multiple other projects where we're facing the same issue -- Lots of people use the software and there are the occasional contributions, but few people who are willing to be core members to really help drive the projects, and I can only drive so many and do my paying job. :/ Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

6 years, 6 months

Re: Error -- postorius Un-handled exception

by Ron / BCLUG

Mark Sapiro wrote on 2024-04-10 14:09: > I suspect you are not starting the services with your venv active. > What's in your mailman3.service and mailman3-web.service files? Thanks for the pointer! I had my mailman3-web.service starting `mailman-web` instead of `uwsgi` somehow. I changed that to match the docs, did the `systemctl daemon-reload`, etc. but still had similar problems. I went through all my config files carefully, I wiped the static folder completely and rebuilt it, and I finally have the last piece of the puzzle in place and my web interface is working! I'm not sure which change actually fixed the issue though. (When looking at "Available lists" my header has "example.com" in the top-left corner?!?) Thank you! I think I need to backup what I have and do a clean re-install so I have a better grasp of what's going on with all these components. Mailman3 looks quite nice, gotta say! rb

2 years, 5 months

Re: Postorius confirmation mails shown in uwsgi-error.log but not sent

by Ralph Seichter

* Mark Sapiro: >> $ grep ^EMAIL settings.py >> EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' >> EMAIL_HOST = 'localhost' >> EMAIL_PORT = 25 > > but is this perhaps overridden in a settings_local.py file? I don't have a local settings file. I cloned the suite from GitLab[1] master branch and modified the existing settings.py, based on Mailman's documentation. [1] https://gitlab.com/mailman/mailman-suite.git I just verified that EMAIL_BACKEND is not redefined anywhere: $ grep -r 'mail\.backends' . Binary file ./__pycache__/settings.cpython-38.pyc matches ./settings.py:EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' ./settings.py:# EMAIL_BACKEND = 'django.core.mail.backends.filebased.EmailBackend' I commented out the "if DEBUG == True" lines, which explains match number three. As far as I can see, the mail backed definition for Django is as it should be. I could provide the differences between my settings.py and the Git HEAD version if you think that might help. -Ralph

6 years, 5 months

Mailman3 Issue

by Danny Weddle

For some weird reason, the file /mailman/var/data/postfix_domains has a whitespace added to every mail domain. I checked the postfix main.cf file for any white spaces, but did not see any. I get errors every time that I try to add a list. The console shows whitespace " example.org" in the list. The web page shows: Environment: Request Method: GET Request URL: http://172.16.11.235/postorius/lists/list11.mail.example2.com/ Django Version: 2.0.5 Python Version: 3.6.5 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'paintstore', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora', 'allauth.socialaccount.providers.openid', 'allauth.socialaccount.providers.github', 'allauth.socialaccount.providers.gitlab', 'allauth.socialaccount.providers.google', 'allauth.socialaccount.providers.twitter', 'allauth.socialaccount.providers.stackexchange') Installed Middleware: ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware'] Traceback: File "/usr/local/lib64/python3.6/site-packages/django/core/handlers/exception.py" in inner 35. response = get_response(request) File "/usr/local/lib64/python3.6/site-packages/django/core/handlers/base.py" in _get_response 128. response = self.process_exception_by_middleware(e, request) File "/usr/local/lib64/python3.6/site-packages/django/core/handlers/base.py" in _get_response 126. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib64/python3.6/site-packages/django/views/generic/base.py" in view 69. return self.dispatch(request, *args, **kwargs) File "/home/weddled/mailman/postorius/src/postorius/views/generic.py" in dispatch 57. return super(MailingListView, self).dispatch(request, *args, **kwargs) File "/usr/local/lib64/python3.6/site-packages/django/views/generic/base.py" in dispatch 89. return handler(request, *args, **kwargs) File "/home/weddled/mailman/postorius/src/postorius/views/list.py" in get 219. 'hyperkitty' in self.mailing_list.archivers and File "/usr/lib64/python3.6/_collections_abc.py" in __contains__ 666. self[key] File "/usr/local/lib/python3.6/site-packages/mailmanclient-3.2.0b1-py3.6.egg/mailmanclient/restbase/base.py" in __getitem__ 151. return self._get(key) File "/usr/local/lib/python3.6/site-packages/mailmanclient-3.2.0b1-py3.6.egg/mailmanclient/restbase/base.py" in _get 90. return self.rest_data[key] File "/usr/local/lib/python3.6/site-packages/mailmanclient-3.2.0b1-py3.6.egg/mailmanclient/restbase/base.py" in rest_data 76. response, content = self._connection.call(self._url) File "/usr/local/lib/python3.6/site-packages/mailmanclient-3.2.0b1-py3.6.egg/mailmanclient/restbase/connection.py" in call 103. raise HTTPError(url, response.status, content, response, None) During handling of the above exception (HTTP Error 500: b'A server error occurred. Please contact the administrator.'), another exception occurred: File "/usr/local/lib64/python3.6/site-packages/django/core/handlers/exception.py" in inner 35. response = get_response(request) File "/usr/local/lib64/python3.6/site-packages/django/utils/deprecation.py" in __call__ 95. response = self.get_response(request) File "/usr/local/lib64/python3.6/site-packages/django/core/handlers/exception.py" in inner 37. response = response_for_exception(request, exc) File "/usr/local/lib64/python3.6/site-packages/django/core/handlers/exception.py" in response_for_exception 87. response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info()) File "/usr/local/lib64/python3.6/site-packages/django/core/handlers/exception.py" in handle_uncaught_exception 122. return debug.technical_500_response(request, *exc_info) File "/usr/local/lib64/python3.6/site-packages/django/views/debug.py" in technical_500_response 89. reporter = ExceptionReporter(request, exc_type, exc_value, tb) File "/usr/local/lib64/python3.6/site-packages/django/views/debug.py" in __init__ 254. self.template_info = getattr(self.exc_value, 'template_debug', None) File "/usr/lib64/python3.6/tempfile.py" in __getattr__ 477. file = self.__dict__['file'] Exception Type: KeyError at /postorius/lists/list11.mail.example2.com/ Exception Value: 'file'

8 years, 4 months

Postorius extremely slow on some of the mailinglists

by Stephen J. Turnbull

Eric Broens via Mailman-users writes: > Mainly "GET /3.1/lists/<mailing_list>@<domain>/requests HTTP/1.1" > takes a long time. The only thing I can think of is that something weird is happening with requests (held messages) on those lists. The runner is waiting on the database server, and eventually timing out, I think. I believe you will find the corresponding messages under $var_dir/messages (var_dir is usually set in mailman.cfg, if not I believe it defaults to /usr/local/mailman3/var). Unfortunately I don't know how to link those entries to the messages in that directory tree. You can "grep -ri '^list-id:.*<your.list.id>$' messages" for held posts via the problem list if you have list ID on (that's the default for recent Mailman) and if that's not going to work you could use "grep -rF 'your(a)list.id' messages" for a less precise search. It's not a good idea to delete them from the file system (I think Mailman is robust to that but I'm not sure). However, if one looks like spam you could try working at a slightly lower level than Postorius by using mailmanclient directly. See https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/…. If that doesn't work (for example listing the requests for a list times out the same way in mailmanclient), you could try directly querying the database server to see what's going on. The check for an unusual number of requests that might be an attack is to connect to the database server with its command line utility. I'm not on CLI terms with MySQL but for PostgreSQL it's $ su - mailman $ psql mailman=> select mailing_list_id, count(mailing_list_id) from _request group by mailing_list_id; mailman=> select id, list_id from mailinglist; and you can append " where id = $mailing_list_id" to the second to the second one to find the List-Id for a specific mailing list. That's just plain SQL including the count function as far as I know, so should work in any backend that Mailman can use. If you're handy with SQL you could do all that with a join but I'm not. ;-) > Any idea why this happens and how it can be solved? Aside from a really astonishingly large number of pending requests (as tested above with count()), perhaps there's something in the requests themselves. If there aren't literally thousands of requests for a problem list you can list them with mailman=> select key, mailing_list_id from _request; Look for odd characters in the key field. One oddity that I'mm pretty sure is not a problem is something of the form "\r\n <message-id>" (my instance lists those fine). There's a bug in Exchange (IIRC) which sometimes includes invalid characters (specifically "[" and "]") in message IDs which I believe we worked around only quite recently, although I don't recall if it affected APIs for requests. Control characters and non-ASCII might also cause trouble, If your database server is MySQL, is it set up to handle the full range of Unicode (I think the option is utf8mbcs)? Lacking that is known to cause problems when headers and maybe body contain emoticons or other extended Unicode characters. Again I don't recall if crashing the REST runner is a known symptom. If none of the above works, there's always the "hit it with a hammmer" approach. Have you tried restarting both Postorius and Mailman? How about restarting the database server? How about all three? As a last resort, it's probably possible to delete the requests from the filesystem and the database by hand. But I don't want to think about that unless absolutely necessary. Steve

1 year, 7 months

Mailman-users search results for query "Postorius"