Search results for "sapiro" - lists.mailman3.org

[MM3-users] Re: Mailman3 throws 500 error (unicode encode) when loading held messages in Postorius

by Mark Sapiro

On 8/27/20 5:44 PM, Simon Handfield via Mailman-users wrote: > > Traceback (most recent call last): > File "/usr/lib/python3.6/wsgiref/handlers.py", line 137, in run > self.result = application(self.environ, self.start_response) > File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper > rtn = function(*args, **kws) > File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 214, in __call__ > return super().__call__(environ, start_response) > File "falcon/api.py", line 215, in falcon.api.API.__call__ (falcon/api.c:2872) > File "falcon/api.py", line 189, in falcon.api.API.__call__ (falcon/api.c:2419) > File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 167, in on_get > resource = self._make_collection(request) > File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in _make_collection > for resource in collection] > File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in <listcomp> > for resource in collection] > File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 157, in _resource_as_dict > resource = self._make_resource(request.id) > File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 78, in _make_resource > resource['msg'] = msg.as_string() > File "/usr/lib/python3.6/email/message.py", line 158, in as_string > g.flatten(self, unixfrom=unixfrom) > File "/usr/lib/python3.6/email/generator.py", line 116, in flatten > self._write(msg) > File "/usr/lib/python3.6/email/generator.py", line 181, in _write > self._dispatch(msg) > File "/usr/lib/python3.6/email/generator.py", line 214, in _dispatch > meth(msg) > File "/usr/lib/python3.6/email/generator.py", line 272, in _handle_multipart > g.flatten(part, unixfrom=False, linesep=self._NL) > File "/usr/lib/python3.6/email/generator.py", line 116, in flatten > self._write(msg) > File "/usr/lib/python3.6/email/generator.py", line 181, in _write > self._dispatch(msg) > File "/usr/lib/python3.6/email/generator.py", line 214, in _dispatch > meth(msg) > File "/usr/lib/python3.6/email/generator.py", line 243, in _handle_text > msg.set_payload(payload, charset) > File "/usr/lib/python3.6/email/message.py", line 315, in set_payload > payload = payload.encode(charset.output_charset) > UnicodeEncodeError: 'ascii' codec can't encode characters in position 8-9: ordinal not in range(128) > > I'm sorry if this is a query that's answered already but I have not been able to find matching thread that seems to relate to my error. Sorry for not answering earlier and for not being able to devote more time to this now as I'm going away within the hour, but this looks as if the message is somehow defective. Basically, the message can't be flattened by its as_string() method because of unicode issues. This can occur because of non-ascii characters is the message headers or in body parts explicitly or implicitly declared as ascii. We have had workarounds for this for some time, but I don't think they are in 3.1.1, See <https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/email/message.…> -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years, 11 months

[MM3-users] Re: Held messages: Long time waiating for response of the Mailman API

by Mark Sapiro

On 1/29/22 05:48, Jacob Sievert via Mailman-users wrote: > Hello, > > we seem to have the same problem right now. > > We are still on python3.6 and postgresql 12.9 > > I looked at our Table pendedkeyvalue and we have roughly 35k rows in > that table, > is that normal or shouldn't those get cleaned up after a while? Yes they should. This is https://gitlab.com/mailman/mailman/-/issues/257 fixed in Mailman core 3.3.5. Also in Mailman 3.3.5 is a new Task runner that runs periodic tasks, one of which is to remove orphaned pendings. This may also be the issue for the OP in this thread if that pendedkeyvalue table is also large. Here is a `mailman shell` script that will clean that up. ``` # Prior to Mailman 3.3.5, some tokens for user confirmations were pended with # too long a lifetime. This script removes those pendings based on when they # were pended and the configured pending_request_life rather than their # expiration. # Also prior to Mailman 3.3.5, pended held_message tokens for email handling # of the message were not removed when the message was handled via REST. This # script removes those pendings too. # This is run with # mailman shell -r delete_orphans_expireds # after saving it as # /opt/mailman/mm/venv/bin/delete_orphans_expireds.py from datetime import datetime from lazr.config import as_timedelta from mailman.config import config from mailman.database.transaction import transactional from mailman.interfaces.pending import IPendings from zope.component import getUtility pendings = getUtility(IPendings) def is_request(id): if config.db.store.execute( 'SELECT * FROM _request WHERE id = {};'.format(id)).rowcount > 0: return True return False then = datetime.now() - as_timedelta(config.mailman.pending_request_life) thenm = datetime.now() - as_timedelta(config.mailman.moderator_request_life) def fromisoformat(x): if hasattr(datetime, 'fromisoformat'): return datetime.fromisoformat(x) try: return datetime.strptime(x, '%Y-%m-%dT%H:%M:%S.%f') except ValueError: return datetime.strptime(x, '%Y-%m-%dT%H:%M:%S') @transactional def delete_orphans_expireds(): count = 0 for token, data in pendings.find(pend_type='held message'): if data and not is_request(data['id']): result = pendings.confirm(token, expunge=True) count += 1 print(f'expunged {count} orphaned pended held messages') count = 0 for token, data in pendings.find(pend_type='data'): if data and data['_mod_hold_date']: when = data['_mod_hold_date'] if isinstance(when, str): when = fromisoformat(when) if when < thenm: result = pendings.confirm(token, expunge=True) count += 1 print(f'expunged {count} expired held messages') pends = list(pendings.find(pend_type='subscription')) pends += list(pendings.find(pend_type='unsubscription')) count = 0 for token, values in pends: if values and values['token_owner'] == 'subscriber': when = values['when'] if isinstance(when, str): when = fromisoformat(when) if when < then: result = pendings.confirm(token, expunge=True) count += 1 print(f'expunged {count} expired (un)subscription confirmations') ``` The above says to save the script at /opt/mailman/mm/venv/bin/delete_orphans_expireds.py but that path may need to be adjusted based in where Mailman's bin/ directory is in your installation. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 6 months

[MM3-users] Re: installing a new instance of mm3

by Christian Stalberg

mailman.log is showing this: ERROR 2021-10-18 22:19:12,385 616 django.security.DisallowedHost Invalid HTTP_HOST header: 'subdomain.domain.tld'. You may need to add 'subdomain.domain.tld' to ALLOWED_HOSTS. ERROR 2021-10-18 23:01:15,318 616 django.security.DisallowedHost Invalid HTTP_HOST header: '192.168.0.1'. You may need to add '192.168.0.1' to ALLOWED_HOSTS. Where do those entries go? -----Original Message----- From: Mark Sapiro <mark(a)msapiro.net> Sent: Friday, October 22, 2021 11:44 AM To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: installing a new instance of mm3 On 10/22/21 6:37 AM, Christian Stalberg via Mailman-users wrote: > Is this document that Brian Carpenter wrote still valid? > > https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10 As Steve said, it should be. > Brian installed my instance. Now its broken. My lists are down. I'm > getting nowhere with diagnostics and troubleshooting (which was easier > with mailman 2). What's broken? Postorius, Hyperkitty, mail delivery, all of them? Do you find any clues in the logs at /opt/mailman/mm/var/logs/? Particularly mailman.log for core and mailmansuite.log for Postorius and Hyperkitty. We will try to help if we have information to help identify the problem. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

4 years, 9 months

[MM3-users] Re: Mailman3 Docker question and importing lists from MM2

by Andrew Hodgson

bin/mailman import21 I saw this in the image, do you have any idea about what may be missing from this? I may shelve this project again for the time being if we can't do the migration, but good to know I can get a production like Docker instance running relatively painlessly. Andrew. -----Original Message----- From: Mark Sapiro [mailto:mark@msapiro.net] Sent: 01 September 2017 16:43 To: mailman-users(a)mailman3.org Subject: [MM3-users] Re: Mailman3 Docker question and importing lists from MM2 On 09/01/2017 07:09 AM, Andrew Hodgson wrote: > > I had a question regarding the parameter: SERVE_FROM_DOMAIN. > > I want to use multiple domains, what is the best way to specify multiple domains in this environment variable? I don't know the answer to this. > Furthermore, is there a recommended way to get data from MM2 (list configs and archives) into MM3? There are two things. For importing list configs there is bin/mailman import21 but it is not up to date and probably doesn't work. For importing archives there is django-admin hyperkitty_import [--settings=if.needed] which works well using the MM 2.1 LISTNAME.mbox/LISTNAME.mbox file as input. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list mailman-users(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

8 years, 11 months

[MM3-users] Re: Mail configuration

by Darren Smith

Mark, This is Mailman3. On Mon, Mar 26, 2018 at 1:53 PM, Mark Sapiro <mark(a)msapiro.net> wrote: > On 03/23/2018 10:05 AM, Darren Smith wrote: > > > > We have been told that "warming up" the IP is the way to go - that is, > > cutting over just some of the traffic a little at a time. So what I am > > looking for is a way for all mail on our domain to be sent to our current > > mailman installation, and based on some configuration (certain mailing > > lists?) certain emails being sent out to the list subscribers will first > be > > forwarded to the new server and be sent from the new IP address. > > > > Does that make sense? > > > > Does anyone know a way that mailman/postfix can be configured to > accomplish > > this? > > > Is this Mailman 3 or Mailman 2.1? The Mailman side of this would be > different depending on which. If it's Mailman 2.1, a better list is > mailman-users(a)python.org > <https://mail.python.org/mailman/listinfo/mailman-users> > > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

8 years, 4 months

[MM3-users] Re: Core, Hyperkitty and Postorius releases

by Odhiambo Washington

On Fri, May 14, 2021 at 11:13 PM Mark Sapiro <mark(a)msapiro.net> wrote: > On 5/14/21 12:50 PM, Odhiambo Washington wrote: > > On Fri, May 14, 2021 at 7:04 PM Mark Sapiro <mark(a)msapiro.net> wrote: > > > >> On 5/14/21 5:02 AM, Odhiambo Washington wrote: > >>> On Fri, May 14, 2021 at 2:52 PM Jeff Steele <jeff(a)steele.com> wrote: > >>> > >>>> > >>>> > >>>> On May 14, 2021, at 7:49 AM, Odhiambo Washington <odhiambo(a)gmail.com> > >>>> wrote: > >>>> > >>>> /bin/sh: sass: Permission denied* > ... > >> Try `which sass` > >> > > > > root@gw:/usr/home/wash # su mailman3 > > (venv) [mailman3@gw /usr/home/wash]$ which sass > > (venv) [mailman3@gw /usr/home/wash]$ > > > > There is no sass. > > > There has to be. You can't get a permissions error on sass if it doesn't > exist (well, maybe if you don't have permission to search a > superordinate directory, but that's unlikely). > > Try `find /usr -name sass` > /usr/local/include/sass That is the only thing found that is in the base system. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v "^$|^.*#" :-)

5 years, 3 months

[MM3-users] Re: converting bulk accept_these_nonmembers in migration from mailman 2 to 3

by Mark Sapiro

On 2/8/22 09:01, Lucio Chiappetti wrote: Snipping to what I thing are the issues. > What I found now (after the migration done by the administrators of the > central seat) is that in mailman3: > > - accept_these_nonmembers is empty > - there are 189 entries listed in Non-members (which look those > formerly in accept_these_nonmembers) And those non-members all have there moderation action set to Defer which means their posts will be accepted but the additional checks such as too big, etc. will still be applied. > - hold_these_nonmembers was copied identical with the same regexp's > as in mailman2 This is all expected. MM 3, unlike MM 2.1 has a concept of nonmembers with moderation action, and this is intended to replace *_these_nonmembers. The legacy *_these nonmembers attributes still exist to support regexps. This is why the addresses were made nonmembers with the appropriate moderation action, but the regexps were retained. > - I have inspected a few of them and found that they have > Administration options Moderation set to List default I'm not sure what you are looking at here. Imported nonmembers should all have moderation action set according to which *_these_nonmembers they came from. Imported members should have their moderation action set to Defer if they were unmoderated and if they were moderated, it should be set based on the 2.1 list's member_moderation_action. > - the effect of this apparently was that a post from some.guy(a)inaf.it > (whose subscription address is guy(a)somewhere.inaf.it) was hold for > moderation > > is this due to a loop with the regexp in hold_these_nonmembers ? It is because all the legacy *_these_nonmember actions are applied before nonmember moderation checks, so if an address matches a regex in hold_these_nonmembers, the post will be held regardless of the nonmember's moderation action. This may actually be a bug. I'll have to think about that. I've filed https://gitlab.com/mailman/mailman/-/issues/978 on this. > the default action would be to let it pass > > Or should we set Administration options Moderation to Accept > immediately ? No. You can avoid this issue by adding the 189 nonmember addresses to accept_these_nonmembers (as addresses, not regexps). Addresses still work, and accept_these_nonmembers takes precedence over hold_these_nonmembers just as in MM 2.1. > - if so, how can we do it automatically for all 189 entries ? > - or move back the 189 addresses to accept_these_nonmembers ? Yes, move them back. > - or for the future have persons posting with alternate addrresses > accepted for the future WITH ONE CLICk ? If they are nonmembers, you can accept the post and set moderation action in one operation in Postorius, but the regexps in hold_these_nonmembers will still take precedence. > - AND LAST BUT NOT LEAST would the syntax @list1 in > accept_these_nonmembers work in mailman3 ? This is https://gitlab.com/mailman/mailman/-/issues/794 (still open) -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 6 months

[MM3-users] Re: Can't post to my own list from HyperKitty

by Mark Sapiro

On 4/8/25 18:45, MegaBrutal via Mailman-users wrote: > > User-profile shows my GMail address as primary and my own-domain address as > secondary. The third address is not present, because I actually wanted to > get rid of it, and it seems I succeeded in that. This is your Django user which has the two addresses associated with it. > Mailman3/users shows me with the address I wanted to get rid of. If I click > Manage, I see all 3 addresses, with (for some reason) only the GMail one > being verified (strange because I'm sure I verified the other as well). And > here's something interesting: it shows me as a member with my GMail address > under subscriptions, but shows me with my megabrutal.com domain under > ownerships. These are your Mailman user and its addresses. They are not the same as your Django user and its addresses. I don't think you can unlink an address from your Mailman user in Postorius/HyperKitty. You can do that in `mailman shell`. ``` $ mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. >>> um = getUtility(IUserManager) >>> usr = um.get_user('user(a)example.com') # any of the three addresses >>> usr.addresses prints a list of the 3 addresses. These are indexed as 0, 1, 2 >>> usr.preferred_address prints the preferred address. If this is the one you want to delete, do >>> usr.preferred_address = usr.addresses[n] # n is the index of one you want >>> del usr.addresses[n] # n is the index of the address you don't want >>> usr.addresses.remove(n) # n is the index of the address you don't want >>> commit() ``` It may not be clear from the above, but the idea is that you are deleting the address you don't want from usr.addresses, but before that if the one you don't want is the preferred_address, you need to set the preferred address to one of the others. Also, `del usr.addresses[n]` deletes the address record, but it is still in the addresses list, thus the `usr.addresses.remove(n)`. This is untested, so before committing you may want to verify that `usr.addresses` now lists just the two addresses you want > This might be the cause? Maybe HyperKitty wants to use my owner identity, > which is not subscribed? Btw, is the owner not automatically considered as > someone who can post to the list, they also need to be subscribed? An owner is not a list member and has no special posting privileges. If only members are allowed to post, the owner address must also be a member to be allowed to post. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 4 months

[MM3-users] Re: Execute assorted Mailman functions via Mailman shell terminal

by Mark Sapiro

On 5/13/25 12:40, Andy Matthews wrote: > When my company migrated from MM2 to MM3, they discovered there were things that didn't work as expected (don't get me started). So they developed a set of Python functions which allowed then to fill in the missing functionality. However, they've discovered that some of those functions either never worked properly, or no longer work. For example, > > ``` > def list_nonmembers(mlist): > for accepted in mlist.accept_these_nonmembers: > print("Accepted: ",accepted) > for held in mlist.hold_these_nonmembers: > print("Held: ",held) > ``` > They're run by shelling into the machine which hosts mailman and executed like so (where scripts is `scripts.py`) > > ``` > sudo -iu mailman mailman shell -l MAILING_LIST_ID --run scripts.list_nonmembers > ``` That should work. What happens when you run it? Possibly it prints nothing because the list's accept_these_nonmembers and hold_these_nonmembers attributes are empty. > I'm recommending that we install Postorius, as I suspect many of the functions they've written are already present in that UI, but it's not as simple as that. Instead they're wanting a handful of these functions (including the one listed above) replaced sooner than we could get Postorius up and running. > > I've gotten this far: > ``` > from mailman.interfaces.listmanager import IListManager > from zope.component import getUtility > mylist_id = 'bravo-open1(a)tena-sda.org' > list_manager = getUtility(IListManager) > mylist = list_manager.get(mylist_id) > ``` > > The mylist variable is of type `mailman.model.mailinglist.MailingList`, and includes an Attributes for nonmembers: > ``` > mylist.nonmembers > <mailman.model.roster.NonmemberRoster object at 0x7ffffa3e65b0> > ``` > > But this nonmembers property doesn't output any users/addresses when printed, and throws a TypeError when I try to iterate over it: First of all I think there may be some confusion between the list's nonmembers and the *_these_nonmembers attributes. *_these_nonmembers are legacy attributes which exist to support `^...` regexps and `@list_id` entries, but the preferred way to accept, hold, reject or discard individual nonmembers is to set the nonmember's moderation action. > ``` > Traceback (most recent call last): > File "<console>", line 1, in <module> > TypeError: 'NonmemberRoster' object is not iterable > ``` You need to iterate over mylist.nonmembers.members, i.e. the roster's members, not the roster. > So, given a mailing list instance, retrieved via `list_manager.get(mylist_id)`, how would I go about printing all nonmembers and their current roles in a list (held, accepted, etc.)? Alternately, is there a different question I should be asking in order to retrieve that information? It's not roles as you've already limited the role to nonmember. It's moderation_action. ``` for nonmember in mylist.nonmembers.members: print(f'Nonmember: {nonmember.address.email},' f' moderation_action: {nonmember.moderation_action}') ``` Note that moderation_action: None means list default. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 3 months

[MM3-users] Re: install v3.3.11

by Jose Bernal

Thank you - that worked and pip also uninstalled the 3.3.10 package. I appreciate the help. --------------------- José Bernal Web/Application Engineer Oregon Criminal Defense Lawyers Association Cell 541-228-8481 jbernal.web.dev(a)gmail.com On Fri, Feb 14, 2025 at 9:05 AM Mark Sapiro <mark(a)msapiro.net> wrote: > On 2/14/25 08:50, Jose Bernal via Mailman-users wrote: > > I'd like to install Mailman v3.3.11. Are there instructions for > installing > > from pip? If not, is there an alternate way to install this latest > > version? When executing: > > > > pip install mailman==3.3.11 > > > > I get: ERROR: No matching distribution found for mailman==3.3.11 > > > 3.3.11 is still in development and hasn't yet been released to PyPI so > if you want to install it in its current state, you have to install from > GitLab. > > pip install git+https://gitlab.com/mailman/mailman@master > > will install the head of the master branch in GitLab. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to jbernal.web.dev(a)gmail.com >

1 year, 6 months

Search results for query "sapiro"