- Mailman-users - lists.mailman3.org

MM3 management tools (for those who need them)
by Dmitry Makovey Sept. 13, 2018

Sept. 13, 2018

Hi, I wanted to share our current development of MM3-related tools that we currently employ on a daily basis. forks, ideas, pointers and issue reports are welcome: https://gitlab.med.stanford.edu/irt-public/mailman-tools Note that it's a "work in progress" bunch of scripts that work, but YMMV. ...so no refunds ;) -- Sr System and DevOps Engineer SoM IRT

1 0

LookupError: unknown encoding: _iso-2022-jp$esc
by Thor Atle Rustad Sept. 7, 2018

Sept. 7, 2018

This happens when I open a page of Held Messages containing an offending message. I get Server Error (500) in Postorius. Here is the mailman.log: Sep 07 15:25:39 2018 (32) REST request handler error: Traceback (most recent call last): File "/usr/local/lib/python3.6/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/local/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", line 214, in __call__ return super().__call__(environ, start_response) File "/usr/local/lib/python3.6/site-packages/falcon/api.py", line 244, in __call__ responder(req, resp, **params) File "/usr/local/lib/python3.6/site-packages/mailman/rest/post_moderation.py", line 160, in on_get resource = self._make_collection(request) File "/usr/local/lib/python3.6/site-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/local/lib/python3.6/site-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/local/lib/python3.6/site-packages/mailman/rest/post_moderation.py", line 150, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/local/lib/python3.6/site-packages/mailman/rest/post_moderation.py", line 77, in _make_resource resource['msg'] = msg.as_string() File "/usr/local/lib/python3.6/site-packages/mailman/email/message.py", line 54, in as_string value = email.message.Message.as_string(self) File "/usr/local/lib/python3.6/email/message.py", line 158, in as_string g.flatten(self, unixfrom=unixfrom) File "/usr/local/lib/python3.6/email/generator.py", line 116, in flatten self._write(msg) File "/usr/local/lib/python3.6/email/generator.py", line 181, in _write self._dispatch(msg) File "/usr/local/lib/python3.6/email/generator.py", line 214, in _dispatch meth(msg) File "/usr/local/lib/python3.6/email/generator.py", line 243, in _handle_text msg.set_payload(payload, charset) File "/usr/local/lib/python3.6/email/message.py", line 315, in set_payload payload = payload.encode(charset.output_charset) LookupError: unknown encoding: _iso-2022-jp$esc

2 1

confusing contents of database (data corruption?)
by Dmitry Makovey Sept. 7, 2018

Sept. 7, 2018

I'm troubleshooting slowness and issues in our mailman (container-based) setup. (See also "ORM exceptions" thread) And I've come across really odd contents of DB: mailmandb=# select * from address, (select email, count(email) as instances from address group by email) as emails where emails.instances>1 and emails.email=address.email ; -[ RECORD 1 ]--+---------------------------------------------- id | 1001 email | some_person(a)some.org _original | Some_Person(a)some.org display_name | Some Person verified_on | registered_on | 2018-07-02 03:23:18.761679 user_id | 22543 preferences_id | 146541 email | some_person(a)some.org instances | 2 -[ RECORD 2 ]--+---------------------------------------------- id | 1002 email | some_person(a)some.org _original | display_name | verified_on | registered_on | 2018-07-02 03:23:19.036667 user_id | preferences_id | 146543 email | some_person(a)some.org instances | 2 mailmandb=# select * from member where address_id in (1002,1001); -[ RECORD 1 ]-----+------------------------------------- id | 73682 _member_id | f4cdxxxx-5c64-xxxx-9ac8-1f5xxxxa0f1c role | 4 list_id | mylist.lists.here.stanford.edu moderation_action | address_id | 1002 preferences_id | 146544 user_id | so: 1. address for the same email gets created twice 2. membership references "empty" address 3. membership does not reference proper user 4. preferences_id in all 3 records are different is there a documentation on the DB contents someplace? I feel like our mailman instance slowness is related to DB content but cleaning up that content seems to be risky at best. -- Sr System and DevOps Engineer SoM IRT

1 0

message file gone, but exists in database?
by Thor Atle Rustad Sept. 6, 2018

Sept. 6, 2018

I'm investigating some problems, and in the process I want to delete some messages. First, listing all messages: >>> messages = list(message_store.messages) Traceback (most recent call last): File "<console>", line 1, in <module> File "/usr/local/lib/python3.6/site-packages/mailman/model/messagestore.py", line 117, in messages yield self._get_message(row) File "/usr/local/lib/python3.6/site-packages/mailman/model/messagestore.py", line 95, in _get_message with open(path, 'rb') as fp: FileNotFoundError: [Errno 2] No such file or directory: '/opt/mailman/var/messages/5X/7W/5X7WMWNORTGVNB6KVWTHHOI2VF2BEQ7K' Since this message should not exist in database, I try to delete it: >>> message_store.delete_message('5X7WMWNORTGVNB6KVWTHHOI2VF2BEQ7K') >>> commit() >>> messages = list(message_store.messages) Traceback (most recent call last): File "<console>", line 1, in <module> File "/usr/local/lib/python3.6/site-packages/mailman/model/messagestore.py", line 117, in messages yield self._get_message(row) File "/usr/local/lib/python3.6/site-packages/mailman/model/messagestore.py", line 95, in _get_message with open(path, 'rb') as fp: FileNotFoundError: [Errno 2] No such file or directory: '/opt/mailman/var/messages/5X/7W/5X7WMWNORTGVNB6KVWTHHOI2VF2BEQ7K' However, it continues to exist in mailman's mind I also cannot delete messages that actually exist. For instance, I find a message in Postorius that I want to delete. I look at the headers and then run >>> message_store.delete_message('6XFISRRFY74NJEE4FTRNWOA2KGKDBZ3T') >>> commit() After that, the message continues to live, both in postorius and as a file: bash-4.3# ls 6XFISRRFY74NJEE4FTRNWOA2KGKDBZ3T What is wrong?

1 0

ORM exceptions
by Dmitry Makovey Sept. 6, 2018

Sept. 6, 2018

I'm tracing slowness of mailman in our infrastructure, and came across ORM exception: Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2837, in one ret = self.one_or_none() File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2816, in one_or_none "Multiple rows were found for one_or_none()") sqlalchemy.orm.exc.MultipleResultsFound: Multiple rows were found for one_or_none() During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.6/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/local/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", line 214, in __call__ return super().__call__(environ, start_response) File "/usr/local/lib/python3.6/site-packages/falcon/api.py", line 244, in __call__ responder(req, resp, **params) File "/usr/local/lib/python3.6/site-packages/mailman/rest/members.py", line 365, in on_post return self._find(request, response) File "/usr/local/lib/python3.6/site-packages/mailman/rest/members.py", line 389, in _find okay(response, etag(resource._make_collection(request))) File "/usr/local/lib/python3.6/site-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/local/lib/python3.6/site-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/local/lib/python3.6/site-packages/mailman/rest/members.py", line 72, in _resource_as_dict user = member.user File "/usr/local/lib/python3.6/site-packages/mailman/model/member.py", line 128, in user else getUtility(IUserManager).get_user(self._address.email)) File "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.6/site-packages/mailman/model/usermanager.py", line 88, in get_user return addresses.one().user File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2840, in one "Multiple rows were found for one()") sqlalchemy.orm.exc.MultipleResultsFound: Multiple rows were found for one() is this a known thing and what's the best path for fixing it? -- Sr System and DevOps Engineer SoM IRT

1 1

log rotation
by Dmitry Makovey Sept. 6, 2018

Sept. 6, 2018

I've been looking for a while and cannot find any information regarding log rotation for Mailman3. We're running containerized deployment which complicates the matter slightly, but having "some" approach for log rotation would be helpful. So before I dig into the code looking for SIGUSR1 or SIGHUP handlers can someone point me to a proper documentation for log management with MM3 ? -- Sr System and DevOps Engineer SoM IRT

3 3

ARC
by David Krantz Aug. 30, 2018

Aug. 30, 2018

Hi, I just found out that gmail uses ARC (sometimes, at least). Is there support for that in Mailman 3? Without having had the time (yet) to read about in depth it it seems as if ARC is intended to solve at least some of the problems that mail lists have with DKIM. Probably causes others... It might be that no support really is needed in mailman so that just milters in sendmail, postfix etc. might do the trick. regards // David

2 1

Migration advice
by Quanah Gibson-Mount Aug. 29, 2018

Aug. 29, 2018

Hi, I'm looking at migrating from an old server running mailman 2.1.15 (Debian wheezy) to a new Ubuntu 18.04 LTS system. I'd like to upgrade to mailman3 as a part of this migration. I was reading the migration docs (<http://docs.mailman3.org/en/latest/migration.html>), which seem to require both mailman2 and mailman3 be installed concurrently on the same system. Is that correct, or can I rsync the mailman2 list information onto the new server without mailman2 itself being present, and import from there? Thanks! Regards, Quanah -- Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: <http://www.symas.com>

2 1

Implicit destination
by David Krantz Aug. 28, 2018

Aug. 28, 2018

Hi, I have a spam-problem like anyone else. Most of the spam my listserver receives has some kind of impilict destination so the implicit rule catches most of the spam that way. This is fine. What is not so fine, however, is that this results in a moderation message to the sender. I'd like to stop that so that the spammer has no confirmation that the mail was received. I can probably stop this in postfix before the mail reaches mailman3, but I'd prefer changing the behaviour of the rule. Is there any setting that could be used or should I just modify the rule? Cheers. // David

4 5

Whitelist non subscriber
by Mark Ziranski Aug. 27, 2018

Aug. 27, 2018

Hello, I want to whitelist an email address on all lists to bypass moderation. The email address is not a subscriber. On google I found accept_these_nonmembers but I do not know how to use it. Please show me some examples how to whitelist someone(a)example.org. Thanks in advance. M.Z.

4 6