old usernames appearing in mailman.log
I have just migrated from a mailman 3.2.2 Debian package installation to a mailman 3.3.8 source install (in a Python virtual environment as mailman user) on an Ubuntu 20.04.6 LTS system.
Several months ago, the system had been upgraded from mailman V2 to mailman V3.2.2. The server has been active for over 20 years.
I've just taken on the project so I'm new to all of this, and one thing I noticed that caught my eye, and has left me curious, is that for the first a couple of seconds starting just after midnight every night (just after log rotation) mailman.log would have many (dozens, or hundreds) of lines entered in it just like this one:
Jun 19 00:00:04 2023 (2266) 127.0.0.1 - - "GET /3.0/users/old-user@domain.tld HTTP/1.1" 404 58
where "old-user@domain.tld" is the email address of a user who either had unsubscribed or changed their email address (I don't know which, I just know they are no longer a member/subscriber to the list). Each successive entry in the log would have a different "old username" in it. I don't understand why these entries would be in the log and what they mean. Can someone shed some light on this for me? This is merely a question of curiosity, and not one of import. (Since migrating from 3.2.2 to 3.3.8 this behavior has stopped, be that GOOD or BAD.)
Ken
On 6/25/23 11:04 AM, Ken Alker wrote:
I've just taken on the project so I'm new to all of this, and one thing I noticed that caught my eye, and has left me curious, is that for the first a couple of seconds starting just after midnight every night (just after log rotation) mailman.log would have many (dozens, or hundreds) of lines entered in it just like this one:
Jun 19 00:00:04 2023 (2266) 127.0.0.1 - - "GET /3.0/users/old-user@domain.tld HTTP/1.1" 404 58
where "old-user@domain.tld" is the email address of a user who either had unsubscribed or changed their email address (I don't know which, I just know they are no longer a member/subscriber to the list). Each successive entry in the log would have a different "old username" in it. I don't understand why these entries would be in the log and what they mean. Can someone shed some light on this for me? This is merely a question of curiosity, and not one of import. (Since migrating from 3.2.2 to 3.3.8 this behavior has stopped, be that GOOD or BAD.)
This is the Django hyperkitty sync_mailman job which is one of the django admin jobs run daily. It goes through the HyperKitty 'sender' database table and tries to find the Mailman core user id for each sender address to add/update its mailman_id field with core's user_id.
The 404s are addresses im HyperKitty's sender table for whih there is no corresponding Mailman user. There are multiple possible reasons for this.
If the behavior has stopped it's probably because the periodic Django jobs are not being run - this is bad.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
--On Sunday, June 25, 2023 1:14 PM -0700 Mark Sapiro <mark@msapiro.net> wrote:
On 6/25/23 11:04 AM, Ken Alker wrote:
I've just taken on the project so I'm new to all of this, and one thing I noticed that caught my eye, and has left me curious, is that for the first a couple of seconds starting just after midnight every night (just after log rotation) mailman.log would have many (dozens, or hundreds) of lines entered in it just like this one:
Jun 19 00:00:04 2023 (2266) 127.0.0.1 - - "GET /3.0/users/old-user@domain.tld HTTP/1.1" 404 58
where "old-user@domain.tld" is the email address of a user who either had unsubscribed or changed their email address (I don't know which, I just know they are no longer a member/subscriber to the list). Each successive entry in the log would have a different "old username" in it. I don't understand why these entries would be in the log and what they mean. Can someone shed some light on this for me? This is merely a question of curiosity, and not one of import. (Since migrating from 3.2.2 to 3.3.8 this behavior has stopped, be that GOOD or BAD.)
This is the Django hyperkitty sync_mailman job which is one of the django admin jobs run daily. It goes through the HyperKitty 'sender' database table and tries to find the Mailman core user id for each sender address to add/update its mailman_id field with core's user_id.
The 404s are addresses im HyperKitty's sender table for whih there is no corresponding Mailman user. There are multiple possible reasons for this.
And that is OK, right?
If the behavior has stopped it's probably because the periodic Django jobs are not being run - this is bad.
I see this in the logs from just after midnight:
Jun 25 00:13:04 2023 (386086) No cached copy of the public suffix list found Jun 25 00:13:04 2023 (386086) ACCEPT: <7500f0e2-1ae1-309d-60d7-6e592c7abf7a@west.net> Jun 25 00:13:04 2023 (386090) HyperKitty failure on https://lists.netlojix.com/hyperkitty/api/mailman/urls:
Followed by a huge amount of HTML (strange).
I'm not sure if the first and last (third) lines in the log excerpt above are due to the same thing. I know why the public suffix list is not found; when I was migrating all my queues from the Debian layout to the "source install" layout, I could not figure out where it should go in the new world order. Can you tell me where it would normally be found on a fresh source install? I'll move it there.
Is there a way to kick it into re-running whatever it does at midnight so I can test this again after putting the public suffix list back? I assume I can just execute "mailman-web runjobs daily", but will this mess things up, or is it OK to run it mid-day without stopping anything first?
Is the last (third) log line excerpt simply a result of the first, or do I have something else messed up?
Thank you! Ken
On 6/25/23 2:27 PM, Ken Alker wrote:
--On Sunday, June 25, 2023 1:14 PM -0700 Mark Sapiro <mark@msapiro.net> wrote:
The 404s are addresses im HyperKitty's sender table for whih there is no corresponding Mailman user. There are multiple possible reasons for this.
And that is OK, right?
Sure. It only means that there are messages in the archive that are From: addresses that aren't currently Mailman users.
If the behavior has stopped it's probably because the periodic Django jobs are not being run - this is bad.
I see this in the logs from just after midnight:
Jun 25 00:13:04 2023 (386086) No cached copy of the public suffix list found Jun 25 00:13:04 2023 (386086) ACCEPT: <7500f0e2-1ae1-309d-60d7-6e592c7abf7a@west.net> Jun 25 00:13:04 2023 (386090) HyperKitty failure on https://lists.netlojix.com/hyperkitty/api/mailman/urls:
Followed by a huge amount of HTML (strange).
This is all from one post.
The first is normal it just says that Mailman's var/public_suffix_list.dat doesn't exist, but it should be retrieved from https://publicsuffix.org/list/public_suffix_list.dat and then saved at var/public_suffix_list.dat so it doesn't need to be retrieved from the web again until the saved copy is older that the configured dmarc.cache_lifetime (default = 7 days, so that message) after which time the message will be 'Cached copy of public suffix list is out of date'
The second is acceptance of the post.
The third is a failure in archiving the message. Th HTML should contain an error message with a traceback which should help determine the problem.
Is there a way to kick it into re-running whatever it does at midnight so I can test this again after putting the public suffix list back? I assume I can just execute "mailman-web runjobs daily", but will this mess things up, or is it OK to run it mid-day without stopping anything first?
Midnight is a coincidence in this case. the above are all due to the post with Message-ID: <7500f0e2-1ae1-309d-60d7-6e592c7abf7a@west.net>
The cached var/public_suffix_list.dat should now be there with a timestamp of Jul 02 00:13:04 (7 days from Jun 25 00:13:04)
Is the last (third) log line excerpt simply a result of the first, or do I have something else messed up?
The third is not because of the first. It is because of some issue with archiving that post.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
--On Sunday, June 25, 2023 3:26 PM -0700 Mark Sapiro <mark@msapiro.net> wrote:
On 6/25/23 2:27 PM, Ken Alker wrote:
--On Sunday, June 25, 2023 1:14 PM -0700 Mark Sapiro <mark@msapiro.net> wrote: I see this in the logs from just after midnight:
Jun 25 00:13:04 2023 (386086) No cached copy of the public suffix list found Jun 25 00:13:04 2023 (386086) ACCEPT: <7500f0e2-1ae1-309d-60d7-6e592c7abf7a@west.net> Jun 25 00:13:04 2023 (386090) HyperKitty failure on https://lists.netlojix.com/hyperkitty/api/mailman/urls:
Followed by a huge amount of HTML (strange).
This is all from one post.
The first is normal it just says that Mailman's var/public_suffix_list.dat doesn't exist, but it should be retrieved from https://publicsuffix.org/list/public_suffix_list.dat and then saved at var/public_suffix_list.dat so it doesn't need to be retrieved from the web again until the saved copy is older that the configured dmarc.cache_lifetime (default = 7 days, so that message) after which time the message will be 'Cached copy of public suffix list is out of date'
The second is acceptance of the post.
The third is a failure in archiving the message. Th HTML should contain an error message with a traceback which should help determine the problem.
Is there a way to kick it into re-running whatever it does at midnight so I can test this again after putting the public suffix list back? I assume I can just execute "mailman-web runjobs daily", but will this mess things up, or is it OK to run it mid-day without stopping anything first?
Midnight is a coincidence in this case. the above are all due to the post with Message-ID: <7500f0e2-1ae1-309d-60d7-6e592c7abf7a@west.net>
The cached var/public_suffix_list.dat should now be there with a timestamp of Jul 02 00:13:04 (7 days from Jun 25 00:13:04)
Yes, thanks.
Is the last (third) log line excerpt simply a result of the first, or do I have something else messed up?
The third is not because of the first. It is because of some issue with archiving that post.
I have studied the output, looked at the code lines referenced in the Traceback, and I can't figure out the problem (I feel like the sad guy on the side of the street with his hood up staring into the engine bay with stream pouring out and not knowing what to do next). I did some analyses on the log file and determined that there are 71 messages it is having problems processing and there are 71 messages "stuck" in /opt/mailman/mm/var/archives/hyperkitty/spool/ (and that it has tried to process each message about 20 times). Since upgrading to V3.3.8 I've not seen new messages get stuck like this (although it's only been running smoothly for 24 hours now and there hasn't been much activity). But I'm suspecting there was something awry between the conversion from V2 to V3 in February and me installing V3.3.8 that caused this (based on message time stamps). It is HIGHLY likely these messages are in the archiving spool as a result of me unshunting the shunt queue and that they are copies of the messages that got delivered to the subscribers when I did that... but are now hung up in the archiving process.
Following is an excerpt from mailman.log. Note that the "first section" (from DOCTYPE at top to 404 at bottom) is repeated four times (maybe not identical; but similar) (so I did not include it to save room) before the Traceback appears.
Jun 25 00:13:04 2023 (386086) No cached copy of the public suffix list found Jun 25 00:13:04 2023 (386086) ACCEPT: <7500f0e2-1ae1-309d-60d7-6e592c7abf7a@west.net> Jun 25 00:13:04 2023 (386090) HyperKitty failure on https://lists.netlojix.com/hyperkitty/api/mailman/urls:
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>lists.netlojix.com</title> <link rel="shortcut icon" href="/static/postorius/img/favicon.ico"> <link rel="stylesheet" href="/static/postorius/libs/bootstrap/css/bootstrap.min.css"> <link rel="stylesheet" href="/static/django-mailman3/css/main.css"> <link rel="stylesheet" href="/static/postorius/css/style.css"> <link rel="stylesheet" href="/static/postorius/libs/fonts/font-awesome/css/font-awesome.min.css" type="text/css" media="all" />
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> <!-- WARNING: Respond.js doesn't work if you view the page via file:// --> <!--[if lt IE 9]> <script src="/static/postorius/libs/html5shiv/html5shiv.min.js"></script> <script src="/static/postorius/libs/respond/respond.min.js"></script> <![endif]--> </head> <body> <nav class="navbar navbar-expand-md navbar-light bg-light navbar-default"> <div class="container"> <div class="navbar-header"> <button type="button" class="navbar-toggler collapsed" data-toggle="collapse" data-target="#header-nav" aria-expanded="false"> <span class="sr-only">Toggle navigation</span> <span class="fa fa-bars"></span> </button> <a class="navbar-brand" href="/mailman3/lists/"><span><img src="/static/postorius/img/mailman_logo_small_trans.png" alt="Mailman logo"/> Postorius</span></a> </div> <div class="collapse navbar-collapse" id="header-nav"> <ul class="nav navbar-nav mr-auto"> <li class="nav-item"><a href="/mailman3/lists/" class="nav-link"> <span class="fa fa-envelope"></span> Lists </a></li>
<li class="nav-item"><a href="/archives/" class="nav-link"> <span class="fa fa-comment"></span> Archives </a></li>
</ul>
<ul class="nav navbar-nav navbar-right">
<li class="nav-item"><a class="nav-link" href="/accounts/login/?next=/hyperkitty/api/mailman/urls"> <span class="fa fa-sign-in"></span> Login </a></li> <li class="nav-item"><a class="nav-link" href="/accounts/signup/?next=/hyperkitty/api/mailman/urls"> <span class="fa fa-user-plus"></span> Sign Up </a></li>
</ul>
</div>
</div>
</nav>
<div class="container" role="main">
<h1>Page not found</h1>
<div class="alert alert-danger">This page either doesn't exist, or it moved somewhere else.</div>
</div>
<footer class="footer">
<div class="container">
<p class="text-center">
<a href="https://postorius.readthedocs.org">Postorius Documentation</a> • <a href="http://list.org">GNU Mailman</a> • Postorius Version 1.3.8
</p>
</div>
</footer>
<script src="/static/postorius/libs/jquery/jquery-3.6.0.min.js"></script> <script src="/static/postorius/libs/popperjs/popper-v1.11.0.min.js"></script> <script src="/static/postorius/libs/bootstrap/js/bootstrap.min.js"></script> <script src="/static/django-mailman3/js/main.js"></script> <script src="/static/postorius/js/script.js"></script> <script> // Add the .js-enabled class to the body so we can style the elements // depending on whether Javascript is enabled. $(document).ready(function(){ $("body").addClass("js-enabled"); }); </script>
</body> </html> (404)
... above portion of log (or similar) repeats three more times ...
Jun 25 00:13:05 2023 (386083) Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.8/site-packages/mailman_hyperkitty/__init__.py", line 158, in _archive_message url = self._send_message(mlist, msg) File "/opt/mailman/venv/lib/python3.8/site-packages/mailman_hyperkitty/__init__.py", line 228, in _send_message raise ValueError(result.text) ValueError:
Still stuck on this, and thought maybe the output of "mailman qfile" of one of the files stuck in var/archives/hyperkitty/spool (with a little obfuscation) would be helpful:
(venv) mailman@speedy:~/mm/var/archives/hyperkitty/spool$ mailman qfile 1687793077.1252093+27a28bb0d3e102c873ea2721df701640d47bcfcf.pck [----- start pickle -----] <----- start object 1 -----> Received: from server.domain.tld (server.domain.tld [127.0.0.1]) by server.domain.tld (Postfix) with ESMTP id 4PnTcS2BcSz6N6gc for <sbarc-list@lists.netlojix.com>; Thu, 30 Mar 2023 09:38:00 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at server.domain.tld Authentication-Results: server.domain.tld (amavisd-new); dkim=pass (1024-bit key) header.d=usda.gov Received: from server.domain.tld ([127.0.0.1]) by server.domain.tld (server.domain.tld [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qx7XLUPCEqzN for <sbarc-list@lists.netlojix.com>; Thu, 30 Mar 2023 09:37:58 -0700 (PDT) Received: from GCC02-BL0-obe.outbound.protection.outlook.com (mail-bl0gcc02on2050.outbound.protection.outlook.com [40.107.89.50]) by server.domain.tld (Postfix) with ESMTPS id 4PnTcQ5fXRz6N46J for <sbarc-list@lists.netlojix.com>; Thu, 30 Mar 2023 09:37:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gyj9ITDfgtjtvsGnEApNOddnSlfRZHF5ESbvfiWgG0tGzYGkTDGnrNSfc0k77Y+mj3nd0MSQeNemzFVUWrvpWFvKoArIqSaddTUxIpraTRUEEHfxsYXOQT4Cte/UO0Hzh+bl4Jl+dW2V/hOUP2mRm7aHVGhNDjkbexSXiOq8oR5Xvru9uWmbB8XZBM86HNbGLfcjciCPw5T92G9p9ts0zWFvln+LyGQVwGYp4i8zdtI0zxpm5bPJHtktOA+rgtKwnqahMRdIeLzqmFq2bAFTcxzapIOMgfK/cdxoyuPxMa1xPJMRwKUrHcTJc8rRgvyJuybI8ZkUKmJMZ5AP54kgfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bT2WvW9iacXqQY9O9fB2p0bjFTC2xa6SYk+sbGogjTA=; b=dGvm5diaA7Vipw0vAKXrXMVCZGuGzO69xj0tHhaz+BPsemsDjaVWpgk40lCpniNUCdmEIXRfKFbzn+fHqVIe/CLI7AO3fI6QvsaV3U1Ye7DfcBNzxBU8XPJSGb2RnWxKBuRMydy2olKFFrdk6E3f596jO2fzi0NbNtBYOAgNibsApN93Jka2Y67CNTcIQw+BhAogxLD40SrKwi7ffevO8VW1zeJ+pOfoQq8yYqC7UZjb5dR8v/jEjPsBKfZIwZ/KYpIA1Oa/TmktA4REMKOKjwN1L/SojGfGg/PqrMtYjkdIhylREiXGVHYh//geE2puSdc5DTQSrCS0irCF5PDmmg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=usda.gov; dmarc=pass action=none header.from=usda.gov; dkim=pass header.d=usda.gov; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usda.gov; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bT2WvW9iacXqQY9O9fB2p0bjFTC2xa6SYk+sbGogjTA=; b=l3r9B3BveGzG0Yu5i6VneqJxjbsV0NK6eyMmbj8IGMNFv25jGeGXZ/LVez/U+pqzlvCChJnWR5Lt8PvzMESvXm8jIF4p7K/piuhSyd7gPGYR/T4s2PnmAvn5lZyOGr4Enr9EdWBJeYc5zUZpTTQPPDtPNCx5mJ+4nkMx+QESyIA= Received: from SJ0PR09MB9767.namprd09.prod.outlook.com (2603:10b6:a03:466::7) by DS0PR09MB11445.namprd09.prod.outlook.com (2603:10b6:8:175::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.22; Thu, 30 Mar 2023 16:37:49 +0000 Received: from SJ0PR09MB9767.namprd09.prod.outlook.com ([fe80::face:4b17:66df:6081]) by SJ0PR09MB9767.namprd09.prod.outlook.com ([fe80::face:4b17:66df:6081%8]) with mapi id 15.20.6254.022; Thu, 30 Mar 2023 16:37:49 +0000 From: "Person <person@domain.tld> To: Person2 <person2@domain.tld>, sbarc list <sbarc-list@lists.netlojix.com> Thread-Topic: [External Email]Thursday at the Peak Thread-Index: AQHZYyTQGrIDUpP+V02LGKG0U1AYtq8ThNmg Message-ID: <SJ0PR09MB976721F1EC9A83F0BC8C7D91BA8E9@SJ0PR09MB9767.namprd09.prod.outlook.com> References: <463e95c9-59e7-10f8-7328-6d953314200a@cox.net> In-Reply-To: <463e95c9-59e7-10f8-7328-6d953314200a@cox.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=usda.gov; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SJ0PR09MB9767:EE_|DS0PR09MB11445:EE_ x-ms-office365-filtering-correlation-id: 3d80552b-5d2d-49f3-964f-08db313d198c x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: CRiPdfHAFKT3JrF2URRLVeBIqavoLsPQlrKiG9N6UjsuHteilxiglM98B6n1I+QG+nhqiEz7AmOCndQz7TivWvhUVq6K196DDwWzgagrQsbDU71LbCkUs+UAZk2g8SM7adLniAfY5JUumYf/CEpVFdGI5A/y7CmIB5OhnfW3qoalHlvzJ0ohFJF4COe4+xPkRFqbB1xZvZ1yzqqf4+bJf0m1xY+7AxSnnxhhON8ZnI38THh9xv0EDKQ2/D8/bpsgmlUoPZSwXwG/M156sFcNy3uNZMZbdvu9rKg/OZCdUaUTnx/9oIieSzyRsyIwaGJhvlPFbFmmz4reCivsPKnuYO2a1XLLSUDY+1owoXPTvw1ZTlC7ARFCgDmhAla5GLZ7JRVD+vGCTl2abyB0knaq9ztZi2mRcS9T1Xf3FuCrVbAO0ud8bnD64zkHOlE7vFWCkw293Dbbl4Lx5PGbFwwyWV9J2vkdLxmbNy580E2ZtltVpkzBsHEzkJLNJGD/ME5kaCVOYdKTVqzXDRwhATY1MEqpNROHLhLbcDtwz1TpqeCKroh+XZNFGh2HUUyDd5kNmu10dBHnC7l9GNVs6xxslzUl2VQkeD4F6lvdv7E1xpTyMM8GYj0HoP9aWu938xsx x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR09MB9767.namprd09.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(366004)(451199021)(83380400001)(122000001)(186003)(26005)(9686003)(86362001)(82960400001)(7696005)(8936002)(498600001)(8676002)(54906003)(110136005)(66556008)(38100700002)(64756008)(66446008)(66476007)(76116006)(66946007)(4326008)(6506007)(71200400001)(5660300002)(53546011)(7416002)(38070700005)(2906002)(33656002)(55016003)(52536014);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?aFF5djZ6eDBvZlJTRmdxK01RZkxsU0JxazNWUG5LUEljdGMvTHhwUXZMaGpl?= =?utf-8?B?QVJkS2x6WjQ0UzVXejJha2ZxUmVEcHB2eDhONDh5aG9wajU2WENxQWp3SDJG?= =?utf-8?B?T2xINU1yOFF1N3d1cnBpNVlzMFRsL0hBakFGVXdnUFNIbFJhTGV6eVp0Wk1M?= =?utf-8?B?QmR2bTlOR2o5cm9OK0J1TU4yaVVLaWpVeTMrcGU0YWFhNjdTMGRhRlRGT3Ew?= =?utf-8?B?RUZ4aDZKaGZNeW5tYUFacytPRW5tSFM4d2t2N2szZVQxaWYrT250MjJkeW9o?= =?utf-8?B?dnlKUEkvRU1iM21LaUdKUVpZNFd4L2d5Sk51bDBpN1htS25lRmJwdlJlVytY?= =?utf-8?B?SjZ3RGFxWTErYWducExlTzN0cG1oTWRvQlVUbzJXdHZSTWdLQXVpeVJ2RUZN?= =?utf-8?B?b0pyc29pOFl2VU12a3g5VTdaZGtoVFVTTHFNMXE5TGQxcjZWeDQ0NHZSb04z?= =?utf-8?B?WTlFS0NDZ3orRkVDcHh6a0dBUkdyL3R1UkVPRmhaSnRQSE5qRllrT2FsWVN1?= =?utf-8?B?U09QN2hFU2h4ZkxwUDJUTC93SzlEZUVlUXVFbjNzVU9Wc1dEb3JMWjc5dVZ3?= =?utf-8?B?TWlsTDJLSzdmTDlWMzhpZFowMVVtQkZSSTdiaW1HYUNYazR2dTFsOVdPMTUr?= =?utf-8?B?VTUrOWVENU1SeVZPNEpqVi9MdmhzZUdoMVQwcEVsS09BTGxySDNUOGMyT0VS?= =?utf-8?B?aXlCUFJnazhZNG1sd1d4K0Z6ZS9qRHRCZFc0WnBHSVZYM1lTbGJrVThQVHpy?= =?utf-8?B?b3F0a0NBQnYrMU9xZ3pZakRpeHR3SEt4eVhORktmeFpiT2ZkQlpXRjAzRzE5?= =?utf-8?B?SkFIUW9Db1IwRFFQd3E5NzI1VlZkUm1UaktEV1d3WUQyMUVsbTNxcjg4bFgz?= =?utf-8?B?aFVGaXhpaTZiczVIMk5NUlRuckZlV1NMeFhNMnQwaXhMMG51bHJDbWxHV3RX?= =?utf-8?B?QVhLcUMxWnRCZUJnNWw2T2lFb0l1L0xZZ0h2bU5CTkhHbGNuZnFwYVBvaXg0?= =?utf-8?B?bng2aWVoUm5Ua2VEZUR4T05VT0R6YWQyS3Ztd2NYclpod1NpSHEvQjRvbU16?= =?utf-8?B?OHNVdEkwZGRSSU9iZ3loa2U2OXhqWERLNnZyRjNSRU03Z3hiS2Myc2kxaG84?= =?utf-8?B?Mm4yVk5ySmpxTGxWUUdTbzZ1VGcwaGh6S3ltRDd4c0RKOVdCc0JZMHA5MStB?= =?utf-8?B?VmdZbVkxaGtIVjdDR2ZaQUZiNURLQ29ZWDBIWjJ6bFpOR1BRcFAvd2Z6ODFm?= =?utf-8?B?c041QVZFS0trOE14M1VZQ2V3cFIvWUxNanpNNmdHYzhUOE9YajAzS0NYMStK?= =?utf-8?B?Y2ZDVVhvOW1yQWYyc3h4MHR4WGh3bXVUdEJvNWZNa29ac3VsbjMybURUZHl4?= =?utf-8?B?UGI4dnFiOUN3WDA2MHR2ZTVld0RJQW0wQ2R2WkhrOWdzSHBVd3lTUndoQ25T?= =?utf-8?B?UklUaTdtc04xOENaMHU5dE1NZndVZ1ZCblQ1VUtsREpOTXNuV3RXMkdTRzV1?= =?utf-8?B?ajZtMTJFVzgyNHlIOGhMNzV3N251c3M2Q3h5RjBZeWZqQjFUbXZrSTBmQW0y?= =?utf-8?B?RUUvSlNyTndRdG8wRFdGNllHZDhJRzN6d0d3Z3lOVmdoa0pocG8wQVFXaGVV?= =?utf-8?B?aXQ3UXBkY216MzZQdUJTc0xFRmdPenZhK3Y3QUpIOUtWcG1YQWdQM1Y1TGpJ?= =?utf-8?B?NU9EcnRtUDl5YzJibnJSWmdVNHZEaEJkKzBGOGN4bEtsV3FzWEQ0Z2pCMnRI?= =?utf-8?B?QW9XdDFSaWV1MmxrRTJCbG5NU25saXdTMzU5NGdBbHdVOXg2OURWNVEzUTVJ?= =?utf-8?B?b0lKYzZjM0cxVDBSWXRlVDVDUzg5bDhxTnBZbmh2b0xXakVML201ZnJnZHFU?= =?utf-8?B?Um1GSlM4ak11NkRoblVEY0dXOE5FU25OMk5INjA2THBxdDhBSFlSVDkrSFgv?= =?utf-8?B?MEZOTXVwck1GT1JiWUx1YThRUkhqSHdiK1RyT3NJY3pyeVdkQzlJWjlSVm4r?= =?utf-8?B?UjFzN1d4alZHN2tsM3dpNjBrVXpQTzhBc2czcVlMdmpUYmIvNVkxVnNtaU1W?= =?utf-8?B?VWZYU3JYdjZKVlNmTzV4U0lJMzhlclBDTjdXM1B6eFZ1K0pnaDhtTHhvSkpK?= =?utf-8?Q?n9sp0sU1WHqEFYHuCPa+zhNgv?= Content-Type: multipart/alternative; boundary="_000_SJ0PR09MB976721F1EC9A83F0BC8C7D91BA8E9SJ0PR09MB9767namp_" MIME-Version: 1.0 X-OriginatorOrg: usda.gov X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SJ0PR09MB9767.namprd09.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3d80552b-5d2d-49f3-964f-08db313d198c X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Mar 2023 16:37:49.4239 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: ed5b36e7-01ee-4ebc-867e-e03cfa0d4697 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR09MB11445 X-MailFrom: person@email.tld X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation Message-ID-Hash: K6EKKTW3DTFGZTYWV2UZG6TDCPBOWF4H X-Message-ID-Hash: K6EKKTW3DTFGZTYWV2UZG6TDCPBOWF4H X-Mailman-Approved-At: Sun, 25 Jun 2023 03:23:55 -0700 CC: MORE EMAIL ADDRESSES WERE HERE X-Mailman-Version: 3.3.8 Precedence: list Subject: [Sbarc-list] Re: [External Email]Thursday at the Peak List-Id: Santa Barbara Amateur Radio Club Listserver <sbarc-list.lists.netlojix.com> Archived-At: <> List-Archive: <> List-Help: <mailto:sbarc-list-request@lists.netlojix.com?subject=help> List-Owner: <mailto:sbarc-list-owner@lists.netlojix.com> List-Post: <mailto:sbarc-list@lists.netlojix.com> List-Subscribe: <mailto:sbarc-list-join@lists.netlojix.com> List-Unsubscribe: <mailto:sbarc-list-leave@lists.netlojix.com> Date: Thu, 30 Mar 2023 16:38:00 X-Original-Date: Thu, 30 Mar 2023 16:37:49 +0000
--_000_SJ0PR09MB976721F1EC9A83F0BC8C7D91BA8E9SJ0PR09MB9767namp_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64
SGkgQmlsbDoNCg0KSWYgd2UgbWFuYWdlZCB0byBtYWtlIGl0IHVwIHRoZXJlIHdlIGNvdWxkIGhh dmUgYSBzbm93YmFsbCBmaWdodCEgIEhvd2V2ZXIsIEFGQUlLIHRoZSByb2FkIGlzIHN0aWxsIGJs b2NrZWQuDQoNCkRhdmUgSi4NCg0KRnJvbTogQmlsbCBUYWxhbmlhbiA8dzF1dXFAY294Lm5ldD4N ClNlbnQ6IFRodXJzZGF5LCBNYXJjaCAzMCwgMjAyMyA5OjI4IEFNDQpUbzogc2JhcmMgbGlzdCA8 c2JhcmMtbGlzdEBsaXN0cy5uZXRsb2ppeC5jb20+DQpDYzogSm9obnNvbiwgRGF2ZSAtIEZTLCBD QSA8RGF2aWQuSm9obnNvbjJAdXNkYS5nb3Y+OyBJYW4gV2lsbGlhbXMgPGlhbl93aWxsaWFtc0Bu cHMuZ292PjsgVWx5c3NlcyBIdWVydGEgPHVseXNzZXNfaHVlcnRhQG5wcy5nb3Y+OyBELiBTY290 dCBTY2hyZWluZXIgPGRzc2NocmVpbmVAY294Lm5ldD47IEJpbGwgVGhlZXJpbmdlciA8dzhwZXlA YW9sLmNvbT47IFBhdWwgU3RyYXVzcyA8cGdzdHJhdXNzQHZlcml6b24ubmV0PjsgTWF0dCBMZWNo bGl0ZXIgPGdyYW5kbmF0aW9uYWxyYWRpb0BnbWFpbC5jb20+OyBOZWxzb24gVHJpY2hsZXIgPG5l bHNvbi50cmljaGxlckBnbWFpbC5jb20+OyBUb255IE5lbHNvbiA8dG9ueS5uZWxzb25AdWNzYi5l ZHU+DQpTdWJqZWN0OiBbRXh0ZXJuYWwgRW1haWxdVGh1cnNkYXkgYXQgdGhlIFBlYWsNCg0KW0V4 dGVybmFsIEVtYWlsXQ0KSWYgdGhpcyBtZXNzYWdlIGNvbWVzIGZyb20gYW4gdW5leHBlY3RlZCBz ZW5kZXIgb3IgcmVmZXJlbmNlcyBhIHZhZ3VlL3VuZXhwZWN0ZWQgdG9waWM7DQpVc2UgY2F1dGlv biBiZWZvcmUgY2xpY2tpbmcgbGlua3Mgb3Igb3BlbmluZyBhdHRhY2htZW50cy4NClBsZWFzZSBz ZW5kIGFueSBjb25jZXJucyBvciBzdXNwaWNpb3VzIG1lc3NhZ2VzIHRvOiBTcGFtLkFidXNlQHVz ZGEuZ292PG1haWx0bzpTcGFtLkFidXNlQHVzZGEuZ292Pg0KVGh1cnNkYXkgbW9ybmluZyBhdCBT YW50YSBZbmV6IFBlYWsuLi4uLi4uIFVTRlMgQmxkZyAxDQoNCg0KDQpbaHR0cHM6Ly9ocHdyZW4u dWNzZC5lZHUvY2FtZXJhcy9ML3N5cC1lLW1vYm8tYy5qcGddDQoNCg0KDQoNClRoaXMgZWxlY3Ry b25pYyBtZXNzYWdlIGNvbnRhaW5zIGluZm9ybWF0aW9uIGdlbmVyYXRlZCBieSB0aGUgVVNEQSBz b2xlbHkgZm9yIHRoZSBpbnRlbmRlZCByZWNpcGllbnRzLiBBbnkgdW5hdXRob3JpemVkIGludGVy Y2VwdGlvbiBvZiB0aGlzIG1lc3NhZ2Ugb3IgdGhlIHVzZSBvciBkaXNjbG9zdXJlIG9mIHRoZSBp bmZvcm1hdGlvbiBpdCBjb250YWlucyBtYXkgdmlvbGF0ZSB0aGUgbGF3IGFuZCBzdWJqZWN0IHRo ZSB2aW9sYXRvciB0byBjaXZpbCBvciBjcmltaW5hbCBwZW5hbHRpZXMuIElmIHlvdSBiZWxpZXZl IHlvdSBoYXZlIHJlY2VpdmVkIHRoaXMgbWVzc2FnZSBpbiBlcnJvciwgcGxlYXNlIG5vdGlmeSB0 aGUgc2VuZGVyIGFuZCBkZWxldGUgdGhlIGVtYWlsIGltbWVkaWF0ZWx5Lg0K
--_000_SJ0PR09MB976721F1EC9A83F0BC8C7D91BA8E9SJ0PR09MB9767namp_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64
PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPCEtLVtp ZiAhbXNvXT48c3R5bGU+dlw6KiB7YmVoYXZpb3I6dXJsKCNkZWZhdWx0I1ZNTCk7fQ0Kb1w6KiB7 YmVoYXZpb3I6dXJsKCNkZWZhdWx0I1ZNTCk7fQ0Kd1w6KiB7YmVoYXZpb3I6dXJsKCNkZWZhdWx0 I1ZNTCk7fQ0KLnNoYXBlIHtiZWhhdmlvcjp1cmwoI2RlZmF1bHQjVk1MKTt9DQo8L3N0eWxlPjwh W2VuZGlmXS0tPjxzdHlsZT48IS0tDQovKiBGb250IERlZmluaXRpb25zICovDQpAZm9udC1mYWNl DQoJe2ZvbnQtZmFtaWx5OkhlbHZldGljYTsNCglwYW5vc2UtMToyIDExIDYgNCAyIDIgMiAyIDIg NDt9DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OiJDYW1icmlhIE1hdGgiOw0KCXBhbm9zZS0x OjIgNCA1IDMgNSA0IDYgMyAyIDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpDYWxpYnJp Ow0KCXBhbm9zZS0xOjIgMTUgNSAyIDIgMiA0IDMgMiA0O30NCi8qIFN0eWxlIERlZmluaXRpb25z ICovDQpwLk1zb05vcm1hbCwgbGkuTXNvTm9ybWFsLCBkaXYuTXNvTm9ybWFsDQoJe21hcmdpbjow aW47DQoJZm9udC1zaXplOjExLjBwdDsNCglmb250LWZhbWlseToiQ2FsaWJyaSIsc2Fucy1zZXJp Zjt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsN Cgljb2xvcjojMDU2M0MxOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQ0Kc3Bhbi5FbWFp bFN0eWxlMTkNCgl7bXNvLXN0eWxlLXR5cGU6cGVyc29uYWwtcmVwbHk7DQoJZm9udC1mYW1pbHk6 IkNhbGlicmkiLHNhbnMtc2VyaWY7DQoJY29sb3I6d2luZG93dGV4dDt9DQouTXNvQ2hwRGVmYXVs dA0KCXttc28tc3R5bGUtdHlwZTpleHBvcnQtb25seTsNCglmb250LXNpemU6MTAuMHB0O30NCkBw YWdlIFdvcmRTZWN0aW9uMQ0KCXtzaXplOjguNWluIDExLjBpbjsNCgltYXJnaW46MS4waW4gMS4w aW4gMS4waW4gMS4waW47fQ0KZGl2LldvcmRTZWN0aW9uMQ0KCXtwYWdlOldvcmRTZWN0aW9uMTt9 DQotLT48L3N0eWxlPjwhLS1baWYgZ3RlIG1zbyA5XT48eG1sPg0KPG86c2hhcGVkZWZhdWx0cyB2 OmV4dD0iZWRpdCIgc3BpZG1heD0iMTAyNiIgLz4NCjwveG1sPjwhW2VuZGlmXS0tPjwhLS1baWYg Z3RlIG1zbyA5XT48eG1sPg0KPG86c2hhcGVsYXlvdXQgdjpleHQ9ImVkaXQiPg0KPG86aWRtYXAg djpleHQ9ImVkaXQiIGRhdGE9IjEiIC8+DQo8L286c2hhcGVsYXlvdXQ+PC94bWw+PCFbZW5kaWZd LS0+DQo8L2hlYWQ+DQo8Ym9keSBsYW5nPSJFTi1VUyIgbGluaz0iIzA1NjNDMSIgdmxpbms9IiM5 NTRGNzIiIHN0eWxlPSJ3b3JkLXdyYXA6YnJlYWstd29yZCI+DQo8ZGl2IGNsYXNzPSJXb3JkU2Vj dGlvbjEiPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+SGkgQmlsbDo8bzpwPjwvbzpwPjwvcD4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+SWYgd2UgbWFuYWdlZCB0byBtYWtlIGl0IHVwIHRoZXJlIHdlIGNvdWxkIGhhdmUgYSBz bm93YmFsbCBmaWdodCEmbmJzcDsgSG93ZXZlciwgQUZBSUsgdGhlIHJvYWQgaXMgc3RpbGwgYmxv Y2tlZC4NCjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8 L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5EYXZlIEouPG86cD48L286cD48L3A+DQo8 cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxkaXY+DQo8ZGl2IHN0 eWxlPSJib3JkZXI6bm9uZTtib3JkZXItdG9wOnNvbGlkICNFMUUxRTEgMS4wcHQ7cGFkZGluZzoz LjBwdCAwaW4gMGluIDBpbiI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48Yj5Gcm9tOjwvYj4gQmls bCBUYWxhbmlhbiAmbHQ7dzF1dXFAY294Lm5ldCZndDsgPGJyPg0KPGI+U2VudDo8L2I+IFRodXJz ZGF5LCBNYXJjaCAzMCwgMjAyMyA5OjI4IEFNPGJyPg0KPGI+VG86PC9iPiBzYmFyYyBsaXN0ICZs dDtzYmFyYy1saXN0QGxpc3RzLm5ldGxvaml4LmNvbSZndDs8YnI+DQo8Yj5DYzo8L2I+IEpvaG5z b24sIERhdmUgLSBGUywgQ0EgJmx0O0RhdmlkLkpvaG5zb24yQHVzZGEuZ292Jmd0OzsgSWFuIFdp bGxpYW1zICZsdDtpYW5fd2lsbGlhbXNAbnBzLmdvdiZndDs7IFVseXNzZXMgSHVlcnRhICZsdDt1 bHlzc2VzX2h1ZXJ0YUBucHMuZ292Jmd0OzsgRC4gU2NvdHQgU2NocmVpbmVyICZsdDtkc3NjaHJl aW5lQGNveC5uZXQmZ3Q7OyBCaWxsIFRoZWVyaW5nZXIgJmx0O3c4cGV5QGFvbC5jb20mZ3Q7OyBQ YXVsIFN0cmF1c3MgJmx0O3Bnc3RyYXVzc0B2ZXJpem9uLm5ldCZndDs7IE1hdHQgTGVjaGxpdGVy DQogJmx0O2dyYW5kbmF0aW9uYWxyYWRpb0BnbWFpbC5jb20mZ3Q7OyBOZWxzb24gVHJpY2hsZXIg Jmx0O25lbHNvbi50cmljaGxlckBnbWFpbC5jb20mZ3Q7OyBUb255IE5lbHNvbiAmbHQ7dG9ueS5u ZWxzb25AdWNzYi5lZHUmZ3Q7PGJyPg0KPGI+U3ViamVjdDo8L2I+IFtFeHRlcm5hbCBFbWFpbF1U aHVyc2RheSBhdCB0aGUgUGVhazxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rpdj4NCjxwIGNs YXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPGRpdiBzdHlsZT0iYm9yZGVy OnNvbGlkIGdyZWVueWVsbG93IDEuNXB0O3BhZGRpbmc6MS4wcHQgMS4wcHQgMS4wcHQgMS4wcHQ7 bWFyZ2luLWxlZnQ6MS41cHQ7bWFyZ2luLXRvcDoxLjVwdDttYXJnaW4tcmlnaHQ6MS41cHQ7bWFy Z2luLWJvdHRvbToxLjVwdCI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0iYmFja2dyb3Vu ZDpnaG9zdHdoaXRlIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjkuMHB0O2NvbG9yOmJsYWNrO2Jh Y2tncm91bmQ6Z3JlZW55ZWxsb3ciPltFeHRlcm5hbCBFbWFpbF08L3NwYW4+PHNwYW4gc3R5bGU9 ImZvbnQtc2l6ZTo5LjBwdDtjb2xvcjpibGFjayI+DQo8YnI+DQpJZiB0aGlzIG1lc3NhZ2UgY29t ZXMgZnJvbSBhbiA8Yj51bmV4cGVjdGVkIHNlbmRlcjwvYj4gb3IgcmVmZXJlbmNlcyBhIDxiPnZh Z3VlL3VuZXhwZWN0ZWQgdG9waWM7PC9iPg0KPGJyPg0KVXNlIGNhdXRpb24gYmVmb3JlIGNsaWNr aW5nIGxpbmtzIG9yIG9wZW5pbmcgYXR0YWNobWVudHMuPGJyPg0KUGxlYXNlIHNlbmQgYW55IGNv bmNlcm5zIG9yIHN1c3BpY2lvdXMgbWVzc2FnZXMgdG86IDxhIGhyZWY9Im1haWx0bzpTcGFtLkFi dXNlQHVzZGEuZ292Ij4NClNwYW0uQWJ1c2VAdXNkYS5nb3Y8L2E+IDwvc3Bhbj48c3BhbiBzdHls ZT0iZm9udC1zaXplOjkuMHB0Ij48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8L2Rpdj4NCjxkaXY+ DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibWFyZ2luLWJvdHRvbToxMi4wcHQiPjxzcGFu IHN0eWxlPSJmb250LXNpemU6MTMuNXB0O2ZvbnQtZmFtaWx5OiZxdW90O0hlbHZldGljYSZxdW90 OyxzYW5zLXNlcmlmIj5UaHVyc2RheSBtb3JuaW5nIGF0IFNhbnRhIFluZXogUGVhay4uLi4uLi4g VVNGUyBCbGRnIDE8YnI+DQo8YnI+DQo8YnI+DQo8L3NwYW4+PGJyPg0KPHNwYW4gc3R5bGU9ImZv bnQtc2l6ZToxMy41cHQ7Zm9udC1mYW1pbHk6JnF1b3Q7SGVsdmV0aWNhJnF1b3Q7LHNhbnMtc2Vy aWYiPjxpbWcgYm9yZGVyPSIwIiB3aWR0aD0iMzA3MyIgaGVpZ2h0PSIyMDQ5IiBzdHlsZT0id2lk dGg6MzIuMDA4M2luO2hlaWdodDoyMS4zNDE2aW4iIGlkPSJfeDAwMDBfaTEwMjUiIHNyYz0iaHR0 cHM6Ly9ocHdyZW4udWNzZC5lZHUvY2FtZXJhcy9ML3N5cC1lLW1vYm8tYy5qcGciPjwvc3Bhbj48 bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8YnI+DQo8YnI+DQo8YnI+DQo8YnI+DQpU aGlzIGVsZWN0cm9uaWMgbWVzc2FnZSBjb250YWlucyBpbmZvcm1hdGlvbiBnZW5lcmF0ZWQgYnkg dGhlIFVTREEgc29sZWx5IGZvciB0aGUgaW50ZW5kZWQgcmVjaXBpZW50cy4gQW55IHVuYXV0aG9y aXplZCBpbnRlcmNlcHRpb24gb2YgdGhpcyBtZXNzYWdlIG9yIHRoZSB1c2Ugb3IgZGlzY2xvc3Vy ZSBvZiB0aGUgaW5mb3JtYXRpb24gaXQgY29udGFpbnMgbWF5IHZpb2xhdGUgdGhlIGxhdyBhbmQg c3ViamVjdCB0aGUgdmlvbGF0b3IgdG8gY2l2aWwNCiBvciBjcmltaW5hbCBwZW5hbHRpZXMuIElm IHlvdSBiZWxpZXZlIHlvdSBoYXZlIHJlY2VpdmVkIHRoaXMgbWVzc2FnZSBpbiBlcnJvciwgcGxl YXNlIG5vdGlmeSB0aGUgc2VuZGVyIGFuZCBkZWxldGUgdGhlIGVtYWlsIGltbWVkaWF0ZWx5Lg0K PC9ib2R5Pg0KPC9odG1sPg0K
--_000_SJ0PR09MB976721F1EC9A83F0BC8C7D91BA8E9SJ0PR09MB9767namp_--
<----- start object 2 -----> { '_parsemsg': False, 'mlist': <mailing list "sbarc-list@lists.netlojix.com" at 0x7f666397be80>, 'version': 3} [----- end pickle -----] (venv) mailman@speedy:~/mm/var/archives/hyperkitty/spool$
On 6/25/23 10:57 PM, Ken Alker wrote:
I have studied the output, looked at the code lines referenced in the Traceback, and I can't figure out the problem (I feel like the sad guy on the side of the street with his hood up staring into the engine bay with stream pouring out and not knowing what to do next). I did some analyses on the log file and determined that there are 71 messages it is having problems processing and there are 71 messages "stuck" in /opt/mailman/mm/var/archives/hyperkitty/spool/ (and that it has tried to process each message about 20 times).
Each time there is a new post, it retries all the messages in the spool/ directory.
Since upgrading to V3.3.8 I've not seen new messages get stuck like this (although it's only been running smoothly for 24 hours now and there hasn't been much activity).
But I'm suspecting there was something awry between the conversion from V2 to V3 in February and me installing V3.3.8 that caused this (based on message time stamps). It is HIGHLY likely these messages are in the archiving spool as a result of me unshunting the shunt queue and that they are copies of the messages that got delivered to the subscribers when I did that... but are now hung up in the archiving process.
Are new messages being archived OK. If so, that is a real mystery. It says mailman-hyperkitty can talk to hyperkitty and archive new messages, but it can't talk to hyperkitty when attempting to process a queued message.
Following is an excerpt from mailman.log. Note that the "first section" (from DOCTYPE at top to 404 at bottom) is repeated four times (maybe not identical; but similar) (so I did not include it to save room) before the Traceback appears.
Jun 25 00:13:04 2023 (386086) No cached copy of the public suffix list found Jun 25 00:13:04 2023 (386086) ACCEPT: <7500f0e2-1ae1-309d-60d7-6e592c7abf7a@west.net> Jun 25 00:13:04 2023 (386090) HyperKitty failure on https://lists.netlojix.com/hyperkitty/api/mailman/urls:
I just went to that URL and got a 404. This is a serious problem. https://lists.netlojix.com/hyperkitty/api/ also returns a 404. It should return some documentation of the API.
https://lists.netlojix.com/archives/api/ does work and returns the expected information.
Your urls.py file probably contains in its urlpatterns list
re_path(r'^mailman3/', include('postorius.urls')),
re_path(r'^archives/', include('hyperkitty.urls')),
The simplest fix is to add
re_path(r'^postorius/', include('postorius.urls')),
re_path(r'^hyperkitty/', include('hyperkitty.urls')),
to that list.
Another way to fix this is in your mailman-hyperkitty.cfg, you have
base_url: https://lists.netlojix.com/hyperkitty
Change that to
base_url: https://lists.netlojix.com/archives
but it wouldn't hurt to add the ^postorius/ and ^hyperkitty/
patterns to urls.py anyway.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
--On Monday, June 26, 2023 5:03 PM -0700 Mark Sapiro <mark@msapiro.net> wrote:
On 6/25/23 10:57 PM, Ken Alker wrote:
I have studied the output, looked at the code lines referenced in the Traceback, and I can't figure out the problem (I feel like the sad guy on the side of the street with his hood up staring into the engine bay with stream pouring out and not knowing what to do next). I did some analyses on the log file and determined that there are 71 messages it is having problems processing and there are 71 messages "stuck" in /opt/mailman/mm/var/archives/hyperkitty/spool/ (and that it has tried to process each message about 20 times).
Each time there is a new post, it retries all the messages in the spool/ directory.
Since upgrading to V3.3.8 I've not seen new messages get stuck like this (although it's only been running smoothly for 24 hours now and there hasn't been much activity). But I'm suspecting there was something awry between the conversion from V2 to V3 in February and me installing V3.3.8 that caused this (based on message time stamps). It is HIGHLY likely these messages are in the archiving spool as a result of me unshunting the shunt queue and that they are copies of the messages that got delivered to the subscribers when I did that... but are now hung up in the archiving process.
Are new messages being archived OK. If so, that is a real mystery. It says mailman-hyperkitty can talk to hyperkitty and archive new messages, but it can't talk to hyperkitty when attempting to process a queued message.
Ahhh! I had convinced myself that new messages were being archived, but now that you ask, they ARE NOT. I may have said they were at some point, and I apologize if I did. Now I feel less confused.
It appears that archiving stopped when I did the migration from "Debian package layout" to "source layout" and upgraded to V3.3.8. I'll bet I munged a config file or two. I was struggling with what to keep from Debian layout and what to bring over from the source install instructions and I haven't gone back through to clean them up. I had previously figured it could not be a configuration issue since some messages were making it into the archive and some were not, but now that I see NONE are making it, this makes the problem less daunting.
Following is an excerpt from mailman.log. Note that the "first section" (from DOCTYPE at top to 404 at bottom) is repeated four times (maybe not identical; but similar) (so I did not include it to save room) before the Traceback appears.
Jun 25 00:13:04 2023 (386086) No cached copy of the public suffix list found Jun 25 00:13:04 2023 (386086) ACCEPT: <7500f0e2-1ae1-309d-60d7-6e592c7abf7a@west.net> Jun 25 00:13:04 2023 (386090) HyperKitty failure on https://lists.netlojix.com/hyperkitty/api/mailman/urls:
I just went to that URL and got a 404. This is a serious problem. https://lists.netlojix.com/hyperkitty/api/ also returns a 404. It should return some documentation of the API.
https://lists.netlojix.com/archives/api/ does work and returns the expected information.
Your urls.py file probably contains in its urlpatterns list
re_path(r'^mailman3/', include('postorius.urls')), re_path(r'^archives/', include('hyperkitty.urls')),The simplest fix is to add
re_path(r'^postorius/', include('postorius.urls')), re_path(r'^hyperkitty/', include('hyperkitty.urls')),to that list.
I see many ursl.py files all over the system, but none that appear to be where I'd expect them to be for processing by any executables. Is this a file I should have copied over and edited, or created from scratch and somehow missed? I don't see anything about it in the instructions at <https://docs.mailman3.org/en/latest/install/virtualenv.html> (although I do see a post from last July from someone stating it got missed in the instructions back then).
Another way to fix this is in your mailman-hyperkitty.cfg, you have
base_url: https://lists.netlojix.com/hyperkittyChange that to
base_url: https://lists.netlojix.com/archives
Done, and restarted both services, but I'm not seeing a difference when I go to <https://lists.netlojix.com/hyperkitty/api/> (I was getting a "Page not found", and still get that).
but it wouldn't hurt to add the
^postorius/and^hyperkitty/patterns to urls.py anyway.
On 6/26/23 10:12 PM, Ken Alker wrote:
--On Monday, June 26, 2023 5:03 PM -0700 Mark Sapiro <mark@msapiro.net> wrote:
Your urls.py file probably contains in its urlpatterns list
re_path(r'^mailman3/', include('postorius.urls')), re_path(r'^archives/', include('hyperkitty.urls')),The simplest fix is to add
re_path(r'^postorius/', include('postorius.urls')), re_path(r'^hyperkitty/', include('hyperkitty.urls')),to that list.
I see many ursl.py files all over the system, but none that appear to be where I'd expect them to be for processing by any executables. Is this a file I should have copied over and edited, or created from scratch and somehow missed? I don't see anything about it in the instructions at <https://docs.mailman3.org/en/latest/install/virtualenv.html> (although I do see a post from last July from someone stating it got missed in the instructions back then).
In your case, it is the one in /opt/mailman/venv/lib/python3.x/site-packages/mailman_web/, but it is not a good idea to change that as changes will be overwritten when mailman_web is updated. It's OK to not change it. Just do the following.
Another way to fix this is in your mailman-hyperkitty.cfg, you have
base_url: https://lists.netlojix.com/hyperkittyChange that to
base_url: https://lists.netlojix.com/archivesDone, and restarted both services, but I'm not seeing a difference when I go to <https://lists.netlojix.com/hyperkitty/api/> (I was getting a "Page not found", and still get that).
That's because going to the 'hyperkitty' url won't work, but nothing should be doing that so it's OK if it doesn't work. Fixing mailman-hyperkitty.cfg will tell mailman-hyperkitty to use the 'archives' url, so now it should be working. The next post after you fixed it should archive OK and all the messages in /opt/mailman/mm/var/archives/hyperkitty/spool/ should also be archived and the files removed.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Ken Alker -
Mark Sapiro