On 9/20/22 14:50, Dan Caballero wrote:

2022-09-20 14:44:17 [FALCON] [ERROR] GET /3.1/lists/jewishlife@caltech.edu/held?count=10&page=1 => Traceback (most recent call last): File "falcon/app.py", line 365, in falcon.app.App.__call__ File "/opt/mailmanve/lib/python3.9/site-packages/mailman/rest/post_moderation.py", line 182, in on_get resource = self._make_collection(request) File "/opt/mailmanve/lib/python3.9/site-packages/mailman/rest/helpers.py", line 182, in _make_collection entries = [as_dict(resource) for resource in collection] File "/opt/mailmanve/lib/python3.9/site-packages/mailman/rest/helpers.py", line 182, in <listcomp> entries = [as_dict(resource) for resource in collection] File "/opt/mailmanve/lib/python3.9/site-packages/mailman/rest/post_moderation.py", line 173, in _resource_as_dict assert resource is not None, resource AssertionError: None

This is most likely a race condition between two browsers, one of which handles the held message while the other is getting the held messages.

Check Mailman's access.log entries with time stamps around 2022-09-20 14:44:17.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

I'm getting an unexpected error when I try to clear all the held messages. Any ideas what could be wrong here?

(mailmanve) mailman@11f52637274a:/var/www/mailman-www$ mailman shell -l jewishlife@caltech.edu Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. The variable 'm' is the jewishlife@caltech.edu mailing list

...

...

from mailman.app.moderator import handle_message requestdb = IListRequests(m) for req in requestdb.held_requests: ... if req.request_type == RequestType.held_message: ... handle_message(m, req.id, Action.discard) ... Traceback (most recent call last): File "<console>", line 3, in <module> File "/opt/mailmanve/lib/python3.9/site-packages/mailman/app/moderator.py", line 112, in handle_message key, msgdata = requestdb.get_request(id) TypeError: cannot unpack non-iterable NoneType object

Looks like there's something missing.

What should I import from mailman.interfaces?

" >>> from mailman.interfaces."

== Dan

On 9/21/22 10:39, Dan Caballero wrote:

I figured out the correct import based on what was done previously. It worked.

Why do you think you need to import anything?

">>> from mailman.app.moderator import handle_message

handle_message is not referenced in the following and doesn't need to be imported here.

>>>> requestdb = IListRequests(m)
>>>> for req in requestdb.held_requests:
> ...    if req.request_type == RequestType.held_message:
> ...        if not requestdb.get_request(req.id):
> ...            print(f'Bad request id: {req.id}, deleting')
> ...            requestdb.delete_request(req.id)
> ...
> Bad request id: 1694, deleting
> Bad request id: 2509, deleting
> Bad request id: 2727, deleting
> Bad request id: 3251, deleting
> Bad request id: 3265, deleting
> Bad request id: 3371, deleting
> Bad request id: 4844, deleting
> Bad request id: 6058, deleting
> Bad request id: 6201, deleting
> Bad request id: 6432, deleting
> Bad request id: 6502, deleting
>>>> commit ()"

I assume after doing this, you were able to access the held messages view for the list.

Note that the only names referenced in the above and not defined there are IListRequests and RequestType, but these are among the many names implicitly imported by mailman shell in interactive mode.

$ mailman shell
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.

 >>> for name in dir():
...     print(name)
...
ALL_TEMPLATES
ALT_TEMPLATE_NAMES
ALWAYS_REPLY
AcceptEvent
AcceptOwnerEvent
Action
AddressAlreadyLinkedError
AddressError
AddressNotLinkedError
AddressVerificationEvent
AlreadySubscribedError
ArchivePolicy
ArchiveRenderingMode
BadDomainSpecificationError
BounceContext
ChainEvent
ClobberDate
ConfigurationUpdatedEvent
ContinueProcessing
DMARCMitigateAction
DatabaseError
DeliveryMode
DeliveryStatus
DigestFrequency
DiscardEvent
DiscardMessage
DomainCreatedEvent
DomainCreatingEvent
DomainDeletedEvent
DomainDeletingEvent
DuplicateStyleError
EmailError
ExistingAddressError
FilterAction
FilterType
HoldEvent
IAPI
IAcceptableAlias
IAcceptableAliasSet
IAddress
IArchiver
IAutoResponseRecord
IAutoResponseSet
IBan
IBanManager
IBounceEvent
IBounceProcessor
ICLISubCommand
ICacheManager
IChain
IChainIterator
IChainLink
IConfiguration
IContentFilter
IDatabase
IDatabaseFactory
IDomain
IDomainManager
IEmailCommand
IEmailResults
IEmailValidator
IHandler
IHeaderMatch
IHeaderMatchList
ILanguage
ILanguageManager
IListArchiver
IListArchiverSet
IListManager
IListRequests
IMailTransportAgentAliases
IMailTransportAgentDelivery
IMailTransportAgentLifecycle
IMailingList
IMember
IMembershipManager
IMessage
IMessageStore
IMutableChain
IOneLastDigest
IPendable
IPended
IPendedKeyValue
IPendings
IPipeline
IPlugin
IPreferences
IRoster
IRule
IRunner
IStyle
IStyleManager
ISubscriptionManager
ISubscriptionService
ISwitchboard
ITemplate
ITemplateLoader
ITemplateManager
IUser
IUserManager
IWorkflowState
IWorkflowStateManager
InvalidConfigurationError
InvalidEmailAddressError
InvalidListNameError
LinkAction
ListAlreadyExistsError
ListCreatedEvent
ListCreatingEvent
ListDeletedEvent
ListDeletingEvent
MailmanError
MemberRole
MembershipChangeEvent
MembershipError
MembershipIsBannedError
MissingConfigurationFileError
MissingPreferredAddressError
MissingUserError
NewsgroupModeration
NoSuchListError
NotAMemberError
PasswordChangeEvent
PendType
Personalization
RejectEvent
RejectMessage
ReplyToMunging
RequestRecord
RequestType
Response
ResponseAction
RunnerCrashEvent
RunnerInterrupt
SomeRecipientsFailed
SubscriptionConfirmationNeededEvent
SubscriptionEvent
SubscriptionInvitationNeededEvent
SubscriptionMode
SubscriptionPendingError
SubscriptionPolicy
TokenOwner
TooManyMembersError
UnrecognizedBounceDisposition
UnsubscriptionConfirmationNeededEvent
UnsubscriptionEvent
UnverifiedAddressError
__builtins__
abort
commit
config
getUtility
m
 >>>

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 9/21/22 16:24, Dan Caballero wrote:

Mark,

I'm seeing this issue pop up this week with several lists. You previously mentioned two conflicting browsers potentially causing the issue. Can you elaborate?

That would only cause a transient issue which would not occur on a subsequent get of the held message view. I gather this is not the case here. I.e. once the error occurs on a get of the held message view, it is persistent.

In addition, is there a way for me to run this cleanup using mailman shell -r "somescript" so that I can check all the lists instead of one by one?

For a -r script, try

# In non-interactive scripts, we need the explicit imports.
from mailman.interfaces.listmanager import IListManager
from mailman.interfaces.requests import IListRequests, RequestType
from zope.component import getUtility

def somescript():
     for mlist in getUtility(IListManager):
         requestdb = IListRequests(mlist)
         for req in requestdb.held_requests:
             if req.request_type == RequestType.held_message:
                 if not requestdb.get_request(req.id):
                     print(f'List: {mlist.list_id}, Bad request id: 
{req.id}, deleting')
                     requestdb.delete_request(req.id)

The real question here is what are these spurious requests and how are they occurring. If you do a query like

SELECT * FROM _request;

Are there entries with empty data_hash or entries with an id that is not the value of any pended_id in the pendedkeyvalue table.

BTW, I understand why you thought an import was needed in the interactive script in my message at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...

• the from mailman.interfaces. line was a mistake on my part and shouldn't have been there.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 9/26/22 11:06, Dan Caballero wrote:

Even after the cleanup using the script above the pendedkeyvalue table still has 648938 rows. Is this all just poor moderation from our owners/moderators not managing their lists?

See https://gitlab.com/mailman/mailman/-/issues/853 and perhaps other issues referenced in that one.

What Mailman core version is this? 3.3.5 fixed a lot of this.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 3/30/23 07:36, Gregory Edigarov wrote:

I have the same issue now. that's happen when I try to open held messages

This is what I have in my logs:

File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 167, in _resource_as_dict assert resource is not None, resource AssertionError: None

What is the full traceback?

that is the default mailman3 debian package.

And your primary support resource should be Debian.

GNU Mailman 3.3.3 (Tom Sawyer)

things were working smoothly for a very long time. it started after disk array disaster. we installed the thing to another server, and restored the database from backups.

I've tried to use this magic:

>    The variable 'm' is the odoo_leads@qarea.org mailing list
>>>> from mailman.app.moderator import handle_message
>>>> requestdb = IListRequests(m)
>>>> for req in requestdb.held_requests:
> ...  if req.request_type == RequestType.held_message:
> ...     if not requestdb.get_request(req.id):
> ...         print(f'Bad request id: {req.id}, deleting')
> ...         requestdb.delete_request(req.id)
> ...
>>>>

but no success. is there any way to fix this?

I think you're on the right track with the above mailman shell, but evidently the issue is different.

You might try

 >>> requestdb = IListRequests(m)
 >>> for req in requestdb.of_type(RequestType.held_message):
...     if not requestdb.get_request(req.id):
...         print(f'Bad request id: {req.id}, deleting')
...         requestdb.delete_request(req.id)
...     else:
...         print(f'Held request: {requestdb.get_request(req.id)}
...
 >>>

To see what the requests look like.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan