- On 4/18/25 04:23, Mark Sapiro wrote:
On 4/17/25 11:35 AM, Mihai Moldovan wrote:
New incoming messages will have the correct data, of course, but imported ones wouldn't, so I'll have to use a more sophisticated approach to handle them, probably by going through mailman's email wrapper, figuring out how to generate a msgdata object for a message, using RFC2369.process and maybe more.
Here's an example script. You need to run this with /opt/mailman/mm/venv/bin/python to get access to the mailman imports.
from mailbox import Maildir, mbox from mailman.email.message import Message from mailman.handlers.rfc2369 import process from mailman.interfaces.listmanager import IListManager from mailman.utilities.email import add_message_hash from zope.component import getUtility mb = mbox('path/to/mbox', factory=Message, create=False) md = Maildir('path/to/maildir', create=False) mlist = getUtility(IListManager).get_by_list_id('your.list.id') for msg in mb: add_message_hash(msg) process(mlist, msg, {}) md.add(msg) mb.close() md.close()
Thank you.
Unfortunately, I'm having a hard time getting this to work correctly. My current approach is (which is modified from yours, for instance to use Prototype.archive_message() instead of writing directly to a Maildir):
import copy
import sys
from mailbox import mbox
from mailman.archiving.prototype import Prototype
from mailman.core import initialize
from mailman.email.message import Message
from mailman.handlers.rfc_2369 import process
from mailman.interfaces.listmanager import IListManager
from mailman.interfaces.mailinglist import IMailingList
from mailman.utilities.email import add_message_hash
from zope.component import getUtility
initialize.initialize()
if (len(sys.argv) < 3):
print('Usage: {0} <list-id> <mbox-file>'.format(sys.argv[0]), file=sys.stderr)
exit(1)
mb = mbox(sys.argv[2], factory=Message, create=False)
mlist = getUtility(IListManager).get_by_list_id(sys.argv[1])
for msg in mb:
try:
add_message_hash(msg)
process(mlist, msg {})
Prototype.archive_message(mlist, msg)
except Exception as e:
print("Error when adding {0}: {1}".format(msg['message-id'], str(e)),
file=sys.stderr)
mb.close()
This returns "Error when adding None: '_PartialFile' object has no attribute 'header_max_count'" for each message.
This, including getting None for the Message-ID, stumped me and I got on to debugging this.
Indeed, even something as simple as
for msg in mb:
print(type(msg))
print(msg['message-id'])
print(msg)
exit(0)
results in getting a None for the Message-ID and a stack trace:
<class 'mailman.email.message.Message'> None Traceback (most recent call last): File "/root/mailman3/prototype-import.py", line 29, in <module> print(msg) File "/usr/lib/python3.12/email/message.py", line 165, in __str__ return self.as_string() ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/email/message.py", line 55, in as_string value = email.message.Message.as_string(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/email/message.py", line 188, in as_string g.flatten(self, unixfrom=unixfrom) File "/usr/lib/python3.12/email/generator.py", line 98, in flatten policy = policy.clone(max_line_length=self.maxheaderlen) ^^^^^^^^^^^^ AttributeError: '_PartialFile' object has no attribute 'clone'. Did you mean: 'close'?
This eventually led me to realize that passing mailman.email.message.Message as the factory to mbox() internally calls factory(msg), and the base class of Message (email.message.Message) has an __init__ function that takes the policy parameter, so it looks as if it's registering the mailbox message incorrectly as the policy handler, which is totally wrong of course.
If I change the call to mb = mbox(sys.argv[2], factory=None, create=False), the output looks more promising, so converting the data to mailman.email.message.Message first seems to have been the wrong idea on my part:
<class 'mailbox.mboxMessage'>
<mailman.1.1399151409.10314.x2go-i18n@lists.x2go.org>
Return-Path: <mailman-bounces@lists.x2go.org>
[...]
and indeed, if I let the import actually happen, it works.
The imported messages, do have a Message-ID-Hash, but the Archived-At and List-Archive headers are empty (literally <>).
Do you have an example message that was archived through Prototype? What are the proper header values for this module? I believe that Archived-At should be the Message-ID-Hash and the List-Archive header contain... well, given that the Prototype archiver is not meant to be publicly available, probably a file:// URL?
Mihai