'Failed adding message' error when migrating a list from mailman2 to mailman3
Hello,
I'm in the process of migrating some of the lists from mailman2 to mailman3. I'm getting the below error message when I migrate a particular list.
*2%Failed adding message <mailman.2.1593699619.32248>: 'ascii' codec can't encode character '\ufffd' in position 252: ordinal not in range(128).*
I ran the check_hk_import script before migrating and it was not giving any error. Any idea how to fix this?
Kind regards, Prasanth
Prasanth Nair writes:
*2%Failed adding message <mailman.2.1593699619.32248>: 'ascii' codec can't encode character '\ufffd' in position 252: ordinal not in range(128).*
This is the Unicode REPLACEMENT CHARACTER, which means that something along the line, possibly Mailman, could not decode some bytes, and replaced them with that character. The message about "ASCII" codec usually means that this occurred in the header of the email, which is typical of spam. The message ID was generated by Mailman not by the originating system, which is also indicative of spam.
If it is spam, you can identify the message and delete it from the archive. If it's not, you will need to find the text that is being replaced and delete it or replace it with ASCII yourself.
I ran the check_hk_import script before migrating and it was not giving any error.
In order to catch this kind of error, the check script would have to pretty much do the import but throw it away.
On 10/14/21 9:04 AM, Stephen J. Turnbull wrote:
Prasanth Nair writes:
*2%Failed adding message <mailman.2.1593699619.32248>: 'ascii' codec can't encode character '\ufffd' in position 252: ordinal not in range(128).* ... I ran the check_hk_import script before migrating and it was not giving any error.
In order to catch this kind of error, the check script would have to pretty much do the import but throw it away.
The check_hk_import script <https://www.msapiro.net/scripts/check_hk_import> is designed to detect and report conditions which would cause the django hyperkitty_import process to abort and stop processing the mbox. It doesn't catch errors like the above, but these are non-fatal and just cause the offending message to be skipped.
As Steve says, the Message-ID was reported so find it in the mbox and if you want it in the archive, extract it from the mbox, fix it and then just rerun the import with just that message as input[1] and a --since=1970-01-01 option.
Note the issue could be in a header, but also could be in a body part implicitly or explicitly declared as charset="us-ascii".
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thank you, Mark.
Kind regards, Prasanth
On Thu, 14 Oct 2021 at 19:17, Mark Sapiro <mark@msapiro.net> wrote:
On 10/14/21 9:04 AM, Stephen J. Turnbull wrote:
Prasanth Nair writes:
*2%Failed adding message <mailman.2.1593699619.32248>: 'ascii' codec can't encode character '\ufffd' in position 252: ordinal not in range(128).* ... I ran the check_hk_import script before migrating and it was not giving any error.
In order to catch this kind of error, the check script would have to pretty much do the import but throw it away.
The check_hk_import script <https://www.msapiro.net/scripts/check_hk_import> is designed to detect and report conditions which would cause the django hyperkitty_import process to abort and stop processing the mbox. It doesn't catch errors like the above, but these are non-fatal and just cause the offending message to be skipped.
As Steve says, the Message-ID was reported so find it in the mbox and if you want it in the archive, extract it from the mbox, fix it and then just rerun the import with just that message as input[1] and a --since=1970-01-01 option.
Note the issue could be in a header, but also could be in a body part implicitly or explicitly declared as charset="us-ascii".
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Thank you, Stephen.
Kind regards, Prasanth
On Thu, 14 Oct 2021 at 17:04, Stephen J. Turnbull < stephenjturnbull@gmail.com> wrote:
Prasanth Nair writes:
*2%Failed adding message <mailman.2.1593699619.32248>: 'ascii' codec can't encode character '\ufffd' in position 252: ordinal not in range(128).*
This is the Unicode REPLACEMENT CHARACTER, which means that something along the line, possibly Mailman, could not decode some bytes, and replaced them with that character. The message about "ASCII" codec usually means that this occurred in the header of the email, which is typical of spam. The message ID was generated by Mailman not by the originating system, which is also indicative of spam.
If it is spam, you can identify the message and delete it from the archive. If it's not, you will need to find the text that is being replaced and delete it or replace it with ASCII yourself.
I ran the check_hk_import script before migrating and it was not giving any error.
In order to catch this kind of error, the check script would have to pretty much do the import but throw it away.
participants (3)
-
Mark Sapiro -
Prasanth Nair -
Stephen J. Turnbull