Migration from 2.1 to 3
Hi,
I am in the process to migrate an old 2.1 installation to 3.2.1 (Debian 10.1) Following the 2.1 to 3 docs:
- Create a new list
- Use the import21 command The command runs successfully (at least no errors in stdout). There is a problem when sending an email to the newly imported list, the message never get send to the members. I do get moderation emails, and if i do a mass subscribe, the welcome email is received by the new members.
In a fresh 3 new list (not imported from 2.1) this does not happen.
I already tried looking for clues in the logs
[logging.smtp] path: debug.log level: debug
[logging.smtp-failure] path: debug.log level: debug
[logging.error] path: debug.log level: debug
But i haven't seen anything useful yet. Any hints on this?
On 12/20/19 7:26 AM, brunol@agro.uba.ar wrote:
Hi,
I am in the process to migrate an old 2.1 installation to 3.2.1 (Debian 10.1) Following the 2.1 to 3 docs:
- Create a new list
- Use the import21 command The command runs successfully (at least no errors in stdout). There is a problem when sending an email to the newly imported list, the message never get send to the members. I do get moderation emails, and if i do a mass subscribe, the welcome email is received by the new members.
Are you saying that when a post is sent to the list, it is held for moderation, and when you approve it, it is not delivered? Or are the moderation messages from some other lists?
Are there any non-digest members with delivery enabled?
Is the MTA (which MTA) delivering the message to Mailman's REST runner?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hello Mark,
Are you saying that when a post is sent to the list, it is held for moderation, and when you approve it, it is not delivered?
This is exactly what is happening.
Are there any non-digest members with delivery enabled?
Yes, there is. Some from the imported pck file and some created trough mailman-web
Is the MTA (which MTA) delivering the message to Mailman's REST runner?
I assume it is (postfix) since the message is being held for moderation.
Dec 23 12:35:29 2019 (694) HOLD: lispruebas@mailman3.agro.uba.ar post from brunol@agro.uba.ar held, message-id=CAOwJ5NX=_bUnfb5ZNqVLFeF3EeOP0GMJW7NeeOWZqVmn5ViyBg@mail.gmail.com: The message comes from a moderated member
Then the user receives the email notification stating that the post awaits moderator approval. The admin receives notification too.
Dec 23 12:35:47 2019 (701) held message approved, message-id: CAOwJ5NX=_bUnfb5ZNqVLFeF3EeOP0GMJW7NeeOWZqVmn5ViyBg@mail.gmail.com
And then nothing. None of the members receives the message
On 12/23/19 7:52 AM, brunol@agro.uba.ar wrote:
Dec 23 12:35:47 2019 (701) held message approved, message-id: CAOwJ5NX=_bUnfb5ZNqVLFeF3EeOP0GMJW7NeeOWZqVmn5ViyBg@mail.gmail.com
And then nothing. None of the members receives the message
Are you sure none of the members receive it or is it just you that doesn't receive it?
If it's just you, I note the message-id is @mail.gmail.com and I wonder if the issue is https://wiki.list.org/x/4030680.
What's in the MTA log for the time immediately after you approve the message?
Are there any errors/tracebacks in the mailman.log after the above approval message?
If a message is posted to the list that isn't held, is that message delivered?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Are you sure none of the members receive it or is it just you that doesn't receive it?
If it's just you, I note the message-id is @mail.gmail.com and I wonder if the issue is https://wiki.list.org/x/4030680.
I am sure of that since we have access to all the recipients (it is a list we use for testing purposes) Plus, a couple of them are hotmail.
What's in the MTA log for the time immediately after you approve the message?
Nothing.
Are there any errors/tracebacks in the mailman.log after the above approval message?
No errors/tracebacks in mailman.log. This is the output of debug.log configured as shown in my first post:
Dec 26 10:02:37 2019 (696) CAOwJ5NW3sYhUCCJAVS7ZdiwtyU8SGrArfnPTt5052m1H+V=+nQ@mail.gmail.com to: lispruebas@mailman3.agro.uba.ar, list: lispruebas, sub: None, dom: mailman3.agro.uba.ar
Dec 26 10:02:37 2019 (696) CAOwJ5NW3sYhUCCJAVS7ZdiwtyU8SGrArfnPTt5052m1H+V=+nQ@mail.gmail.com subaddress: None, queue: in
Dec 26 10:02:37 2019 (696) b'250 Ok\r\n'
Dec 26 10:02:37 2019 (696) _handle_client readline: b'QUIT\r\n'
Dec 26 10:02:37 2019 (696) ('127.0.0.1', 43354) Data: b'QUIT'
Dec 26 10:02:37 2019 (696) b'221 Bye\r\n'
Dec 26 10:02:37 2019 (696) ('127.0.0.1', 43354) connection lost
Dec 26 10:02:37 2019 (696) Connection lost during _handle_client()
Dec 26 10:02:38 2019 (699) Using agent:
Those "Using agent: If a message is posted to the list that isn't held, is that message
delivered? This is how i am testing right now, but still the same. Is there any relevant logger i should look on debug too besides the ones i already configured?
I've also noticed that the messages are stuck in the "message" table (postgresql) and the corresponding files exists under /var/lib/mailman3/messages/
On 12/26/19 5:22 AM, brunol@agro.uba.ar wrote:
No errors/tracebacks in mailman.log. This is the output of debug.log configured as shown in my first post:
Dec 26 10:02:37 2019 (696) CAOwJ5NW3sYhUCCJAVS7ZdiwtyU8SGrArfnPTt5052m1H+V=+nQ@mail.gmail.com to: lispruebas@mailman3.agro.uba.ar, list: lispruebas, sub: None, dom: mailman3.agro.uba.ar Dec 26 10:02:37 2019 (696) CAOwJ5NW3sYhUCCJAVS7ZdiwtyU8SGrArfnPTt5052m1H+V=+nQ@mail.gmail.com subaddress: None, queue: in Dec 26 10:02:37 2019 (696) b'250 Ok\r\n' Dec 26 10:02:37 2019 (696) _handle_client readline: b'QUIT\r\n' Dec 26 10:02:37 2019 (696) ('127.0.0.1', 43354) Data: b'QUIT' Dec 26 10:02:37 2019 (696) b'221 Bye\r\n' Dec 26 10:02:37 2019 (696) ('127.0.0.1', 43354) connection lost Dec 26 10:02:37 2019 (696) Connection lost during _handle_client() Dec 26 10:02:38 2019 (699) Using agent:
Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Dec 26 10:02:38 2019 (699) Using agent: Those "Using agent:
What do you mean by constantly appearing? Do you mean endlessly repeating or that they just repeat a number of times equal to the number of expected recipients of the post?
On 12/26/19 10:24 AM, brunol@agro.uba.ar wrote:
I've also noticed that the messages are stuck in the "message" table (postgresql) and the corresponding files exists under /var/lib/mailman3/messages/
This indicates the held message has not been removed from the held message queue. You previously posted that you saw a log message
Dec 23 12:35:47 2019 (701) held message approved, message-id: CAOwJ5NX=_bUnfb5ZNqVLFeF3EeOP0GMJW7NeeOWZqVmn5ViyBg@mail.gmail.com
This message comes from mailman.app.moderator.handle_message() (in mailman/app/moderator.py). After logging that message, it requeues it in the 'pipeline' queue for further processing and deletes it from the message table and /var/lib/mailman3/messages/.
This indicates that possibly subsequent processing is holding the message again.
Is there any relevant logger i should look on debug too besides the ones i already configured?
set
[logging.debug] level: debug
in mailman.cfg to get more in debug.log.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
What do you mean by constantly appearing? Do you mean endlessly repeating or that they just repeat a number of times equal to the number of expected recipients of the post?
Sorry, I meant endlessly repeating.
This is a piece from debug after sending an email to the list from a recipient that does not need moderation:
Dec 27 11:41:10 2019 (11473) [outgoing]
This is endlessly repeating with different filebase
On 12/27/19 6:54 AM, brunol@agro.uba.ar wrote:
What do you mean by constantly appearing? Do you mean endlessly repeating or that they just repeat a number of times equal to the number of expected recipients of the post?
Sorry, I meant endlessly repeating.
This is a piece from debug after sending an email to the list from a recipient that does not need moderation:
Dec 27 11:41:10 2019 (11473) [outgoing]
: CAOwJ5NX+K3CORg9riHc2T1dbdKZLZojHT1MVs44Wd3SbTJju6A@mail.gmail.com Dec 27 11:41:10 2019 (11473) Using agent: Dec 27 11:41:10 2019 (11468) [BounceRunner] starting oneloop Dec 27 11:41:10 2019 (11468) [BounceRunner] ending oneloop: 0 Dec 27 11:41:10 2019 (11473) Cannot connect to SMTP server 127.0.0.1 on port 25
This message is logged only once, but the issue occurs every time.
deliver is getting a socket.error exception on its attempt to connect to the MTA. Thus, sending of this message fails and the message is requeued and the process repeats.
Is an MTA listening on 127.0.0.1 on port 25?
The only question is if we can't send this mail, how can we send the 'held for moderation' notices?. These are sent the same way.
Dec 27 11:41:10 2019 (11473) [OutgoingRunner] finishing filebase: 1577457670.0510046+6fe6039c1b0b65672aade3bfced1aca2fe8afa16 Dec 27 11:41:10 2019 (11473) [OutgoingRunner] doing periodic Dec 27 11:41:10 2019 (11473) [OutgoingRunner] committing transaction Dec 27 11:41:10 2019 (11473) [OutgoingRunner] checking short circuit Dec 27 11:41:10 2019 (11473) [OutgoingRunner] ending oneloop: 1 Dec 27 11:41:10 2019 (11473) [OutgoingRunner] starting oneloop Dec 27 11:41:10 2019 (11473) [OutgoingRunner] processing filebase: 1577457670.5784276+6f5454f4a57da4dc15912e789f0719be53c74616 Dec 27 11:41:10 2019 (11473) [OutgoingRunner] processing onefile Dec 27 11:41:10 2019 (11473) [outgoing]
: CAOwJ5NX+K3CORg9riHc2T1dbdKZLZojHT1MVs44Wd3SbTJju6A@mail.gmail.com Dec 27 11:41:10 2019 (11473) Using agent: Dec 27 11:41:10 2019 (11470) [IncomingRunner] starting oneloop Dec 27 11:41:10 2019 (11470) [IncomingRunner] ending oneloop: 0 Dec 27 11:41:10 2019 (11473) [OutgoingRunner] finishing filebase: 1577457670.5784276+6f5454f4a57da4dc15912e789f0719be53c74616 Dec 27 11:41:10 2019 (11473) [OutgoingRunner] doing periodic Dec 27 11:41:10 2019 (11473) [OutgoingRunner] committing transaction Dec 27 11:41:10 2019 (11473) [OutgoingRunner] checking short circuit Dec 27 11:41:10 2019 (11473) [OutgoingRunner] ending oneloop: 1 Dec 27 11:41:10 2019 (11473) [OutgoingRunner] starting oneloop Dec 27 11:41:10 2019 (11473) [OutgoingRunner] processing filebase: 1577457670.61572+a2844469514ead14dedcee020ad8e2a34036f8ef Dec 27 11:41:10 2019 (11473) [OutgoingRunner] processing onefile Dec 27 11:41:10 2019 (11473) [outgoing] : CAOwJ5NX+K3CORg9riHc2T1dbdKZLZojHT1MVs44Wd3SbTJju6A@mail.gmail.com Dec 27 11:41:10 2019 (11473) Using agent: Dec 27 11:41:10 2019 (11473) [OutgoingRunner] finishing filebase: 1577457670.61572+a2844469514ead14dedcee020ad8e2a34036f8ef Dec 27 11:41:10 2019 (11473) [OutgoingRunner] doing periodic Dec 27 11:41:10 2019 (11473) [OutgoingRunner] committing transaction Dec 27 11:41:10 2019 (11473) [OutgoingRunner] checking short circuit Dec 27 11:41:10 2019 (11473) [OutgoingRunner] ending oneloop: 1 Dec 27 11:41:10 2019 (11473) [OutgoingRunner] starting oneloop Dec 27 11:41:10 2019 (11473) [OutgoingRunner] processing filebase: 1577457670.6512213+13fe7854257bb1a494f797f480cb10733bd440ac Dec 27 11:41:10 2019 (11473) [OutgoingRunner] processing onefile Dec 27 11:41:10 2019 (11473) [outgoing] : CAOwJ5NX+K3CORg9riHc2T1dbdKZLZojHT1MVs44Wd3SbTJju6A@mail.gmail.com Dec 27 11:41:10 2019 (11473) Using agent: Dec 27 11:41:10 2019 (11469) [CommandRunner] starting oneloop Dec 27 11:41:10 2019 (11469) [CommandRunner] ending oneloop: 0 This is endlessly repeating with different filebase
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
That message is also present when i post to the other list (the one that is not imported from 2.1) and the message is delivered, that's odd.
It is listening on 127.0.0.1 on port 25 and i've tried changing it to an upper port (2525) with no luck.
Thanks for the help so far, i'll keep digging.
I've found something, the imported list preferred language is Spanish (es), changing it to 'en' :
mailman shell -l lispruebas@mailman3.agro.uba.ar m._preferred_language 'es' m._preferred_language = 'en'
seems to fix everything, as soon as i hit ctrl+d the messages are delivered properly.
Even when i set it to french which has the same charset as spanish it works as expected. Any ideas why?
On 1/3/20 7:55 AM, brunol@agro.uba.ar wrote:
I've found something, the imported list preferred language is Spanish (es), changing it to 'en' : ... seems to fix everything, as soon as i hit ctrl+d the messages are delivered properly.
Even when i set it to french which has the same charset as spanish it works as expected. Any ideas why?
Actually, since Mailman 3.2.1, the default charset for 'fr' is utf-8, not iso-8859-1, although I don't offhand understand why the language would affect this even if the encoding is different. In any case, at this level the message should be pure ascii with any non-ascii having been base64 or quoted_printable encoded.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I'm running into the same issue, but after some troubleshooting I think the issue isn't with the import21 command. I'm running into the same issue when I create a list using 'mailman create' without importing anything. New lists created through Postorius work fine.
On 1/8/20 6:38 AM, Ryan Morash wrote:
I'm running into the same issue, but after some troubleshooting I think the issue isn't with the import21 command. I'm running into the same issue when I create a list using 'mailman create' without importing anything. New lists created through Postorius work fine.
Can you please post the mailman create command options you are using
to create the list.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I figured it out. It was a stupid mistake on my part, running the ‘mailman create’ commands as root… chown’ing the /var/lib/mailman3/lists directories to lists:lists solved the issues.
From: Mark Sapiro mark@msapiro.net Date: Wednesday, January 8, 2020 at 1:20 PM To: mailman-users@mailman3.org mailman-users@mailman3.org Subject: [MM3-users] Re: Migration from 2.1 to 3 On 1/8/20 6:38 AM, Ryan Morash wrote:
I'm running into the same issue, but after some troubleshooting I think the issue isn't with the import21 command. I'm running into the same issue when I create a list using 'mailman create' without importing anything. New lists created through Postorius work fine.
Can you please post the mailman create command options you are using
to create the list.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://linkprotect.cudasvc.com/url?a=https%3a%2f%2flists.mailman3.org%2fmailman3%2flists%2fmailman-users.mailman3.org%2f&c=E,1,h15CfHaOdoO_ah1ID2WFa90FdJqrh3TMHkipShbxTwqnSUtPwoySMcv5o1ahU8KCaJGOlf8KYGScId7zFUqIOEHa06wwK74ZmXIbaNDz7tbSlLjjJ5DncEG8Bw,,&typo=1 Confidential Transmission This correspondence is confidential and may contain privileged materials or information. It is intended for the use of the addressee only and all others are prohibited from copying or disseminating this message or its contents. If you have received this transmission in error, delete it immediately and notify the sender.
participants (4)
-
brunol＠agro.uba.ar -
Mark Sapiro -
Ryan Morash -
Ryan Morash