Dear Robert.

Thanks so much for your prompt answer. I am afraid this is not feasible, as this MTA is also serving other purposes, hence impacting not only the messages addressed to distribution lists, but actually all of them. Besides, we would not like to resort to relaying in mail servers we do not control for retries, as they may have so heterogeneous behaviours regarding how the subsequent attempts are dealt.

I would you to also please confirm whether there is a way to stop just Hyperkitty, or Django maintenance tasks. I would like you to please confirm whether the runners need the backend available just to deliver mails to the members of the distribution lists.

Many thanks for your time once more. Best regards.

-----"Robert Moody" <[1]robert@kneedrag.org> wrote: -----

To: [2]eugenio.jordan@esa.int From: "Robert Moody" <[3]robert@kneedrag.org> Date: 08/23/2021 06:09AM Cc: [4]mailman-users@mailman3.org Subject: Re: [MM3-users] Mailman backend maintenance task A simple solution would be to stop the MTA in front of mailman. That is assuming the MTA is only delivering to mailman and nothing else. This would force any mails being sent during the maintenance to queue up on the sending server and once the maintenance is done starting the MTA will allow mails that were queued on the sending MTAs to get delivered. Downside is mails will be delayed based on the sending servers retry settings. Get [5]BlueMail for Android On 23 Aug 2021, at 05:55, [6]eugenio.jordan@esa.int wrote:

Dear all. My apologies if the answer to this question is obvious, but I am pretty newbie for 3 version, and to be very honest I haven't had enough time yet to have a deep look into the documentation. Our customer is currently using PostGRESQL as backend, and we would like to perform some maintenance tasks, namely running vacuum full, or at least trying to rebuild hyperkitty_email primary key related index. We have been asked on the real impact of putting in place such initiative. Though the latter is related to archiving, I haven't found a way to stop just Hyperkitty or Django related processes other than stopping Mailman's core, hence preventing mails addressed to distribution lists from being delivered, could you please confirm if I am correct? Regarding the former, as far as I have read, the "mappings" lists -> addresses are stored just in the database, so if we run some kind of procedure or task like vacuum which will lock exclusively tables, or want anyway to have the database stopped for a cold backup or whatever, Mailman willl not work, that is, again the mails addressed to the distribution lists will not be delivered. Will you please confirm this point, too? Really wish we had a testing environment to have been able to check this ourselves and not bothering others, but unfortunately we lack it, and we need to provide an answer as soon as possible. Thanks a lot for your time. Best regards. __________________________________________________________________

Mailman-users mailing list -- [7]mailman-users@mailman3.org To unsubscribe send an email to [8]mailman-users-leave@mailman3.org [9]https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org /

This message is intended only for the recipient(s) named above. It may contain p roprietary information and/or protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received this e-mail in error, please notify the sender immediately. ESA applies appropri ate organisational measures to protect personal data, in case of data privacy queries, please contact the ESA Data Prot ection Officer (dpo@esa.int).

References

1. mailto:robert@kneedrag.org
2. mailto:eugenio.jordan@esa.int
3. mailto:robert@kneedrag.org
4. mailto:mailman-users@mailman3.org
5. https://bluemail.me/
6. mailto:eugenio.jordan@esa.int
7. mailto:mailman-users@mailman3.org
8. mailto:mailman-users-leave@mailman3.org
9. https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org

Dear Raj.

Thanks a lot for your tips.

I would like you to please confirm whether you mean I can stop Hiperkitty by just shutting down the Web components instance (actually, a Docker container). Does this mean, therefore, Hiperkitty is to be considered as part of the Web Components?

Just for my information, because I'm afraid that both Core and Hiperkitty share the very same database:

[database] class: mailman.database.postgresql.PostgreSQLDatabase url: postgres://mailman:mailmanpass@database/mailmandb

This is the very same database used by Hyperkitty, so I am afraid that I will have to end up stopping also the Core components, or, anyway, if they cannot access the database "normally" for whatever reason, they will not be delivering the mails to the different distribution lists members and we will have to rely on the external mail servers retries. Could you please confirm this assumption is right?

Once more, I really appreciate you kindly took your time to answer and explain about Mailman architecture. Best regards.

-----"Abhilash Raj" <[1]maxking@asynchronous.in> wrote: -----

Dear all.

My apologies if the answer to this question is obvious, but I am

newbie for 3 version, and to be very honest I haven't had enough time yet to have a deep look into the documentation.

Our customer is currently using PostGRESQL as backend, and we would like to perform some maintenance tasks, namely running vacuum full, or at least trying to rebuild hyperkitty_email primary key related index. We have been asked on the real impact of putting in place such initiative. Though the latter is related to archiving, I haven't found a way to stop just Hyperkitty or Django related processes other than stopping Mailman's core, hence preventing mails addressed to distribution lists from being delivered, could you please confirm if I am correct? You can indeed stop Hyperkitty separate from stopping Core. Depending on how your client installed Mailman, there are two separate

To: [2]eugenio.jordan@esa.int From: "Abhilash Raj" <[3]maxking@asynchronous.in> Date: 08/23/2021 07:31PM Cc: "Mailman users" <[4]mailman-users@mailman3.org> Subject: Re: [MM3-users] Mailman backend maintenance task On Sun, Aug 22, 2021, at 10:54 AM, [5]eugenio.jordan@esa.int wrote: pretty scripts/processes/commands to start and stop Mailman Core and Web components. You should be able to safely stop Hyperkitty, run maintenance and re-start it. Core will continue to function during this duration, sending out emails and such. The archiving will obviously not happen, but it will be queued and Core we retry to archive that, which should succeed whenever HK comes back up.

Regarding the former, as far as I have read, the "mappings" lists -> addresses are stored just in the database, so if we run some kind of procedure or task like vacuum which will lock exclusively tables, or want anyway to have the database stopped for a cold backup or

Mailman willl not work, that is, again the mails addressed to the distribution lists will not be delivered. Will you please confirm

whatever, this

point, too?

Both Core and Web components have separate databases. Although, technically speaking, the database name is passed in through configs and you can pass in the same database name to both and since they use separate table names, there is a good chance it would work. Considering that your client has configured separate databases for both, you can run maintenance for the Web database without affecting any operation of Core (other than archiving). Core stores the mappings and data related to MailingLists. While Hyperkitty also has a partial copy of that data in its database that it needs to maintain the archives, that data can be synchronized with a command and is also done routinely using a cron job. A database backup would be a good idea either way. Hope this helps!

 thanks,
 Abhilash Raj (maxking)

This message is intended only for the recipient(s) named above. It may contain p roprietary information and/or protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received this e-mail in error, please notify the sender immediately. ESA applies appropri ate organisational measures to protect personal data, in case of data privacy queries, please contact the ESA Data Prot ection Officer (dpo@esa.int).

References

1. mailto:maxking@asynchronous.in
2. mailto:eugenio.jordan@esa.int
3. mailto:maxking@asynchronous.in
4. mailto:mailman-users@mailman3.org
5. mailto:eugenio.jordan@esa.int

Dear Steve:

I really appreciate you also took your time to provide more insights on how Mailman works, and how their different components interact each other.

There are two "minor" tasks and one major task that we would need to accomplish within PostGRESQL, namely rebuilding the index related to hyperkitty_email table primary key, rebuilding also another index on the very same table supporting an uniqueness constraint (these would be the minor); and running vacuum full on the whole database to effectively reduce the size of precisely hyperkitty_email table, which is the largest table in this database, and if possible retrieve back to the SAN the extra space we had to add to cope with that very table size increasing.

The way vacuum full locks the tables are, among all the locking mechanisms provided by PostGRESQL to guarantee the transactional integrity, one of the strongest, if not the strongest one (as far as I know, the tables are more ore less recreated when running vacuum full).

The idea we were considering was more or less stopping all of the components, then starting up only PostGRESQL, and then performing each one of the mentioned tasks. Actually our PG release allow "online" indices rebuild (drop + rebuild concurrently), but we did not want to assume the risk if, for whatever reason, new records were added to the table breaking the uniqueness of both constraints (fixing this cases turns out to be a real pain in most cases). Alternatively, we could lock the table ourselves to get more or less the same effect, true, although, after all, the final impact on potential connections willing to run DML would be the same. In other words, rather than having lots of transactions coming from whether core component processes or web component processes fail, we prefer to stop everything. But it's got of course the worst downside: we would receive

Let me please now go back to your kind explanation to see if I did get things correctly. You mention that regardless the connectivity that Mailman's core processes might have with the database, the emails will be received by and queued until the connection is back again and they can be delivered to the distribution lists' members. Could you please confirm whether this task would be actually delegated to Postfix (or any MTA integrated with Mailman)? If so, this is actually what we need: we could afford delaying the reception a couple of days, whilst we are sure that no mail will be lost.

Once more, thanks a lot for your kind help. Best regards.

-----"Stephen J. Turnbull" <[1]stephenjturnbull@gmail.com> wrote: -----

To: [2]eugenio.jordan@esa.int From: "Stephen J. Turnbull" <[3]stephenjturnbull@gmail.com> Date: 08/24/2021 05:33PM Cc: [4]mailman-users@mailman3.org Subject: [MM3-users] Mailman backend maintenance task Hi, Eugenio, I wrote this earlier but am in the middle of moving my office so my mail has been intermittent. I have been following your discussion with Abhilash, and he's definitely the expert, especially if you are using the containers he creates and distributes. But there's some information here he hasn't mentioned yet. Please consider the following discussion to be background that allows you to understand some of the considerations. I've only run the Mailman 3 suite with all three running constantly on the same host, so I have no experience with this kind of issue. [5]eugenio.jordan@esa.int writes: > Our customer is currently using PostGRESQL as backend, and we would > like to perform some maintenance tasks, namely running vacuum full, > or at least trying to rebuild hyperkitty_email primary key related > index. We have been asked on the real impact of putting in place > such initiative. Though the latter is related to archiving, I > haven't found a way to stop just Hyperkitty or Django related > processes other than stopping Mailman's core, hence preventing > mails addressed to distribution lists from being delivered, could > you please confirm if I am correct? Mailman core can certainly run without either Postorius or HyperKitty. Controlling Mailman core (moderation, helping users) without Postorius is annoying, but it can be done. If you stop the HyperKitty process, what should happen, I believe, is that posts for archiving will accumulate in the 'archive' queue until HyperKitty is available again. It's been a while since I studied this so I could be completely wrong, but as I understand it HyperKitty and Postorius are not daemon processes. Rather, they are WSGI applications, which means they are subprocesses spawned by your webserver, and controlled by it. In order to keep them from running, you would reconfigure the webserver to not call those WSGI applications. How that is done is specific to the webserver and the WSGI module. If you are running from Docker containers, most likely, you can just stop their containers. The larger problem is that Mailman core uses a RDBMS. Normally both Django and Mailman are referring to the same RDBMS, PostgreSQL in your case. I'm not familiar with the vaccuum operation; if it requires taking the whole RDBMS down, and Mailman is running on that RDBMS, you're out of luck. Mailman can accept posts and queue them, but it can't deliver them to subscribers without access to the RDBMS tables. If it's just a matter of locking some tables while other are available, then it should work because the tables used by core and HyperKitty are disjoint as far as I know. (I think Postorius and HyperKitty both use Django's user tables for authorization, so at least for those tables both Postorius and HyperKitty will have to be down, but core can continue running because its database is completely separate.) > Regarding the former, as far as I have read, the "mappings" lists > -> addresses are stored just in the database, so if we run some > kind of procedure or task like vacuum which will lock exclusively > tables, or want anyway to have the database stopped for a cold > backup or whatever, Mailman willl not work, that is, again the > mails addressed to the distribution lists will not be > delivered. Will you please confirm this point, too? That depends on how "vacuum" works. If you can tell the RDBMS not to lock Mailman core's tables, it can continue to run. Definitely if the database is not running, Mailman will continue to receive and store the posts, but it won't be able to distribute mail to subscribers until its tables are available again. At that point the queued posts will be processed normally, except that if there's a large backlog, they won't go out in a quick burst, it may take a while. One possible worry, depending on how you are connected to the Internet, is that your provider may interpret the sudden burst of outgoing mail when Mailman comes back on line as spam or some other sort of mischief. Mailman has no way to throttle this: it feeds messages to the MTA until the MTA says "stop". Then it waits until the MTA is ready again. Regards, Steve

This message is intended only for the recipient(s) named above. It may contain p roprietary information and/or protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received this e-mail in error, please notify the sender immediately. ESA applies appropri ate organisational measures to protect personal data, in case of data privacy queries, please contact the ESA Data Prot ection Officer (dpo@esa.int).

References

1. mailto:stephenjturnbull@gmail.com
2. mailto:eugenio.jordan@esa.int
3. mailto:stephenjturnbull@gmail.com
4. mailto:mailman-users@mailman3.org
5. mailto:eugenio.jordan@esa.int

On Aug 24, 2021, at 11:22 AM, Stephen J. Turnbull <stephenjturnbull@gmail.com> wrote:

Eugenio Jordan (external) writes:

...

The idea we were considering was more or less stopping all of the components, then starting up only PostGRESQL, and then performing each one of the mentioned tasks.

In that case, your Mailman installation will basically be down (no web presence, no deliveries) for the duration of the database maintenance.

...

You mention that regardless the connectivity that Mailman's core processes might have with the database, the emails will be received by and queued until the connection is back again and they can be delivered to the distribution lists' members. Could you please confirm whether this task would be actually delegated to Postfix (or any MTA integrated with Mailman)?

If Mailman core is running, it will accept the mail and store it in its own queue. If not, the MTA will store it in its deferred queue until Mailman comes back up.

The big problem with delegating this to the MTA is that after two days the MTA probably won't retry for whole days, but if you do an MTA queue flush, it will probably stop other mail delivery for quite a while, since it will be occupied with processing the Mailman backlog.

The way Mailman works is that there is a master process, which manages a suite of runners. Each runner is responsible for a particular stage. The LMTP runner does nothing except accept messages from the MTA, and store each one in a file for the next runner in the chain.

Although, looking at the code1, it seems like LMTP runner will actually try to verify that the incoming email in destined to a valid email address before it will accept the email.

So, if the database isn’t reachable, I think LMTP runner will start rejecting emails from MTA. This is perhaps handling the situation where the transport maps are out-of-date and Mailman receives an email from MTA for a list that has been deleted.

While I haven’t tested this situation, I don’t think it would work. So far I am seeing that holding emails off at MTA level and letting it retry after the database maintenance has been complete might be the only thing that works if the entire Postgresql instance has to be brought down.

If MTA will retry continuously for 2 days, would that not be enough time to complete the migration and bring the database back up?

-- thanks, Abhilash Raj (maxking)