Is there a way to oerhaps custom-plugin my way through to set this up as an integration into Core?  And specify the custom handler process or such to allow that integration?  I see that there are ways to code plugins into existence so it may be a custom-processor plugin or such for this purpose.

Replies to threads would still have the corresponding RFC headers and such for replies and would properly get Hyperkitty to index threads properly, but subject munging on this case might be something I would custom-code into the system if that's doable.

Thomas

On 10/18/23 18:03, Mark Sapiro wrote:

On 10/18/23 14:54, Thomas Ward via Mailman-users wrote:

...

Is there any way to have Subject based thread tagging, etc.?  I ask this only because we're trying to thread items for easy ID in a system right now that does thread ID tagging in the subject line (i.e. "[Test-0000004] Something" and "Re: [Test-0000004] Something" are identified as the same thread, but a brand new message "Something" would end up as "[Test-0000005] Something" in the chain if it was a new message.)

It is not possible with the current design.

The issue is the tagging is done by Mailman core and Mailman core knows nothing about threads.

On 10/18/23 20:25, Mark Sapiro wrote:

On 10/18/23 15:06, Thomas Ward via Mailman-users wrote:

...

Is there a way to oerhaps custom-plugin my way through to set this up as an integration into Core?  And specify the custom handler process or such to allow that integration?  I see that there are ways to code plugins into existence so it may be a custom-processor plugin or such for this purpose.

The issue is Mailman core would have to keep track of the message-ids and thread number of all prior posts to know whether the current post is a reply to some other post or not.

You would need a new database table or some other mechanism to keep track of this and then you may need to keep everything forever because you can't know that there won't be a future post in reply to some very old post.

I think this is doable in our (read: Employer) use case, but without needing to modify Mailman Core much beyond a custom plugin and some fussing around.

This is a side effect of a custom implementation of "Tagging" data by thread ID in a separate data / intel tagging system that reads from received emails in a custom intel processing mailbox. That has zero care as to how Mailman or Hyperkitty index items, and actually is pretty trivially implemented with PREFIX format being [ListName-NNNNNNN]:

1. If subject does not match PREFIX format:
   • Get next thread number from DB (In postgres, this is a serial matched to {list_id}_thread we call with nextval())
   • Add subject prefix as [ListName-{ZeroPaddedThreadNumFromDB}]
2. If subject matches PREFIX form:
   • Do nothing

As for how HyperKitty handles this, it replies entirely on RFC 5322 fields (see section 3.6.4 in that RFC) for identifying reply to, etc. to identify replies to things, which in turn it seems is fully capable of handling that.  We simply need the message to be sent *with* a thread ID that members can hit reply to and keep the prefix in the subject line.  We don't care how that's done in the Hyperkitty side of things, we only care about it being sent in the subject line.

I noticed that Mailman3 has the capacity for external / custom plugins, and with some work I believe I can achieve this with a customized pipeline with a custom handler of thread-prefix instead of subject-prefix, otherwise calling the other handlers done by the traditional system, and just handle prefix differently in the thread-prefix handler in the custom pipeline.  I'm following this attempt currently, but will keep it internal to employer.  It'll also be kept internal to our use-case because it isn't easily exportable to other Mailman3 implementations.

But yes, you're right it needs additional DB table to map list_id <--> thread_id and then would need some kind of "Subject prefix uses thread IDs instead of Post IDs config option", or only handle based on a custom pipeline.

I"m going to deep dive into this on my end and see if I can come up with a solution that relies direct on the DB calls.  Which hopefully should be safe to do, but we'll see in testing.

Thomas