Question about message bounce stats
When I used Mailgun some time back, it had a feature of, for any specific outgoing mailing, being able to show you stats and other details of which mails were received ok, which bounced, and which were undeliverable, which I found helpful to check every so often. Their system also kept track of long-term trends, eg for addresses that frequently failed.
Does Mailman3 keep such stats, and if so is there any way to get access to them, even if only API? I would think it useful to do so (and yes, for individual outgoing messages at least for a period).
Regards,
Ruth
-- Tel: 01223 414180 Blog: http://www.ivimey.org/blog LinkedIn: http://uk.linkedin.com/in/ruthivimeycook/
Ruth Ivimey-Cook writes:
Does Mailman3 keep such stats, and if so is there any way to get access to them, even if only API? I would think it useful to do so (and yes, for individual outgoing messages at least for a period).
In released versions, it currently does not, as far as I know. This kind of thing should be available from your MTA, of course, and I think there was a visceral opposition to keeping track of people's behavior that way from the early days. So those are our excuses.
However, it's something that comes up occasionally on the list as well as as a GSoC proposal, and I think there's some code on a branch for this that Abhilash has been threatening to release. And of course HyperKitty now publishes stats on posting behavior. So there's really no reason to oppose it, now it's just a matter of time to implement.
Abhilash's participation has been kind of spotty recently (and apologetic about it, he's not going anywhere), but hopefully he'll pick up the thread shortly and at least confirm or deny what I wrote here.
Steve
Stephen,
Thanks for your comments. I kinda-thought that was the state of affairs but am glad there is at least some work on it in the pipeline.
If Abhilash has a starting point, and there is acceptance from others to integrate something of this sort, then I could possibly help make it happen.
Ruth
On 30/08/2021 16:46, Stephen J. Turnbull wrote:
Ruth Ivimey-Cook writes:
Does Mailman3 keep such stats, and if so is there any way to get access to them, even if only API? I would think it useful to do so (and yes, for individual outgoing messages at least for a period).
In released versions, it currently does not, as far as I know. This kind of thing should be available from your MTA, of course, and I think there was a visceral opposition to keeping track of people's behavior that way from the early days. So those are our excuses.
However, it's something that comes up occasionally on the list as well as as a GSoC proposal, and I think there's some code on a branch for this that Abhilash has been threatening to release. And of course HyperKitty now publishes stats on posting behavior. So there's really no reason to oppose it, now it's just a matter of time to implement.
Abhilash's participation has been kind of spotty recently (and apologetic about it, he's not going anywhere), but hopefully he'll pick up the thread shortly and at least confirm or deny what I wrote here.
Steve
-- Tel: 01223 414180 Blog: http://www.ivimey.org/blog LinkedIn: http://uk.linkedin.com/in/ruthivimeycook/
Ruth Ivimey-Cook writes:
If Abhilash has a starting point,
IIRC it was quite primitive, and probably not at the the individual message level, more like list level.
and there is acceptance from others to integrate something of this sort, then I could possibly help make it happen.
If you're that interested, I'd say get started, at least on sketching out requirements. There will probably be some nitpicking about code style and the like, and there may be delays in integrating stuff due to the peanut gallery contributing requirements that we decide we want in the feature, but I don't think there's a problem with the basic idea.
We won't be putting tracking pixels and the like in outgoing posts, but as long as it's internal data already generated by normal Mailman operation, I would be somewhere between very surprised and shocked (SHOCKED, I tell you!) if it weren't accepted.
Steve
On Aug 30, 2021, at 11:27 AM, Stephen J. Turnbull stephenjturnbull@gmail.com wrote:
We won't be putting tracking pixels and the like in outgoing posts, but as long as it's internal data already generated by normal Mailman operation, I would be somewhere between very surprised and shocked (SHOCKED, I tell you!) if it weren't accepted.
This would be a huge win for list admins everywhere. Thank you.
To be honest, you could probably just hard-code the bounce stats for any Oath property (Aol, Yahoo, Verizon, etc). 😂
- Mark
mark@pdc-racing.net | 408-348-2878
On Aug 30, 2021, at 11:27 AM, Stephen J. Turnbull stephenjturnbull@gmail.com wrote:
Ruth Ivimey-Cook writes:
If Abhilash has a starting point,
IIRC it was quite primitive, and probably not at the the individual message level, more like list level.
We do keep a bounce score for Member (email, mailinglist pair) that is right now only available in Mailman’s database and is used for the purposes of un-subscriptions or disabling delivery.
We haven’t had a chance to expose these values through REST API, but it is something I’d be happy to accept a patch for and isn’t too difficult if someone wants to pick up the task. It might be helpful at some point to expose some of those stats to the list owner or site admin as well in Postorius.
and there is acceptance from others to integrate something of this sort, then I could possibly help make it happen.
If you're that interested, I'd say get started, at least on sketching out requirements. There will probably be some nitpicking about code style and the like, and there may be delays in integrating stuff due to the peanut gallery contributing requirements that we decide we want in the feature, but I don't think there's a problem with the basic idea.
Yep, +1. It would be good to first figure out what more stats do we need/want and if they just need to be exposed through the API or we need to setup some offline setup for calculating those metrics with some mechanism and then expose.
We won't be putting tracking pixels and the like in outgoing posts, but as long as it's internal data already generated by normal Mailman operation, I would be somewhere between very surprised and shocked (SHOCKED, I tell you!) if it weren't accepted.
Yeah, anything that is possible to generate with regular Mailman operation is already possible for a site admin to get from logs with some scripting. So, the idea to expose them in a better UX I guess should be okay. Only thing we might want to consider is how much of that data is exposed to non-server-admins (list owners, mods etc) and the policy enforcement for that if users don’t like the default.
-- thanks, Abhilash Raj (maxking)
On Mon, Aug 30, 2021, at 8:46 AM, Stephen J. Turnbull wrote:
Ruth Ivimey-Cook writes:
Does Mailman3 keep such stats, and if so is there any way to get access to them, even if only API? I would think it useful to do so (and yes, for individual outgoing messages at least for a period).
In released versions, it currently does not, as far as I know. This kind of thing should be available from your MTA, of course, and I think there was a visceral opposition to keeping track of people's behavior that way from the early days. So those are our excuses.
However, it's something that comes up occasionally on the list as well as as a GSoC proposal, and I think there's some code on a branch for this that Abhilash has been threatening to release. And of course HyperKitty now publishes stats on posting behavior. So there's really no reason to oppose it, now it's just a matter of time to implement.
I don't think we have any changes since last release related to enhancements in bounce processing, there have been some bug fixes through which occasionally can cause bounce runner to cause some issues by keeping open transactions for quite long.
-- thanks, Abhilash Raj (maxking)
On Mon, 30 Aug 2021, Ruth Ivimey-Cook wrote:
When I used Mailgun some time back, it had a feature of, for any specific outgoing mailing, being able to show you stats and other details of which mails were received ok, which bounced, and which were undeliverable, which I found helpful to check every so often. Their system also kept track of long-term trends, eg for addresses that frequently failed.
Was mailgun forwarding messages with user-specific trackers embedded in them, or just reporting stats based on mail envelopes?
If you are referring to a 'list manager' that is used for one-way communication and specifically tracks individual users, possibly by using a tracking image in the content, you may want to look at phpList.org, which does generates various statisics. phpList is not suitable for a discussion list or many-to-many communications.
Brett
Brett,
Mailgun did use tracker images and other techniques, but also simple status codes; you don't need a tracker to notice a DSN or 500-bounce from the MTA. I'm not looking for something like phplist.org.
What I am asking for/hoping for is something that wraps up whatever sources of info are available and at least associates it with a list, if not per message.
Regards,
Ruth
On 30/08/2021 17:34, Brett Delmage wrote:
On Mon, 30 Aug 2021, Ruth Ivimey-Cook wrote:
When I used Mailgun some time back, it had a feature of, for any specific outgoing mailing, being able to show you stats and other details of which mails were received ok, which bounced, and which were undeliverable, which I found helpful to check every so often. Their system also kept track of long-term trends, eg for addresses that frequently failed.
Was mailgun forwarding messages with user-specific trackers embedded in them, or just reporting stats based on mail envelopes?
If you are referring to a 'list manager' that is used for one-way communication and specifically tracks individual users, possibly by using a tracking image in the content, you may want to look at phpList.org, which does generates various statisics. phpList is not suitable for a discussion list or many-to-many communications.
Brett
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- Tel: 01223 414180 Blog: http://www.ivimey.org/blog LinkedIn: http://uk.linkedin.com/in/ruthivimeycook/
On Mon, 30 Aug 2021, Ruth Ivimey-Cook wrote:
Mailgun did use tracker images and other techniques, but also simple status codes; you don't need a tracker to notice a DSN or 500-bounce from the MTA.
Ruth,
I totally agree.
What I am asking for/hoping for is something that wraps up whatever sources of info are available and at least associates it with a list, if not per message.
The script pflogsumm did a lot of that (if not everything), but your MTA needs to be Postfix and it needs access to the MTA log file. It may not even be around anymore, as I have not used it for a few years (but it was in perl so maybe still runs fine?)
Please let the list know if you find something suitable.
Brett
participants (5)
-
Abhilash Raj -
Brett Delmage -
Mark Dadgar -
Ruth Ivimey-Cook -
Stephen J. Turnbull