(Custom) Plugins: Proper file path(s) for adding a handler?
I want to add a custom handler that implements customized thread IDs in the system. I already have the DB logic worked out, but the documentation for plugins is NOT clear on how to define a handler in a plugin, nor does the example plugin in the documentation on Gitlab have any custom handlers.
I'm looking to add a custom handler so that the private pipeline for threaded-subject-list can use thread-subject (the custom handler) instead of subject-prefix (builtin).
I'm unclear how to add this properly in a plugin that would be read properly by Mailman, though.
Any tips on the filestructure needed for the Python code/plugin package in order to properly be set up and work?
Thomas
On 10/23/23 20:37, Thomas Ward via Mailman-users wrote:
I want to add a custom handler that implements customized thread IDs in the system. I already have the DB logic worked out, but the documentation for plugins is NOT clear on how to define a handler in a plugin, nor does the example plugin in the documentation on Gitlab have any custom handlers.
I'm looking to add a custom handler so that the private pipeline for threaded-subject-list can use thread-subject (the custom handler) instead of subject-prefix (builtin).
https://gitlab.com/mailman/mailman/-/tree/master/src/mailman/plugins/testing...
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/plugins/doc...
These are two places you can find docs for an example plugin that ships with Mailman's code. Adding a new "component", like a Handler, works very similar to the "rules" in the example shown above.
You will probably need a directory tree like:
example/
__init__.py
hooks.py
handlers/
__init__.py
handlers.py
Installing the handler is same as per shown in docs using below (do note that the above example is the name of the package that is importable):
[plugin.example]
class: example.hooks.ExamplePlugin
enabled: yes
[logging.plugins]
propagate: yes
Finally, you can verify that your handler was loaded properly using something along the lines of:
$ mailman shell
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.
>>> for each in config.handlers:
... print(each)
...
member-recipients
to-archive
cleanse-dkim
file-recipients
to-outgoing
mime-delete
after-delivery
decorate
to-digest
avoid-duplicates
arc-sign
cook-headers
replybot
tagger
acknowledge
dmarc
subject-prefix
validate-authenticity
owner-recipients
to-usenet
cleanse
rfc-2369
-- thanks, Abhilash Raj (maxking)
On 10/23/23 22:57, Abhilash Raj wrote:
On 10/23/23 20:37, Thomas Ward via Mailman-users wrote:
I want to add a custom handler that implements customized thread IDs in the system. I already have the DB logic worked out, but the documentation for plugins is NOT clear on how to define a handler in a plugin, nor does the example plugin in the documentation on Gitlab have any custom handlers.
I'm looking to add a custom handler so that the private pipeline for threaded-subject-list can use thread-subject (the custom handler) instead of subject-prefix (builtin).
https://gitlab.com/mailman/mailman/-/tree/master/src/mailman/plugins/testing...
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/plugins/doc...
These are two places you can find docs for an example plugin that ships with Mailman's code. Adding a new "component", like a Handler, works very similar to the "rules" in the example shown above. Yeah these're helpful, but the problem I was running into was file naming schema - I was not aware it's always PLUGIN/COMPONENT/COMPONENT.py in the naming, which it was not entirely clear about. It makes sense now that I look back, but it never hurts to get clarification.
You will probably need a directory tree like:
example/ __init__.py hooks.py handlers/ __init__.py handlers.pyInstalling the handler is same as per shown in docs using below (do note that the above example is the name of the package that is importable):
<snip>
Finally, you can verify that your handler was loaded properly using something along the lines of:
$ mailman shell Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. >>> for each in config.handlers: ... print(each) ... <snip>
The only other thing that was missing is how to set up the proper code for a Handler implementation/class, which I used an existing mailman3 core plugin as an example to just run with for testing purposes.
There are notes in the example plugin on Gitlab and the documentation that "other components will be added as examples eventually" but that was well over a year ago, so we'll see how that ages.
Thanks for the assist!
Thomas
On 10/23/23 23:03, Thomas Ward via Mailman-users wrote:
On 10/23/23 22:57, Abhilash Raj wrote:
On 10/23/23 20:37, Thomas Ward via Mailman-users wrote:
I want to add a custom handler that implements customized thread IDs in the system. I already have the DB logic worked out, but the documentation for plugins is NOT clear on how to define a handler in a plugin, nor does the example plugin in the documentation on Gitlab have any custom handlers.
I'm looking to add a custom handler so that the private pipeline for threaded-subject-list can use thread-subject (the custom handler) instead of subject-prefix (builtin).
https://gitlab.com/mailman/mailman/-/tree/master/src/mailman/plugins/testing...
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/plugins/doc...
These are two places you can find docs for an example plugin that ships with Mailman's code. Adding a new "component", like a Handler, works very similar to the "rules" in the example shown above. Yeah these're helpful, but the problem I was running into was file naming schema - I was not aware it's always PLUGIN/COMPONENT/COMPONENT.py in the naming, which it was not entirely clear about. It makes sense now that I look back, but it never hurts to get clarification.
You will probably need a directory tree like:
example/ __init__.py hooks.py handlers/ __init__.py handlers.pyInstalling the handler is same as per shown in docs using below (do note that the above example is the name of the package that is importable):
<snip>
Finally, you can verify that your handler was loaded properly using something along the lines of:
$ mailman shell Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. >>> for each in config.handlers: ... print(each) ... <snip>The only other thing that was missing is how to set up the proper code for a Handler implementation/class, which I used an existing mailman3 core plugin as an example to just run with for testing purposes.
There are notes in the example plugin on Gitlab and the documentation that "other components will be added as examples eventually" but that was well over a year ago, so we'll see how that ages.
Thanks for the assist!
Thomas
One last question:
This is in the Plugins documentation:
Mailman defines a plugin as a Python package on sys.path that provides components matching the IPlugin interface.
This means I can install the code to anywhere on sys.path that the
system can detect and it does not need to be in the Mailman3
library/directory space in the venv?
Thomas
On 10/24/23 8:20 AM, Thomas Ward via Mailman-users wrote:
This means I can install the code to anywhere on
sys.paththat the system can detect and it does not need to be in the Mailman3 library/directory space in the venv?
Yes, as long as Mailman can import the class defined for the plugin in mailman.cfg, it will work.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Perfect, that should make things easier for me to install.
Thank you kindly for all your support everyone!
Thomas
On 10/24/23 11:54, Mark Sapiro wrote:
On 10/24/23 8:20 AM, Thomas Ward via Mailman-users wrote:
This means I can install the code to anywhere on
sys.paththat the system can detect and it does not need to be in the Mailman3 library/directory space in the venv?Yes, as long as Mailman can import the class defined for the plugin in mailman.cfg, it will work.
Running into some nasty issues.
I have a structure like this, installed as a package in the workspace once installed by pip in the venv:
mailman_customizations/ __init__.py hooks.py <-- plugin definition in here as MailmanCustomizations for class name handlers/ __init__.py handlers.py <-- custom handler in here pipelines/ __init__.py pipelines.py <-- custom pipelines which implement the handlers here.
However, whenever I specify this in config with this clause:
[plugin.MailmanCustomizations] class: mailman_customizations.hooks.MailmanCustomizations enabled: no
... I get a hard load error on "ModuleNotFoundError: No module named 'MailmanCustomizations'"
To confirm this is a Python module/class, I did a pure python shell with
from mailman_customizations.hooks import MailmanCustomizations and it
properly imports that class, so I'm not sure where to go from here in
debugging.
Thomas
On 10/24/23 11:55, Thomas Ward via Mailman-users wrote:
Perfect, that should make things easier for me to install.
Thank you kindly for all your support everyone!
Thomas
On 10/24/23 11:54, Mark Sapiro wrote:
On 10/24/23 8:20 AM, Thomas Ward via Mailman-users wrote:
This means I can install the code to anywhere on
sys.paththat the system can detect and it does not need to be in the Mailman3 library/directory space in the venv?Yes, as long as Mailman can import the class defined for the plugin in mailman.cfg, it will work.
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to teward@thomas-ward.net
On 10/24/23 12:34 PM, Thomas Ward via Mailman-users wrote:
... I get a hard load error on "ModuleNotFoundError: No module named 'MailmanCustomizations'"
To confirm this is a Python module/class, I did a pure python shell with
from mailman_customizations.hooks import MailmanCustomizationsand it properly imports that class, so I'm not sure where to go from here in debugging.
What if you try this with mailman-web shell?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 10/24/23 18:29, Mark Sapiro wrote:
On 10/24/23 12:34 PM, Thomas Ward via Mailman-users wrote:
... I get a hard load error on "ModuleNotFoundError: No module named 'MailmanCustomizations'"
To confirm this is a Python module/class, I did a pure python shell with
from mailman_customizations.hooks import MailmanCustomizationsand it properly imports that class, so I'm not sure where to go from here in debugging.What if you try this with
mailman-web shell?
Doesn't error on load, but can't access configs in order to see if my custom handler is present. Which Abhilash said has to be done in the mailman shell - as does the documentation. Mailman can't even start with this added. Which is a problem.
Thomas
On October 24, 2023 3:47:10 PM PDT, Thomas Ward via Mailman-users <mailman-users@mailman3.org> wrote:
On 10/24/23 18:29, Mark Sapiro wrote:
What if you try this with
mailman-web shell?Doesn't error on load, but can't access configs in order to see if my custom handler is present. Which Abhilash said has to be done in the mailman shell - as does the documentation. Mailman can't even start with this added. Which is a problem.
I was confused. I should have said mailman shell, not mailman-web shell.
-- Mark Sapiro <mark@msapiro.net> Sent from my Not_an_iThing with standards compliant, open source software.
On 10/24/23 18:53, Mark Sapiro wrote:
On October 24, 2023 3:47:10 PM PDT, Thomas Ward via Mailman-users <mailman-users@mailman3.org> wrote:
On 10/24/23 18:29, Mark Sapiro wrote:
What if you try this with
mailman-web shell?Doesn't error on load, but can't access configs in order to see if my custom handler is present. Which Abhilash said has to be done in the mailman shell - as does the documentation. Mailman can't even start with this added. Which is a problem.
I was confused. I should have said
mailman shell, notmailman-web shell.
Insert "circular problem" here.
When I DO NOT have the module loaded in the configuration, mailman shell works. Since that was functional, I went into the shell and
executed this:
from mailman_customizations.hooks import MailmanCustomizations
This worked fine, but obviously importing the class doesn't necessarily *activate* the class. So something is up with how I'm *supposed* to activate the plugin in the config, and we know that it's visible to Mailman in its shell, but it can't autoload it.
So, again, not sure where to look next for debugging.
Thomas
On 10/24/23 3:56 PM, Thomas Ward via Mailman-users wrote:
When I DO NOT have the module loaded in the configuration,
mailman shellworks. Since that was functional, I went into the shell and executed this:from mailman_customizations.hooks import MailmanCustomizations
This worked fine, but obviously importing the class doesn't necessarily *activate* the class. So something is up with how I'm *supposed* to activate the plugin in the config, and we know that it's visible to Mailman in its shell, but it can't autoload it.
I'm past the limit of my knowledge here and only stabbing in the dark, but what happens after you import it. Can you instantiate it, e.g.
instance = MailmanCustomizations()
dir(instance)
I'm not saying this should succeed, just that it may possibly provide useful information.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I was able to instantiate the class AND get the expected three objects from it - pre_hook, post_hook, and resource.
from mailman_customizations.hooks import MailmanCustomizations instance = MailmanCustomizations() dir(instance) ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__implemented__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__providedBy__', '__provides__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'post_hook', 'pre_hook', 'resource']
I know Abhilash Raj is probably well versed in this from statements made elsewhere, so I've added them in the CC field. Abhilash Raj, any ideas?
Thomas
-----Original Message----- From: Mark Sapiro <mark@msapiro.net> Sent: Tuesday, October 24, 2023 10:36 PM To: mailman-users@mailman3.org Subject: [MM3-users] Re: (Custom) Plugins: Proper file path(s) for adding a handler?
On 10/24/23 3:56 PM, Thomas Ward via Mailman-users wrote:
When I DO NOT have the module loaded in the configuration,
mailman shellworks. Since that was functional, I went into the shell and executed this:from mailman_customizations.hooks import MailmanCustomizations
This worked fine, but obviously importing the class doesn't necessarily *activate* the class. So something is up with how I'm *supposed* to activate the plugin in the config, and we know that it's visible to Mailman in its shell, but it can't autoload it.
I'm past the limit of my knowledge here and only stabbing in the dark, but what happens after you import it. Can you instantiate it, e.g.
instance = MailmanCustomizations()
dir(instance)
I'm not saying this should succeed, just that it may possibly provide useful information.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to teward@thomas-ward.net
On 10/25/23 01:04, Thomas Ward via Mailman-users wrote:
Running into some nasty issues.
I have a structure like this, installed as a package in the workspace once installed by pip in the venv:
mailman_customizations/ __init__.py hooks.py <-- plugin definition in here as MailmanCustomizations for class name handlers/ __init__.py handlers.py <-- custom handler in here pipelines/ __init__.py pipelines.py <-- custom pipelines which implement the handlers here.
However, whenever I specify this in config with this clause:
[plugin.MailmanCustomizations] class: mailman_customizations.hooks.MailmanCustomizations enabled: no
Change this to
[plugin.mailman_customizations] class: mailman_customizations.hooks.MailmanCustomizations enabled: yes
And I think it should work.
This isn't documented properly (needs an issue : -), but the way plugin system works is that for each component, it will try to import various components from "from <plugin_name>.<component>" and if the name isn't the name of the plugin package, then obvously the import fails.
The "ModuleNotFound" error was coming from the
[plugin.MailmanCustomizations] line and not from class: mailman_customizations.hooks.MailmanCustomizations which I think was
causing confusion.
I also _just_ remembered that i had created an example plugin, which can hopefully serve as a basis for plugins.
https://gitlab.com/mailman/example-mailman-plugin/-/tree/master
-- thanks, Abhilash Raj (maxking)
On Wed, 25 Oct 2023, at 11:23 PM, Abhilash Raj wrote:
On 10/25/23 01:04, Thomas Ward via Mailman-users wrote:
Running into some nasty issues.
I have a structure like this, installed as a package in the workspace once installed by pip in the venv:
mailman_customizations/ __init__.py hooks.py <-- plugin definition in here as MailmanCustomizations for class name handlers/ __init__.py handlers.py <-- custom handler in here pipelines/ __init__.py pipelines.py <-- custom pipelines which implement the handlers here.
However, whenever I specify this in config with this clause:
[plugin.MailmanCustomizations] class: mailman_customizations.hooks.MailmanCustomizations enabled: no
Change this to
[plugin.mailman_customizations] class: mailman_customizations.hooks.MailmanCustomizations enabled: yes
FWIW You can also do:
[plugin.MailmanCustomizations]
class: mailman_customizations.hooks.MailmanCustomizations
enabled: no
component_package: mailman_customizations
In ^ case, the "name" of the plugin, "MailmanCustomizations" in this case, doesn't matter and can be anything and mailman will use "component_package" as the base module to look for components.
-- thanks, Abhilash Raj (maxking)
participants (3)
-
Abhilash Raj -
Mark Sapiro -
Thomas Ward