Problems with New MM3 Installation
After using Mailman 2 (and possibly 1 though I don’t remember) for almost 20 years, I have just upgraded to Mailman 3. I followed the instructions here:
https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta...
Things generally work, but I have run into a couple of small problems:
The uwsgi-error.log is filling with errors that seem to culminate in "ModuleNotFoundError: No module named ‘settings’”. My settings.py is located in “/etc/mailman3” and seems to be found by other components of MM3. This error must be coming from the cron jobs. Does anyone have any ideas how to fix this problem?
While old messages were imported without issue, new emails are not being archives. I see them in:
mailman3/mm/var/archives/prototype/[listname]@[domain]/new
But they are not being delivered. I have added this to my mailman.cfg:
[archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /usr/local/mailman3/mailman-hyperkitty.cfg
And created the "/usr/local/mailman3/mailman-hyperkitty.cfg” as such:
base_url: http://localhost/archives/ api_key: [key that matches “MAILMAN_ARCHIVER_KEY” in my settings.py]
Some documentation says that the url should be "http://localhost/hyperkitty/“, but url generates a page not found error on my setup while the archives are available at the “archives” url.
I would greatly appreciate any assistance with these two issues.
Thanks much,
Jeff
On May 10, 2021, at 7:38 AM, Jeff <mm3@steele.com> wrote:
After using Mailman 2 (and possibly 1 though I don’t remember) for almost 20 years, I have just upgraded to Mailman 3. I followed the instructions here:
https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta...
Things generally work, but I have run into a couple of small problems:
- The uwsgi-error.log is filling with errors that seem to culminate in "ModuleNotFoundError: No module named ‘settings’”. My settings.py is located in “/etc/mailman3” and seems to be found by other components of MM3. This error must be coming from the cron jobs. Does anyone have any ideas how to fix this problem?
I replied to another message on the list with the same issue, lets discuss it over there since it looks very similar.
- While old messages were imported without issue, new emails are not being archives. I see them in:
mailman3/mm/var/archives/prototype/[listname]@[domain]/new
But they are not being delivered. I have added this to my mailman.cfg:
[archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /usr/local/mailman3/mailman-hyperkitty.cfg
And created the "/usr/local/mailman3/mailman-hyperkitty.cfg” as such:
base_url: http://localhost/archives/ api_key: [key that matches “MAILMAN_ARCHIVER_KEY” in my settings.py]
Some documentation says that the url should be "http://localhost/hyperkitty/“, but url generates a page not found error on my setup while the archives are available at the “archives” url.
Are you seeing any errors in the logs?
Mailman Core’s logs are at /opt/mailman/mm/var/logs/mailman.log.
You should also check if this is something wrong at the Archiver end,
and the logs for those are available at /opt/mailman/web/logs/mailmanweb.log.
-- thanks, Abhilash Raj (maxking)
I have found work around for the settings issue but continue to have an issue with archiving new messages. New messages are stuck here;
mailman3/mm/var/archives/prototype/[listname]@[domain]/new
I have added this to my mailman.cfg:
[archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /usr/local/mailman3/mailman-hyperkitty.cfg
And created the "/usr/local/mailman3/mailman-hyperkitty.cfg” as such:
base_url: http://localhost/archives/ api_key: [key that matches “MAILMAN_ARCHIVER_KEY” in my settings.py]
Some documentation says that the url should be "http://localhost/hyperkitty/ but url generates a page not found error on my setup while the archives are available at the “archives” url.
But, now that I think about it, "localhost" doesn't make sense here because I am virtual hosts and "localhost" will probably just connect to the first virtual host rather than the mailmanweb virtual host.
Any advice on what do do here?
On 5/11/21 7:13 AM, mm3@steele.com wrote:
I have found work around for the settings issue but continue to have an issue with archiving new messages. New messages are stuck here;
mailman3/mm/var/archives/prototype/[listname]@[domain]/new
They aren't stuck there. That is where they are archived by the
prototype archiver. This is separate from HyperKitty.
I have added this to my mailman.cfg:
[archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /usr/local/mailman3/mailman-hyperkitty.cfg
And created the "/usr/local/mailman3/mailman-hyperkitty.cfg” as such:
base_url: http://localhost/archives/ api_key: [key that matches “MAILMAN_ARCHIVER_KEY” in my settings.py]
Some documentation says that the url should be "http://localhost/hyperkitty/ but url generates a page not found error on my setup while the archives are available at the “archives” url.
But, now that I think about it, "localhost" doesn't make sense here because I am virtual hosts and "localhost" will probably just connect to the first virtual host rather than the mailmanweb virtual host.
Any advice on what do do here?
Set base_url: in /usr/local/mailman3/mailman-hyperkitty.cfg to a url that will reliably access HyperKitty, probably the same base URL that a user with a web browser would use.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On May 11, 2021, at 1:18 PM, Mark Sapiro <mark@msapiro.net> wrote:
Set base_url: in /usr/local/mailman3/mailman-hyperkitty.cfg to a url that will reliably access HyperKitty, probably the same base URL that a user with a web browser would use.
Thanks Mark,
I really appreciate your assistance. New messages are now being archived as expected, but I still have a few small remaining issues to work out with my MM3 installation:
- some emails that arrived before I got the archive working are still sitting in:
/usr/local/mailman3/mm/var/archives/prototype/[listname]@[domain]/new
Strangely this includes at least one email that was archived. Should this directly be cleared as messages are archived? Is there a way to force it to be cleared?
- Some messages that arrived while I was having other problems with the setup have been stored in:
/usr/local/mailman3/mm/var/queue/shunt
What should I do to get these messages processed?
- Similar to above, some messages are in:
/usr/local/mailman3/mm/var/queue/bad
How can I get these messages processed?
4 Modified the template for “list:admin:action:post” through Postorious and emails that arrived after that were put in the “shunt” queue. I traced this problem to POSTORIUS_TEMPLATE_BASE_URL. By default, it is set to "http://localhost:8000 <http://localhost:8000/>” which is where uwsgi is listening. However, uwsigi is listening for uwsgi:/ not http://. Should I change POSTORIUS_TEMPLATE_BASE_URL to “uwsgi://localhost:8000 <uwsgi://localhost:8000>” or keep http but make it something else?
Thanks again for any help anyone can provide.
Jeff
On 5/11/21 7:46 PM, Jeff wrote:
I really appreciate your assistance. New messages are now being archived as expected, but I still have a few small remaining issues to work out with my MM3 installation:
- some emails that arrived before I got the archive working are still sitting in:
/usr/local/mailman3/mm/var/archives/prototype/[listname]@[domain]/new
Strangely this includes at least one email that was archived. Should this directly be cleared as messages are archived? Is there a way to force it to be cleared?
/usr/local/mailman3/mm/var/archives/prototype/[listname]@[domain] is a collection of messages in maildir format. This is how the prototype archiver stores messages. This has nothing to do with HyperKitty. It is a separate archiver. If you don't want messages to be stored there, you can disable the prototype archiver for the list by unchecking prototype in Postorius -> Settings -> Archiving -> Active archivers or disable it entirely for the installation by setting
[archiver.prototype]
enable: no
in mailman.cfg
- Some messages that arrived while I was having other problems with the setup have been stored in:
/usr/local/mailman3/mm/var/queue/shunt
What should I do to get these messages processed?
Assuming the issues that caused them to be shunted have been fixed, just run the command
mailman unshunt
- Similar to above, some messages are in:
/usr/local/mailman3/mm/var/queue/bad
How can I get these messages processed?
First examine the messages with
mailman qfile /usr/local/mailman3/mm/var/queue/bad/<file>
To be sure you want them. Also examine mailman.log for the times they
were stored in the bad queue. You are looking for 'Skipping and
preserving unparseable message: <file>' and/or 'SHUNTING FAILED,
preserving original entry: <file> messages and you want to determine
which runner preserved the message. There may be a 'whichq' entry in the
message's metadata in which case, that's the answer.
Once you have determined the queue, you can just move the .psv file from the bad queue to the appropriate queue, renaming it from .psv to .pck in the process.
4 Modified the template for “list:admin:action:post” through Postorious and emails that arrived after that were put in the “shunt” queue. I traced this problem to POSTORIUS_TEMPLATE_BASE_URL. By default, it is set to "http://localhost:8000 <http://localhost:8000/>” which is where uwsgi is listening. However, uwsigi is listening for uwsgi:/ not http://. Should I change POSTORIUS_TEMPLATE_BASE_URL to “uwsgi://localhost:8000 <uwsgi://localhost:8000>” or keep http but make it something else?
I know little about uwsgi. setting it to uwsgi://localhost:8000 might
work. You could also set it to the base URL that a user would use to
access the web UI, e.g. https://example.com and that would presumably
be proxied to uwsgi. For example, on the server for this list we have
POSTORIUS_TEMPLATE_BASE_URL = 'https://lists.mailman3.org'
which gets proxied to gunicorn, but the idea is the same.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, May 11, 2021, at 8:43 PM, Mark Sapiro wrote:
On 5/11/21 7:46 PM, Jeff wrote:
I really appreciate your assistance. New messages are now being archived as expected, but I still have a few small remaining issues to work out with my MM3 installation:
- some emails that arrived before I got the archive working are still sitting in:
/usr/local/mailman3/mm/var/archives/prototype/[listname]@[domain]/new
Strangely this includes at least one email that was archived. Should this directly be cleared as messages are archived? Is there a way to force it to be cleared?
/usr/local/mailman3/mm/var/archives/prototype/[listname]@[domain] is a collection of messages in maildir format. This is how the prototype archiver stores messages. This has nothing to do with HyperKitty. It is a separate archiver. If you don't want messages to be stored there, you can disable the prototype archiver for the list by unchecking prototype in Postorius -> Settings -> Archiving -> Active archivers or disable it entirely for the installation by setting
[archiver.prototype] enable: noin mailman.cfg
- Some messages that arrived while I was having other problems with the setup have been stored in:
/usr/local/mailman3/mm/var/queue/shunt
What should I do to get these messages processed?
Assuming the issues that caused them to be shunted have been fixed, just run the command
mailman unshunt
- Similar to above, some messages are in:
/usr/local/mailman3/mm/var/queue/bad
How can I get these messages processed?
First examine the messages with
mailman qfile /usr/local/mailman3/mm/var/queue/bad/<file>To be sure you want them. Also examine mailman.log for the times they were stored in the
badqueue. You are looking for 'Skipping and preserving unparseable message: <file>' and/or 'SHUNTING FAILED, preserving original entry: <file> messages and you want to determine which runner preserved the message. There may be a 'whichq' entry in the message's metadata in which case, that's the answer.Once you have determined the queue, you can just move the .psv file from the bad queue to the appropriate queue, renaming it from .psv to .pck in the process.
4 Modified the template for “list:admin:action:post” through Postorious and emails that arrived after that were put in the “shunt” queue. I traced this problem to POSTORIUS_TEMPLATE_BASE_URL. By default, it is set to "http://localhost:8000 <http://localhost:8000/>” which is where uwsgi is listening. However, uwsigi is listening for uwsgi:/ not http://. Should I change POSTORIUS_TEMPLATE_BASE_URL to “uwsgi://localhost:8000 <uwsgi://localhost:8000>” or keep http but make it something else?
I know little about uwsgi. setting it to
uwsgi://localhost:8000might work. You could also set it to the base URL that a user would use to access the web UI, e.g.https://example.comand that would presumably be proxied to uwsgi. For example, on the server for this list we havePOSTORIUS_TEMPLATE_BASE_URL = 'https://lists.mailman3.org'which gets proxied to gunicorn, but the idea is the same.
Using the full external URL should work. Core doesn’t speak uwsgi protocol so the uwsgi:// URL won’t actually work.
I am starting to think it was a very bad idea to use uwsgi in the official docs :(
-- thanks, Abhilash Raj (maxking)
Thanks Mark and Abhilash (who responded separately).
I have worked out all the issues I was having and now the list is running smoothly. The trick for POSTORIUS_TEMPLATE_BASE_URL with uwsgi was "http://[hostname]/mailman3/ <http://[hostname]/mailman3/>“.
I appreciate all the help I’ve received from this mailing list.
On May 11, 2021, at 11:43 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 5/11/21 7:46 PM, Jeff wrote:
I really appreciate your assistance. New messages are now being archived as expected, but I still have a few small remaining issues to work out with my MM3 installation:
- some emails that arrived before I got the archive working are still sitting in:
/usr/local/mailman3/mm/var/archives/prototype/[listname]@[domain]/new
Strangely this includes at least one email that was archived. Should this directly be cleared as messages are archived? Is there a way to force it to be cleared?
/usr/local/mailman3/mm/var/archives/prototype/[listname]@[domain] is a collection of messages in maildir format. This is how the prototype archiver stores messages. This has nothing to do with HyperKitty. It is a separate archiver. If you don't want messages to be stored there, you can disable the prototype archiver for the list by unchecking prototype in Postorius -> Settings -> Archiving -> Active archivers or disable it entirely for the installation by setting
[archiver.prototype] enable: noin mailman.cfg
- Some messages that arrived while I was having other problems with the setup have been stored in:
/usr/local/mailman3/mm/var/queue/shunt
What should I do to get these messages processed?
Assuming the issues that caused them to be shunted have been fixed, just run the command
mailman unshunt
- Similar to above, some messages are in:
/usr/local/mailman3/mm/var/queue/bad
How can I get these messages processed?
First examine the messages with
mailman qfile /usr/local/mailman3/mm/var/queue/bad/<file>To be sure you want them. Also examine mailman.log for the times they were stored in the
badqueue. You are looking for 'Skipping and preserving unparseable message: <file>' and/or 'SHUNTING FAILED, preserving original entry: <file> messages and you want to determine which runner preserved the message. There may be a 'whichq' entry in the message's metadata in which case, that's the answer.Once you have determined the queue, you can just move the .psv file from the bad queue to the appropriate queue, renaming it from .psv to .pck in the process.
4 Modified the template for “list:admin:action:post” through Postorious and emails that arrived after that were put in the “shunt” queue. I traced this problem to POSTORIUS_TEMPLATE_BASE_URL. By default, it is set to "http://localhost:8000 <http://localhost:8000/>” which is where uwsgi is listening. However, uwsigi is listening for uwsgi:/ not http://. Should I change POSTORIUS_TEMPLATE_BASE_URL to “uwsgi://localhost:8000 <uwsgi://localhost:8000>” or keep http but make it something else?
I know little about uwsgi. setting it to
uwsgi://localhost:8000might work. You could also set it to the base URL that a user would use to access the web UI, e.g.https://example.comand that would presumably be proxied to uwsgi. For example, on the server for this list we havePOSTORIUS_TEMPLATE_BASE_URL = 'https://lists.mailman3.org'which gets proxied to gunicorn, but the idea is the same.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
participants (4)
-
Abhilash Raj -
Jeff -
Mark Sapiro -
mm3＠steele.com