Hi,

Tamas HOLCZER via Mailman-users <mailman-users@mailman3.org> 於 2019年10月28日 週一 20:32 寫道：

Hi Franklin,

it seems you have a brand new install, right? By default (at least in Debian) the archivers are not enabled. You should add something like this to your mailman.cfg:

[archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /etc/mailman3/mailman-hyperkitty.cfg

Please check this site:

https://docs.mailman3.org/en/latest/devsetup.html#set-up-and-run-hyperkitty

Best regards,

Tamás

Yes, I did follow the document and add it in the mailman.cfg, but seems not worked. How to check if it really is working or not?

Regards, Franklin

Hi Stephen,

Thanks for your explanation.

Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> 於 2019年10月29日 週二 下午1:35寫道：

Franklin Weng writes:

...

However in the Archive section I couldn't see anything and all the lists were "inactive" and hence were hidden by default.

If you can see that much, HyperKitty is running.

Have you sent mail to the lists? Has it been delivered to subscribers?

Yes, mails to subscribers were delivered successfully.

...

I confirmed that api_key set in the /etc/mailman3/mailman-hyperkitty.cfg and /etc/mailman3/mailman-web.py is the same. I also confirmed that the apache2 wsgi module has been loaded.

In the mailman.cfg database is set to mysql and in the mailman-web.py database is set to sqlite3. Does it matter?

I don't think they need to be the same. The HyperKitty and Postorius databases are a Django database backed by an RDBMS (typically sqlite3, PostgreSQL, or MySQL). The databases used by the Mailman core and the Django webservices HyperKitty and Postorius are conceptually separate, although they can be backed by the same RDBMS server.

As Tamas has said in the previous reply, my mailman is a fresh new install on Debian buster. I followed Debian.README and didn't change much. In Debian's setting, under /usr/share/mailman3-web/ are these files:

manage.py settings_local.py -> /etc/mailman3/mailman-web.py settings_local.py.sample settings.py urls.py wsgi.py

In the top of settings_locas.py there are comments saying that this file is to override default settings in /usr/share/mailman3-web/settings.py, so I suppose hyperkitty database should define here, though it is different from /usr/share/mailman3-web/settings.py.

However, sqlite3 is likely to result in poor performance compared to PostgreSQL or MySQL.

You have to configure HyperKitty's database. In my sqlite3 test installation it looks like this:

# Top of settings.py. BASE_DIR is the directory containing settings.py. BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# 30-50 lines later: DATABASES = { 'default': { 'ENGINE': 'django.db.backends.sqlite3', # This database file is in the same directory as settings.py. 'NAME': os.path.join(BASE_DIR, 'hyperkitty.db'), # Not used with sqlite3: 'USER': '', 'PASSWORD': '', 'HOST': '', 'PORT': '', } }

Does the database file named by 'NAME' exist?

In the settings_local.py, it defines sqliet3 and the file of 'NAME' /var/lib/mailman3/web/mailman3web.db exists and is a sqlite3 format file. But in settings.py (which is supposed to be overridden by setting_local.py) it defines using postgresql, which doesn't exist in my Debian system.

There is no systemd service for Hyperkitty. systemd runs Apache, which loads mod_wsgi, which runs Python, which imports the HyperKitty package, which responds to web requests by building pages dynamically using Django from information from Django's database, which may be backed by MySQL, PostgreSQL, or sqlite3. Sounds complicated, but it performs well once you get it properly configured.

Is there any log I can trace the executing path? It looks not complicated but I didn't know and couldn't find out how to check if it works correctly. For Apache2 mod_wsgi, the wsgi.conf in mods-enabled does not enable any configuration (all commented out) and when I use ps to check, uwsgi runs as:

14233 ? Ss 0:01 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini 14236 ? Sl 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini

The /var/log/mailman3/web/mailman-web.log is at https://pastebin.com/U3jHR6JS There seems to be something running but in the interface I still didn't see any results.

Franklin

Yes, it is checked.

Mark Sapiro <mark@msapiro.net> 於 2019年10月30日 週三 11:19 寫道：

I haven't read this thread in detail. I've been off line for 3 days due to a Public Safety Power Shutoff, but my first question is if you go the the Pustorius web UI for the list(s) and look at Settings -> Archiving -> Active archivers, is hyperkitty checked?

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

Hi,

It looks like to have some clues now.

Mark Sapiro <mark@msapiro.net> 於 2019年10月30日 週三 13:07 寫道：

On 10/29/19 8:21 PM, Franklin Weng wrote:

...

Yes, it is checked.

Have you checked Mailman's logs - Mailman's var/logs/mailman.log in particular?

I found some hyperkitty URL error:

The default settings of base_url in mailman-hyperkitty.cfg is defined as

base_url: http://localhost/mailman/hyperkitty/

In the mailman.log there are messages like:

Oct 30 13:53:43 2019 (14259) HyperKitty failure on http://localhost/mailman3/hyperkitty/api/mailman/archive: <!DOCTYPE HTML PUBLIC "-//IETF// DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Found</h1> <p>The requested URL was not found on this server.</p>

I changed it to

base_url: https://localhost/mailman3/hyperkitty/

then the log became

File "/usr/lib/python3/dist-packages/requests /adapters.py", line 514, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /mailman3/hyperkitty/api/mailman/archive?key=xxx (Caused by SSLError(SSLCertVerification Error("hostname 'localhost' doesn't match ' lists.slat.org'")))

which makes sense because the SSL certificate applied from Letsencrypt is for lists.slat.org.

If I use

base_url: https://lists.slat.org/mailman3/hyperkitty/

The log became

Oct 30 15:25:50 2019 (6256) HyperKitty failure on https://lists.slat.org/mailman3/hyperkitty/api/mailman/urls: <html><title>Auth required</title><body>

            <h1>Authorization Required</h1>

</body></html> (401)

In my mailman-web.py:

MAILMAN_ARCHIVER_KEY = 'my-key,same as in mailman-hyperkitty.cfg' MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1')

Not sure what to change here.

Do messages received from the list have an Archived-At: header?

Yes, but it's empty

List-Id: 測試論壇 <slat-list.lists.slat.org> Archived-At: <> List-Archive: <> List-Help: <mailto:slat-list-request@lists.slat.org?subject=help> List-Post: <mailto:slat-list@lists.slat.org> List-Subscribe: <mailto:slat-list-join@lists.slat.org> List-Unsubscribe: <mailto:slat-list-leave@lists.slat.org>

What's in /etc/mailman3/mailman-hyperkitty.cfg? In particular does the [general] setting base_url define a URL that accesses hyperkitty

It seems to be the question…

and

does the [general] setting api_key match the MAILMAN_ARCHIVER_KEY setting in /etc/mailman3/mailman-web.py, but note that the MAILMAN_ARCHIVER_KEY setting is quotes because it is a Python string assignment and the api_key setting is not quoted

Yes, it's single quoted.

Thanks for all your help!

Franklin

Hi Tamas,

Tamas HOLCZER via Mailman-users <mailman-users@mailman3.org> 於 2019年10月30日 週三 20:09 寫道：

Hi Franklin,

when I installed mailman3 on Debian I had some issues. One was about MAILMAN_ARCHIVER_FROM. It turned out, that the request to Hyperkitty was sent from one of the real IPs of the machine (not from localhost). This depends on the interfaces and the routing of that machine. In my case the solution was something like:

MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1', '1.2.3.4', '1.2.3.5') , where 1.2.3.4 and 1.2.3.5 are the IP addresses of the machine running mailman3 (I have not debugged if it is using .4 or .5 in reality I left both of them there). Give it a try with your IPs.

Tamás

Thanks. However my server does not have a fixed IP. It uses pppoe and gets its IP from ISP and I set up ddns for accessing it.

Not test if it would work or not using 'lists.slat.org' here yet.

Franklin

Hi,

Sorry, I found that port 8000/8001 is working so the REST_API_URL should be working fine.

Franklin Weng <franklin@goodhorse.idv.tw> 於 2019年10月30日 週三 下午10:17寫道：

Hi,

Then I found that

MAILMAN_REST_API_URL = 'http://localhost:8001'

the URL does not work either (404 Page not found), but I didn't find any error log about that.

My apache setting is: 000-default.conf: port 80, didn't change the default setting (/var/www/html/index.html replaced) 001-lists.conf: server name "lists.slat.org", port 80 Redirect "/" to " https://lists.slat.org/". Port 443 has all the SSL and mailman alias set.

In this case how should I set the API_URL and hyperkitty URL?

Thanks, Franklin

Tamas HOLCZER via Mailman-users <mailman-users@mailman3.org> 於 2019年10月30日 週三 下午8:09寫道：

...

Hi Franklin,

when I installed mailman3 on Debian I had some issues. One was about MAILMAN_ARCHIVER_FROM. It turned out, that the request to Hyperkitty was sent from one of the real IPs of the machine (not from localhost). This depends on the interfaces and the routing of that machine. In my case the solution was something like:

MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1', '1.2.3.4', '1.2.3.5') , where 1.2.3.4 and 1.2.3.5 are the IP addresses of the machine running mailman3 (I have not debugged if it is using .4 or .5 in reality I left both of them there). Give it a try with your IPs.

Tamás

-- Franklin Weng 中華民國軟體自由協會常務理事 LibreOffice 導入專家 LibreOffice 法人代表文件基金會董事、認證委員會委員 KDE 協會成員、歐洲自由軟體基金會成員

Hi Mark,

Great thanks to you and everyone in this discussion.

Mark Sapiro <mark@msapiro.net> 於 2019年10月30日 週三 23:13 寫道：

On 10/30/19 12:52 AM, Franklin Weng wrote:

...

If I use

base_url: https://lists.slat.org/mailman3/hyperkitty/

The log became

Oct 30 15:25:50 2019 (6256) HyperKitty failure on https://lists.slat.org/mailman3/hyperkitty/api/mailman/urls: <html><title>Auth required</title><body>

            <h1>Authorization Required</h1>

</body></html> (401)

In my mailman-web.py:

MAILMAN_ARCHIVER_KEY = 'my-key,same as in mailman-hyperkitty.cfg' MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1')

Not sure what to change here.

There have been other reports of authorization failures even though the keys match. See the last few posts at < https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/M...

...

. The solution there was to change both the api_key setting in /etc/mailman3/mailman-hyperkitty.cfg and the MAILMAN_ARCHIVER_KEY setting in /etc/mailman3/mailman-web.py to new values.

I didn't change the key. When I follow the thread and print the keys getting from web and in the settings I found that one with single quote and the other not. In the mailman-web.py the value of MAILMAN_ARCHIVER_KEY should be single quoted, **but** the api_key in [general] SHOULD NOT single quoted.

Therefore I set my hyperkitty base URL to https://lists.slat.org/mailman's/hyperkitty/ and removed the quote in mailman-hyperkitty.cfg it finally works.

Maybe adding a comment to explain it a bit more would help later people not to step on the mine again.

Thanks for your help!

Regards, Franklin

On 9/23/20 4:59 AM, cristian.bianchi@usi.ch wrote:

I have a similar problem with mailman 3.2.1 (debian package):

# mailman info GNU Mailman 3.2.1 (La Villa Strangiato) Python 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]

the messages are not archived even if the key is set and hyperkitty is checked as active archiver.

To analyze the problem I set DEBUG = True in /etc/mailman3/mailman-web.py. In debug mode the problem does not occur, but this second problem appears: https://gitlab.com/mailman/hyperkitty/-/issues/84

I don't understand. https://gitlab.com/mailman/hyperkitty/-/issues/84 is supposed to be fixed. Are you saying it isn't.

How can I fix it?

Start by finding all the log messages relating to this archiving failure. If there are exceptions with tracebacks, what are they?

Also, are there messages in var/archives/hyperkitty/spool/? If so, there will also be ''Exception in the HyperKitty archiver:' messages with an exception logged in mailman.log for each such message every time there is a list post.

Note that HyperKitty is not a service that runs like Mailman core. The only check is can you access it via the web?

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan