Hyperkitty stuck files
Hi,
I have a mailman3 setup in venv with python 3.11.2 mailman 3.3.9 mailman-hyperkitty 1.2.1
I feel I have read a lot about issues with stucked files in spool directory still I am not able to proceed with debugging in my case.
Ideas would be welcome and appreciated.
Thanks
This is the error in archiver.log
Jul 09 10:56:12 2024 (47531) HyperKitty archiver processing queued filebase: 1720522497.980487+a88492922feca24ced15594a5ec16556195 6f3ad Jul 09 10:56:12 2024 (47531) hyperkitty archiver: sending message <B9E91239-ED71-4CC6-8FFF-3308C5C4DE2E@me.com> Jul 09 10:56:12 2024 (47531) HyperKitty failure on http://127.0.0.1:8000/archives/api/mailman/archive:
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Server Error</title> <link rel="shortcut icon" href="/static/postorius/img/favicon.ico"> <link rel="stylesheet" href="/static/postorius/libs/bootstrap/css/bootstrap.min.css"> <link rel="stylesheet" href="/static/django-mailman3/css/main.css"> <link rel="stylesheet" href="/static/postorius/css/style.css">
</head> <body> <div class="container"> <h1>Server error</h1> <div class="alert alert-danger">An error occurred while processing your request.</div> </div> </body> </html> (500) Jul 09 10:56:12 2024 (47531) Could not archive the message with id <B9E91239-ED71-4CC6-8FFF-3308C5C4DE2E@me.com> Jul 09 10:56:12 2024 (47531) archiving failed, re-queuing (mailing-list act.lists.contextualscience.org, message <B9E91239-ED71-4CC6-8FFF-3308C5C4DE2E@me.com>) Jul 09 10:56:12 2024 (47531) Exception in the HyperKitty archiver:
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Server Error</title> <link rel="shortcut icon" href="/static/postorius/img/favicon.ico"> <link rel="stylesheet" href="/static/postorius/libs/bootstrap/css/bootstrap.min.css"> <link rel="stylesheet" href="/static/django-mailman3/css/main.css"> <link rel="stylesheet" href="/static/postorius/css/style.css">
</head> <body> <div class="container"> <h1>Server error</h1> <div class="alert alert-danger">An error occurred while processing your request.</div> </div> </body> </html> Jul 09 10:56:12 2024 (47531) Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.11/site-packages/mailman_hyperkitty/__init__.py", line 158, in _archive_message url = self._send_message(mlist, msg) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailman_hyperkitty/__init__.py", line 228, in _send_message raise ValueError(result.text) ValueError:
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Server Error</title> <link rel="shortcut icon" href="/static/postorius/img/favicon.ico"> <link rel="stylesheet" href="/static/postorius/libs/bootstrap/css/bootstrap.min.css"> <link rel="stylesheet" href="/static/django-mailman3/css/main.css"> <link rel="stylesheet" href="/static/postorius/css/style.css">
</head> <body> <div class="container"> <h1>Server error</h1> <div class="alert alert-danger">An error occurred while processing your request.</div> </div> </body> </html>
On 7/9/24 7:05 AM, zoltan@circle-interactive.co.uk wrote:
This is the error in archiver.log
Jul 09 10:56:12 2024 (47531) HyperKitty archiver processing queued filebase: 1720522497.980487+a88492922feca24ced15594a5ec165561956f3ad Jul 09 10:56:12 2024 (47531) hyperkitty archiver: sending message <B9E91239-ED71-4CC6-8FFF-3308C5C4DE2E@me.com> Jul 09 10:56:12 2024 (47531) HyperKitty failure on http://127.0.0.1:8000/archives/api/mailman/archive:
It looks like hyperkitty is throwing an uncaught exception in attempting to archive this message. Try moving 1720522497.980487+a88492922feca24ced15594a5ec165561956f3ad.pck out of the spool/ directory and see if it processes the rest of the messages there.
You can examine that message with the mailman qfile command.
There might also be a traceback in the Django logs that will say more about why HyperKitty is failing.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thank you Mark!
I think I tracked down the related django error for that message. mailmanweb.log says
MySQLdb.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\x8A. ...' for column mailmanweb.hyperkitty_email.content at row 1")
but collation is utf8mb4_general_ci on the table.
Do I need to maybe specify it in settings file?
On Wed, Jul 10, 2024 at 1:36 PM <zoltan@circle-interactive.co.uk> wrote:
Thank you Mark!
I think I tracked down the related django error for that message. mailmanweb.log says
MySQLdb.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\x8A. ...' for column
mailmanweb.hyperkitty_email.contentat row 1")but collation is utf8mb4_general_ci on the table.
Do I need to maybe specify it in settings file?
Does your Django settings for the DATABASE look like the below?
DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'DBNAME', # Example, change as needed 'USER': 'USER', 'PASSWORD': 'XXXXXXXXX', 'HOST': '127.0.0.1', 'PORT': '', 'OPTIONS': { 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4', }, } }
I think the OPTIONS bit was required to mitigate that error.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
Thank you again for the OPTIONS
I do not have errors in log any more and I had 273 stuck mails which are gone now.
Can this maybe be added to the installation guide? https://docs.mailman3.org/en/latest/install/virtualenv.html
Thanks, Zoltan
On Wed, Jul 10, 2024 at 4:55 PM <zoltan@circle-interactive.co.uk> wrote:
Thank you again for the OPTIONS
I do not have errors in log any more and I had 273 stuck mails which are gone now.
Can this maybe be added to the installation guide? https://docs.mailman3.org/en/latest/install/virtualenv.html
Since that official HOWTO only focuses more on PostgreSQL than MySQL, there are so many bits about MySQL that aren't mentioned there, For example, there's no mention of "mysqlclient|pymysql". Maybe those using MySQL as the DB backend need to figure out those :)
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
On 7/10/24 6:55 AM, zoltan@circle-interactive.co.uk wrote:
Can this maybe be added to the installation guide? https://docs.mailman3.org/en/latest/install/virtualenv.html
That guide is for an installation using PostgreSQL. We do mention the OPTIONS setting for MySQL in multiple places. E.g.,
https://gitlab.com/mailman/mailman-suite/-/blame/master/mailman-suite_projec...
https://gitlab.com/mailman/hyperkitty/-/blob/master/example_project/settings...
https://gitlab.com/mailman/mailman-web/-/blame/master/mailman_web/settings/b...
However, I just added a comment to https://docs.mailman3.org/en/latest/install/virtualenv.html#initial-configur... about this.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Mark Sapiro -
Odhiambo Washington -
zoltan＠circle-interactive.co.uk