Internal Server Error: /archives/api/mailman/archive OperationalError at /archives/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\x8A\\x0A\\x0A...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1")
Someone sent a short E-Mail with an Emoition in it. Is there something known about problems?
The mails was sent via mailman3, but not archived due to the error. I can bring more details, please let me know, if this is requested.
Internal Server Error:
/archives/api/mailman/archive
OperationalError at /archives/api/mailman/archive
(1366, "Incorrect string value: '\\xF0\\x9F\\x98\\x8A\\x0A\\x0A...' for column mailman3web.hyperkitty_email.content at row 1")
Hyperkitty 1.3.12 GNU Mailman 3.3.10 (Tom Sawyer) Python 3.11.2 (main, Apr 28 2025, 14:11:48) [GCC 12.2.0]
Now, every new mail triggers again this error.
On 9/9/25 13:45, Wikinaut wrote:
Someone sent a short E-Mail with an Emoition in it. Is there something known about problems?
Yes, the issue is your database is MySQL or MariaDB and your tables are not utf8mb4 so they don't accept 4-byte utf8 encodings. See the post at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/... for a way to fix this.
Now, every new mail triggers again this error.
Because the offending message is in the queue at /opt/mailman/mm/var/archives/hyperkitty/spool/ and it gets retried on every post. Setting the character set to utf8mb4 on the hyperkitty_email table will allow the retries to succeed.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I changed the table hyperkitty_email.
The 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3user', # TODO: Replace this with the password. 'PASSWORD': '****', 'HOST': 'localhost', 'PORT': '5432', # For MySQL/MariaDB, in addition to changing the 'ENGINE' setting, # uncomment the following to enable utf8 4-byte encodings. 'OPTIONS': { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3user', # TODO: Replace this with the password. 'PASSWORD': '883AGfuadS7gg', 'HOST': 'localhost', 'PORT': '5432', # For MySQL/MariaDB, in addition to changing the 'ENGINE' setting, # uncomment the following to enable utf8 4-byte encodings. 'OPTIONS': { 'charset': 'utf8mb4', 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", } } }
'charset': 'utf8mb4',
is already present.
The Error message is still coming.
What do I have to fix next?
MariaDB [mailman3web]> SHOW FULL COLUMNS FROM hyperkitty_email; +-----------------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+ | Field | Type | Collation | Null | Key | Default | Extra | Privileges | Comment | +-----------------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+ | id | int(11) | NULL | NO | PRI | NULL | auto_increment | select,insert,update,references | | | message_id | varchar(255) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | message_id_hash | varchar(255) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | subject | varchar(512) | utf8mb4_unicode_ci | YES | MUL | NULL | | select,insert,update,references | | | content | longtext | utf8mb4_unicode_ci | YES | | NULL | | select,insert,update,references | | | date | datetime(6) | NULL | NO | MUL | NULL | | select,insert,update,references | | | timezone | smallint(6) | NULL | NO | | NULL | | select,insert,update,references | | | in_reply_to | varchar(255) | utf8mb4_general_ci | YES | MUL | NULL | | select,insert,update,references | | | archived_date | datetime(6) | NULL | NO | MUL | NULL | | select,insert,update,references | | | thread_depth | int(11) | NULL | NO | | NULL | | select,insert,update,references | | | thread_order | int(11) | NULL | YES | MUL | NULL | | select,insert,update,references | | | mailinglist_id | int(11) | NULL | NO | MUL | NULL | | select,insert,update,references | | | parent_id | int(11) | NULL | YES | MUL | NULL | | select,insert,update,references | | | sender_id | varchar(255) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | thread_id | int(11) | NULL | NO | MUL | NULL | | select,insert,update,references | | | sender_name | varchar(255) | utf8mb4_general_ci | YES | | NULL | | select,insert,update,references | | +-----------------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+
Looks to be okay, doesn't it?
Mark,
my database **was** already on utf8mb.
I checked with the server (with the problem) from which I moved to a new server, the sam issue. Server 1: SHOW FULL COLUMNS FROM hyperkitty_email; +-----------------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+ | Field | Type | Collation | Null | Key | Default | Extra | Privileges | Comment | +-----------------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+ | id | int(11) | NULL | NO | PRI | NULL | auto_increment | select,insert,update,references | | | message_id | varchar(255) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | message_id_hash | varchar(255) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | subject | varchar(512) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | content | longtext | utf8mb4_general_ci | NO | | NULL | | select,insert,update,references | | | date | datetime(6) | NULL | NO | MUL | NULL | | select,insert,update,references | | | timezone | smallint(6) | NULL | NO | | NULL | | select,insert,update,references | | | in_reply_to | varchar(255) | utf8mb4_general_ci | YES | MUL | NULL | | select,insert,update,references | | | archived_date | datetime(6) | NULL | NO | MUL | NULL | | select,insert,update,references | | | thread_depth | int(11) | NULL | NO | | NULL | | select,insert,update,references | | | thread_order | int(11) | NULL | YES | MUL | NULL | | select,insert,update,references | | | mailinglist_id | int(11) | NULL | NO | MUL | NULL | | select,insert,update,references | | | parent_id | int(11) | NULL | YES | MUL | NULL | | select,insert,update,references | | | sender_id | varchar(255) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | thread_id | int(11) | NULL | NO | MUL | NULL | | select,insert,update,references | | | sender_name | varchar(255) | utf8mb4_general_ci | YES | | NULL | | select,insert,update,references | | +-----------------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+ 16 rows in set (0.004 sec)
This looks fine, too, doesn't it?
How can I fix the Internal Server Error? Lost.....
Please close this issue now.
The problem could be solved now. No idea, what was wrong.
On 9/9/25 14:47, Wikinaut wrote:
Mark,
my database **was** already on utf8mb.
Do you mean utf8mb4? I don't think there is a utf8mb encoding, but if there is it may be an alias for utf8mb3.
I checked with the server (with the problem) from which I moved to a new server, the sam issue. Server 1: SHOW FULL COLUMNS FROM hyperkitty_email; +-----------------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+ | Field | Type | Collation | Null | Key | Default | Extra | Privileges | Comment | +-----------------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+ | id | int(11) | NULL | NO | PRI | NULL | auto_increment | select,insert,update,references | | | message_id | varchar(255) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | message_id_hash | varchar(255) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | subject | varchar(512) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | content | longtext | utf8mb4_general_ci | NO | | NULL | | select,insert,update,references | | | date | datetime(6) | NULL | NO | MUL | NULL | | select,insert,update,references | | | timezone | smallint(6) | NULL | NO | | NULL | | select,insert,update,references | | | in_reply_to | varchar(255) | utf8mb4_general_ci | YES | MUL | NULL | | select,insert,update,references | | | archived_date | datetime(6) | NULL | NO | MUL | NULL | | select,insert,update,references | | | thread_depth | int(11) | NULL | NO | | NULL | | select,insert,update,references | | | thread_order | int(11) | NULL | YES | MUL | NULL | | select,insert,update,references | | | mailinglist_id | int(11) | NULL | NO | MUL | NULL | | select,insert,update,references | | | parent_id | int(11) | NULL | YES | MUL | NULL | | select,insert,update,references | | | sender_id | varchar(255) | utf8mb4_general_ci | NO | MUL | NULL | | select,insert,update,references | | | thread_id | int(11) | NULL | NO | MUL | NULL | | select,insert,update,references | | | sender_name | varchar(255) | utf8mb4_general_ci | YES | | NULL | | select,insert,update,references | | +-----------------+--------------+--------------------+------+-----+---------+----------------+---------------------------------+---------+ 16 rows in set (0.004 sec)
This looks fine, too, doesn't it?
utf8mb4_general_ci is only a collation sequence. It is not the character set of the database.
The bottom line is if you are still getting the subject error it is because the character set of the database and/or the hyperkitty_email table is utf8mb3.
There several threads on this in the archives (search for utf8mb4) and at least one issue at https://gitlab.com/mailman/hyperkitty/-/issues/248 that have information about fixing this.
I don't know for sure about MariaDB, but for MySQL you can also add
[mysqld]
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
to the MySQL configuration file.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Mark Sapiro -
Wikinaut