I'm running a custom image which is based on an older version of the one here: https://github.com/maxking/docker-mailman. I attached it below. But I separated postorius and hyperkitty, so hyperkitty is running in its own container. I'm deploying the image with a plain 'docker run' behind nginx. I made fulltext_index persistent now, but it didn't get populated with anything yet. I don't really have an error traceback because there is never an error thrown. The only thing with some content is uwsgi-error.log, which you can find below. I'm also still getting the "A string literal cannot contain NUL (0x00) characters." messages. I also noticed that it takes incredibly long for the webinterface to load (several minutes) even though there doesn't seem to be any process consuming notable resources apart from the minutely job.
Funnily enough, I have the exact same image deployed on a second server as well for testing. On that one everything works fine. The only difference is that on the problematic one I have a lot more mailing lists/archives and that I imported them from mailman2. Could something have gone wrong during the import? I used the regular hyperkitty_import command.
uwsgi-error.log:
*** Starting uWSGI 2.0.18 (64bit) on [Sat Apr 27 22:50:17 2019] ***
compiled with version: 6.4.0 on 27 April 2019 22:48:42
os: Linux-4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19)
nodename: hyperkitty.docker
machine: x86_64
clock source: unix
detected number of CPU cores: 4
current working directory: /home/hyperkitty
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
setgid() to 82
setuid() to 82
chdir() to /home/hyperkitty
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 0.0.0.0:8081 fd 8
uwsgi socket 1 bound to TCP address 0.0.0.0:8080 fd 9
Python version: 3.6.8 (default, Jan 30 2019, 23:54:38) [GCC 6.4.0]
Python main interpreter initialized at 0x55dfaa41c980
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
[uwsgi-cron] command "./manage.py runjobs minutely" registered as
cron task
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" registered
as cron task
[uwsgi-cron] command "./manage.py runjobs hourly" registered as cron
task
[uwsgi-cron] command "./manage.py runjobs daily" registered as cron task
[uwsgi-cron] command "./manage.py runjobs monthly" registered as
cron task
[uwsgi-cron] command "./manage.py runjobs weekly" registered as cron
task
[uwsgi-cron] command "./manage.py runjobs yearly" registered as cron
task
mapped 208576 bytes (203 KB) for 4 cores
*** Operational MODE: threaded ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter
0x55dfaa41c980 pid: 1 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1)
spawned uWSGI worker 1 (pid: 40, cores: 4)
Sat Apr 27 22:50:18 2019 - [uwsgi-cron] running "./manage.py runjobs
minutely" (pid 45)
[uwsgi-daemons] spawning "./manage.py qcluster" (uid: 82 gid: 82)
22:50:21 [Q] INFO Q Cluster-47 starting.
22:50:21 [Q] INFO Process-1:1 ready for work at 59
22:50:21 [Q] INFO Process-1:2 ready for work at 60
22:50:21 [Q] INFO Process-1:3 ready for work at 61
22:50:21 [Q] INFO Process-1:4 ready for work at 62
22:50:21 [Q] INFO Process-1:5 monitoring at 63
22:50:21 [Q] INFO Process-1 guarding cluster at 58
22:50:21 [Q] INFO Process-1:6 pushing tasks at 64
22:50:21 [Q] INFO Q Cluster-47 running.
22:59:31 [Q] INFO Enqueued 3403
22:59:31 [Q] INFO Process-1:1 processing [update_from_mailman]
22:59:33 [Q] INFO Processed [update_from_mailman]
Sat Apr 27 23:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
quarter_hourly" (pid 73)
Sat Apr 27 23:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
hourly" (pid 74)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
with pid 73 exited after 64 second(s)
23:01:28 [Q] INFO Enqueued 3404
23:01:29 [Q] INFO Process-1:2 processing
[rebuild_mailinglist_cache_recent]
[uwsgi-cron] command "./manage.py runjobs hourly" running with pid
74 exited after 91 second(s)
Sat Apr 27 23:01:36 2019 - uwsgi_response_write_body_do(): Broken
pipe [core/writer.c line 341] during GET / (212.203.58.154)
OSError: write error
23:01:36 [Q] INFO Processed [rebuild_mailinglist_cache_recent]
Sat Apr 27 23:15:00 2019 - [uwsgi-cron] running "./manage.py runjobs
quarter_hourly" (pid 88)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
with pid 88 exited after 4 second(s)
23:28:24 [Q] INFO Enqueued 3405
23:28:24 [Q] INFO Process-1:3 processing [update_from_mailman]
23:28:25 [Q] INFO Processed [update_from_mailman]
Sat Apr 27 23:30:00 2019 - [uwsgi-cron] running "./manage.py runjobs
quarter_hourly" (pid 96)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
with pid 96 exited after 4 second(s)
23:44:40 [Q] INFO Enqueued 3406
23:44:40 [Q] INFO Process-1:4 processing [update_from_mailman]
23:44:41 [Q] INFO Processed [update_from_mailman]
Sat Apr 27 23:45:00 2019 - [uwsgi-cron] running "./manage.py runjobs
quarter_hourly" (pid 104)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
with pid 104 exited after 4 second(s)
Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
quarter_hourly" (pid 113)
Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
hourly" (pid 114)
Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
daily" (pid 115)
Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
weekly" (pid 116)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
with pid 113 exited after 55 second(s)
[uwsgi-cron] command "./manage.py runjobs weekly" running with pid
116 exited after 55 second(s)
00:01:36 [Q] INFO Enqueued 3407
00:01:36 [Q] INFO Process-1:1 processing
[rebuild_mailinglist_cache_recent]
[uwsgi-cron] command "./manage.py runjobs hourly" running with pid
114 exited after 99 second(s)
00:01:50 [Q] INFO Processed [rebuild_mailinglist_cache_recent]
00:04:52 [Q] INFO Enqueued 3408
00:04:52 [Q] INFO Process-1:2 processing [update_from_mailman]
00:04:54 [Q] INFO Processed [update_from_mailman]Dockerfile:
FROM python:3.6-alpine3.7 # Add startup script to container COPY
assets/docker-entrypoint.sh /usr/local/bin/ # Install packages and
dependencies for hyperkitty and add user for executing apps. # It's
important that the user has the UID/GID 82 so nginx can access the
files. RUN set -ex \&& apk add --no-cache --virtual .build-deps gcc
libc-dev linux-headers git \postgresql-dev \&& apk add --no-cache
--virtual .mailman-rundeps bash sassc mailcap \postgresql-client
curl \&& pip install -U django==2.2 \&& pip install
git+https://gitlab.com/eestec/mailmanclient
\git+https://gitlab.com/mailman/hyperkitty@c9fa4d4bfc295438d3e01cd93090064d004cf44d
\git+https://gitlab.com/eestec/django-mailman3 \whoosh \uwsgi
\psycopg2 \dj-database-url \typing \&& apk del .build-deps \&&
addgroup -S -g 82 hyperkitty \&& adduser -S -u 82 -G hyperkitty
hyperkitty \&& chmod u+x /usr/local/bin/docker-entrypoint.sh# Add
needed files for uwsgi server + settings for django COPY
assets/__init__.py /home/hyperkittyCOPY assets/manage.py
/home/hyperkittyCOPY assets/urls.py /home/hyperkittyCOPY
assets/wsgi.py /home/hyperkittyCOPY assets/uwsgi.ini
/home/hyperkittyCOPY assets/settings.py /home/hyperkitty# Change
ownership for uwsgi+django files and set execution rights for
management script RUN chown -R hyperkitty /home/hyperkitty && chmod
u+x /home/hyperkitty/manage.py# Make sure we are in the correct
working dir WORKDIR /home/hyperkittyEXPOSE 8080 8081# Use stop
signal for uwsgi server STOPSIGNAL SIGINTENTRYPOINT
["docker-entrypoint.sh"]CMD ["uwsgi", "--ini",
"/home/hyperkitty/uwsgi.ini"]On 4/27/19 7:58 PM, Abhilash Raj wrote:
On Sat, Apr 27, 2019, at 9:40 AM, Alain Kohli wrote:
I have run "python manage.py rebuild_index" before, doesn't that do clear_index as well? Apart from that, I run hyperkitty in a docker container and didn't know fulltext_index should be persistent, so that got deleted after every version update for sure. Which images are you using and how are you deploying them?
You should persist fulltext_index, yes, and possibly logs if you need them for debugging later.
Can you paste the entire error traceback?
On 4/26/19 10:18 PM, Mark Sapiro wrote:
On 4/26/19 11:14 AM, Alain Kohli wrote:
I see loads of "A string literal cannot contain NUL (0x00) characters." messages, but I haven't found missing messages in the archives yet. Not sure how that could be related, though. Apart from that I don't see anything unusual. The other jobs (quarter_hourly, hourly, etc.) seem to run and finish normally. Did you upgrade from a Python 2.7 version of HyperKitty to a Python 3 version? The Haystack/Whoosh search engine databases are not compatible between the two and "A string literal cannot contain NUL (0x00) characters." is the symptom.
You need to run 'python manage.py clear_index' or just remove all the files from the directory defined as 'PATH' under HAYSTACK_CONNECTIONS in your settings file (normally 'fulltext_index' in the same directory that contains your settings.py.
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/