On our installation, we are often gettting Internal Server Error (500). The logs look like this:
ERROR 2018-01-09 13:52:57,640 1604 django.request Internal Server Error: /hyperkitty/list/test@seznami.cjvt.si/
Traceback (most recent call last):
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/exception.py", line 42, in inner
response = get_response(request)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/HyperKitty-1.1.0-py2.7.egg/hyperkitty/lib/view_helpers.py", line 133, in inner
mlist = MailingList.objects.get(name=mlist_fqdn)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 379, in get
num = len(clone)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 238, in __len__
self._fetch_all()
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 1087, in _fetch_all
self._result_cache = list(self.iterator())
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 54, in __iter__
results = compiler.execute_sql()
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/sql/compiler.py", line 835, in execute_sql
cursor.execute(sql, params)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/sqlite3/base.py", line 337, in execute
return Database.Cursor.execute(self, query, params)
OperationalError: database is locked
Does this suggest I should switch from sqlite to postgres (https://stackoverflow.com/a/26864360)? I'm happy to do that, but how do I migrate the data between the databases (after installing postgres and creating the database)? I've found this doc (http://mailman.readthedocs.io/en/latest/src/mailman/docs/database.html#datab...), but I don't think I understand this and I haven't used alembic: these instructions are just for when you modify the model within the same database, right?
Thanks in advance!
Hi
I've had this problem when mailman was importing new lists (I had script running) and I wanted to access those lists. I am using sqlite as well.
Sten
On 11/01/2018 11:21, cyp@trojina.si wrote:
On our installation, we are often gettting Internal Server Error (500). The logs look like this:
ERROR 2018-01-09 13:52:57,640 1604 django.request Internal Server Error: /hyperkitty/list/test@seznami.cjvt.si/ Traceback (most recent call last): File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/exception.py", line 42, in inner response = get_response(request) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/HyperKitty-1.1.0-py2.7.egg/hyperkitty/lib/view_helpers.py", line 133, in inner mlist = MailingList.objects.get(name=mlist_fqdn) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 379, in get num = len(clone) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 238, in __len__ self._fetch_all() File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 1087, in _fetch_all self._result_cache = list(self.iterator()) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/query.py", line 54, in __iter__ results = compiler.execute_sql() File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/models/sql/compiler.py", line 835, in execute_sql cursor.execute(sql, params) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/utils.py", line 94, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/opt/mailman/venv2/local/lib/python2.7/site-packages/Django-1.10.7-py2.7.egg/django/db/backends/sqlite3/base.py", line 337, in execute return Database.Cursor.execute(self, query, params) OperationalError: database is locked
Does this suggest I should switch from sqlite to postgres (https://stackoverflow.com/a/26864360)? I'm happy to do that, but how do I migrate the data between the databases (after installing postgres and creating the database)? I've found this doc (http://mailman.readthedocs.io/en/latest/src/mailman/docs/database.html#datab...), but I don't think I understand this and I haven't used alembic: these instructions are just for when you modify the model within the same database, right?
Thanks in advance!
Mailman-users mailing list mailman-users@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
I've had this problem when mailman was importing new lists (I had script running) and I wanted to access those lists. I am using sqlite as well.
I'm not sure this is our case, as we only have a few mailing lists and don't manually run any scripts. And yet this error comes up a lot (on the order of several times a day). I think it generally comes up when the user is clicking in the browser, getting a 500 error. After a reload or 3, it tends to go away, which is why I've been shamelessly ignoring this for months, but I can't get away with this anymore. :)
On 01/11/2018 11:16 AM, cyp@trojina.si wrote:
I've had this problem when mailman was importing new lists (I had script running) and I wanted to access those lists. I am using sqlite as well. I'm not sure this is our case, as we only have a few mailing lists and don't manually run any scripts. And yet this error comes up a lot (on the order of several times a day). I think it generally comes up when the user is clicking in the browser, getting a 500 error. After a reload or 3, it tends to go away, which is why I've been shamelessly ignoring this for months, but I can't get away with this anymore. :) Assuming you set up everything as mentioned in the docs, you have a couple of cron jobs that interact with the database...
Basically any script that interacts with sqlite will lock it for as long as the script runs. Web workers appear to always time out when a script is running...
Actually, setting up the cronjobs is also something that awaits me ... I initially used this guide (http://docs.mailman3.org/en/latest/devsetup.html), as it seemed most explicit, but as such I overlooked the cronjobs (which I assume should be http://docs.mailman3.org/en/latest/config-core.html#configuring-cron-jobs and http://docs.mailman3.org/en/latest/config-web.html#scheduled-tasks-required). I wonder if this could also be somehow indirectly responsible?
I (finally) set up the standard cronjobs (http://docs.mailman3.org/en/latest/config-web.html#scheduled-tasks-required) last week, but this doesn't seem to have any obvious effect on this problem (i.e., it's not clearly either better or worse). But I wonder whether the fact that I had overlooked this (so that our installation didn't have these cronjobs installed for months) could have resulted in some hard-to-fix corrupt state?
We also looked into the other logs a little and found that uwsgi-error.log is getting filled constantly. Even if the cronjobs are commented out, and even in the middle of the night, and even though we only have a few mailing lists, all with limited activity, there's probably more than a line per second in this log. Here's the last screenful, which seems as representative as any:
14:45:15 [Q] INFO Process-1:7687 processing [compute_thread_positions] 14:45:15 [Q] ERROR Failed [update_from_mailman] - MailingList matching query does not exist. 14:45:16 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:16 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_recent] 14:45:16 [Q] INFO Process-1:7688 processing [compute_thread_positions] 14:45:16 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:17 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:17 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_recent] 14:45:17 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:18 [Q] INFO Process-1:7688 processing [rebuild_mailinglist_cache_for_month] 14:45:18 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. 14:45:19 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_for_month] 14:45:19 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. 14:45:20 [Q] INFO Process-1:7688 processing [compute_thread_positions] 14:45:20 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:20 [Q] INFO Process-1:7687 processing [check_orphans] 14:45:21 [Q] INFO Process-1:7688 processing [compute_thread_positions] 14:45:21 [Q] ERROR Failed [check_orphans] - Email matching query does not exist. 14:45:21 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:22 [Q] INFO Process-1:7688 processing [update_from_mailman] 14:45:22 [Q] INFO Process-1:7687 processing [compute_thread_positions] 14:45:23 [Q] INFO Process-1:7688 processing [check_orphans] 14:45:23 [Q] ERROR Failed [update_from_mailman] - MailingList matching query does not exist. 14:45:23 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:24 [Q] ERROR Failed [check_orphans] - Email matching query does not exist. 14:45:24 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_recent] 14:45:24 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:25 [Q] INFO Process-1:7687 processing [update_from_mailman] 14:45:25 [Q] INFO Process-1:7688 processing [compute_thread_positions] 14:45:25 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_recent] 14:45:26 [Q] INFO Process-1:7688 processing [rebuild_mailinglist_cache_for_month] 14:45:26 [Q] INFO Process-1:7687 processing [check_orphans] 14:45:26 [Q] ERROR Failed [update_from_mailman] - MailingList matching query does not exist. 14:45:26 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:27 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:27 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. 14:45:27 [Q] ERROR Failed [check_orphans] - Email matching query does not exist. 14:45:27 [Q] INFO Process-1:7688 processing [rebuild_mailinglist_cache_for_month] 14:45:28 [Q] INFO Process-1:7687 processing [check_orphans] 14:45:28 [Q] INFO Process-1:7688 processing [update_from_mailman] 14:45:29 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. 14:45:29 [Q] ERROR Failed [check_orphans] - Email matching query does not exist. 14:45:29 [Q] ERROR Failed [update_from_mailman] - MailingList matching query does not exist. 14:45:29 [Q] INFO Process-1:7687 processing [compute_thread_positions] 14:45:30 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:30 [Q] INFO Process-1:7688 processing [rebuild_mailinglist_cache_recent] 14:45:30 [Q] INFO Process-1:7688 processing [rebuild_mailinglist_cache_recent] 14:45:30 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:31 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:31 [Q] INFO Process-1:7688 processing [check_orphans] 14:45:31 [Q] ERROR Failed [check_orphans] - Email matching query does not exist. 14:45:31 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_for_month] 14:45:31 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. 14:45:32 [Q] INFO Process-1:7688 processing [update_from_mailman] 14:45:32 [Q] ERROR Failed [update_from_mailman] - MailingList matching query does not exist.
We also noticed that there's almost constant writing going on. A typical line from iotop:
7430 be/4 www-data 0.00 B/s 31.34 K/s 0.00 % 44.14 % python ./manage.py qcluster
We will try a MM3 upgrade on Monday (to realign all the suite components with the 3.1 versions here: https://wiki.list.org/Mailman3), and maybe migrate to postgresql, but I have a feeling that there's a more "basic" problem. Help or even speculation would be appreciated. :)
I (finally) set up the standard cronjobs (http://docs.mailman3.org/en/latest/config-web.html#scheduled-tasks-required) last week, but this doesn't seem to have any obvious effect on this problem (i.e., it's not clearly either better or worse). But I wonder whether the fact that I had overlooked this (so that our installation didn't have these cronjobs installed for months) could have resulted in some hard-to-fix corrupt state?
On 01/19/2018 03:53 PM, Cyprian Laskowski wrote: that shouldn't be the case. The cronjobs fix issues for all exisiting entries. They are executed as often as deemed necessary to keep state somehow stable without impacting performance too much...
We also looked into the other logs a little and found that uwsgi-error.log is getting filled constantly. Even if the cronjobs are commented out, and even in the middle of the night, and even though we only have a few mailing lists, all with limited activity, there's probably more than a line per second in this log. Here's the last screenful, which seems as representative as any:
14:45:15 [Q] INFO Process-1:7687 processing [compute_thread_positions] 14:45:15 [Q] ERROR Failed [update_from_mailman] - MailingList matching query does not exist. 14:45:16 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:16 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_recent] 14:45:16 [Q] INFO Process-1:7688 processing [compute_thread_positions] 14:45:16 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:17 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:17 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_recent] 14:45:17 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:18 [Q] INFO Process-1:7688 processing [rebuild_mailinglist_cache_for_month] 14:45:18 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. 14:45:19 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_for_month] 14:45:19 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. 14:45:20 [Q] INFO Process-1:7688 processing [compute_thread_positions] 14:45:20 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:20 [Q] INFO Process-1:7687 processing [check_orphans] 14:45:21 [Q] INFO Process-1:7688 processing [compute_thread_positions] 14:45:21 [Q] ERROR Failed [check_orphans] - Email matching query does not exist. 14:45:21 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:22 [Q] INFO Process-1:7688 processing [update_from_mailman] 14:45:22 [Q] INFO Process-1:7687 processing [compute_thread_positions] 14:45:23 [Q] INFO Process-1:7688 processing [check_orphans] 14:45:23 [Q] ERROR Failed [update_from_mailman] - MailingList matching query does not exist. 14:45:23 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:24 [Q] ERROR Failed [check_orphans] - Email matching query does not exist. 14:45:24 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_recent] 14:45:24 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:25 [Q] INFO Process-1:7687 processing [update_from_mailman] 14:45:25 [Q] INFO Process-1:7688 processing [compute_thread_positions] 14:45:25 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_recent] 14:45:26 [Q] INFO Process-1:7688 processing [rebuild_mailinglist_cache_for_month] 14:45:26 [Q] INFO Process-1:7687 processing [check_orphans] 14:45:26 [Q] ERROR Failed [update_from_mailman] - MailingList matching query does not exist. 14:45:26 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:27 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:27 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. 14:45:27 [Q] ERROR Failed [check_orphans] - Email matching query does not exist. 14:45:27 [Q] INFO Process-1:7688 processing [rebuild_mailinglist_cache_for_month] 14:45:28 [Q] INFO Process-1:7687 processing [check_orphans] 14:45:28 [Q] INFO Process-1:7688 processing [update_from_mailman] 14:45:29 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. 14:45:29 [Q] ERROR Failed [check_orphans] - Email matching query does not exist. 14:45:29 [Q] ERROR Failed [update_from_mailman] - MailingList matching query does not exist. 14:45:29 [Q] INFO Process-1:7687 processing [compute_thread_positions] 14:45:30 [Q] ERROR Failed [compute_thread_positions] - Thread matching query does not exist. 14:45:30 [Q] INFO Process-1:7688 processing [rebuild_mailinglist_cache_recent] 14:45:30 [Q] INFO Process-1:7688 processing [rebuild_mailinglist_cache_recent] 14:45:30 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:31 [Q] ERROR Failed [rebuild_mailinglist_cache_recent] - MailingList matching query does not exist. 14:45:31 [Q] INFO Process-1:7688 processing [check_orphans] 14:45:31 [Q] ERROR Failed [check_orphans] - Email matching query does not exist. 14:45:31 [Q] INFO Process-1:7687 processing [rebuild_mailinglist_cache_for_month] 14:45:31 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. 14:45:32 [Q] INFO Process-1:7688 processing [update_from_mailman] 14:45:32 [Q] ERROR Failed [update_from_mailman] - MailingList matching query does not exist.
We also noticed that there's almost constant writing going on. A typical line from iotop:
7430 be/4 www-data 0.00 B/s 31.34 K/s 0.00 % 44.14 % python ./manage.py qcluster
We will try a MM3 upgrade on Monday (to realign all the suite components with the 3.1 versions here: https://wiki.list.org/Mailman3), and maybe migrate to postgresql, but I have a feeling that there's a more "basic" problem. Help or even speculation would be appreciated. :) migrating to postgresql should at least take care of the database locking problem.
Make sure you are actually using either the master branch (with the latest commit) of hyperkitty or the latest release of it.
Now comes the "speculation": To me it looks like all this output is generated by the django-q tasks. I'm not that familiar with it, but I guess it's either some configuration error or a bug in Hyperkitty. The errors produced seem to corespond to unhandled exceptions, however a quick glance at the current master shows that they are handled, so I'm not sure what's going on here... Compare your settings to the settings in mailman-suite (or better the example-project of Hyperkitty). Update your settings if you spot differences that might have something to do with it, if that doesn't help open an issue about that here https://gitlab.com/mailman/hyperkitty/issues
Thanks for your feedback again. It turned out that upgrading our components to the listed 3.1 suite versions *and* fixing permissions on /var/www/.cache helped significantly. We still get database locked errors, but much more sporadically, and the application is more responsive in general.
However, we haven't been able to switch to postgresql. We did do a django "migrate" for sqlite due to the new version, but when we then backed up the sqlite and tried to load it into the new postgresql database, we get:
hyperkitty.models.email.DoesNotExist: Problem installing fixture '/opt/mailman/mailman-suite/mailman-suite_project/dump_sqlite_updated.json': Email matching query does not exist.
Interestingly, this seems to be the same error as perhaps the most common one in our uwsgi-error.log (see my previous message), which is still seeing constant activity. Any suggestion of how to overcome this?
I have tried to compare our settings to the example ones, and haven't found obvious culprits, but I will take a closer look.
OperationalError: database is locked
Does this suggest I should switch from sqlite to postgres (https://stackoverflow.com/a/26864360)? I'm happy to do that, but how do I migrate the data between the databases (after installing postgres and creating the database)? I've found this doc (http://mailman.readthedocs.io/en/latest/src/mailman/docs/database.html#datab...), but I don't think I understand this and I haven't used alembic: these instructions are just for when you modify the model within the same database, right?
sqlite isn't really good with constant load on it.
Assuming your database is not very big, you can just use the dumpdata and loaddata commands of django.
Otherwise you'll either have to do it "manually" using database dumps or programatically by using two databases for django.
All aproaches are mentioned in this stackoverflow question:
https://stackoverflow.com/questions/3476606/django-what-are-the-best-practic...
When it comes to low level stuff, anything you find for django will work for Hyperkitty.
As for the doc link, this is Mailman core specific and not related to Hyperkitty
Assuming your database is not very big, you can just use the dumpdata and loaddata commands of django. ... All aproaches are mentioned in this stackoverflow question:
https://stackoverflow.com/questions/3476606/django-what-are-the-best-prac...
Aha, that looks easy enough. Thanks for the explanation and link, I can try that.
participants (4)
-
cyp＠trojina.si -
Cyprian Laskowski
-
Simon Hanna -
Sten Aus