API timeout for one list
I apologize, because I think this was talked about before, but I could not find it in the archives.
For one list, the REST API times out. Doesn’t seem to matter which page I try to go to. The uwsgi-error.log entry:
--- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 96, in inner return func(*args, **kwargs) psycopg2.ProgrammingError: no results to fetch
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 332, in pusher task_set = broker.dequeue() File "/usr/lib/python3.8/site-packages/django_q/brokers/orm.py", line 67, in dequeue if tasks: File "/usr/lib/python3.8/site-packages/django/db/models/query.py", line 278, in __bool__ self._fetch_all() File "/usr/lib/python3.8/site-packages/django/db/models/query.py", line 1242, in _fetch_all self._result_cache = list(self._iterable_class(self)) File "/usr/lib/python3.8/site-packages/django/db/models/query.py", line 55, in __iter__ results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size) File "/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql return list(result) File "/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1554, in cursor_iter for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel): File "/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1554, in <lambda> for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel): File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 96, in inner return func(*args, **kwargs) File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 89, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 96, in inner return func(*args, **kwargs) django.db.utils.ProgrammingError: no results to fetch
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3.8/logging/__init__.py", line 1081, in emit msg = self.format(record) File "/usr/lib/python3.8/logging/__init__.py", line 925, in format return fmt.format(record) File "/usr/lib/python3.8/logging/__init__.py", line 664, in format record.message = record.getMessage() File "/usr/lib/python3.8/logging/__init__.py", line 369, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "./manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/usr/lib/python3.8/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line utility.execute() File "/usr/lib/python3.8/site-packages/django/core/management/__init__.py", line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/usr/lib/python3.8/site-packages/django/core/management/base.py", line 323, in run_from_argv self.execute(*args, **cmd_options) File "/usr/lib/python3.8/site-packages/django/core/management/base.py", line 364, in execute output = self.handle(*args, **options) File "/usr/lib/python3.8/site-packages/django_q/management/commands/qcluster.py", line 22, in handle q.start() File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 67, in start self.sentinel.start() File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/usr/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch code = process_obj._bootstrap(parent_sentinel=child_r) File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 157, in __init__ self.start() File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 162, in start self.guard() File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 268, in guard self.reincarnate(self.pusher) File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 210, in reincarnate self.pusher = self.spawn_pusher() File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 190, in spawn_pusher return self.spawn_process(pusher, self.task_queue, self.event_out, self.broker) File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 186, in spawn_process p.start() File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/usr/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch code = process_obj._bootstrap(parent_sentinel=child_r) File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 334, in pusher logger.error(e, traceback.format_exc()) Message: ProgrammingError('no results to fetch') Arguments: ('Traceback (most recent call last):\n File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 96, in inner\n return func(*args, **kwargs)\npsycopg2.ProgrammingError: no results to fetch\n \nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/usr/lib/python3.8/site-packages/django_q/cluster.py", line 332, in pusher\n task_set = bro ker.dequeue()\n File "/usr/lib/python3.8/site-packages/django_q/brokers/orm.py", line 67, in dequeue\n if tasks:\n File "/usr/lib/python3.8/site-packages/django/db/models/query.py", line 278, in __bool__\n self._fetch_all()\n File "/usr/lib/python3.8/site-packages/django/db/models/query.py", line 1242, in _fetch_all\n self._result_cache = list(self._iterable_class(self))\n File "/usr/lib/python3.8/site-packa ges/django/db/models/query.py", line 55, in __iter__\n results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)\n File "/usr/lib/python3.8/site-packages/django/db/models/sq l/compiler.py", line 1175, in execute_sql\n return list(result)\n File "/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1554, in cursor_iter\n for rows in iter((lambda: cursor.fetc hmany(itersize)), sentinel):\n File "/usr/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1554, in <lambda>\n for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):\n File "/us r/lib/python3.8/site-packages/django/db/utils.py", line 96, in inner\n return func(*args, **kwargs)\n File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 89, in __exit__\n raise dj_exc_value.w ith_traceback(traceback) from exc_value\n File "/usr/lib/python3.8/site-packages/django/db/utils.py", line 96, in inner\n return func(*args, **kwargs)\ndjango.db.utils.ProgrammingError: no results to fetch\n' ,)
The other lists work fine.
Thank you, Seth
On 9/22/20 12:45 PM, Seth Seeger wrote:
I apologize, because I think this was talked about before, but I could not find it in the archives.
For one list, the REST API times out. Doesn’t seem to matter which page I try to go to. The uwsgi-error.log entry:
...
The other lists work fine.
This can happen on lists with lots of members.
Try increasing the timeout. See <https://wiki.list.org/x/17892072>
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sep 22, 2020, at 4:30 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 9/22/20 12:45 PM, Seth Seeger wrote:
I apologize, because I think this was talked about before, but I could not find it in the archives.
For one list, the REST API times out. Doesn’t seem to matter which page I try to go to. The uwsgi-error.log entry:
...
The other lists work fine.
This can happen on lists with lots of members.
Try increasing the timeout. See <https://wiki.list.org/x/17892072>
Thank you - that did solve my timeout problem.
However, each page for this list takes between 40 and 80 seconds to load. The list has 465 members, and it’s running (via Docker) on a 2 core, 4 GB RAM Linode VM. I’ve watch htop during the load - one CPU is around 20%, and the other hovers around 80%, mostly with the process “runner”.
Is there a way to optimize this list? There are no held messages, no subscription requests, no templates.
The next biggest list has 438 members, and all those pages load very quickly (0-2 seconds). I can’t figure out what the difference between the two lists.
Thanks, Seth
On Wed, Sep 23, 2020, at 6:33 AM, Seth Seeger wrote:
On Sep 22, 2020, at 4:30 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 9/22/20 12:45 PM, Seth Seeger wrote:
I apologize, because I think this was talked about before, but I could not find it in the archives.
For one list, the REST API times out. Doesn’t seem to matter which page I try to go to. The uwsgi-error.log entry:
...
The other lists work fine.
This can happen on lists with lots of members.
Try increasing the timeout. See <https://wiki.list.org/x/17892072>
Thank you - that did solve my timeout problem.
However, each page for this list takes between 40 and 80 seconds to load. The list has 465 members, and it’s running (via Docker) on a 2 core, 4 GB RAM Linode VM. I’ve watch htop during the load - one CPU is around 20%, and the other hovers around 80%, mostly with the process “runner”.
Is there a way to optimize this list? There are no held messages, no subscription requests, no templates.
Are there any subscription requests pending confirmation from the User. If you go to Subscription Requests -> Pending Confirmation from list's admin page, you'll see the list there.
In the past, I've noticed this caused a significant delay in every list page. This has since been fixed in the master branch, but yet to be released.
The next biggest list has 438 members, and all those pages load very quickly (0-2 seconds). I can’t figure out what the difference between the two lists.
Thanks, Seth
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- thanks, Abhilash Raj (maxking)
On Sep 23, 2020, at 11:45 AM, Abhilash Raj <maxking@asynchronous.in> wrote:
Is there a way to optimize this list? There are no held messages, no subscription requests, no templates.
Are there any subscription requests pending confirmation from the User. If you go to Subscription Requests -> Pending Confirmation from list's admin page, you'll see the list there.
In the past, I've noticed this caused a significant delay in every list page. This has since been fixed in the master branch, but yet to be released.
Ah hah! There are 2,095. Thank you.
Seth
participants (3)
-
Abhilash Raj -
Mark Sapiro -
Seth Seeger