On Mon, Dec 3, 2018, at 11:44 AM, Stefan Tatschner wrote:
On Mon, 2018-12-03 at 10:01 -0800, Abhilash Raj wrote:
So, if you are using Django's ORM to keep the task states in Django Q, this is bound to happen depending on how frequently do you run tasks. In Hyperkitty, we have some minutely tasks, which means the polling is going to be high.
You can however, configure Django Q to store the tasks outside of database in a better high-speed in-memory database like Redis for example[1] for queuing the tasks. Other options like MongoDB, Amazon SQS is also available.
Is it really necessary to have these high complexity? Reducing database load by introducing another database seems wrong to me.
Redis is not exactly a database, it is used as a message broker.
The ORM backend in DjangoQ exists for low performance workloads where you can afford to poll the database that frequently and not have negative affects on rest of your system. It also lets you avoid installing a message broker like Redis.
If your system does handle a ton of traffic, then it is recommended to use Redis or similar higher-performant systems.
-- thanks, Abhilash Raj (maxking)