Issues encountered while upgrading to 3.3.7
Hello everyone, I am starting a new thread on this so that the issues I encountered do not get mixed up with the other discussions. Over the weekend I upgraded an MM3 installation to the latest versions of the components. I started off by creating a new venv, installing and then migrating the configurations and the scripts. In the end, I ended up with MM3 Core ==3.3.7, postorius==1.3.7, HyperKitty==1.3.6, django==4.1.3. 1. SQLAlchemy - At the stage where one runs 'mailman info', I encountered the first error: : sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgres This is solved by a change in mailman.cfg so that the URI starts with postgresql:// instead of postgres://. SQLAlchemy used to accept both but has removed support for the postgres name. <https://github.com/sqlalchemy/sqlalchemy/issues/6083#issuecomment-801478013> 2. Django==4.x At the point of running the post-update processes, particularly the "migrate" bit, I got the following error: : ImportError: cannot import name 'url' from 'django.conf.urls' (/opt/mailman/mm/venv/lib/python3.9/site-packages/django/conf/urls/__init__.py) I understand this is a consequence of the django.conf.urls.url() having been deprecated starting version 3 and has been removed in version 4 of Django. https://docs.djangoproject.com/en/4.1/releases/4.0/#features-removed-in-4-0 The upgrade process seems to have brought in the newer version of Django because now I have 4.1.3 up from 3.x If anyone wants, I found an elaborate explanation at https://bobbyhadz.com/blog/python-importerror-cannot-import-name-url-from-dj... . My Google-fu found suggestions that in urls.py, I should import re_path and use it, or alias it for url for the migrate to work. The solution I used was to alias url to re_path, but I am considering just doing away with url altogether. Not sure which is the better option between using re_path or path. So here is the patch I have used: (venv) mailman@lists:~/mm$ diff -u urls.py.bak urls.py --- urls.py.bak 2022-12-03 12:20:28.028090659 +0300 +++ urls.py 2022-12-03 12:36:20.298875624 +0300 @@ -17,7 +17,8 @@ # Postorius. If not, see <http://www.gnu.org/licenses/>. -from django.conf.urls import include, url +from django.conf.urls import include +from django.urls import re_path as url from django.contrib import admin from django.urls import reverse_lazy from django.views.generic import RedirectView 3. The last issue that I encountered is one I kind of solved, but completely don't understand the consequences of the solution I applied: /opt/mailman/mm/bin/django-admin migrate <CUT> System check identified some issues: WARNINGS: django_mailman3.MailDomain: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the DjangoMailman3Config.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. django_mailman3.Profile: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the DjangoMailman3Config.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Attachment: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Email: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Favorite: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.LastView: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.MailingList: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Profile: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Tag: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Tagging: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Thread: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.ThreadCategory: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Vote: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. postorius.EmailTemplate: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostoriusConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. Operations to perform: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount </CUT> Here again, Google-fu came to the rescue. I added the following line to settings_local.py and the warnings were suppressed. DEFAULT_AUTO_FIELD = 'django.db.models.AutoField' I need to understand how this affects everything and how this is likely to impact the functionality of MM3. What is the proper solution for this? And should it be django.db.models.BigAutoField or django.db.models.AutoField? Which is which? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
Odhiambo Washington wrote:
- SQLAlchemy - At the stage where one runs 'mailman info', I encountered the first error: : sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgres
This is solved by a change in mailman.cfg so that the URI starts with postgresql:// instead of postgres://. SQLAlchemy used to accept both but has removed support for the postgres name. <https://github.com/sqlalchemy/sqlalchemy/issues/6083#issuecomment-801478013>
This is covered here: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.h... "BREAKING CHANGE: When using PostgreSQL database with Mailman, starting with Mailman Core 3.3.7 (due to bump in SQLAlchemy 1.4+), the urls in [database] section will require postgresql as the scheme."
- Django==4.x At the point of running the post-update processes, particularly the "migrate" bit, I got the following error: : ImportError: cannot import name 'url' from 'django.conf.urls' (/opt/mailman/mm/venv/lib/python3.9/site-packages/django/conf/urls/__init__.py)
This was recently covered here: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/N...
When I came to upgrade my actual installation I migrated to the Mailman-Web package so this file is now maintained by the Mailman developers.
- The last issue that I encountered is one I kind of solved, but completely don't understand the consequences of the solution I applied: /opt/mailman/mm/bin/django-admin migrate
System check identified some issues:
I got these as well. I have safely ignored these but didn't suppress the warnings like you have done. My Googling and continued usage since upgrading to Django 4.x indicates these are safe to ignore but need to be dealt with by the developers at some point.
Thanks. Andrew.
On Mon, Dec 5, 2022 at 1:57 PM Andrew Hodgson <andrew@hodgson.io> wrote:
Odhiambo Washington wrote:
- SQLAlchemy - At the stage where one runs 'mailman info', I encountered the first error: : sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgres
This is solved by a change in mailman.cfg so that the URI starts with postgresql:// instead of postgres://. SQLAlchemy used to accept both but has removed support for the postgres name. < https://github.com/sqlalchemy/sqlalchemy/issues/6083#issuecomment-801478013
This is covered here: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.h... "BREAKING CHANGE: When using PostgreSQL database with Mailman, starting with Mailman Core 3.3.7 (due to bump in SQLAlchemy 1.4+), the urls in [database] section will require postgresql as the scheme."
- Django==4.x At the point of running the post-update processes, particularly the "migrate" bit, I got the following error: : ImportError: cannot import name 'url' from 'django.conf.urls'
(/opt/mailman/mm/venv/lib/python3.9/site-packages/django/conf/urls/__init__.py)
This was recently covered here: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/N...
When I came to upgrade my actual installation I migrated to the Mailman-Web package so this file is now maintained by the Mailman developers.
- The last issue that I encountered is one I kind of solved, but completely don't understand the consequences of the solution I applied: /opt/mailman/mm/bin/django-admin migrate
System check identified some issues:
I got these as well. I have safely ignored these but didn't suppress the warnings like you have done. My Googling and continued usage since upgrading to Django 4.x indicates these are safe to ignore but need to be dealt with by the developers at some point.
Thanks. Andrew.
Hi Andrew,
Thanks for the quick response. I must have missed reading from the list on Saturday for obvious reasons - weekend :-) And of course, I kinda missed the BREAKING CHANGE also.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 12/5/22 02:37, Odhiambo Washington wrote:
Here again, Google-fu came to the rescue. I added the following line to settings_local.py and the warnings were suppressed. DEFAULT_AUTO_FIELD = 'django.db.models.AutoField' I need to understand how this affects everything and how this is likely to impact the functionality of MM3. What is the proper solution for this? And should it be django.db.models.BigAutoField or django.db.models.AutoField? Which is which?
Prior to Django 3.2 Django set default primary keys to 32 bit integers. In 3.2 Django introduced the possibility to set default primary keys to 64 bit integers by setting the type to django.db.models.BigAutoField.
Thus, putting
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
in your settings will maintain compatibility with pre 3.2 versions and existing tables and this is the correct thing to do.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Mon, Dec 5, 2022 at 8:54 PM Mark Sapiro <mark@msapiro.net> wrote:
On 12/5/22 02:37, Odhiambo Washington wrote:
Here again, Google-fu came to the rescue. I added the following line to settings_local.py and the warnings were suppressed. DEFAULT_AUTO_FIELD = 'django.db.models.AutoField' I need to understand how this affects everything and how this is likely
to
impact the functionality of MM3. What is the proper solution for this? And should it be django.db.models.BigAutoField or django.db.models.AutoField? Which is which?
Prior to Django 3.2 Django set default primary keys to 32 bit integers. In 3.2 Django introduced the possibility to set default primary keys to 64 bit integers by setting the type to django.db.models.BigAutoField.
Thus, putting
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
in your settings will maintain compatibility with pre 3.2 versions and existing tables and this is the correct thing to do.
Thank you very much for the clarification.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 12/5/22 10:02, Odhiambo Washington wrote:
On Mon, Dec 5, 2022 at 8:54 PM Mark Sapiro <mark@msapiro.net> wrote:
Thus, putting
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
in your settings will maintain compatibility with pre 3.2 versions and existing tables and this is the correct thing to do.
Thank you very much for the clarification.
Also note that for the past year or so, the files postorius/example_project/settings.py:, mailman_web/settings/base.py, hyperkitty/example_project/settings.py and mailman-suite/mailman-suite_project/settings.py have all contained
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
and if you follow https://docs.mailman3.org/en/latest/install/virtualenv.html#initial-configur... you will import mailman_web/settings/base.py and pick this up from mailman-web 0.0.6 and both https://wiki.list.org/x/17892066 and https://wiki.list.org/x/17891998 suggest that your settings.py should be a copy of the mailman-suite settings.py file but with DEBUG = False, the implication being that when that file changes, yours should too.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Andrew Hodgson
-
Mark Sapiro
-
Odhiambo Washington