Migrating mailman3 to latest ubuntu lts
Hi,
I'm trying to migrate several lists from an ubuntu 18.04, where it was installed via packages (mailman 3.1.1), to ubuntu 22.04.
Since ubuntu 22.04 is broken in terms of packages for mailman3, I'm trying to install manually following the site instructions.
I just dumped the mysql DB we have nowadays, mailman3 and mailman3-web, and loaded in the new server. I also copied the configurations and fixed the paths since now mailman3 lives on /local/mailman and it uses venv. I also copied /var/lib/mailman3 into /local/mailman/var/lib/mailman3.
mailman seems to accept pretty much of the work done.
(venv) mailman@new-server ~ (v3.3.9)> mailman info GNU Mailman 3.3.9 (Tom Sawyer) Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] config file: /etc/mailman3/mailman.cfg db url: mysql+pymysql://mailman3:********@localhost/mailman3?charset=utf8mb4&use_unicode=1 devmode: DISABLED REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:***********
Even configuration is working fine.
(venv) mailman@new-server ~ (v3.3.9)> mailman conf [ARC] authserv_id: [ARC] dkim: yes [ARC] dmarc: yes [ARC] domain: [ARC] enabled: no [ARC] privkey: [ARC] selector: [ - a lot of stuff - ] [shell] banner: Welcome to the GNU Mailman shell [shell] history_file: [shell] prompt: >>> [shell] use_ipython: no [styles] default: legacy-default [webservice] admin_pass: ************************* [webservice] admin_user: restadmin [webservice] api_version: 3.1 [webservice] configuration: python:mailman.config.gunicorn [webservice] hostname: localhost [webservice] port: 8001 [webservice] show_tracebacks: yes [webservice] use_https: no [webservice] workers: 2
But it is failing to start displaying the domain name from the previous machine.
(venv) mailman@new-server ~ (v3.3.9)> mailman start Starting Mailman's master runner Generating MTA alias maps Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute result = self._query(query) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query conn.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result result.read() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read first_packet = self.connection._read_packet() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet packet.raise_for_error() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error err.raise_mysql_exception(self._data) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception raise errorclass(errno, errval) pymysql.err.OperationalError: (1054, "Unknown column 'domain.alias_domain' in 'field list'")
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/local/mailman/venv/bin/mailman", line 33, in <module> sys.exit(load_entry_point('mailman==3.3.9', 'console_scripts', 'mailman')()) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/bin/mailman.py", line 69, in invoke return super().invoke(ctx) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/commands/cli_control.py", line 109, in start call_name(config.mta.incoming).regenerate() File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 109, in regenerate self._generate_lmtp_file(fp) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 149, in _generate_lmtp_file mlist = _FakeList(list_name, mail_host) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 75, in __init__ self.mail_host = _get_alias_domain(mail_host) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 63, in _get_alias_domain d = domain_manager.get(domain) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/database/transaction.py", line 106, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/model/domain.py", line 162, in get if domains.count() < 1: File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 3132, in count self._legacy_from_self(col).enable_eagerloads(False).scalar() File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2825, in scalar ret = self.one() File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2798, in one return self._iter().one() # type: ignore File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2847, in _iter result: Union[ScalarResult[_T], Result[_T]] = self.session.execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2308, in execute return self._execute_internal( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2190, in _execute_internal result: Result[Any] = compile_state_cls.orm_execute_statement( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement result = conn.execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1416, in execute return meth( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 516, in _execute_on_connection return connection._execute_clauseelement( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1639, in _execute_clauseelement ret = self._execute_context( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1848, in _execute_context return self._exec_single_context( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1988, in _exec_single_context self._handle_dbapi_exception( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2343, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute result = self._query(query) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query conn.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result result.read() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read first_packet = self.connection._read_packet() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet packet.raise_for_error() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error err.raise_mysql_exception(self._data) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception raise errorclass(errno, errval) sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1054, "Unknown column 'domain.alias_domain' in 'field list'") [SQL: SELECT count(*) AS count_1 FROM (SELECT domain.id AS domain_id, domain.mail_host AS domain_mail_host, domain.description AS domain_description, domain.alias_domain AS domain_alias_domain FROM domain WHERE domain.mail_host = %(mail_host_1)s) AS anon_1] [parameters: {'mail_host_1': 'old-server.mydomain.com'}] (Background on this error at: https://sqlalche.me/e/20/e3q8)
But I can see some processes are starting.
(venv) mailman@new-server ~ (v3.3.9)> ps auxwww | grep "[p]ython" | grep mailman mailman 1411605 0.0 0.0 32180 22212 ? Ss 10:44 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/gunicorn -c /etc/mailman3/gunicorn.conf mailman_web.wsgi:application mailman 1411607 0.0 0.0 44244 32412 ? S 10:44 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/gunicorn -c /etc/mailman3/gunicorn.conf mailman_web.wsgi:application mailman 1411671 1.0 0.2 98584 83004 ? Ss 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/master -C /etc/mailman3/mailman.cfg mailman 1411681 1.0 0.2 98392 82504 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=archive:0:1 mailman 1411682 97.7 10.1 3396160 3346868 ? R 10:45 1:58 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=bounces:0:1 mailman 1411683 1.0 0.2 98588 82816 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=command:0:1 mailman 1411684 1.0 0.2 98580 82904 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=in:0:1 mailman 1411685 1.0 0.2 172312 82772 ? Sl 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=lmtp:0:1 mailman 1411686 1.1 0.2 98588 82652 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=nntp:0:1 mailman 1411687 1.1 0.2 98608 83016 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=out:0:1 mailman 1411688 1.0 0.2 98584 82936 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=pipeline:0:1 mailman 1411689 1.1 0.2 107040 90972 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 mailman 1411690 1.0 0.2 98592 82784 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=retry:0:1 mailman 1411691 41.3 0.5 187380 171012 ? S 10:45 0:50 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=task:0:1 mailman 1411692 1.0 0.2 98472 83028 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=virgin:0:1 mailman 1411693 1.0 0.2 98448 82608 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=digest:0:1 mailman 1411705 0.0 0.2 107040 75968 ? S 10:45 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 mailman 1411709 0.0 0.2 107040 75968 ? S 10:45 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 mailman 1413629 0.0 0.0 17644 8812 ? Rs 10:47 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/mailman start
Best Regards, Helio Loureiro http://helio.loureiro.eng.br https://github.com/helioloureiro
Did you run the upgrade steps?
https://docs.mailman3.org/en/latest/upgrade-guide.html
For schema upgrades you will at least need to do "mailman-web migrate", but please read the entire section as 3.1.1 to 3.3.9 is a big step.
--Jered
----- On Dec 14, 2023, at 4:48 AM, Helio Loureiro helio@loureiro.eng.br wrote:
Hi,
I'm trying to migrate several lists from an ubuntu 18.04, where it was installed via packages (mailman 3.1.1), to ubuntu 22.04.
Since ubuntu 22.04 is broken in terms of packages for mailman3, I'm trying to install manually following the site instructions.
I just dumped the mysql DB we have nowadays, mailman3 and mailman3-web, and loaded in the new server. I also copied the configurations and fixed the paths since now mailman3 lives on /local/mailman and it uses venv. I also copied /var/lib/mailman3 into /local/mailman/var/lib/mailman3.
mailman seems to accept pretty much of the work done.
(venv) mailman@new-server ~ (v3.3.9)> mailman info GNU Mailman 3.3.9 (Tom Sawyer) Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] config file: /etc/mailman3/mailman.cfg db url: mysql+pymysql://mailman3:********@localhost/mailman3?charset=utf8mb4&use_unicode=1 devmode: DISABLED REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:***********
Even configuration is working fine.
(venv) mailman@new-server ~ (v3.3.9)> mailman conf [ARC] authserv_id: [ARC] dkim: yes [ARC] dmarc: yes [ARC] domain: [ARC] enabled: no [ARC] privkey: [ARC] selector: [ - a lot of stuff - ] [shell] banner: Welcome to the GNU Mailman shell [shell] history_file: [shell] prompt: >>> [shell] use_ipython: no [styles] default: legacy-default [webservice] admin_pass: ************************* [webservice] admin_user: restadmin [webservice] api_version: 3.1 [webservice] configuration: python:mailman.config.gunicorn [webservice] hostname: localhost [webservice] port: 8001 [webservice] show_tracebacks: yes [webservice] use_https: no [webservice] workers: 2
But it is failing to start displaying the domain name from the previous machine.
(venv) mailman@new-server ~ (v3.3.9)> mailman start Starting Mailman's master runner Generating MTA alias maps Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute result = self._query(query) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query conn.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result result.read() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read first_packet = self.connection._read_packet() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet packet.raise_for_error() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error err.raise_mysql_exception(self._data) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception raise errorclass(errno, errval) pymysql.err.OperationalError: (1054, "Unknown column 'domain.alias_domain' in 'field list'")
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/local/mailman/venv/bin/mailman", line 33, in <module> sys.exit(load_entry_point('mailman==3.3.9', 'console_scripts', 'mailman')()) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/bin/mailman.py", line 69, in invoke return super().invoke(ctx) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/commands/cli_control.py", line 109, in start call_name(config.mta.incoming).regenerate() File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 109, in regenerate self._generate_lmtp_file(fp) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 149, in _generate_lmtp_file mlist = _FakeList(list_name, mail_host) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 75, in __init__ self.mail_host = _get_alias_domain(mail_host) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 63, in _get_alias_domain d = domain_manager.get(domain) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/database/transaction.py", line 106, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/model/domain.py", line 162, in get if domains.count() < 1: File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 3132, in count self._legacy_from_self(col).enable_eagerloads(False).scalar() File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2825, in scalar ret = self.one() File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2798, in one return self._iter().one() # type: ignore File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2847, in _iter result: Union[ScalarResult[_T], Result[_T]] = self.session.execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2308, in execute return self._execute_internal( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2190, in _execute_internal result: Result[Any] = compile_state_cls.orm_execute_statement( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement result = conn.execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1416, in execute return meth( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 516, in _execute_on_connection return connection._execute_clauseelement( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1639, in _execute_clauseelement ret = self._execute_context( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1848, in _execute_context return self._exec_single_context( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1988, in _exec_single_context self._handle_dbapi_exception( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2343, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute result = self._query(query) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query conn.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result result.read() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read first_packet = self.connection._read_packet() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet packet.raise_for_error() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error err.raise_mysql_exception(self._data) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception raise errorclass(errno, errval) sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1054, "Unknown column 'domain.alias_domain' in 'field list'") [SQL: SELECT count(*) AS count_1 FROM (SELECT domain.id AS domain_id, domain.mail_host AS domain_mail_host, domain.description AS domain_description, domain.alias_domain AS domain_alias_domain FROM domain WHERE domain.mail_host = %(mail_host_1)s) AS anon_1] [parameters: {'mail_host_1': 'old-server.mydomain.com'}] (Background on this error at: https://sqlalche.me/e/20/e3q8)
But I can see some processes are starting.
(venv) mailman@new-server ~ (v3.3.9)> ps auxwww | grep "[p]ython" | grep mailman mailman 1411605 0.0 0.0 32180 22212 ? Ss 10:44 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/gunicorn -c /etc/mailman3/gunicorn.conf mailman_web.wsgi:application mailman 1411607 0.0 0.0 44244 32412 ? S 10:44 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/gunicorn -c /etc/mailman3/gunicorn.conf mailman_web.wsgi:application mailman 1411671 1.0 0.2 98584 83004 ? Ss 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/master -C /etc/mailman3/mailman.cfg mailman 1411681 1.0 0.2 98392 82504 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=archive:0:1 mailman 1411682 97.7 10.1 3396160 3346868 ? R 10:45 1:58 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=bounces:0:1 mailman 1411683 1.0 0.2 98588 82816 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=command:0:1 mailman 1411684 1.0 0.2 98580 82904 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=in:0:1 mailman 1411685 1.0 0.2 172312 82772 ? Sl 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=lmtp:0:1 mailman 1411686 1.1 0.2 98588 82652 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=nntp:0:1 mailman 1411687 1.1 0.2 98608 83016 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=out:0:1 mailman 1411688 1.0 0.2 98584 82936 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=pipeline:0:1 mailman 1411689 1.1 0.2 107040 90972 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 mailman 1411690 1.0 0.2 98592 82784 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=retry:0:1 mailman 1411691 41.3 0.5 187380 171012 ? S 10:45 0:50 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=task:0:1 mailman 1411692 1.0 0.2 98472 83028 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=virgin:0:1 mailman 1411693 1.0 0.2 98448 82608 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=digest:0:1 mailman 1411705 0.0 0.2 107040 75968 ? S 10:45 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 mailman 1411709 0.0 0.2 107040 75968 ? S 10:45 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 mailman 1413629 0.0 0.0 17644 8812 ? Rs 10:47 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/mailman start
Best Regards, Helio Loureiro http://helio.loureiro.eng.br https://github.com/helioloureiro
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to jered@convivian.com
On 12/14/23 10:19, Jered Floyd wrote:
Did you run the upgrade steps?
https://docs.mailman3.org/en/latest/upgrade-guide.html
For schema upgrades you will at least need to do "mailman-web migrate", but please read the entire section as 3.1.1 to 3.3.9 is a big step.
These are all important and necessary steps, but "mailman-web migrate" updates schemas for the mailman-web database. It does not update the alembic migrations for the mailman database.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi,
I forgot to mention that, but there is no mailman-web on ubuntu packages on 18.04. So I can't find a way to run it.
Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro
On Thu, 14 Dec 2023 at 18:50, Mark Sapiro <mark@msapiro.net> wrote:
On 12/14/23 10:19, Jered Floyd wrote:
Did you run the upgrade steps?
https://docs.mailman3.org/en/latest/upgrade-guide.html
For schema upgrades you will at least need to do "mailman-web migrate",
but please read the entire section as 3.1.1 to 3.3.9 is a big step.
These are all important and necessary steps, but "mailman-web migrate" updates schemas for the mailman-web database. It does not update the alembic migrations for the mailman database.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to helio@loureiro.eng.br
On 12/15/23 05:45, Helio Loureiro wrote:
Hi,
I forgot to mention that, but there is no mailman-web on ubuntu packages on 18.04. So I can't find a way to run it.
It is the normal Django admin command. It may be called django-admin or manage.py. The Debian/Ubuntu package docs should say how to run it.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 12/14/23 01:48, Helio Loureiro wrote:
But it is failing to start displaying the domain name from the previous machine.
(venv) mailman@new-server ~ (v3.3.9)> mailman start Starting Mailman's master runner Generating MTA alias maps Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute result = self._query(query) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query conn.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result result.read() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read first_packet = self.connection._read_packet() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet packet.raise_for_error() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error err.raise_mysql_exception(self._data) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception raise errorclass(errno, errval) pymysql.err.OperationalError: (1054, "Unknown column 'domain.alias_domain' in 'field list'")
The issue is your domains table and possibly others are not up to the
current schema. This should be accomplished by Mailman core running all
the necessary migrations in mailman/database/alembic/versions/. For some
reason, alembic's MigrationContext thinks the migrations have been run,
but they haven't. In the mailman database, the alembic_version table
has the version_num of the putative current version. The migration that
added alias_domain is dfe82cf73702_add_alias_domain.py. If there is no
alias_domain column in the domain table, alembic_version.version_num
should be a version older than 'dfe82cf73702'.
I suggest querying the prior mailman database for the alembic_version.version_num value and setting that in the current database, assuming that that migration exists in mailman/database/alembic/versions/.
This may be sufficient, but there may be other inconsistencies depending on exactly what you did.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi, I looked to the version I in the old installation:
mysql> select * from alembic_version; +--------------+ | version_num | +--------------+ | 4bd95c99b2e7 | +--------------+ 1 row in set (0.00 sec)
I set the same version on the new. Actually I loaded a fresh dump from DB to be sure it was the same.
And now it is starting without errors. I guess that was it.
Thanks a lot!
Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro
On Thu, 14 Dec 2023 at 18:47, Mark Sapiro <mark@msapiro.net> wrote:
On 12/14/23 01:48, Helio Loureiro wrote:
But it is failing to start displaying the domain name from the previous machine.
(venv) mailman@new-server ~ (v3.3.9)> mailman start Starting Mailman's master runner Generating MTA alias maps Traceback (most recent call last): File
"/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py",
line 1969, in _exec_single_context self.dialect.do_execute( File
line 922, in do_execute cursor.execute(statement, parameters) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py",
153, in execute result = self._query(query) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py",
"/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line line
322, in _query conn.query(q) File
"/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py",
line 558, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File
"/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py",
line 822, in _read_query_result result.read() File
"/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py",
line 1200, in read first_packet = self.connection._read_packet() File
"/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py",
line 772, in _read_packet packet.raise_for_error() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error err.raise_mysql_exception(self._data) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception raise errorclass(errno, errval) pymysql.err.OperationalError: (1054, "Unknown column 'domain.alias_domain' in 'field list'")
The issue is your domains table and possibly others are not up to the current schema. This should be accomplished by Mailman core running all the necessary migrations in mailman/database/alembic/versions/. For some reason, alembic's MigrationContext thinks the migrations have been run, but they haven't. In the
mailmandatabase, the alembic_version table has the version_num of the putative current version. The migration that added alias_domain is dfe82cf73702_add_alias_domain.py. If there is no alias_domain column in the domain table, alembic_version.version_num should be a version older than 'dfe82cf73702'.I suggest querying the prior mailman database for the alembic_version.version_num value and setting that in the current database, assuming that that migration exists in mailman/database/alembic/versions/.
This may be sufficient, but there may be other inconsistencies depending on exactly what you did.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to helio@loureiro.eng.br
Hi,
Well... not that good. Now it is mailman3-web that doesn't work.
mailman@new-machine ~ (v3.3.9)> source venv/bin/activate.fish (venv) mailman@new-machine ~ (v3.3.9)> mailman-web
Type 'mailman-web help <subcommand>' for help on a specific subcommand.
Available subcommands:
[django] check compilemessages createcachetable dbshell diffsettings dumpdata flush inspectdb loaddata makemessages makemigrations migrate optimizemigration runserver sendtestemail shell showmigrations sqlflush sqlmigrate sqlsequencereset squashmigrations startapp startproject test testserver (venv) mailman@new-machine ~ (v3.3.9)> mailman-web check System check identified no issues (0 silenced). (venv) mailman@new-machine ~ (v3.3.9)> mailman-web check migrate Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/django/apps/registry.py", line 158, in get_app_config return self.app_configs[app_label] KeyError: 'migrate'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/local/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py", line 90, in main execute_from_command_line(sys.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line utility.execute() File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 402, in run_from_argv self.execute(*args, **cmd_options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 448, in execute output = self.handle(*args, **options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/check.py", line 56, in handle app_configs = [apps.get_app_config(app_label) for app_label in app_labels] File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/check.py", line 56, in <listcomp> app_configs = [apps.get_app_config(app_label) for app_label in app_labels] File "/local/mailman/venv/lib/python3.10/site-packages/django/apps/registry.py", line 165, in get_app_config raise LookupError(message) LookupError: No installed app with label 'migrate'.
Any idea why migrate isn't working and saying it is installed?
Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro
On Sat, 16 Dec 2023 at 07:51, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Helio Loureiro writes:
And now it is starting without errors. I guess that was it.
Thank you for the success report! We love to hear it.
Please do continue to watch for strange behavior and feel free to report here if you have futher problems.
Steve
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to helio@loureiro.eng.br
Helio Loureiro writes:
Well... not that good. Now it is mailman3-web that doesn't work.
(venv) mailman@new-machine ~ (v3.3.9)> mailman-web check System check identified no issues (0 silenced). (venv) mailman@new-machine ~ (v3.3.9)> mailman-web check migrate Traceback (most recent call last): [omitted] LookupError: No installed app with label 'migrate'.
Any idea why migrate isn't working and saying it is installed?
Management commands aren't applications, so they wouldn't be valid arguments to "mailman-web check" in any case. Try "mailman-web check postorius" or "mailman-web check hyperkitty" to see how the application argument works. You should get the report "System check identified no issues (0 silenced)." as above.
Then try "mailman-web migrate -h". If that returns help output, the migrate command is installed and can be run (the help string is part of the migrate module, not part of mailman-web itself).
Why do you think "mailman-web migrate" isn't working? Most likely "migrate" is working as designed, but it is doing nothing because something about the way you transported the database originally caused mailman-web to think its database schema is up-to-date even though it is not (as happened with the mailman database).
Unfortunately this database is managed by Django, not by Alembic, and I don't know offhand how to discover and tell Django the actual schema version and then ask it to run any new migrations.
Steve
Hi,
Not a very good response.
(venv) mailman@new-server ~ (v3.3.9) [0|1]> mailman-web check postorius Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/django/apps/registry.py", line 158, in get_app_config return self.app_configs[app_label] KeyError: 'postorius'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/local/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py", line 90, in main execute_from_command_line(sys.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line utility.execute() File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 402, in run_from_argv self.execute(*args, **cmd_options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 448, in execute output = self.handle(*args, **options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/check.py", line 56, in handle app_configs = [apps.get_app_config(app_label) for app_label in app_labels] File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/check.py", line 56, in <listcomp> app_configs = [apps.get_app_config(app_label) for app_label in app_labels] File "/local/mailman/venv/lib/python3.10/site-packages/django/apps/registry.py", line 165, in get_app_config raise LookupError(message) LookupError: No installed app with label 'postorius'. (venv) mailman@new-server ~ (v3.3.9) [0|1]> mailman-web check hyperkitty Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/django/apps/registry.py", line 158, in get_app_config return self.app_configs[app_label] KeyError: 'hyperkitty'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/local/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py", line 90, in main execute_from_command_line(sys.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line utility.execute() File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 402, in run_from_argv self.execute(*args, **cmd_options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 448, in execute output = self.handle(*args, **options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/check.py", line 56, in handle app_configs = [apps.get_app_config(app_label) for app_label in app_labels] File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/check.py", line 56, in <listcomp> app_configs = [apps.get_app_config(app_label) for app_label in app_labels] File "/local/mailman/venv/lib/python3.10/site-packages/django/apps/registry.py", line 165, in get_app_config raise LookupError(message) LookupError: No installed app with label 'hyperkitty'. (venv) mailman@new-server ~ (v3.3.9) [0|1]> mailman-web migrate -h usage: mailman-web migrate [-h] [--noinput] [--database DATABASE] [--fake] [--fake-initial] [--plan] [--run-syncdb] [--check] [--prune] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] [--skip-checks] [app_label] [migration_name]
Updates database schema. Manages both apps with migrations and those without.
positional arguments: app_label App label of an application to synchronize the state. migration_name Database state will be brought to the state after that migration. Use the name "zero" to unapply all migrations.
options: -h, --help show this help message and exit --noinput, --no-input Tells Django to NOT prompt the user for input of any kind. --database DATABASE Nominates a database to synchronize. Defaults to the "default" database. --fake Mark migrations as run without actually running them. --fake-initial Detect if tables already exist and fake-apply initial migrations if so. Make sure that the current database schema matches your initial migration before using this flag. Django will only check for an existing table name. --plan Shows a list of the migration actions that will be performed. --run-syncdb Creates tables for apps without migrations. --check Exits with a non-zero status if unapplied migrations exist. --prune Delete nonexistent migrations from the django_migrations table. --version Show program's version number and exit. -v {0,1,2,3}, --verbosity {0,1,2,3} Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. --pythonpath PYTHONPATH A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". --traceback Raise on CommandError exceptions. --no-color Don't colorize the command output. --force-color Force colorization of the command output. --skip-checks Skip system checks.
Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro
On Sun, 17 Dec 2023 at 04:15, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Helio Loureiro writes:
Well... not that good. Now it is mailman3-web that doesn't work.
(venv) mailman@new-machine ~ (v3.3.9)> mailman-web check System check identified no issues (0 silenced). (venv) mailman@new-machine ~ (v3.3.9)> mailman-web check migrate Traceback (most recent call last): [omitted] LookupError: No installed app with label 'migrate'.
Any idea why migrate isn't working and saying it is installed?
Management commands aren't applications, so they wouldn't be valid arguments to "mailman-web check" in any case. Try "mailman-web check postorius" or "mailman-web check hyperkitty" to see how the application argument works. You should get the report "System check identified no issues (0 silenced)." as above.
Then try "mailman-web migrate -h". If that returns help output, the migrate command is installed and can be run (the help string is part of the migrate module, not part of mailman-web itself).
Why do you think "mailman-web migrate" isn't working? Most likely "migrate" is working as designed, but it is doing nothing because something about the way you transported the database originally caused mailman-web to think its database schema is up-to-date even though it is not (as happened with the mailman database).
Unfortunately this database is managed by Django, not by Alembic, and I don't know offhand how to discover and tell Django the actual schema version and then ask it to run any new migrations.
Steve
On 12/16/23 10:12 AM, Helio Loureiro wrote:
Hi,
Well... not that good. Now it is mailman3-web that doesn't work.
mailman@new-machine ~ (v3.3.9)> source venv/bin/activate.fish (venv) mailman@new-machine ~ (v3.3.9)> mailman-web
Type 'mailman-web help <subcommand>' for help on a specific subcommand.
Available subcommands:
[django] check compilemessages createcachetable dbshell diffsettings dumpdata flush inspectdb loaddata makemessages makemigrations migrate optimizemigration runserver sendtestemail shell showmigrations sqlflush sqlmigrate sqlsequencereset squashmigrations startapp startproject test testserver
This response indicates that mailman-web is not getting settings.py.
Have you configured /etc/mailman3/settings.py? See
<https://docs.mailman3.org/en/latest/install/virtualenv.html#initial-configuration>.
(venv) mailman@new-machine ~ (v3.3.9)> mailman-web check System check identified no issues (0 silenced). (venv) mailman@new-machine ~ (v3.3.9)> mailman-web check migrate
You should run mailman-web migrate not mailman-web check migrate
plus others. See
<https://docs.mailman3.org/en/latest/install/virtualenv.html#run-database-migrations>
et.seq. but first you need the appropriate settings so mailman-web
returns something like
$ mailman-web
Type 'mailman-web help <subcommand>' for help on a specific subcommand.
Available subcommands:
[account]
account_unsetmultipleprimaryemails
[auth]
changepassword
createsuperuser
[compressor]
compress
mtime_cache
[contenttypes]
remove_stale_contenttypes
[django]
check
compilemessages
createcachetable
dbshell
diffsettings
dumpdata
flush
inspectdb
loaddata
makemessages
makemigrations
migrate
optimizemigration
sendtestemail
shell
showmigrations
sqlflush
sqlmigrate
sqlsequencereset
squashmigrations
startapp
startproject
test
testserver
[django_extensions]
admin_generator
clean_pyc
clear_cache
compile_pyc
create_command
create_jobs
create_template_tags
delete_squashed_migrations
describe_form
drop_test_database
dumpscript
export_emails
find_template
generate_password
generate_secret_key
graph_models
list_model_info
list_signals
mail_debug
managestate
merge_model_instances
notes
pipchecker
print_settings
print_user_for_session
raise_test_exception
reset_db
reset_schema
runjob
runjobs
runprofileserver
runscript
runserver_plus
set_default_site
set_fake_emails
set_fake_passwords
shell_plus
show_template_tags
show_urls
sqlcreate
sqldiff
sqldsn
sync_s3
syncdata
unreferenced_files
update_permissions
validate_templates
[django_q]
qcluster
qinfo
qmemory
qmonitor
[haystack]
build_solr_schema
clear_index
haystack_info
rebuild_index
update_index
[hyperkitty]
attachments_to_file
hyperkitty_import
hyperkitty_warm_up_cache
mailman_sync
update_index_one_list
[postorius]
mmclient
reset_passwords
[rest_framework]
generateschema
[sessions]
clearsessions
[staticfiles]
collectstatic
findstatic
runserver
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi,
Indeed it was the configuration. It was placed into /etc/mailman3/mailman-web.py. After a I changed to /etc/mailman3/settings.py a few things advanced a little bit more.
I had to figure out how to fix mysqlclient installation since there isn't a mention about it and the simple "pip install mysqclient" was breaking with pkg-config issues. But it did work at the end.
Now I can see further messages on mailman3-web than before.
(venv) mailman@new-server ~ (v3.3.9)> mailman-web migrate System check identified some issues:
WARNINGS: account.EmailAddress: (models.W036) MariaDB does not support unique constraints with conditions. HINT: A constraint won't be created. Silence this warning if you don't care about it. account.EmailAddress: (models.W043) MariaDB does not support indexes on expressions. HINT: An index won't be created. Silence this warning if you don't care about it. Operations to perform: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount Running migrations: Applying account.0004_alter_emailaddress_drop_unique_email...Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 75, in execute return self.cursor.execute(query, args) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 179, in execute res = self._query(mogrified_query) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 330, in _query db.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 257, in query _mysql.connection.query(self, query) MySQLdb.OperationalError: (2013, 'Lost connection to MySQL server during query')
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/local/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py", line 90, in main execute_from_command_line(sys.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line utility.execute() File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 402, in run_from_argv self.execute(*args, **cmd_options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 448, in execute output = self.handle(*args, **options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 96, in wrapped res = handle_func(*args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 349, in handle post_migrate_state = executor.migrate( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 135, in migrate state = self._migrate_all_forwards( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards state = self.apply_migration( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 252, in apply_migration state = migration.apply(state, schema_editor) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/migration.py", line 130, in apply operation.database_forwards( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/operations/fields.py", line 235, in database_forwards schema_editor.alter_field(from_model, from_field, to_field) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 788, in alter_field self._alter_field( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 858, in _alter_field self.execute(self._delete_unique_sql(model, constraint_name)) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 199, in execute cursor.execute(sql, params) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers return executor(sql, params, many, context) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute with self.db.wrap_database_errors: File "/local/mailman/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 75, in execute return self.cursor.execute(query, args) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 179, in execute res = self._query(mogrified_query) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 330, in _query db.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 257, in query _mysql.connection.query(self, query) django.db.utils.OperationalError: (2013, 'Lost connection to MySQL server during query')
Which seems to be complaining with DB credentials. But they're properly set and it works via mysql command line.
(venv) mailman@new-server ~ (v3.3.9)> mysql -umailman3web -p mailman3web Enter password: Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A
Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 32 Server version: 10.6.12-MariaDB-0ubuntu0.22.04.1 Ubuntu 22.04
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [mailman3web]> show tables; +-------------------------------+ | Tables_in_mailman3web | +-------------------------------+ | account_emailaddress | | account_emailconfirmation | | auth_group | | auth_group_permissions | | auth_permission | | auth_user | | auth_user_groups | | auth_user_user_permissions | | django_admin_log | | django_content_type | | django_mailman3_maildomain | | django_mailman3_profile | | django_migrations | | django_q_ormq | | django_q_schedule | | django_q_task | | django_session | | django_site | | hyperkitty_attachment | | hyperkitty_email | | hyperkitty_favorite | | hyperkitty_lastview | | hyperkitty_mailinglist | | hyperkitty_profile | | hyperkitty_sender | | hyperkitty_tag | | hyperkitty_tagging | | hyperkitty_thread | | hyperkitty_threadcategory | | hyperkitty_vote | | socialaccount_socialaccount | | socialaccount_socialapp | | socialaccount_socialapp_sites | | socialaccount_socialtoken | +-------------------------------+ 34 rows in set (0.000 sec)
On the /etc/mailman3/settings.py:
(venv) mailman@seliius26224 ~ (v3.3.9)> cat /etc/mailman3/settings.py | grep -v "#" | grep -v '^$' from mailman_web.settings.base import * from mailman_web.settings.mailman import * ADMINS = ( ('Mailman Suite Admin', 'root@localhost'), ) DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': '*******************', 'HOST': 'localhost', 'PORT': '3306', } } STATIC_ROOT = '/local/mailman/web/static' COMPRESS_ENABLED = True LOGGING['handlers']['file']['filename'] = '/local/mailman/web/logs/mailmanweb.log' ALLOWED_HOSTS = [ "127.0.0.1", "*" ] CSRF_TRUSTED_ORIGINS = [ ] SITE_ID = 1 SECRET_KEY = '****************************************' DEFAULT_FROM_EMAIL = 'admin@mailman.domain.com' SERVER_EMAIL = 'admin@mailman.domain.com' MAILMAN_REST_API_URL = 'http://localhost:8001' MAILMAN_REST_API_USER = 'restadmin' MAILMAN_REST_API_PASS = '***************************' MAILMAN_ARCHIVER_KEY = '****************************' MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1', '1.2.3.4') HAYSTACK_CONNECTIONS = { 'default': { 'PATH': "/opt/mailman/web/xapian_index", 'ENGINE': 'xapian_backend.XapianEngine' }, }
And /local/mailman/web/logs/mailmanweb.log is empty.
Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro
On Sun, 17 Dec 2023 at 16:35, Mark Sapiro <mark@msapiro.net> wrote:
On 12/16/23 10:12 AM, Helio Loureiro wrote:
Hi,
Well... not that good. Now it is mailman3-web that doesn't work.
mailman@new-machine ~ (v3.3.9)> source venv/bin/activate.fish (venv) mailman@new-machine ~ (v3.3.9)> mailman-web
Type 'mailman-web help <subcommand>' for help on a specific subcommand.
Available subcommands:
[django] check compilemessages createcachetable dbshell diffsettings dumpdata flush inspectdb loaddata makemessages makemigrations migrate optimizemigration runserver sendtestemail shell showmigrations sqlflush sqlmigrate sqlsequencereset squashmigrations startapp startproject test testserver
This response indicates that
mailman-webis not getting settings.py. Have you configured /etc/mailman3/settings.py? See < https://docs.mailman3.org/en/latest/install/virtualenv.html#initial-configur....
(venv) mailman@new-machine ~ (v3.3.9)> mailman-web check System check identified no issues (0 silenced). (venv) mailman@new-machine ~ (v3.3.9)> mailman-web check migrate
You should run
mailman-web migratenotmailman-web check migrateplus others. See < https://docs.mailman3.org/en/latest/install/virtualenv.html#run-database-migrations>et.seq. but first you need the appropriate settings so
mailman-webreturns something like$ mailman-web Type 'mailman-web help <subcommand>' for help on a specific subcommand. Available subcommands: [account] account_unsetmultipleprimaryemails [auth] changepassword createsuperuser [compressor] compress mtime_cache [contenttypes] remove_stale_contenttypes [django] check compilemessages createcachetable dbshell diffsettings dumpdata flush inspectdb loaddata makemessages makemigrations migrate optimizemigration sendtestemail shell showmigrations sqlflush sqlmigrate sqlsequencereset squashmigrations startapp startproject test testserver [django_extensions] admin_generator clean_pyc clear_cache compile_pyc create_command create_jobs create_template_tags delete_squashed_migrations describe_form drop_test_database dumpscript export_emails find_template generate_password generate_secret_key graph_models list_model_info list_signals mail_debug managestate merge_model_instances notes pipchecker print_settings print_user_for_session raise_test_exception reset_db reset_schema runjob runjobs runprofileserver runscript runserver_plus set_default_site set_fake_emails set_fake_passwords shell_plus show_template_tags show_urls sqlcreate sqldiff sqldsn sync_s3 syncdata unreferenced_files update_permissions validate_templates [django_q] qcluster qinfo qmemory qmonitor [haystack] build_solr_schema clear_index haystack_info rebuild_index update_index [hyperkitty] attachments_to_file hyperkitty_import hyperkitty_warm_up_cache mailman_sync update_index_one_list [postorius] mmclient reset_passwords [rest_framework] generateschema [sessions] clearsessions [staticfiles] collectstatic findstatic runserver-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to helio@loureiro.eng.br
On 12/18/23 6:24 AM, Helio Loureiro wrote:
Hi,
Indeed it was the configuration. It was placed into /etc/mailman3/mailman-web.py. After a I changed to /etc/mailman3/settings.py a few things advanced a little bit more.
I had to figure out how to fix mysqlclient installation since there isn't a mention about it and the simple "pip install mysqclient" was breaking with pkg-config issues. But it did work at the end.
Now I can see further messages on mailman3-web than before.
(venv) mailman@new-server ~ (v3.3.9)> mailman-web migrate System check identified some issues:
WARNINGS: account.EmailAddress: (models.W036) MariaDB does not support unique constraints with conditions. HINT: A constraint won't be created. Silence this warning if you don't care about it. account.EmailAddress: (models.W043) MariaDB does not support indexes on expressions. HINT: An index won't be created. Silence this warning if you don't care about it. Operations to perform: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount Running migrations: Applying account.0004_alter_emailaddress_drop_unique_email...Traceback (most recent call last):
I'm not sure why there would be an issue with this migration, but there is a possible compatibility issue depending on how you installed things.
django-mailman3<=1.3.11 is not compatible with django-allauth>=0.58.
In your venv, try
pip install django-allauth\<0.58
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi,
No luck :(
(venv) mailman@new-server ~ (v3.3.9)> *pip freeze | egrep "mailman-web|django-mailman3|django-allauth"* django-allauth==0.59.0 django-mailman3==1.3.11 mailman-web==0.0.8 (venv) mailman@new-server ~ (v3.3.9)> *pip install -U django-allauth==0.58.0* Collecting django-allauth==0.58.0 Downloading django-allauth-0.58.0.tar.gz (861 kB) ---------------------------------------- 861.7/861.7 KB 9.4 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: requests-oauthlib>=0.3.0 in ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (1.3.1) Requirement already satisfied: Django>=3.2 in ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (4.1.13) Requirement already satisfied: pyjwt[crypto]>=1.7 in ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (2.8.0) Requirement already satisfied: requests>=2.0.0 in ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (2.31.0) Requirement already satisfied: python3-openid>=3.0.8 in ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (3.2.0) Requirement already satisfied: asgiref<4,>=3.5.2 in ./venv/lib/python3.10/site-packages (from Django>=3.2->django-allauth==0.58.0) (3.7.2) Requirement already satisfied: sqlparse>=0.2.2 in ./venv/lib/python3.10/site-packages (from Django>=3.2->django-allauth==0.58.0) (0.4.4) Requirement already satisfied: cryptography>=3.4.0 in ./venv/lib/python3.10/site-packages (from pyjwt[crypto]>=1.7->django-allauth==0.58.0) (41.0.7) Requirement already satisfied: defusedxml in ./venv/lib/python3.10/site-packages (from python3-openid>=3.0.8->django-allauth==0.58.0) (0.7.1) Requirement already satisfied: urllib3<3,>=1.21.1 in ./venv/lib/python3.10/site-packages (from requests>=2.0.0->django-allauth==0.58.0) (2.1.0) Requirement already satisfied: charset-normalizer<4,>=2 in ./venv/lib/python3.10/site-packages (from requests>=2.0.0->django-allauth==0.58.0) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in ./venv/lib/python3.10/site-packages (from requests>=2.0.0->django-allauth==0.58.0) (3.6) Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.10/site-packages (from requests>=2.0.0->django-allauth==0.58.0) (2023.11.17) Requirement already satisfied: oauthlib>=3.0.0 in ./venv/lib/python3.10/site-packages (from requests-oauthlib>=0.3.0->django-allauth==0.58.0) (3.2.2) Requirement already satisfied: typing-extensions>=4 in ./venv/lib/python3.10/site-packages (from asgiref<4,>=3.5.2->Django>=3.2->django-allauth==0.58.0) (4.9.0) Requirement already satisfied: cffi>=1.12 in ./venv/lib/python3.10/site-packages (from cryptography>=3.4.0->pyjwt[crypto]>=1.7->django-allauth==0.58.0) (1.16.0) Requirement already satisfied: pycparser in ./venv/lib/python3.10/site-packages (from cffi>=1.12->cryptography>=3.4.0->pyjwt[crypto]>=1.7->django-allauth==0.58.0) (2.21) Building wheels for collected packages: django-allauth Building wheel for django-allauth (pyproject.toml) ... done Created wheel for django-allauth: filename=django_allauth-0.58.0-py3-none-any.whl size=1157319 sha256=a430c552101d1ad47bc00b16d1c1d6df728afacdd13823927b4cbfb02c35dbfc Stored in directory: /local/mailman/.cache-ubuntu-22.04/pip/wheels/55/0a/79/e199827a18f310906c2a90b0e92b89c41daf21d2a502db6710 Successfully built django-allauth Installing collected packages: django-allauth Attempting uninstall: django-allauth Found existing installation: django-allauth 0.59.0 Uninstalling django-allauth-0.59.0: Successfully uninstalled django-allauth-0.59.0 Successfully installed django-allauth-0.58.0 (venv) mailman@new-server ~ (v3.3.9)> *mailman-web migrate* System check identified some issues:
WARNINGS: account.EmailAddress: (models.W036) MariaDB does not support unique constraints with conditions. HINT: A constraint won't be created. Silence this warning if you don't care about it. account.EmailAddress: (models.W043) MariaDB does not support indexes on expressions. HINT: An index won't be created. Silence this warning if you don't care about it. Operations to perform: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount Running migrations: Applying account.0004_alter_emailaddress_drop_unique_email...Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 75, in execute return self.cursor.execute(query, args) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 179, in execute res = self._query(mogrified_query) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 330, in _query db.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 257, in query _mysql.connection.query(self, query) MySQLdb.OperationalError: (2013, 'Lost connection to MySQL server during query')
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/local/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py", line 90, in main execute_from_command_line(sys.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line utility.execute() File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 402, in run_from_argv self.execute(*args, **cmd_options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 448, in execute output = self.handle(*args, **options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 96, in wrapped res = handle_func(*args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 349, in handle post_migrate_state = executor.migrate( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 135, in migrate state = self._migrate_all_forwards( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards state = self.apply_migration( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 252, in apply_migration state = migration.apply(state, schema_editor) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/migration.py", line 130, in apply operation.database_forwards( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/operations/fields.py", line 235, in database_forwards schema_editor.alter_field(from_model, from_field, to_field) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 788, in alter_field self._alter_field( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 858, in _alter_field self.execute(self._delete_unique_sql(model, constraint_name)) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 199, in execute cursor.execute(sql, params) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers return executor(sql, params, many, context) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute with self.db.wrap_database_errors: File "/local/mailman/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 75, in execute return self.cursor.execute(query, args) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 179, in execute res = self._query(mogrified_query) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 330, in _query db.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 257, in query _mysql.connection.query(self, query) django.db.utils.OperationalError: (2013, 'Lost connection to MySQL server during query') (venv) mailman@new-server ~ (v3.3.9) [0|1]> *more /etc/mailman3/settings.py* # Mailman Web configuration file. # /etc/mailman3/settings.py
# Get the default settings. from mailman_web.settings.base import * from mailman_web.settings.mailman import *
# Settings below supplement or override the defaults.
#: Default list of admins who receive the emails from error logging. ADMINS = ( ('Mailman Suite Admin', 'root@localhost'), )
# Postgresql database setup. DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', # TODO: Replace this with the password. 'PASSWORD': '***********', 'HOST': 'localhost', # PORT: set to empty string for default. 'PORT': '3306', # OPTIONS: Extra parameters to use when connecting to the database. #'OPTIONS': { # Set sql_mode to 'STRICT_TRANS_TABLES' for MySQL. See # https://docs.djangoproject.com/en/1.11/ref/ # databases/#setting-sql-mode # 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", # 'charset': 'utf8mb4', #},
}}
# 'collectstatic' command will copy all the static files here.
# Alias this location from your webserver to /static
STATIC_ROOT = '/local/mailman/web/static'
# enable the 'compress' command. COMPRESS_ENABLED = True
# Make sure that this directory is created or Django will fail on start. LOGGING['handlers']['file']['filename'] = '/local/mailman/web/logs/mailmanweb.log'
#: See https://docs.djangoproject.com/en/dev/ref/settings/#allowed-hosts ALLOWED_HOSTS = [ "localhost", # Archiving API from Mailman, keep it. "127.0.0.1", # "lists.your-domain.org", # Add here all production domains you have. "*" ]
#: See https://docs.djangoproject.com/en/dev/ref/settings/#csrf-trusted-origins (venv) mailman@new-server ~ (v3.3.9)> *mysql -umailman3web -p -h localhost mailman3web* Enter password: Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A
Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 32 Server version: 10.6.12-MariaDB-0ubuntu0.22.04.1 Ubuntu 22.04
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [mailman3web]> show tables; +-------------------------------+ | Tables_in_mailman3web | +-------------------------------+ | account_emailaddress | | account_emailconfirmation | | auth_group | | auth_group_permissions | | auth_permission | | auth_user | | auth_user_groups | | auth_user_user_permissions | | django_admin_log | | django_content_type | | django_mailman3_maildomain | | django_mailman3_profile | | django_migrations | | django_q_ormq | | django_q_schedule | | django_q_task | | django_session | | django_site | | hyperkitty_attachment | | hyperkitty_email | | hyperkitty_favorite | | hyperkitty_lastview | | hyperkitty_mailinglist | | hyperkitty_profile | | hyperkitty_sender | | hyperkitty_tag | | hyperkitty_tagging | | hyperkitty_thread | | hyperkitty_threadcategory | | hyperkitty_vote | | socialaccount_socialaccount | | socialaccount_socialapp | | socialaccount_socialapp_sites | | socialaccount_socialtoken | +-------------------------------+ 34 rows in set (0.000 sec)
Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro
On Mon, 18 Dec 2023 at 17:11, Mark Sapiro <mark@msapiro.net> wrote:
On 12/18/23 6:24 AM, Helio Loureiro wrote:
Hi,
Indeed it was the configuration. It was placed into /etc/mailman3/mailman-web.py. After a I changed to /etc/mailman3/settings.py a few things advanced a little bit more.
I had to figure out how to fix mysqlclient installation since there isn't a mention about it and the simple "pip install mysqclient" was breaking with pkg-config issues. But it did work at the end.
Now I can see further messages on mailman3-web than before.
(venv) mailman@new-server ~ (v3.3.9)> mailman-web migrate System check identified some issues:
WARNINGS: account.EmailAddress: (models.W036) MariaDB does not support unique constraints with conditions. HINT: A constraint won't be created. Silence this warning if you don't care about it. account.EmailAddress: (models.W043) MariaDB does not support indexes on expressions. HINT: An index won't be created. Silence this warning if you don't care about it. Operations to perform: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount Running migrations: Applying account.0004_alter_emailaddress_drop_unique_email...Traceback (most recent call last):
I'm not sure why there would be an issue with this migration, but there is a possible compatibility issue depending on how you installed things.
django-mailman3<=1.3.11 is not compatible with django-allauth>=0.58.
In your venv, try
pip install django-allauth\<0.58-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to helio@loureiro.eng.br
And using the module manually works...
(venv) mailman@new-server ~ (v3.3.9) [0|127]> *ipython3* /usr/lib/python3/dist-packages/IPython/core/interactiveshell.py:949: UserWarning: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv. warn( Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] Type 'copyright', 'credits' or 'license' for more information IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: *import MySQLdb*
(venv) mailman@new-sever ~ (v3.3.9)> *fg* Send job 1, 'ipython3' to foreground In [3]: *con = MySQLdb.connect("localhost", "mailman3web", "*****************", "mailman3web")*
In [4]: *cursor = con.cursor()* In [5]: *cursor.execute("show tables")* Out[5]: 34
In [6]: *print(cursor.fetchall())* (('account_emailaddress',), ('account_emailconfirmation',), ('auth_group',), ('auth_group_permissions',), ('auth_permission',), ('auth_user',), ('auth_user_groups',), ('auth_user_user_permissions',), ('django_admin_log',), ('django_content_type',), ('django_mailman3_maildomain',), ('django_mailman3_profile',), ('django_migrations',), ('django_q_ormq',), ('django_q_schedule',), ('django_q_task',), ('django_session',), ('django_site',), ('hyperkitty_attachment',), ('hyperkitty_email',), ('hyperkitty_favorite',), ('hyperkitty_lastview',), ('hyperkitty_mailinglist',), ('hyperkitty_profile',), ('hyperkitty_sender',), ('hyperkitty_tag',), ('hyperkitty_tagging',), ('hyperkitty_thread',), ('hyperkitty_threadcategory',), ('hyperkitty_vote',), ('socialaccount_socialaccount',), ('socialaccount_socialapp',), ('socialaccount_socialapp_sites',), ('socialaccount_socialtoken',))
In [7]: *cursor.execute("select * from account_emailaddress")* Out[7]: 2530
In [8]: *print(cursor.fetchall())*
So what can I do? Is there a way to troubleshoot the root cause of this issue?
Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro
On Tue, 19 Dec 2023 at 11:11, Helio Loureiro <helio@loureiro.eng.br> wrote:
Hi,
No luck :(
(venv) mailman@new-server ~ (v3.3.9)> *pip freeze | egrep "mailman-web|django-mailman3|django-allauth"* django-allauth==0.59.0 django-mailman3==1.3.11 mailman-web==0.0.8 (venv) mailman@new-server ~ (v3.3.9)> *pip install -U django-allauth==0.58.0* Collecting django-allauth==0.58.0 Downloading django-allauth-0.58.0.tar.gz (861 kB) ---------------------------------------- 861.7/861.7 KB 9.4 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: requests-oauthlib>=0.3.0 in ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (1.3.1) Requirement already satisfied: Django>=3.2 in ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (4.1.13) Requirement already satisfied: pyjwt[crypto]>=1.7 in ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (2.8.0) Requirement already satisfied: requests>=2.0.0 in ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (2.31.0) Requirement already satisfied: python3-openid>=3.0.8 in ./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (3.2.0) Requirement already satisfied: asgiref<4,>=3.5.2 in ./venv/lib/python3.10/site-packages (from Django>=3.2->django-allauth==0.58.0) (3.7.2) Requirement already satisfied: sqlparse>=0.2.2 in ./venv/lib/python3.10/site-packages (from Django>=3.2->django-allauth==0.58.0) (0.4.4) Requirement already satisfied: cryptography>=3.4.0 in ./venv/lib/python3.10/site-packages (from pyjwt[crypto]>=1.7->django-allauth==0.58.0) (41.0.7) Requirement already satisfied: defusedxml in ./venv/lib/python3.10/site-packages (from python3-openid>=3.0.8->django-allauth==0.58.0) (0.7.1) Requirement already satisfied: urllib3<3,>=1.21.1 in ./venv/lib/python3.10/site-packages (from requests>=2.0.0->django-allauth==0.58.0) (2.1.0) Requirement already satisfied: charset-normalizer<4,>=2 in ./venv/lib/python3.10/site-packages (from requests>=2.0.0->django-allauth==0.58.0) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in ./venv/lib/python3.10/site-packages (from requests>=2.0.0->django-allauth==0.58.0) (3.6) Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.10/site-packages (from requests>=2.0.0->django-allauth==0.58.0) (2023.11.17) Requirement already satisfied: oauthlib>=3.0.0 in ./venv/lib/python3.10/site-packages (from requests-oauthlib>=0.3.0->django-allauth==0.58.0) (3.2.2) Requirement already satisfied: typing-extensions>=4 in ./venv/lib/python3.10/site-packages (from asgiref<4,>=3.5.2->Django>=3.2->django-allauth==0.58.0) (4.9.0) Requirement already satisfied: cffi>=1.12 in ./venv/lib/python3.10/site-packages (from cryptography>=3.4.0->pyjwt[crypto]>=1.7->django-allauth==0.58.0) (1.16.0) Requirement already satisfied: pycparser in ./venv/lib/python3.10/site-packages (from cffi>=1.12->cryptography>=3.4.0->pyjwt[crypto]>=1.7->django-allauth==0.58.0) (2.21) Building wheels for collected packages: django-allauth Building wheel for django-allauth (pyproject.toml) ... done Created wheel for django-allauth: filename=django_allauth-0.58.0-py3-none-any.whl size=1157319 sha256=a430c552101d1ad47bc00b16d1c1d6df728afacdd13823927b4cbfb02c35dbfc Stored in directory: /local/mailman/.cache-ubuntu-22.04/pip/wheels/55/0a/79/e199827a18f310906c2a90b0e92b89c41daf21d2a502db6710 Successfully built django-allauth Installing collected packages: django-allauth Attempting uninstall: django-allauth Found existing installation: django-allauth 0.59.0 Uninstalling django-allauth-0.59.0: Successfully uninstalled django-allauth-0.59.0 Successfully installed django-allauth-0.58.0 (venv) mailman@new-server ~ (v3.3.9)> *mailman-web migrate* System check identified some issues:
WARNINGS: account.EmailAddress: (models.W036) MariaDB does not support unique constraints with conditions. HINT: A constraint won't be created. Silence this warning if you don't care about it. account.EmailAddress: (models.W043) MariaDB does not support indexes on expressions. HINT: An index won't be created. Silence this warning if you don't care about it. Operations to perform: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount Running migrations: Applying account.0004_alter_emailaddress_drop_unique_email...Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 75, in execute return self.cursor.execute(query, args) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 179, in execute res = self._query(mogrified_query) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 330, in _query db.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 257, in query _mysql.connection.query(self, query) MySQLdb.OperationalError: (2013, 'Lost connection to MySQL server during query')
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/local/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py", line 90, in main execute_from_command_line(sys.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line utility.execute() File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 402, in run_from_argv self.execute(*args, **cmd_options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 448, in execute output = self.handle(*args, **options) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py", line 96, in wrapped res = handle_func(*args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 349, in handle post_migrate_state = executor.migrate( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 135, in migrate state = self._migrate_all_forwards( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards state = self.apply_migration( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 252, in apply_migration state = migration.apply(state, schema_editor) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/migration.py", line 130, in apply operation.database_forwards( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/operations/fields.py", line 235, in database_forwards schema_editor.alter_field(from_model, from_field, to_field) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 788, in alter_field self._alter_field( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 858, in _alter_field self.execute(self._delete_unique_sql(model, constraint_name)) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 199, in execute cursor.execute(sql, params) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers( File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers return executor(sql, params, many, context) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute with self.db.wrap_database_errors: File "/local/mailman/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) File "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 75, in execute return self.cursor.execute(query, args) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 179, in execute res = self._query(mogrified_query) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 330, in _query db.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 257, in query _mysql.connection.query(self, query) django.db.utils.OperationalError: (2013, 'Lost connection to MySQL server during query') (venv) mailman@new-server ~ (v3.3.9) [0|1]> *more /etc/mailman3/settings.py* # Mailman Web configuration file. # /etc/mailman3/settings.py
# Get the default settings. from mailman_web.settings.base import * from mailman_web.settings.mailman import *
# Settings below supplement or override the defaults.
#: Default list of admins who receive the emails from error logging. ADMINS = ( ('Mailman Suite Admin', 'root@localhost'), )
# Postgresql database setup. DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', # TODO: Replace this with the password. 'PASSWORD': '***********', 'HOST': 'localhost', # PORT: set to empty string for default. 'PORT': '3306', # OPTIONS: Extra parameters to use when connecting to the database. #'OPTIONS': { # Set sql_mode to 'STRICT_TRANS_TABLES' for MySQL. See # https://docs.djangoproject.com/en/1.11/ref/ # databases/#setting-sql-mode # 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", # 'charset': 'utf8mb4', #},
}}
# 'collectstatic' command will copy all the static files here. # Alias this location from your webserver to
/staticSTATIC_ROOT = '/local/mailman/web/static'# enable the 'compress' command. COMPRESS_ENABLED = True
# Make sure that this directory is created or Django will fail on start. LOGGING['handlers']['file']['filename'] = '/local/mailman/web/logs/mailmanweb.log'
#: See https://docs.djangoproject.com/en/dev/ref/settings/#allowed-hosts ALLOWED_HOSTS = [ "localhost", # Archiving API from Mailman, keep it. "127.0.0.1", # "lists.your-domain.org", # Add here all production domains you have. "*" ]
#: See https://docs.djangoproject.com/en/dev/ref/settings/#csrf-trusted-origins (venv) mailman@new-server ~ (v3.3.9)> *mysql -umailman3web -p -h localhost mailman3web* Enter password: Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A
Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 32 Server version: 10.6.12-MariaDB-0ubuntu0.22.04.1 Ubuntu 22.04
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [mailman3web]> show tables; +-------------------------------+ | Tables_in_mailman3web | +-------------------------------+ | account_emailaddress | | account_emailconfirmation | | auth_group | | auth_group_permissions | | auth_permission | | auth_user | | auth_user_groups | | auth_user_user_permissions | | django_admin_log | | django_content_type | | django_mailman3_maildomain | | django_mailman3_profile | | django_migrations | | django_q_ormq | | django_q_schedule | | django_q_task | | django_session | | django_site | | hyperkitty_attachment | | hyperkitty_email | | hyperkitty_favorite | | hyperkitty_lastview | | hyperkitty_mailinglist | | hyperkitty_profile | | hyperkitty_sender | | hyperkitty_tag | | hyperkitty_tagging | | hyperkitty_thread | | hyperkitty_threadcategory | | hyperkitty_vote | | socialaccount_socialaccount | | socialaccount_socialapp | | socialaccount_socialapp_sites | | socialaccount_socialtoken | +-------------------------------+ 34 rows in set (0.000 sec)
Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro
On Mon, 18 Dec 2023 at 17:11, Mark Sapiro <mark@msapiro.net> wrote:
On 12/18/23 6:24 AM, Helio Loureiro wrote:
Hi,
Indeed it was the configuration. It was placed into /etc/mailman3/mailman-web.py. After a I changed to /etc/mailman3/settings.py a few things advanced a little bit more.
I had to figure out how to fix mysqlclient installation since there isn't a mention about it and the simple "pip install mysqclient" was breaking with pkg-config issues. But it did work at the end.
Now I can see further messages on mailman3-web than before.
(venv) mailman@new-server ~ (v3.3.9)> mailman-web migrate System check identified some issues:
WARNINGS: account.EmailAddress: (models.W036) MariaDB does not support unique constraints with conditions. HINT: A constraint won't be created. Silence this warning if you don't care about it. account.EmailAddress: (models.W043) MariaDB does not support indexes on expressions. HINT: An index won't be created. Silence this warning if you don't care about it. Operations to perform: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount Running migrations: Applying account.0004_alter_emailaddress_drop_unique_email...Traceback (most recent call last):
I'm not sure why there would be an issue with this migration, but there is a possible compatibility issue depending on how you installed things.
django-mailman3<=1.3.11 is not compatible with django-allauth>=0.58.
In your venv, try
pip install django-allauth\<0.58-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to helio@loureiro.eng.br
On 12/19/23 3:11 AM, Helio Loureiro wrote:
Hi,
No luck :(
(venv) mailman@new-server ~ (v3.3.9)> *pip install -U django-allauth==0.58.0*
You need to install django-allauth<0.58.0 as in
pip install django-allauth\<0.58
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi,
And here we have it:
(venv) mailman@new=server ~ (v3.3.9)> mailman-web migrate System check identified some issues:
WARNINGS: account.EmailAddress: (models.W036) MariaDB does not support unique constraints with conditions. HINT: A constraint won't be created. Silence this warning if you don't care about it. account.EmailAddress: (models.W043) MariaDB does not support indexes on expressions. HINT: An index won't be created. Silence this warning if you don't care about it. Operations to perform: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount Running migrations: Applying account.0004_alter_emailaddress_drop_unique_email... OK Applying account.0005_emailaddress_idx_upper_email... OK Applying admin.0003_logentry_add_action_flag_choices... OK Applying auth.0009_alter_user_last_name_max_length... OK Applying auth.0010_alter_group_name_max_length... OK Applying auth.0011_update_proxy_permissions... OK Applying auth.0012_alter_user_first_name_max_length... OK Applying django_mailman3.0003_sessions... OK Applying django_q.0010_auto_20200610_0856... OK Applying django_q.0011_auto_20200628_1055... OK Applying django_q.0012_auto_20200702_1608... OK Applying django_q.0013_task_attempt_count... OK Applying django_q.0014_schedule_cluster... OK Applying hyperkitty.0016_auto_20180309_0056... OK
tl;dr: it was the database.
So the django was spitting out absolutely horrible traceback errors to diagnose the issue. I modified the code to print the query and then raise exception (what is expected from an error, not only "I just failed, sorry do whatever").
Then I got the query and tried manually. And it failed.
(venv) mailman@new-server ~ (v3.3.9)> mailman-web migrate System check identified some issues:
WARNINGS:
account.EmailAddress: (models.W036) MariaDB does not support unique
constraints with conditions.
HINT: A constraint won't be created. Silence this warning if you don't care
about it.
account.EmailAddress: (models.W043) MariaDB does not support indexes on
expressions.
HINT: An index won't be created. Silence this warning if you don't care
about it.
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes,
django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
socialaccount
Running migrations:
Applying account.0004_alter_emailaddress_drop_unique_email...Error
running query: b'*ALTER** TABLE account_emailaddress DROP INDEX email'*
Traceback (most recent call last):
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 89, in _execute
return self.cursor.execute(sql, params)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
[...]
(venv) mailman@new-server ~ (v3.3.9) [0|1]> mysql -umailman3web -p -h localhost mailman3web Enter password: Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A
Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 32 Server version: 10.6.12-MariaDB-0ubuntu0.22.04.1 Ubuntu 22.04
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [mailman3web]> *ALTER TABLE account_emailaddress DROP INDEX
email;*
ERROR 2013 (HY000): Lost connection to server during query
MariaDB [mailman3web]> *ALTER TABLE account_emailaddress DROP INDEX
email;*
ERROR 2006 (HY000): Server has gone away
No connection. Trying to reconnect...
Connection id: 32
Current database: mailman3web
ERROR 2013 (HY000): Lost connection to server during query
So I looked to the mysql error logs and searched Internet for the issue. It recommended to run a "mysql_upgrade", which failed with another error:
root@seliius26224 /e/mailman3# mysql_upgrade -u root --force Phase 1/7: Checking and upgrading mysql database Processing databases mysql mysql.column_stats OK mysql.columns_priv OK mysql.db OK mysql.engine_cost Error : Table 'mysql.engine_cost' doesn't exist status : Operation failed mysql.event OK mysql.func OK [...] sys.x$wait_classes_global_by_avg_latency OK sys.x$wait_classes_global_by_latency OK sys.x$waits_by_host_by_latency OK sys.x$waits_by_user_by_latency OK sys.x$waits_global_by_latency OK Phase 4/7: Running 'mysql_fix_privilege_tables' ERROR 1932 (42S02) at line 477: Table 'mysql.innodb_index_stats' doesn't exist in engine FATAL ERROR: Upgrade failed
I searched again the Internet and found a fix: https://stackoverflow.com/questions/37856155/mysql-upgrade-failed-innodb-tab...
Now migration is progressing.
Thanks for all the patience and support.
Best Regards, Helio Loureiro https://helio.loureiro.eng.br https://github.com/helioloureiro https://mastodon.social/@helioloureiro
On Tue, 19 Dec 2023 at 15:23, Mark Sapiro <mark@msapiro.net> wrote:
On 12/19/23 3:11 AM, Helio Loureiro wrote:
Hi,
No luck :(
(venv) mailman@new-server ~ (v3.3.9)> *pip install -U django-allauth==0.58.0*
You need to install django-allauth<0.58.0 as in
pip install django-allauth\<0.58-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to helio@loureiro.eng.br
participants (4)
-
Helio Loureiro -
Jered Floyd -
Mark Sapiro -
Stephen J. Turnbull