I recently upgraded to mailman3 on two Debian servers and I am getting a cron error periodically sent to me as depicted below. It looks like a permission problem, however, /var/lib/mailman3/locks and /var/lib/mailman3/* all have user = "list" and group = "list" and the cron job runs as user = "list" so I don't quite understand why I am getting these errors.
Hopefully, someone might reply with what the problem is.
Thanks,
Gordon
Here is some additional information:
"ls -ld locks" output: drwxr-xr-x 2 list list 4096 Sep 28 08:51 locks
ls -l locks" output: total 12 -rw-rw---- 1 list list 80 Sep 28 00:00 'dbcreate.lck|slc.mailhub4u.com|24435|4056142050254595815' -rw-rw---- 2 list list 77 Sep 28 2020 master.lck -rw-rw---- 2 list list 77 Sep 28 2020 'master.lck|slc.mailhub4u.com|5291|3708878559945320758'
Here is the error message that is emailed to me:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 216, in lock os.link(self._claimfile, self._lockfile) FileExistsError: [Errno 17] File exists: '/var/lib/mailman3/locks/dbcreate.lck|slc.mailhub4u.com|24435|4056142050254595815' -> '/var/lib/mailman3/locks/dbcreate.lck'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/bin/mailman", line 11, in <module> load_entry_point('mailman==3.2.1', 'console_scripts', 'mailman')() File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3/dist-packages/click/core.py", line 716, in main with self.make_context(prog_name, args, **extra) as ctx: File "/usr/lib/python3/dist-packages/click/core.py", line 641, in make_context self.parse_args(ctx, args) File "/usr/lib/python3/dist-packages/click/core.py", line 1089, in parse_args rest = Command.parse_args(self, ctx, args) File "/usr/lib/python3/dist-packages/click/core.py", line 940, in parse_args value, args = param.handle_parse_result(ctx, opts, args) File "/usr/lib/python3/dist-packages/click/core.py", line 1477, in handle_parse_result self.callback, ctx, self, value) File "/usr/lib/python3/dist-packages/click/core.py", line 96, in invoke_param_callback return callback(ctx, param, value) File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 94, in initialize_config initialize(value) File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 218, in initialize initialize_2(propagate_logs=propagate_logs) File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 177, in initialize_2 config.db = getUtility(IDatabaseFactory, utility_name).create() File "/usr/lib/python3/dist-packages/mailman/database/factory.py", line 50, in create with Lock(os.path.join(config.LOCK_DIR, 'dbcreate.lck')): File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 334, in __enter__ self.lock() File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 248, in lock elif self._read() == self._claimfile: File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 442, in _read with open(self._lockfile) as fp: PermissionError: [Errno 13] Permission denied: '/var/lib/mailman3/locks/dbcreate.lck'
On 9/28/20 7:38 AM, gordon@dickens.com wrote:
I recently upgraded to mailman3 on two Debian servers and I am getting a cron error periodically sent to me as depicted below. It looks like a permission problem, however, /var/lib/mailman3/locks and /var/lib/mailman3/* all have user = "list" and group = "list" and the cron job runs as user = "list" so I don't quite understand why I am getting these errors.
ls -l locks" output: total 12 -rw-rw---- 1 list list 80 Sep 28 00:00 'dbcreate.lck|slc.mailhub4u.com|24435|4056142050254595815'
Does PID 24435 exist and if so, what is it?
This is strange because it says PID 24435 has claimed the lock, but there is no lock which would be a file named dbcreate.lck if there is no PID 24435 or if it isn't related to Mailman, just remove the 'dbcreate.lck|slc.mailhub4u.com|24435|4056142050254595815' file.
-rw-rw---- 2 list list 77 Sep 28 2020 master.lck -rw-rw---- 2 list list 77 Sep 28 2020 'master.lck|slc.mailhub4u.com|5291|3708878559945320758'
Here is the error message that is emailed to me:
What command is cron running? Although it is some mailman command and
the sub-command isn't relevant as it never gets that far, but the
-C/--config option may be.
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 216, in lock os.link(self._claimfile, self._lockfile) FileExistsError: [Errno 17] File exists: '/var/lib/mailman3/locks/dbcreate.lck|slc.mailhub4u.com|24435|4056142050254595815' -> '/var/lib/mailman3/locks/dbcreate.lck'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/bin/mailman", line 11, in <module> load_entry_point('mailman==3.2.1', 'console_scripts', 'mailman')() File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3/dist-packages/click/core.py", line 716, in main with self.make_context(prog_name, args, **extra) as ctx: File "/usr/lib/python3/dist-packages/click/core.py", line 641, in make_context self.parse_args(ctx, args) File "/usr/lib/python3/dist-packages/click/core.py", line 1089, in parse_args rest = Command.parse_args(self, ctx, args) File "/usr/lib/python3/dist-packages/click/core.py", line 940, in parse_args value, args = param.handle_parse_result(ctx, opts, args) File "/usr/lib/python3/dist-packages/click/core.py", line 1477, in handle_parse_result self.callback, ctx, self, value) File "/usr/lib/python3/dist-packages/click/core.py", line 96, in invoke_param_callback return callback(ctx, param, value) File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 94, in initialize_config initialize(value) File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 218, in initialize initialize_2(propagate_logs=propagate_logs) File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 177, in initialize_2 config.db = getUtility(IDatabaseFactory, utility_name).create() File "/usr/lib/python3/dist-packages/mailman/database/factory.py", line 50, in create with Lock(os.path.join(config.LOCK_DIR, 'dbcreate.lck')): File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 334, in __enter__ self.lock() File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 248, in lock elif self._read() == self._claimfile: File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 442, in _read with open(self._lockfile) as fp: PermissionError: [Errno 13] Permission denied: '/var/lib/mailman3/locks/dbcreate.lck'
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark, Thanks for the reply! I have removed the lock file as you recommended. The command in the subject header of the error email that mailman3 sent was as follows: Cron <list@slc> if [ -x /usr/bin/mailman ]; then /usr/bin/mailman digests --send; fi So, I believe that cron was running the following script in /etc/cron.d/mailman3:    # /etc/cron.d/mailman3: crontab entries for the mailman3 package    SHELL=/bin/sh    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin    0 0 * * *      list   if [ -x /usr/bin/mailman ]; then /usr/bin/mailman digests --send; fi Please let me know if you have any other ideas.... Thanks, Gordon
On 9/29/20 4:59 AM, Gordon Dickens wrote:
Hi Mark, Thanks for the reply! I have removed the lock file as you recommended.
Has the error recurred? If so, what is the content of var/lib/mailman3/locks/?
What do you see if you run
/usr/bin/mailman
manually with no arguments? It should print some help text and a list of commands.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 9/29/20 2:04 PM, Mark Sapiro wrote:
On 9/29/20 4:59 AM, Gordon Dickens wrote:
Hi Mark, Thanks for the reply! I have removed the lock file as you recommended.
Has the error recurred? If so, what is the content of var/lib/mailman3/locks/?
What do you see if you run
/usr/bin/mailman
manually with no arguments? It should print some help text and a list of commands.
If the error has not recurred, and the output from /usr/bin/mailman is
as expected, I'm guessing that the cause of the issue may have been some
other mailman command that was run manually and left a stale lock
behind for the cron of mailman digests --send to trip over.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
The error has not re-occurred in the last 24 hours since I removed the lock file as Mark recommended. So, I am hopefull that the problem is solved.
FYI,
Gordon
On 9/30/20 8:21 AM, gordon@dickens.com wrote:
The error has not re-occurred in the last 24 hours since I removed the lock file as Mark recommended. So, I am hopefull that the problem is solved.
As I suggested at
<https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...>,
I don't think the underlying issue was with the mailman digests --send
run by cron. I think any mailman command would have failed as long as
the dbcreate.db lock residue was in /var/lib/mailman3/locks/. The
question is what left that there in the first place. Did you manually
run any mailman commands that terminated abnormally or were killed in
the day before this started?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark, and other helpful souls, This cron error continues to occur at 12AMÂ several times each week but not each day .Here is the output from the error message that is sent to the mailman3 admin:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 216, in lo ck os.link(self._claimfile, self._lockfile) FileExistsError: [Errno 17] File exists: '/var/lib/mailman3/locks/dbcreate.lck|s lc.mailhub4u.com|19728|1804933663424130229' -> '/var/lib/mailman3/locks/dbcreate .lck'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/bin/mailman", line 11, in <module> load_entry_point('mailman==3.2.1', 'console_scripts', 'mailman')() File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3/dist-packages/click/core.py", line 716, in main with self.make_context(prog_name, args, **extra) as ctx: File "/usr/lib/python3/dist-packages/click/core.py", line 641, in make_context self.parse_args(ctx, args) File "/usr/lib/python3/dist-packages/click/core.py", line 1089, in parse_args rest = Command.parse_args(self, ctx, args) File "/usr/lib/python3/dist-packages/click/core.py", line 940, in parse_args value, args = param.handle_parse_result(ctx, opts, args) File "/usr/lib/python3/dist-packages/click/core.py", line 1477, in handle_pars e_result self.callback, ctx, self, value) File "/usr/lib/python3/dist-packages/click/core.py", line 96, in invoke_param_ callback return callback(ctx, param, value) File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 94, in init ialize_config initialize(value) File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 218, in initialize initialize_2(propagate_logs=propagate_logs) File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 177, in initialize_2 config.db = getUtility(IDatabaseFactory, utility_name).create() File "/usr/lib/python3/dist-packages/mailman/database/factory.py", line 50, in create with Lock(os.path.join(config.LOCK_DIR, 'dbcreate.lck')): File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 334, in __ enter__ self.lock() File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 248, in lo ck elif self._read() == self._claimfile: File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 442, in _r ead with open(self._lockfile) as fp: PermissionError: [Errno 13] Permission denied: '/var/lib/mailman3/locks/dbcreate .lck'
Then, the contents of the /var/lib/mailman3/locks directory is as follows:
-rw-rw---- 1 list list 80 Oct 25 00:00
'dbcreate.lck|slc.mailhub4u.com|19728|1804933663424130229'
-rw-rw---- 2 list list 78 Oct 26Â 2020Â master.lck
-rw-rw---- 2 list list 78 Oct 26Â 2020
'master.lck|slc.mailhub4u.com|25024|3254783301248355113'I have since removed the file 'dbcreate.lck|slc.mailhub4u.com|19728|1804933663424130229'. . Also, I had previously changed the permissions on the /var/lib/mailman3/locks directory to give universal read/write permissions::
drwxrwxrwx 2 list    list      4096 Oct 25 08:11 locksAlso, as user = list, I am able to create, edit and delete files in that directory which really doesn't make any sense because mailman3 is also running as user = list when it creates the Permission denied error. Does anybody have an idea as to what is going on here and how to fix it? Thanks, Gordon Dickens
On 10/25/20 6:14 AM, Gordon Dickens wrote:
Hi Mark, and other helpful souls, This cron error continues to occur at 12AM several times each week but not each day .Here is the output from the error message that is sent to the mailman3 admin:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 216, in lo ck os.link(self._claimfile, self._lockfile) FileExistsError: [Errno 17] File exists: '/var/lib/mailman3/locks/dbcreate.lck|s lc.mailhub4u.com|19728|1804933663424130229' -> '/var/lib/mailman3/locks/dbcreate .lck'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/bin/mailman", line 11, in <module> load_entry_point('mailman==3.2.1', 'console_scripts', 'mailman')() File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3/dist-packages/click/core.py", line 716, in main with self.make_context(prog_name, args, **extra) as ctx: File "/usr/lib/python3/dist-packages/click/core.py", line 641, in make_context self.parse_args(ctx, args) File "/usr/lib/python3/dist-packages/click/core.py", line 1089, in parse_args rest = Command.parse_args(self, ctx, args) File "/usr/lib/python3/dist-packages/click/core.py", line 940, in parse_args value, args = param.handle_parse_result(ctx, opts, args) File "/usr/lib/python3/dist-packages/click/core.py", line 1477, in handle_pars e_result self.callback, ctx, self, value) File "/usr/lib/python3/dist-packages/click/core.py", line 96, in invoke_param_ callback return callback(ctx, param, value) File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 94, in init ialize_config initialize(value) File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 218, in initialize initialize_2(propagate_logs=propagate_logs) File "/usr/lib/python3/dist-packages/mailman/core/initialize.py", line 177, in initialize_2 config.db = getUtility(IDatabaseFactory, utility_name).create() File "/usr/lib/python3/dist-packages/mailman/database/factory.py", line 50, in create with Lock(os.path.join(config.LOCK_DIR, 'dbcreate.lck')): File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 334, in __ enter__ self.lock() File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 248, in lo ck elif self._read() == self._claimfile: File "/usr/lib/python3/dist-packages/flufl/lock/_lockfile.py", line 442, in _r ead with open(self._lockfile) as fp: PermissionError: [Errno 13] Permission denied: '/var/lib/mailman3/locks/dbcreate .lck'
Then, the contents of the /var/lib/mailman3/locks directory is as follows:
-rw-rw---- 1 list list 80 Oct 25 00:00 'dbcreate.lck|slc.mailhub4u.com|19728|1804933663424130229' -rw-rw---- 2 list list 78 Oct 26 2020 master.lck -rw-rw---- 2 list list 78 Oct 26 2020 'master.lck|slc.mailhub4u.com|25024|3254783301248355113'I have since removed the file 'dbcreate.lck|slc.mailhub4u.com|19728|1804933663424130229'. . Also, I had previously changed the permissions on the /var/lib/mailman3/locks directory to give universal read/write permissions::
drwxrwxrwx 2 list list 4096 Oct 25 08:11 locksAlso, as user = list, I am able to create, edit and delete files in that directory which really doesn't make any sense because mailman3 is also running as user = list when it creates the Permission denied error. Does anybody have an idea as to what is going on here and how to fix it?
The FileExistsError is possibly due to some kind of race condition
between two mailman commands. Are you perhaps running both mailman digests --periodic and mailman notify via cron at midnight each
night? If so, try running one if them a minute later.
The permissions error is on reading the lock file itself. I don't understand what is causing that, but I doubt permissions on the locks directory are involved.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Gordon Dickens -
gordon＠dickens.com
-
Mark Sapiro