Re: statistics monitoring for Mailman3
On 3/12/21 11:22 AM, Mohsen Masoudfar wrote:
I was checking on log rotating on smtp.log. On my server it does not rotate. I could use log-rotator to rotate that, but could not find any information how to inform the mailman process of the rotation, e.g. which signal to send him?
The mailman reopen
command is the recommended way, or you can send the
master
a SIGUSR1. The master's PID is in the file var/master.pid or
/var/run/mailman/master.pid if you've configured layout: fhs
.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark,
Thanks again for the nice Python script. There is an issue with outlook.com. They add a '^M' (carriage Return) before msgid!! Here is part of the log that shows that. This is not in every log, though. I updated the get_log_data function (please see below), I believe it is not the smartest solution, but it seems to be working for me.
Mar 14 12:53:19 2021 (11696) ('127.0.0.1', 55180) Data: b'QUIT' Mar 14 12:53:19 2021 (11696) ('127.0.0.1', 55180) connection lost Mar 14 12:53:19 2021 (11696) Connection lost during _handle_client() Mar 14 12:53:21 2021 (11701) <MN2PR20MB324587763E970CC56351F8C5B66D9@MN2PR20MB3245.namprd20.prod.outlook.com> smtp to testlist3@newlistserv.aaas.org for 1 recips, completed in 0.013000726699829102 seconds Mar 14 12:53:21 2021 (11701) <MN2PR20MB324587763E970CC56351F8C5B66D9@MN2PR20MB3245.namprd20.prod.outlook.com> post to testlist3@newlistserv.aaas.org from testlist3@newlistserv.aaas.org, 7940 bytes
LOGRETS = ('^(?P<time_stamp>[^(]*) \(\d+\) (?:\n|\r\n)') LOGREMI = (' (?P<msgid><[^>]*>) smtp to (?P<listname>[^ ]*) for (?P<count>\d+) recips, completed ')
def get_log_data(log): data = [] with open(log) as fp: for line in fp.readlines(): mo = re.match(LOGRE, line) if not mo: mts = re.match(LOGRETS, line) if mts: dt = datetime.datetime.strptime(mts.group('time_stamp'), '%b %d %H:%M:%S %Y')
if not mts:
mmi = re.match(LOGREMI, line)
if mmi:
mid = mmi.group('msgid')
lname = mmi.group('listname')
count = int(mmi.group('count'))
data.append([dt, mid, lname, count])
continue
continue
dt = datetime.datetime.strptime(mo.group('time_stamp'),
'%b %d %H:%M:%S %Y')
mid = mo.group('msgid')
lname = mo.group('listname')
count = int(mo.group('count'))
data.append([dt, mid, lname, count])
return data
Thanks Mohsen
-----Original Message----- From: Mark Sapiro <mark@msapiro.net> Sent: Friday, March 12, 2021 2:42 PM To: MM3 Users <mailman-users@mailman3.org> Subject: [MM3-users] Re: statistics monitoring for Mailman3
[EXTERNAL EMAIL]
On 3/12/21 11:22 AM, Mohsen Masoudfar wrote:
I was checking on log rotating on smtp.log. On my server it does not rotate. I could use log-rotator to rotate that, but could not find any information how to inform the mailman process of the rotation, e.g. which signal to send him?
The mailman reopen
command is the recommended way, or you can send the master
a SIGUSR1. The master's PID is in the file var/master.pid or /var/run/mailman/master.pid if you've configured layout: fhs
.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mail...
On 3/15/21 12:39 PM, Mohsen Masoudfar wrote:
Hi Mark,
Thanks again for the nice Python script. There is an issue with outlook.com. They add a '^M' (carriage Return) before msgid!! Here is part of the log that shows that. This is not in every log, though. I updated the get_log_data function (please see below), I believe it is not the smartest solution, but it seems to be working for me.
Does the script with your change produce a different result than the original?
This has nothing to do with outlook.com per se. It only has to do with the length of the log message. Further, I do not think there are actually newlines in these log messages. I think that is just an artifact of how you are viewing the log.
Mar 14 12:53:21 2021 (11701) <MN2PR20MB324587763E970CC56351F8C5B66D9@MN2PR20MB3245.namprd20.prod.outlook.com> smtp to testlist3@newlistserv.aaas.org for 1 recips, completed in 0.013000726699829102 seconds
I suspect if you do
grep namprd20.prod.outlook.com /path/to/var/logs/smtp.log
The result will be lines like
> Mar 14 12:53:21 2021 (11701) <MN2PR20MB324587763E970CC56351F8C5B66D9@MN2PR20MB3245.namprd20.prod.outlook.com> smtp to testlist3@newlistserv.aaas.org for 1 recips, completed in 0.013000726699829102 seconds
whereas if they are really split, the result would be
> <MN2PR20MB324587763E970CC56351F8C5B66D9@MN2PR20MB3245.namprd20.prod.outlook.com> smtp to testlist3@newlistserv.aaas.org for 1 recips, completed in 0.013000726699829102 seconds
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 3/15/21 2:01 PM, Mark Sapiro wrote:
This has nothing to do with outlook.com per se. It only has to do with the length of the log message. Further, I do not think there are actually newlines in these log messages. I think that is just an artifact of how you are viewing the log.
Actually, I am incorrect. Sorry for any confusion. I agree that the long lines are folded in the log and the script needs updating. I will do that.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 3/15/21 2:01 PM, Mark Sapiro wrote:
This has nothing to do with outlook.com per se. It only has to do with the length of the log message. Further, I do not think there are actually newlines in these log messages. I think that is just an artifact of how you are viewing the log.
Actually, I'm wrong again. It does have to do with outlook.com. It is because the Message-ID: header in the original message is folded as in
Message-ID: <crlf> <MN2PR20MB324587763E970CC56351F8C5B66D9@MN2PR20MB3245.namprd20.prod.outlook.com>
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 3/15/21 2:38 PM, Mark Sapiro wrote:
On 3/15/21 2:01 PM, Mark Sapiro wrote:
This has nothing to do with outlook.com per se. It only has to do with the length of the log message. Further, I do not think there are actually newlines in these log messages. I think that is just an artifact of how you are viewing the log.
Actually, I'm wrong again. It does have to do with outlook.com. It is because the Message-ID: header in the original message is folded as in
Message-ID: <crlf> <MN2PR20MB324587763E970CC56351F8C5B66D9@MN2PR20MB3245.namprd20.prod.outlook.com>
Sorry for the premature send. I'm having a bad time with this thread. :-(
I meant to say
It does have to do with outlook.com. It is because the Message-ID: header in the original message is folded as in
Message-ID: <crlf> <MN2PR20MB324587763E970CC56351F8C5B66D9@MN2PR20MB3245.namprd20.prod.outlook.com>
and the added <crlf><space> actually gets included in the log message. Anyway I have updated the script at <https://www.msapiro.net/scripts/smtp_report>. My update is much simpler than the one at <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/Y2ULGDOI2VOFU5BTE65APPXHOAOWS4BH/>, but I have tested it, and it works.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Mark Sapiro
-
Mohsen Masoudfar