Hi!
I have now tested ARC signing of messages a bit more and have found (I think) all settings so that it works in that it generates the signed headers that ARC is. But the signature is not accepted; gmail says "body hash mismatch" and hotmail says "arc=fail (47)". I have set the following settings under the [ARC] section. I have tried both with the sig_headers set as displayed below and with it commented to lean against the defaults. I use the same key and selector as for DKIM-signing with the server's domain. The DKIM-signature is accepted. Is there any known issues with ARC that I should be aware of? I run the latest release of mailman (3.3.0 of core) on Python 3.7.6 and I use postfix as MTA.
enabled: yes privkey: <my key path> selector: <my selector> domain: <my server's domain>
dmarc: yes dkim: yes authserv_id: <my server's domain> trusted_authserv_ids: <my hosted domains>
#sig_headers: From, Sender, Reply-To, Subject, Date, Message-ID, To, Cc, MIME-Version, Content-Type, Content-Transfer-Encoding, Content-ID, Content-Description, Resent-Date, Resent-From, Resent-Sender, Resent-To, Resent-Cc, Resent-Message-ID, In-Reply-To, References, List-Id, List-Help, List-Unsubscribe, List-Subscribe, List-Post, List-Owner, List-Archive
I hope that this is something simple that I have forgot.
Cheers // David
David Krantz writes:
Is there any known issues with ARC that I should be aware of?
Doesn't Postfix do ARC yet? :-( rspamd seems to be recommended by a couple of Linux distros, supports the milter interface that Postfix uses to call out to 3rd party processes, and the website looks good. See rspamd.com. That said, if you want to use Mailman's support...
Our support doesn't have issues that I know of. You can check the issue tracker: https://gitlab.com/mailman/mailman/issues. On second thought, don't bother. I did and "ARC" picks up scads of irrelevant junk, "authenticated received chain" nothing, and a couple of other tries don't pick up anything useful either.
I run the latest release of mailman (3.3.0 of core) on Python 3.7.6 and I use postfix as MTA.
[various data deleted]
I haven't looked at ARC in some months, so it's possible that our current implementation is out of date with best current practice. However, in a quick scan I see no difference in RFC 8617 vs. ID-ARC-23, so I don't know what might have changed there.
Are the messages being refused by those sites because of the ARC errors, or are they being delivered anyway? We probably need to see the message as received by Mailman, the message as emitted by Mailman, and the message as received at the validating site to have a hope of diagnosing this.
The things that I can see from the data you've provided that might be related are
If it's just the body hash that's a problem, there may be one or more MXes that convert Content-Transfer-Encoding between your Mailman and the receiving sites. That would be easy to check if you have the messages as sent and as received.
You have an awfully aggressive list of signed header fields. In
particular, the Resent-* fields are treated as trace fields: there may be multiple instances of each. I don't understand the semantics of DKIM/ARC signing for those headers very well, but it's something that I would think easy to mess up. You might try deleting those.
You are supposed to sign any existing DKIM-Signature fields, but that's not in the list.
The ARC RFC 8617 says that an ARC Sealer must add any DKIM-Signature *before* ARC sealing. I suppose this could be a problem, since our ARC implementation doesn't DKIM sign. (On the other hand, I don't see how it would be a problem, since any MX along the way might add a DKIM signature, and you're not signing DKIM-Signature according to your list of signed fields.)
It would be helpful if instead of quoting small parts of error messages, you gave us the entire body of a delivery status notification message or the entire MTA log. If there's stuff (domains, mailboxes, IP addresses) you want to redact, feel free, just make sure that it's obvious what was redacted.
I will check our implementation, principally to make sure that the authentication results used are computed on the raw incoming message and the ARC sealing is done on the raw outgoing message after all other message modifications are done. I will also see if I can conjure up tests that test the site's MTA(s) as well as just our implementation, as we currently test our implementation using static data from the ARC implementation we use (ie, we ARC-process a given message and then do a byte-by-byte comparison on an allegedly correct ARC-Sealed version of the same).
Sorry I can't give more precise help. Maybe something here will be of use.
Steve
On 3/2/20 7:45 AM, Stephen J. Turnbull wrote:
David Krantz writes:
Is there any known issues with ARC that I should be aware of?
Doesn't Postfix do ARC yet? :-( rspamd seems to be recommended by a couple of Linux distros, supports the milter interface that Postfix uses to call out to 3rd party processes, and the website looks good. See rspamd.com. That said, if you want to use Mailman's support...
I've been trying openarc <https://github.com/trusteddomainproject/OpenARC> with Postfix as well as Mailman's ARC. I still haven't got it all worked out - I have a fairly convoluted mail flow with the possibility of messages being altered before they ever get to Mailman.
There is an issue with Mailman's ARC signing in a supporting library. See <https://github.com/ValiMail/authentication-headers/issues/5>.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Mon, Mar 2, 2020 at 7:07 PM Mark Sapiro <mark@msapiro.net> wrote:
I've been trying openarc <https://github.com/trusteddomainproject/OpenARC> with Postfix as well as Mailman's ARC. I still haven't got it all worked out - I have a fairly convoluted mail flow with the possibility of messages being altered before they ever get to Mailman.
I do not have that problem. The ARC implementation in Mailman accepts incoming DKIM signatures etc. so that is not a problem.
There is an issue with Mailman's ARC signing in a supporting library. See <https://github.com/ValiMail/authentication-headers/issues/5>.
I have not had that issue yet, I do not think any of my users format their name that way. Good to know, though.
Cheers. // David
On Mon, Mar 2, 2020 at 4:45 PM Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Doesn't Postfix do ARC yet? :-( rspamd seems to be recommended by a couple of Linux distros, supports the milter interface that Postfix uses to call out to 3rd party processes, and the website looks good. See rspamd.com. That said, if you want to use Mailman's support...
To me, it seems easier to use Mailman's implementation if it is possible. Rspamd seems very nice as an alternative if I find time to substitute my spamassassin, greylister and, I guess, opendkim setup. As the feature of rspamd set is large with a web UI it probably is as time consuming to set up as mailman, so I'd rather not if I can avoid it. When I migrate to another server in due time it seems interesting.
Our support doesn't have issues that I know of. You can check the issue tracker: https://gitlab.com/mailman/mailman/issues. On second thought, don't bother. I did and "ARC" picks up scads of irrelevant junk, "authenticated received chain" nothing, and a couple of other tries don't pick up anything useful either.
I run the latest release of mailman (3.3.0 of core) on Python 3.7.6 and I use postfix as MTA.
I haven't looked at ARC in some months, so it's possible that our current implementation is out of date with best current practice. However, in a quick scan I see no difference in RFC 8617 vs. ID-ARC-23, so I don't know what might have changed there.
Are the messages being refused by those sites because of the ARC errors, or are they being delivered anyway? We probably need to see the message as received by Mailman, the message as emitted by Mailman, and the message as received at the validating site to have a hope of diagnosing this.
So far, they are delivered. Probably for the same reason that my messages are accepted without ARC. IPs with good reputation and a generally well-behaved setup.
The things that I can see from the data you've provided that might be related are
- If it's just the body hash that's a problem, there may be one or more MXes that convert Content-Transfer-Encoding between your Mailman and the receiving sites. That would be easy to check if you have the messages as sent and as received.
I do not think they do, but I'll examine that. My DKIM signing works just fine but postfix might have changed something before opendkim get the message. I'll have to find a way to capture the sent message leaving mailman before it enters postfix.
You have an awfully aggressive list of signed header fields. In particular, the Resent-* fields are treated as trace fields: there may be multiple instances of each. I don't understand the semantics of DKIM/ARC signing for those headers very well, but it's something that I would think easy to mess up. You might try deleting those.
You are supposed to sign any existing DKIM-Signature fields, but that's not in the list.
I just used the default list of fields from the Mailman documentation. Now I did try a shorter list of fields but that did not help.
- The ARC RFC 8617 says that an ARC Sealer must add any DKIM-Signature *before* ARC sealing. I suppose this could be a problem, since our ARC implementation doesn't DKIM sign. (On the other hand, I don't see how it would be a problem, since any MX along the way might add a DKIM signature, and you're not signing DKIM-Signature according to your list of signed fields.)
It would be a problem if I'd add the DKIM fields to the field list of course. I see another potential problem with that list in that it includes fields that are not present, I had understood it as that would fail if a field is missing.
It would be helpful if instead of quoting small parts of error messages, you gave us the entire body of a delivery status notification message or the entire MTA log. If there's stuff (domains, mailboxes, IP addresses) you want to redact, feel free, just make sure that it's obvious what was redacted.
I can do that. For that I will have to re-run my test which I do not have time for today, alas. It is easier to me to send an entire delivered message (with some domains and IPs redacted). I'll add such an example at the bottom of this mail as it gets a bit unwieldy.
I will check our implementation, principally to make sure that the authentication results used are computed on the raw incoming message and the ARC sealing is done on the raw outgoing message after all other message modifications are done. I will also see if I can conjure up tests that test the site's MTA(s) as well as just our implementation, as we currently test our implementation using static data from the ARC implementation we use (ie, we ARC-process a given message and then do a byte-by-byte comparison on an allegedly correct ARC-Sealed version of the same).
Is there any good tool to verify an ARC signature? I could of course set another list as a subscriber to feed the result back to OpenDKIM and Mailman-ARC to see what happens but that is a bit convoluted.
Sorry I can't give more precise help. Maybe something here will be of use.
Of much use as it got me thinking, if nothing else.
// David
Delivered-To: gazorg@gmail.com Received: by 2002:ab3:6e8a:0:0:0:0:0 with SMTP id d10csp504094ltp; Sat, 29 Feb 2020 08:27:23 -0800 (PST) X-Google-Smtp-Source: APXvYqy2N4DctBE7sJ5iPliD4tFTap/Ag38FlQknzbgepRIeBhGi4BkvVeX+otJg+Q8EqS3L12L0 X-Received: by 2002:a05:6402:36d:: with SMTP id s13mr8975006edw.304.1582993643529; Sat, 29 Feb 2020 08:27:23 -0800 (PST) Return-Path: <test-bounces@example2.com> Received: from lists.example1.com (lists.example1.com. [192.168.0.1redactedIP]) by mx.google.com with ESMTPS id q7si4234768edn.147.2020.02.29.08.27.23 for <gazorg@gmail.com> (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 29 Feb 2020 08:27:23 -0800 (PST) Received-SPF: pass (google.com: domain of test-bounces@example2.com designates 192.168.0.1redactedIP as permitted sender) client-ip=192.168.0.1redactedIP; Authentication-Results: mx.google.com; dkim=pass header.i=@example2.com header.s=dkim header.b=aUkUFDX3; arc=fail (body hash mismatch); spf=pass (google.com: domain of test-bounces@example2.com designates 192.168.0.1redactedIP as permitted sender) smtp.mailfrom=test-bounces@example2.com Received: from lists.example1.com (localhost [127.0.0.1]) by lists.example1.com (Postfix) with ESMTP id 37B37405C6; Sat, 29 Feb 2020 16:27:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=example2.com; s=dkim; t=1582993643; bh=QtvH55+E9IsKYbQtbq3Ctq/SJNg1OC15t8DzRRa/xtM=; h=Date:To:Subject:List-Id:List-Help:List-Post:List-Subscribe: List-Unsubscribe:From:Reply-To:From; b=aUkUFDX3gmRAu4yffKt1a93D0zQ5SHOVM9IBgn9Eds02d99fTpDPGPrLURXm5nMsm 5Czjhyv12hwt/sDcqjkgot7XNZgpYu+ygeNhfFr/fNDR3I+/OJhQJFl7pGFH6/LkpI eZqCZuGFTTZjLU5/kPZ3Bxul9NsWUpFoxkoDj8bumbVnkAqBpFNqKF7vojUs/0A8lU MNZRRpvFzdpJPrMOWq2ZW/SfLZJaXJEKQPBX+cxtLTadwRooG4tJ58hGbjuk5qIlHH 0rohDoyI6zFO1IZLUub+LltmX8Z9CjGyfOp7FJeXs4i/F2Y0NtZfdATjGr9ODWYsxl Do7WWaCt/IagQ== ARC-Seal: i=1; cv=none; a=rsa-sha256; d=example1.com; s=dkim; t=1582993642; b=gcSal3MqawEf89KBCDi3s7mNkPZfJhN7XJ0SRlaB6fTT1LSVbEDb6JXa0cM4owpyvPPWU VG6/bOqHJeuACWqGQ6/wnXJCuRCJJkKzv0FVJCg61xfXcSUtKKCeLFarZtx59WUGxkkmCqI oDJufbsX56AYM0awVkHn9xu+eezG87Q= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=example1.com; s=dkim; t=1582993642; h=from : sender : reply-to : subject : date : message-id : to : cc : mime-version : content-type : content-transfer-encoding : content-id : content-description : resent-date : resent-from : resent-sender : resent-to : resent-cc : resent-message-id : in-reply-to : references : list-id : list-help : list-unsubscribe : list-subscribe : list-post : list-owner : list-archive; bh=Dphqgp/+wTnNNbrrDg3NfnpjXIJyK09lbeEWp6UaizQ=; b=W3oTWN8gDlllat7/QAXOe0lQk6IHDhIhwgu4KOjkFN8nOdRQUbnv0dt9LoZt2NrjY20wj g1IKXDuyfHitDwSqloxrYl8woZji+qy2fYyws4ATmzHckqBgfGQbDSxu3Lu3iaGf8cjEy11 Mj+rUGOg0sWoXXNCqjvMGMU3SWhQBRs= ARC-Authentication-Results: i=1; example1.com; dkim=pass header.d=example3.com header.i=@example3.com header.b="RgU2p/NV"; dkim-atps=neutral; arc=none; dmarc=none Authentication-Results: example1.com; dkim=pass header.d=example3.com header.i=@example3.com header.b="RgU2p/NV"; dkim-atps=neutral; arc=none; dmarc=none Received: from mail-lj1-f180.google.com (mail-lj1-f180.google.com [209.85.208.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.example1.com (Postfix) with ESMTPS id 720B9405C5 for <test@example2.com>; Sat, 29 Feb 2020 16:27:20 +0000 (UTC) Received: by mail-lj1-f180.google.com with SMTP id q23so6847929ljm.4 for <test@example2.com>; Sat, 29 Feb 2020 08:27:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=Dphqgp/+wTnNNbrrDg3NfnpjXIJyK09lbeEWp6UaizQ=; b=VK+DeP8PfxjHWKPnE/VeVQCBOfi7xubKqXJ+qIokd8osR8zolWOQoljLgKgjmllrYi ZjRTWt4/f9imHKTtu1V+8YVdQ/yfAQ1LR9aYxTnBS0Baz9BV22ghXOuTAgm77uPOPZnQ BWRG6W01Fwb5VjlGMSQrn/+nQa5m+gBjKnF4F34HqtM3c0Igx9459lL++O3CC0M4vjnb sg0JVmQbbWu1anXQRnx2Ghy8/tkcKuJVDE9BhTXmMQCjJSzwRvLp7jHTlNk1tXW11TQ5 ZBnzgg+oN5GcgudjSMnBhstsCUqvEQttV5FOj1gNBV+EOw0B+b+A4iLGyUn/c+bbrVwi HXjg== X-Gm-Message-State: ANhLgQ0MEeEPN51okIGKQNjjbGpV/RHmPT8h47RTCG9U5MwsQWT9UZHg XzxoNg3sC2PYicdeBLor1wLThcLqWerLvsxOW5jMIm6SrSU= X-Received: by 2002:a2e:7d07:: with SMTP id y7mr5916044ljc.7.1582993639657; Sat, 29 Feb 2020 08:27:19 -0800 (PST) MIME-Version: 1.0 Date: Sat, 29 Feb 2020 17:27:08 +0100 Message-ID: <CAFstx=9nwFZeaE_cZ+69YAGY-cCE_6maP62A7X0VMTBC6Mwhqg@mail.gmail.com> To: test@example2.com X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.11 (lists.example1.com [192.168.0.1redactedIP]); Sat, 29 Feb 2020 16:27:20 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.11 (lists.example1.com [192.168.0.1redactedIP]); Sat, 29 Feb 2020 16:27:20 +0000 (UTC) for IP:'209.85.208.180' DOMAIN:'mail-lj1-f180.google.com' HELO:'mail-lj1-f180.google.com' FROM:'david@example3.com' RCPT:'' X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on lists.example1.com Message-ID-Hash: 2WKWCRI6GNTHTKGUKXSV3QI7JSG4BCCS X-Message-ID-Hash: 2WKWCRI6GNTHTKGUKXSV3QI7JSG4BCCS X-MailFrom: david@example3.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header X-Mailman-Version: 3.3.0 Precedence: list Subject: [Test] ARC-test med aktiverad modul List-Id: Test for mail setup <test.example2.com> List-Help: <mailto:test-request@example2.com?subject=help> List-Post: <mailto:test@example2.com> List-Subscribe: <mailto:test-join@example2.com> List-Unsubscribe: <mailto:test-leave@example2.com> From: David Krantz via Test <test@example2.com> Reply-To: David Krantz <david@example3.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.11 (lists.example1.com [192.168.0.1redactedIP]); Sat, 29 Feb 2020 16:27:23 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.11 (lists.example1.com [192.168.0.1redactedIP]); Sat, 29 Feb 2020 16:27:23 +0000 (UTC) for IP:'127.0.0.1' DOMAIN:'localhost' HELO:'lists.example1.com' FROM:'test-bounces@example2.com' RCPT:''
RGV0IGjDpHIga2Fuc2tlIGZ1bmthci4gVmkgZsOlciBzZS4NCg0KLy8gREF2aWQKX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KVGVzdGdhbmcgbWFpbGluZyBs aXN0CnRlc3RnYW5nQGtsYW1nYW5nZXQuc2UKaHR0cHM6Ly9saXN0cy56ZXJvdGguc2UvbWFpbG1h bjMvbGlzdHMvdGVzdGdhbmcua2xhbWdhbmdldC5zZS8K
participants (3)
-
David Krantz -
Mark Sapiro -
Stephen J. Turnbull