[mlmmj] mlmmj and spf

[mlmmj] mlmmj and spf

[theo borm]

Dear list members,

We operate a small, closed, moderated mailing list that recently stopped 
working for a large part of its subscribers. The organization of which 
these subscribers are a member maintains an SPF record which denies 
access to all servers except a named few, which seems to be the cause of 
these problems.

As a work-around I set mlmmj to use a different from address in the 
"From:" header. This solution is, however, plainly bad as it removes the 
original sender from the headers. I have seen other lists use "Sender:" 
header, but results are a mixed bag. With strict SPF checking of the 
"From:" header in place these mails also don't pass.

What would be the correct solution?

any help, pointers greatly appreciated.

regards, Theo

Re: [mlmmj] mlmmj and spf

[Franky Van Liedekerke]

On 2012-05-11 13:41, theo borm wrote:
> Dear list members,
>
> We operate a small, closed, moderated mailing list that recently
> stopped working for a large part of its subscribers. The organization
> of which these subscribers are a member maintains an SPF record which
> denies access to all servers except a named few, which seems to be 
> the
> cause of these problems.
>
> As a work-around I set mlmmj to use a different from address in the
> "From:" header. This solution is, however, plainly bad as it removes
> the original sender from the headers. I have seen other lists use
> "Sender:" header, but results are a mixed bag. With strict SPF
> checking of the "From:" header in place these mails also don't pass.
>
> What would be the correct solution?
>
> any help, pointers greatly appreciated.
>
> regards, Theo

You have 2 options:

- get your mailserver in the SPF records for the domains for which you 
send mail (probably not a valid option, depending on the number of 
domains and the willingness of the DNS admins for those)
- make it so that the SMTP header "mail from" is set to the original 
sender, the "From:" part is in fact only data, it is the SMTP "mail 
from" that's being checked AFAIK.

I can't check it from here, so I can't say if rewriting just the SMTP 
"mail from" is possible in mlmmj.


Franky

Re: [mlmmj] mlmmj and spf

[theo borm]

Thanks

I already contacted the organization and they replied: your server is 
not under our control, so therefore we will not put it in our SPF 
records (...). Check of the actual spf records reveals that microsoft, 2 
/16 subnets, a few ISPs and a couple of others are already in there. 
Must be a very powerful organization to control all these.

Mlmmj as-is doesn't appear to have a way to *rewrite* headers. I can 
delete and insert headers no problem, but nothing in the way of 
rewriting or variable substitution on headers.

I just had a look at the sources. Hacking support for this into 
mlmmj-process.c/do_all_the_voodoo_here.c looks doable.

regards, Theo



do_all_the_voodoo_here

On 05/11/2012 01:46 PM, Marc MAURICE wrote:
> Hello Theo,
>
> The facts that the organization put this SPF means that the 
> organization doesn't want to allow mail comming from @organization.org 
> to be sent by external, non whitelisted servers.
>
> Either ask the organization to add your server to the SPF, if possible.
> Or install the mlmmj on a server already whitelisted (probably not 
> possible).
>
> Otherwise the only solution I see is rewriting the From header as you 
> said. You can always add the original sender in a header, or in the 
> mail body (is it possible to add the sender in the body with mlmmj?)
>
> From my little knowledge...
>
> Regards,
> Marc
>
>
>
>
>
> On 11/05/2012 13:41, theo borm wrote:
>> Dear list members,
>>
>> We operate a small, closed, moderated mailing list that recently 
>> stopped working for a large part of its subscribers. The organization 
>> of which these subscribers are a member maintains an SPF record which 
>> denies access to all servers except a named few, which seems to be 
>> the cause of these problems.
>>
>> As a work-around I set mlmmj to use a different from address in the 
>> "From:" header. This solution is, however, plainly bad as it removes 
>> the original sender from the headers. I have seen other lists use 
>> "Sender:" header, but results are a mixed bag. With strict SPF 
>> checking of the "From:" header in place these mails also don't pass.
>>
>> What would be the correct solution?
>>
>> any help, pointers greatly appreciated.
>>
>> regards, Theo
>>
>>
>>
>>
>

Re: [mlmmj] mlmmj and spf

[Christian Laursen]

On 05/11/12 13:41, theo borm wrote:
 >
> We operate a small, closed, moderated mailing list that recently stopped
> working for a large part of its subscribers. The organization of which
> these subscribers are a member maintains an SPF record which denies
> access to all servers except a named few, which seems to be the cause of
> these problems.

It sounds like their SPF implementation is broken.

> As a work-around I set mlmmj to use a different from address in the
> "From:" header. This solution is, however, plainly bad as it removes the
> original sender from the headers. I have seen other lists use "Sender:"
> header, but results are a mixed bag. With strict SPF checking of the
> "From:" header in place these mails also don't pass.

SPF checking should be done on the enevelope FROM address, not the From: 
header contained inside the mail.

When mlmmj send out mails to the list subscribers the envelope from 
looks something like this:
listname+bounces-XXXX-recipient=example.com@listowner.tld

So the SPF checking is done against the domain that the list is running on.

The correct cause of action would be to get the organization with broken 
SPF checking to fix that.

-- 
Christian Laursen

Re: [mlmmj] mlmmj and spf

[Ben Schmidt]

I second what Christian says. It sounds like the SPF checking is broken. It should 
be against the envelope-from. Checking against the "From:" header is wrong and bad 
for a number of reasons.

If they want to validate the "From:" and/or other headers, they should use DKIM. 
It will actually be effective and more efficient for validating headers.

I think it's pretty unlikely I would accept a change to Mlmmj to work around a 
buggy SPF implementation.

Cheers,

Ben.



On 11/05/12 10:43 PM, Christian Laursen wrote:
> On 05/11/12 13:41, theo borm wrote:
>  >
>> We operate a small, closed, moderated mailing list that recently stopped
>> working for a large part of its subscribers. The organization of which
>> these subscribers are a member maintains an SPF record which denies
>> access to all servers except a named few, which seems to be the cause of
>> these problems.
>
> It sounds like their SPF implementation is broken.
>
>> As a work-around I set mlmmj to use a different from address in the
>> "From:" header. This solution is, however, plainly bad as it removes the
>> original sender from the headers. I have seen other lists use "Sender:"
>> header, but results are a mixed bag. With strict SPF checking of the
>> "From:" header in place these mails also don't pass.
>
> SPF checking should be done on the enevelope FROM address, not the From: header
> contained inside the mail.
>
> When mlmmj send out mails to the list subscribers the envelope from looks
> something like this:
> listname+bounces-XXXX-recipient=example.com@listowner.tld
>
> So the SPF checking is done against the domain that the list is running on.
>
> The correct cause of action would be to get the organization with broken SPF
> checking to fix that.
>

Re: [mlmmj] mlmmj and spf

[Franky Van Liedekerke]

On 2012-05-11 15:20, Ben Schmidt wrote:
> I second what Christian says. It sounds like the SPF checking is
> broken. It should be against the envelope-from. Checking against the
> "From:" header is wrong and bad for a number of reasons.

yes, I forgot that mlmmj already implements verp for the envelope from 
(or "mail from" as I simplisticly called it) :-)
So I agree here as well.

Franky

Re: [mlmmj] mlmmj and spf

[theo borm]

On 05/11/2012 03:20 PM, Ben Schmidt wrote:
>
> I think it's pretty unlikely I would accept a change to Mlmmj to work 
> around a buggy SPF implementation.

Hi Ben,

I would be the last to suggest a patch to work around a Sender-Id 
misconfiguration. ;-)

However, a patch allowing slightly more advanced header manipulations 
(say regexp match/replace, between delheader and customheader), would 
/that/ be welcomed?

regards,

Theo

Re: [mlmmj] mlmmj and spf

[theo borm]

[-- Attachment #1: Type: text/plain, Size: 2216 bytes --]

Hi Christian,

Thanks for the answer.

On 05/11/2012 02:43 PM, Christian Laursen wrote:
> On 05/11/12 13:41, theo borm wrote:
> >
>> We operate a small, closed, moderated mailing list that recently stopped
>> working for a large part of its subscribers. The organization of which
>> these subscribers are a member maintains an SPF record which denies
>> access to all servers except a named few, which seems to be the cause of
>> these problems.
>
> It sounds like their SPF implementation is broken.

microsoft?

>
>> As a work-around I set mlmmj to use a different from address in the
>> "From:" header. This solution is, however, plainly bad as it removes the
>> original sender from the headers. I have seen other lists use "Sender:"
>> header, but results are a mixed bag. With strict SPF checking of the
>> "From:" header in place these mails also don't pass.
>
> SPF checking should be done on the enevelope FROM address, not the 
> From: header contained inside the mail.

http://www.openspf.org/SPF_vs_Sender_ID summarizes this nicely:

<quote>
How will /Sender ID/ implementations violating the /SPF/ specification 
affect me?

If you have published an |v=spf1| policy to protect the use of your 
domain in the MAIL FROM and HELO addresses, /Sender ID/ implementations 
that apply your policy to /PRA/ (per RFC 4406) will reject your mail if 
you use your domain in the "|From|" (or generally /PRA/) header field 
while sending from (MAIL FROM) another system.
</quote>

organization has an |v=spf1| policy in place. Mail is outsourced to 
microsoft, which uses sender ID.

It's the receiving server which has to implement spf/sender-ID, so 
delivery is erratic to say the least.


>
> When mlmmj send out mails to the list subscribers the envelope from 
> looks something like this:
> listname+bounces-XXXX-recipient=example.com@listowner.tld

listowner.tld doesn't have an spf record.

this is not the problem. problem is the "From" field/

>
> So the SPF checking is done against the domain that the list is 
> running on.
>
> The correct cause of action would be to get the organization with 
> broken SPF checking to fix that.
>

I (and others) have told them a few times to no avail.

regards, Theo


[-- Attachment #2: Type: text/html, Size: 3391 bytes --]

Re: [mlmmj] mlmmj and spf

[Franky Van Liedekerke]

On 2012-05-11 15:55, theo borm wrote:

>> It sounds like their SPF implementation is broken.

> microsoft?

Well, you mentioned SPF in your original mail, but MS indeed uses 
Sender ID :-)

> organization has an v=spf1 policy in place. Mail is outsourced to
> microsoft, which uses sender ID.
>
> It's the receiving server which has to implement spf/sender-ID, so
> delivery is erratic to say the least.

Well, not the receiving end, but the one doing the DNS records. So try 
to convince the DNS owners for those domains to change their 
"spf2.0/mfrom,pra" to just "spf2.0/mfrom"

>
> I (and others) have told them a few times to no avail.
>
> regards, Theo

Of course MS won't listen :-) But if the DNS owners are reasonable ...

Franky

Re: [mlmmj] mlmmj and spf

[theo borm]

On 05/11/2012 03:00 PM, Franky Van Liedekerke wrote:
> On 2012-05-11 14:57, theo borm wrote:
>> Hi Franky,
>>
>> Thanks for the answer.
>>
>> There are two types of "from"'s: the envelope sender as specified in
>> the SMTP communication with the "mail from" command, and the one
>> embedded in the mail data itself using the "From" header. Either can
>> be checked with SPF and the microsoft variant (Sender ID), and both
>> have a different way of interpreting these info.
>> Normall the envelope sender is used by mlmmj to implement "VERP",
>> which allows it to efficiently detect which email addresses bounce and
>> take appropriate action. It is also normally not shown to end-users,
>> so it is not much use here. Besides, I don't want to break VERP
>> processing.
>>
>> regards, Theo
>
> yes, I forgot about the VERP ...
> Let me brush off my SPF knowledge again, I thought there was an 
> exception for mailing lists and the Return-Path header or so ...
>
> Franky

Hi Franky,

Apologies for accidentally taking this off-list....

As it stands, broken or misconfigured sender-id / spf 
implementations/interpretations are being used. I doubt if reliance on 
headers that can just as easily be faked to define exceptions would be 
more reliable.

regards, Theo

Re: [mlmmj] mlmmj and spf

[theo borm]

On 05/11/2012 03:56 PM, Franky Van Liedekerke wrote:
> On 2012-05-11 15:55, theo borm wrote:
>
>>> It sounds like their SPF implementation is broken.
>
>> microsoft?
>
> Well, you mentioned SPF in your original mail, but MS indeed uses 
> Sender ID :-)

sorry for the confusion.

>
>> organization has an v=spf1 policy in place. Mail is outsourced to
>> microsoft, which uses sender ID.
>>
>> It's the receiving server which has to implement spf/sender-ID, so
>> delivery is erratic to say the least.
>
> Well, not the receiving end, but the one doing the DNS records. So try 
> to convince the DNS owners for those domains to change their 
> "spf2.0/mfrom,pra" to just "spf2.0/mfrom"

It's microsoft who recommends using v=spf1 record as-if it is a 
spf2.0/mfrom,pra record. It is the receiving end (microsioft) using a 
sender ID implementation that is configured to interpret it this way. It 
is the DNS owner who *might* be able to circumvent this stupidity by 
using a spf2.0/mfrom record instead. But it may even be reasonable for 
the DNS owner to refuse this on the fundamental grounds that they 
shouldn't be fixing other peoples' problem.

>
>>
>> I (and others) have told them a few times to no avail.
>>
>> regards, Theo
>
> Of course MS won't listen :-) But if the DNS owners are reasonable ...

I have warned the DNS owners ~ 2 years ago. Their response was that 
"they would discuss this with the "software vendor" (Wietse Venema), but 
that I should not expect a resolution". Until recently they operated 
their own mail servers, which applied the spf record correctly. Recently 
they have outsourced their mail to microsoft (externally their only MX 
is mail.messaging.microsoft.com), and that is when the trouble started.

Of course there has been trouble before, but these were "fringe users" 
on providers that used sender id, and the problem was simply solved by 
instructing them to use gmail or whatever. Now I have to tell the 
majority of list users to go to gmail because their IT department is what?.

well. It's a nuisance.

cheers, Theo

Re: [mlmmj] mlmmj and spf

[Ben Schmidt]

> http://www.openspf.org/SPF_vs_Sender_ID summarizes this nicely:
>
> <quote>
> How will /Sender ID/ implementations violating the /SPF/ specification
> affect me?
>
> If you have published an |v=spf1| policy to protect the use of your
> domain in the MAIL FROM and HELO addresses, /Sender ID/
> implementations that apply your policy to /PRA/ (per RFC 4406) will
> reject your mail if you use your domain in the "|From|" (or generally
> /PRA/) header field while sending from (MAIL FROM) another system.
> </quote>
>
> organization has an |v=spf1| policy in place. Mail is outsourced to
> microsoft, which uses sender ID.
>
> It's the receiving server which has to implement spf/sender-ID, so
> delivery is erratic to say the least.

Yeah. This kind of stuff is a PITA.

As well as the solution by publishing an spf2.0 record, the document you
referenced suggests using the "Sender:" header. I can't really remember,
but did you say you already tried that and it didn't help?

>> When mlmmj send out mails to the list subscribers the envelope from
>> looks something like this:
>> listname+bounces-XXXX-recipient=example.com@listowner.tld
>
> listowner.tld doesn't have an spf record.
>
> this is not the problem. problem is the "From" field/

Hmm. Well, when an implementation is broken, who knows how broken? Maybe
the combination of a "Sender:" header and a published SPF record for
that domain (which will also be the domain of the envelope-from) will
help...if it can be done without too much hassle....

Ben.

Re: [mlmmj] mlmmj and spf

[Ben Schmidt]

On 12/05/12 12:04 AM, theo borm wrote:
> On 05/11/2012 03:20 PM, Ben Schmidt wrote:
>>
>> I think it's pretty unlikely I would accept a change to Mlmmj to work
>> around a buggy SPF implementation.
>
> I would be the last to suggest a patch to work around a Sender-Id
> misconfiguration. ;-)
>
> However, a patch allowing slightly more advanced header manipulations
> (say regexp match/replace, between delheader and customheader), would
> /that/ be welcomed?

Hi, Theo,

Yes, I'd quite probably accept such a change, provided it is kept
simple, and particularly if some specifics are floated past the mailing
list prior to preparing a patch, so people have a chance to raise any
issues.

There are also some pending changes to header stuff in the bug tracker,
so it would be good to at least keep those in mind, or even implement
them along with the new code (probably in separate patches, but
nonetheless, the work is related and might be good to be implemented
together).

Cheers,

Ben.