Getting rid of infradead.org - corrupted subjects

Getting rid of infradead.org - corrupted subjects

[Krzysztof Kozlowski]

Hi all,

The address list is semi-random as I don't know whom to approach.

Problem: infradead.org corrupts email subjects by changing:
s/,/, /

Examples:
1. Previous discussion with Uwe about linux-arm-kernel:
https://lore.kernel.org/all/20220215125856.es2euyoqo6mp4y2t@pengutronix.de/

2. My last email for MTD
Modified email:
https://lore.kernel.org/linux-mtd/20220408063720.12826-1-krzysztof.kozlowski@linaro.org/
Proper email:
https://lore.kernel.org/all/3e550deb-7c27-894b-a0b5-62609a60f17f@microchip.com/

When people use Patchwork or b4, the email from the list is being used,
thus the one with a changed subject. Example:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9547c4e74f6b4b56c2f9e4e7a286ce126fd333fa
The subject should be "dt-bindings: mtd: jedec,spi-nor: remove unneeded
properties" (no space after coma)

This leads to real commits with a bit odd subject. At least not the
original subject.

Can we do something about infradead? Get rid of it and move the mailing
lists to vger.kernel.org? Or fix it?

Best regards,
Krzysztof

Re: Getting rid of infradead.org - corrupted subjects

[Geert Uytterhoeven]

CC dwmw2

On Wed, May 25, 2022 at 10:07 AM Krzysztof Kozlowski
<krzysztof.kozlowski@linaro.org> wrote:
> The address list is semi-random as I don't know whom to approach.
>
> Problem: infradead.org corrupts email subjects by changing:
> s/,/, /
>
> Examples:
> 1. Previous discussion with Uwe about linux-arm-kernel:
> https://lore.kernel.org/all/20220215125856.es2euyoqo6mp4y2t@pengutronix.de/
>
> 2. My last email for MTD
> Modified email:
> https://lore.kernel.org/linux-mtd/20220408063720.12826-1-krzysztof.kozlowski@linaro.org/
> Proper email:
> https://lore.kernel.org/all/3e550deb-7c27-894b-a0b5-62609a60f17f@microchip.com/
>
> When people use Patchwork or b4, the email from the list is being used,
> thus the one with a changed subject. Example:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9547c4e74f6b4b56c2f9e4e7a286ce126fd333fa
> The subject should be "dt-bindings: mtd: jedec,spi-nor: remove unneeded
> properties" (no space after coma)
>
> This leads to real commits with a bit odd subject. At least not the
> original subject.
>
> Can we do something about infradead? Get rid of it and move the mailing
> lists to vger.kernel.org? Or fix it?
>
> Best regards,
> Krzysztof

Re: Getting rid of infradead.org - corrupted subjects

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 2657 bytes --]

On Wed, 2022-05-25 at 10:34 +0200, Geert Uytterhoeven wrote:
> CC dwmw2

Thanks, Geert.

> On Wed, May 25, 2022 at 10:07 AM Krzysztof Kozlowski
> <krzysztof.kozlowski@linaro.org> wrote:
> 
> > The address list is semi-random as I don't know whom to approach.
> > Problem: infradead.org corrupts email subjects by changing:
> > s/,/, /

I found an old thread on mailman-users which appears to be claiming
that it's just standard folding as described in RFC2822 §2.2.3.

https://mail.python.org/pipermail/mailman-users/2007-May/057119.html

2.2.3. Long Header Fields

   Each header field is logically a single line of characters comprising
   the field name, the colon, and the field body.  For convenience
   however, and to deal with the 998/78 character limitations per line,
   the field body portion of a header field can be split into a multiple
   line representation; this is called "folding".  The general rule is
   that wherever this standard allows for folding white space (not
   simply WSP characters), a CRLF may be inserted before any WSP.  For
   example, the header field:

           Subject: This is a test

   can be represented as:

           Subject: This
            is a test

But this is folding at a *comma*, not at whitespace. The original
subject line was (in a single line):

Subject: [PATCH v2] dt-bindings: mtd: jedec,spi-nor: remove unneeded properties

... and Mailman 'folded' it in the wrong place to:

Subject: [PATCH v2] dt-bindings: mtd: jedec,
 spi-nor: remove unneeded properties

That *isn't* proper folding because it didn't happen at a whitespace.

I do need to upgrade to Mailman 3 at some point; I've been
procrastinating on the basis that it "ain't broke". For the time being
I've just disabled folding in MM2 with a trivial hack:

--- Message.py.orig	2022-05-25 02:51:05.917145192 -0700
+++ Message.py	2022-05-25 02:50:44.137142069 -0700
@@ -49,7 +49,7 @@ class Generator(email.Generator.Generato
        Headers will by default _not_ be folded in attachments.
     """
     def __init__(self, outfp, mangle_from_=True,
-                 maxheaderlen=78, children_maxheaderlen=0):
+                 maxheaderlen=0, children_maxheaderlen=0):
         email.Generator.Generator.__init__(self, outfp,
                 mangle_from_=mangle_from_, maxheaderlen=maxheaderlen)
         self.__children_maxheaderlen = children_maxheaderlen

Looking at http://lists.infradead.org/pipermail/testlist/2022-May/subject.html
it seems to have had the desired effect. Please let me know if not (or
if there are other side-effects).

Thanks for the report.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

Re: Getting rid of infradead.org - corrupted subjects

[Rob Herring]

On Wed, May 25, 2022 at 4:55 AM David Woodhouse <dwmw2@infradead.org> wrote:
>
> On Wed, 2022-05-25 at 10:34 +0200, Geert Uytterhoeven wrote:
> > CC dwmw2
>
> Thanks, Geert.
>
> > On Wed, May 25, 2022 at 10:07 AM Krzysztof Kozlowski
> > <krzysztof.kozlowski@linaro.org> wrote:
> >
> > > The address list is semi-random as I don't know whom to approach.
> > > Problem: infradead.org corrupts email subjects by changing:
> > > s/,/, /
>
> I found an old thread on mailman-users which appears to be claiming
> that it's just standard folding as described in RFC2822 §2.2.3.
>
> https://mail.python.org/pipermail/mailman-users/2007-May/057119.html
>
> 2.2.3. Long Header Fields
>
>    Each header field is logically a single line of characters comprising
>    the field name, the colon, and the field body.  For convenience
>    however, and to deal with the 998/78 character limitations per line,
>    the field body portion of a header field can be split into a multiple
>    line representation; this is called "folding".  The general rule is
>    that wherever this standard allows for folding white space (not
>    simply WSP characters), a CRLF may be inserted before any WSP.  For
>    example, the header field:
>
>            Subject: This is a test
>
>    can be represented as:
>
>            Subject: This
>             is a test
>
> But this is folding at a *comma*, not at whitespace. The original
> subject line was (in a single line):
>
> Subject: [PATCH v2] dt-bindings: mtd: jedec,spi-nor: remove unneeded properties
>
> ... and Mailman 'folded' it in the wrong place to:
>
> Subject: [PATCH v2] dt-bindings: mtd: jedec,
>  spi-nor: remove unneeded properties
>
> That *isn't* proper folding because it didn't happen at a whitespace.
>
> I do need to upgrade to Mailman 3 at some point; I've been
> procrastinating on the basis that it "ain't broke". For the time being
> I've just disabled folding in MM2 with a trivial hack:

Yay!!! I should have asked years ago...

Any chance the appending of footers can be stopped as it causes lore
to report duplicates and b4 dmarc attestation to fail?

Rob

Re: Getting rid of infradead.org - corrupted subjects

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 349 bytes --]

On Wed, 2022-05-25 at 09:30 -0500, Rob Herring wrote:
> 
> Yay!!! I should have asked years ago...
> 
> 
> 
> Any chance the appending of footers can be stopped as it causes lore
> to report duplicates and b4 dmarc attestation to fail?

That's up to individual list owners; it's part of the per-list config.
It's very easy to turn off.


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

Re: Getting rid of infradead.org - corrupted subjects

[Chen-Yu Tsai]

On Wed, May 25, 2022 at 11:17 PM David Woodhouse <dwmw2@infradead.org> wrote:
>
> On Wed, 2022-05-25 at 09:30 -0500, Rob Herring wrote:
> >
> > Yay!!! I should have asked years ago...
> >
> >
> >
> > Any chance the appending of footers can be stopped as it causes lore
> > to report duplicates and b4 dmarc attestation to fail?
>
> That's up to individual list owners; it's part of the per-list config.
> It's very easy to turn off.

Could you share who the owners for linux-arm-kernel@, linux-mtd@ and
linux-mediatek@, just to name a few, are?

Thanks
ChenYu

Re: Getting rid of infradead.org - corrupted subjects

[Uwe Kleine-König]

[-- Attachment #1: Type: text/plain, Size: 2954 bytes --]

On Wed, May 25, 2022 at 10:55:18AM +0100, David Woodhouse wrote:
> On Wed, 2022-05-25 at 10:34 +0200, Geert Uytterhoeven wrote:
> > CC dwmw2
> 
> Thanks, Geert.
> 
> > On Wed, May 25, 2022 at 10:07 AM Krzysztof Kozlowski
> > <krzysztof.kozlowski@linaro.org> wrote:
> > 
> > > The address list is semi-random as I don't know whom to approach.
> > > Problem: infradead.org corrupts email subjects by changing:
> > > s/,/, /
> 
> I found an old thread on mailman-users which appears to be claiming
> that it's just standard folding as described in RFC2822 §2.2.3.
> 
> https://mail.python.org/pipermail/mailman-users/2007-May/057119.html
> 
> 2.2.3. Long Header Fields
> 
>    Each header field is logically a single line of characters comprising
>    the field name, the colon, and the field body.  For convenience
>    however, and to deal with the 998/78 character limitations per line,
>    the field body portion of a header field can be split into a multiple
>    line representation; this is called "folding".  The general rule is
>    that wherever this standard allows for folding white space (not
>    simply WSP characters), a CRLF may be inserted before any WSP.  For
>    example, the header field:
> 
>            Subject: This is a test
> 
>    can be represented as:
> 
>            Subject: This
>             is a test
> 
> But this is folding at a *comma*, not at whitespace. The original
> subject line was (in a single line):
> 
> Subject: [PATCH v2] dt-bindings: mtd: jedec,spi-nor: remove unneeded properties
> 
> ... and Mailman 'folded' it in the wrong place to:
> 
> Subject: [PATCH v2] dt-bindings: mtd: jedec,
>  spi-nor: remove unneeded properties
> 
> That *isn't* proper folding because it didn't happen at a whitespace.
> 
> I do need to upgrade to Mailman 3 at some point; I've been
> procrastinating on the basis that it "ain't broke". For the time being
> I've just disabled folding in MM2 with a trivial hack:
> 
> --- Message.py.orig	2022-05-25 02:51:05.917145192 -0700
> +++ Message.py	2022-05-25 02:50:44.137142069 -0700
> @@ -49,7 +49,7 @@ class Generator(email.Generator.Generato
>         Headers will by default _not_ be folded in attachments.
>      """
>      def __init__(self, outfp, mangle_from_=True,
> -                 maxheaderlen=78, children_maxheaderlen=0):
> +                 maxheaderlen=0, children_maxheaderlen=0):
>          email.Generator.Generator.__init__(self, outfp,
>                  mangle_from_=mangle_from_, maxheaderlen=maxheaderlen)
>          self.__children_maxheaderlen = children_maxheaderlen

For the record: Some time ago I debugged that myself, too and found an
equivalent change. So this looks good to me.

Best regards and thanks for caring,
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-König            |
Industrial Linux Solutions                 | https://www.pengutronix.de/ |

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Getting rid of infradead.org - corrupted subjects

[Uwe Kleine-König]

[-- Attachment #1: Type: text/plain, Size: 3695 bytes --]

Hello,

On Wed, May 25, 2022 at 09:23:32PM +0200, Uwe Kleine-König wrote:
> On Wed, May 25, 2022 at 10:55:18AM +0100, David Woodhouse wrote:
> > On Wed, 2022-05-25 at 10:34 +0200, Geert Uytterhoeven wrote:
> > > CC dwmw2
> > 
> > Thanks, Geert.
> > 
> > > On Wed, May 25, 2022 at 10:07 AM Krzysztof Kozlowski
> > > <krzysztof.kozlowski@linaro.org> wrote:
> > > 
> > > > The address list is semi-random as I don't know whom to approach.
> > > > Problem: infradead.org corrupts email subjects by changing:
> > > > s/,/, /
> > 
> > I found an old thread on mailman-users which appears to be claiming
> > that it's just standard folding as described in RFC2822 §2.2.3.
> > 
> > https://mail.python.org/pipermail/mailman-users/2007-May/057119.html
> > 
> > 2.2.3. Long Header Fields
> > 
> >    Each header field is logically a single line of characters comprising
> >    the field name, the colon, and the field body.  For convenience
> >    however, and to deal with the 998/78 character limitations per line,
> >    the field body portion of a header field can be split into a multiple
> >    line representation; this is called "folding".  The general rule is
> >    that wherever this standard allows for folding white space (not
> >    simply WSP characters), a CRLF may be inserted before any WSP.  For
> >    example, the header field:
> > 
> >            Subject: This is a test
> > 
> >    can be represented as:
> > 
> >            Subject: This
> >             is a test
> > 
> > But this is folding at a *comma*, not at whitespace. The original
> > subject line was (in a single line):
> > 
> > Subject: [PATCH v2] dt-bindings: mtd: jedec,spi-nor: remove unneeded properties
> > 
> > ... and Mailman 'folded' it in the wrong place to:
> > 
> > Subject: [PATCH v2] dt-bindings: mtd: jedec,
> >  spi-nor: remove unneeded properties
> > 
> > That *isn't* proper folding because it didn't happen at a whitespace.
> > 
> > I do need to upgrade to Mailman 3 at some point; I've been
> > procrastinating on the basis that it "ain't broke". For the time being
> > I've just disabled folding in MM2 with a trivial hack:
> > 
> > --- Message.py.orig	2022-05-25 02:51:05.917145192 -0700
> > +++ Message.py	2022-05-25 02:50:44.137142069 -0700
> > @@ -49,7 +49,7 @@ class Generator(email.Generator.Generato
> >         Headers will by default _not_ be folded in attachments.
> >      """
> >      def __init__(self, outfp, mangle_from_=True,
> > -                 maxheaderlen=78, children_maxheaderlen=0):
> > +                 maxheaderlen=0, children_maxheaderlen=0):
> >          email.Generator.Generator.__init__(self, outfp,
> >                  mangle_from_=mangle_from_, maxheaderlen=maxheaderlen)
> >          self.__children_maxheaderlen = children_maxheaderlen
> 
> For the record: Some time ago I debugged that myself, too and found an
> equivalent change. So this looks good to me.

small supplement: I found the change I did back then:

--- /usr/lib/python2.7/email/header.py    2018-09-26 20:42:22.000000000 +0200
+++ /usr/lib/python2.7/email/header.py    2018-09-26 20:42:22.000000000 +0200
@@ -371,7 +371,7 @@
         joiner = NL + self._continuation_ws
         return joiner.join(chunks)

-    def encode(self, splitchars=';, '):
+    def encode(self, splitchars=' '):
         """Encode a message header into an RFC-compliant format.

         There are many issues involved in converting a given string for use in

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-König            |
Industrial Linux Solutions                 | https://www.pengutronix.de/ |

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]