b4 v0.3.4 released

b4 v0.3.4 released

[Konstantin Ryabitsev]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

Hello:

I pushed v0.3.4 to pypi, so if you're using it from there, you can run 
"pip install --user --upgrade b4" to grab the latest version.

Notable changes compared to 0.3.3:

- deals properly with series versions that are only mentioned in the 
  cover letter
- adds a b4.sh wrapper that lets you run b4 without needing to install 
  with pip
- fixes a couple of backtraces due to bugs with patch attestation
- adds a caching layer to avoid hitting lore.kernel.org on repeating 
  runs (e.g. when rerunning with -t, -s, -l, and similar)
- properly refuses to install on python version < 3.5
- uses more descriptive filenames for "b4 am" mboxes

Thanks to the following people for their feedback:

- Geoff Levand
- Mark Brown
- Kees Cook
- Geert Uytterhoeven
- James Bottomley

-K

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: [kernel.org users] b4 v0.3.4 released

[Jason A. Donenfeld]

Hi Konstantin,

Still not a fan at all of the attestation stuff, but the lore
integration is a great convenience. Thanks for this great tool. I've
packaged it up for Gentoo:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8374a4402f40ad1ff3613e281db5bb897708fcea

Users can now run:

`$ emerge b4`

Please let me know if any of the metadata I've used in that commit is
off. As well, at the bottom of this email is a listing of all the
files this winds up installing, in case you spot something as being
wrong.

Will you continue to send release announcements here, or should I set
up a notifier through other means?

I'm curious to learn: where did this name "b4" come from? Is it like
the millennial meme phrase "inb4" being a deconstruction of "inbox"
for a postmodern command line work of art?

Regards,
Jason

----8<--------------------------

zx2c4@thinkpad ~ $ equery f b4
* Searching for b4 ...
* Contents of net-mail/b4-0.3.4:
/usr
/usr/bin
/usr/bin/b4 -> ../lib/python-exec/python-exec2
/usr/lib
/usr/lib/python-exec
/usr/lib/python-exec/python3.6
/usr/lib/python-exec/python3.6/b4
/usr/lib64
/usr/lib64/python3.6
/usr/lib64/python3.6/site-packages
/usr/lib64/python3.6/site-packages/b4
/usr/lib64/python3.6/site-packages/b4-0.3.4-py3.6.egg-info
/usr/lib64/python3.6/site-packages/b4-0.3.4-py3.6.egg-info/PKG-INFO
/usr/lib64/python3.6/site-packages/b4-0.3.4-py3.6.egg-info/SOURCES.txt
/usr/lib64/python3.6/site-packages/b4-0.3.4-py3.6.egg-info/dependency_links.txt
/usr/lib64/python3.6/site-packages/b4-0.3.4-py3.6.egg-info/entry_points.txt
/usr/lib64/python3.6/site-packages/b4-0.3.4-py3.6.egg-info/requires.txt
/usr/lib64/python3.6/site-packages/b4-0.3.4-py3.6.egg-info/top_level.txt
/usr/lib64/python3.6/site-packages/b4/__init__.py
/usr/lib64/python3.6/site-packages/b4/__pycache__
/usr/lib64/python3.6/site-packages/b4/__pycache__/__init__.cpython-36.opt-1.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/__init__.cpython-36.opt-2.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/__init__.cpython-36.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/attest.cpython-36.opt-1.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/attest.cpython-36.opt-2.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/attest.cpython-36.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/command.cpython-36.opt-1.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/command.cpython-36.opt-2.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/command.cpython-36.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/mbox.cpython-36.opt-1.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/mbox.cpython-36.opt-2.pyc
/usr/lib64/python3.6/site-packages/b4/__pycache__/mbox.cpython-36.pyc
/usr/lib64/python3.6/site-packages/b4/attest.py
/usr/lib64/python3.6/site-packages/b4/command.py
/usr/lib64/python3.6/site-packages/b4/mbox.py
/usr/share
/usr/share/doc
/usr/share/doc/b4-0.3.4
/usr/share/doc/b4-0.3.4/README.rst.bz2

Re: [kernel.org users] b4 v0.3.4 released

[Jason A. Donenfeld]

I induced a crash:

$ b4 am -c 20200324153624.23109-1-jiaxun.yang@flygoat.com
Looking up https://lore.kernel.org/r/20200324153624.23109-1-jiaxun.yang@flygoat.com
Grabbing thread from lore.kernel.org
Reduced thread to strict matches only (138->107)
Checking for newer revisions on https://lore.kernel.org/linux-devicetree/
Analyzing 107 messages in the thread
Found new series v1
Found new series v2
Traceback (most recent call last):
 File "/usr/lib/python-exec/python3.6/b4", line 11, in <module>
   load_entry_point('b4==0.3.4', 'console_scripts', 'b4')()
 File "/usr/lib64/python3.6/site-packages/b4/command.py", line 134, in cmd
   cmdargs.func(cmdargs)
 File "/usr/lib64/python3.6/site-packages/b4/command.py", line 40, in cmd_am
   b4.mbox.main(cmdargs)
 File "/usr/lib64/python3.6/site-packages/b4/mbox.py", line 407, in main
   mbox_to_am(threadmbox, config, cmdargs)
 File "/usr/lib64/python3.6/site-packages/b4/mbox.py", line 114, in mbox_to_am
   lmbx.add_message(msg)
 File "/usr/lib64/python3.6/site-packages/b4/__init__.py", line 232,
in add_message
   lmsg = LoreMessage(msg)
 File "/usr/lib64/python3.6/site-packages/b4/__init__.py", line 572, in __init__
   if diffstatre.search(self.body):
TypeError: expected string or bytes-like object

However, starting from other message IDs works:

zx2c4@thinkpad ~/Projects/wireguard-tools $ b4 am -c
https://lore.kernel.org/lkml/20190905144316.12527-1-jiaxun.yang@flygoat.com/
Looking up https://lore.kernel.org/r/20190905144316.12527-1-jiaxun.yang@flygoat.com
Grabbing thread from lore.kernel.org
Reduced thread to strict matches only (138->100)
Checking for newer revisions on https://lore.kernel.org/lkml/
Analyzing 100 messages in the thread
Found new series v1
Found new series v3
Found new series v6
Will use the latest revision: v6
You can pick other revisions using the -vN flag
---
Writing ./v6_20200324_jiaxun_yang_modernize_loongson64_machine_v6.mbx
 [PATCH v6 01/11] irqchip: Add driver for Loongson I/O Local Interrupt
Controller
 [PATCH v6 02/11] irqchip: loongson-liointc: Workaround LPC IRQ Errata
 [PATCH v6 03/11] dt-bindings: interrupt-controller: Add Loongson LIOINTC
 [PATCH v6 04/11] irqchip: Add driver for Loongson-3 HyperTransport
PIC controller
 [PATCH v6 05/11] dt-bindings: interrupt-controller: Add Loongson-3 HTPIC
 [PATCH v6 06/11] irqchip: mips-cpu: Convert to simple domain
 [PATCH v6 07/11] MIPS: Loongson64: Drop legacy IRQ code
 [PATCH v6 08/11] dt-bindings: mips: Add loongson boards
 [PATCH v6 09/11] MIPS: Loongson64: Add generic dts
 [PATCH v6 10/11] MIPS: Loongson64: Load built-in dtbs
 [PATCH v6 11/11] MIPS: Loongson64: Move MIPS_CPU_IRQ_BASE
---
Total patches: 11
---
NOTE: Some trailers were sent to the cover letter:
     Reviewed-by: Marc Zyngier <maz@kernel.org>
NOTE: Rerun with -t to apply them to all patches
---
Cover: ./v6_20200324_jiaxun_yang_modernize_loongson64_machine_v6.cover
Link: https://lore.kernel.org/r/20200324153624.23109-1-jiaxun.yang@flygoat.com
Base: not found, sorry
      git checkout -b v6_20200324_jiaxun_yang_flygoat_com master
      git am ./v6_20200324_jiaxun_yang_modernize_loongson64_machine_v6.mbx

Re: [kernel.org users] b4 v0.3.4 released

[Konstantin Ryabitsev]

On Tue, Mar 24, 2020 at 08:19:00PM -0600, Jason A. Donenfeld wrote:
> I induced a crash:
> 
> $ b4 am -c 20200324153624.23109-1-jiaxun.yang@flygoat.com
> Looking up https://lore.kernel.org/r/20200324153624.23109-1-jiaxun.yang@flygoat.com
> Grabbing thread from lore.kernel.org
> Reduced thread to strict matches only (138->107)
> Checking for newer revisions on https://lore.kernel.org/linux-devicetree/
> Analyzing 107 messages in the thread
> Found new series v1
> Found new series v2
> Traceback (most recent call last):
>  File "/usr/lib/python-exec/python3.6/b4", line 11, in <module>
>    load_entry_point('b4==0.3.4', 'console_scripts', 'b4')()
>  File "/usr/lib64/python3.6/site-packages/b4/command.py", line 134, in cmd
>    cmdargs.func(cmdargs)
>  File "/usr/lib64/python3.6/site-packages/b4/command.py", line 40, in cmd_am
>    b4.mbox.main(cmdargs)
>  File "/usr/lib64/python3.6/site-packages/b4/mbox.py", line 407, in main
>    mbox_to_am(threadmbox, config, cmdargs)
>  File "/usr/lib64/python3.6/site-packages/b4/mbox.py", line 114, in mbox_to_am
>    lmbx.add_message(msg)
>  File "/usr/lib64/python3.6/site-packages/b4/__init__.py", line 232,
> in add_message
>    lmsg = LoreMessage(msg)
>  File "/usr/lib64/python3.6/site-packages/b4/__init__.py", line 572, in __init__
>    if diffstatre.search(self.body):
> TypeError: expected string or bytes-like object

Thanks for the report -- the latest commit no longer breaks on messages 
containing no usable text/plain parts (that message is html-only).

I'll release 0.3.5 in the near future to fix this and a few other bugs.

-K

Re: [kernel.org users] b4 v0.3.4 released

[Konstantin Ryabitsev]

On Tue, Mar 24, 2020 at 08:11:10PM -0600, Jason A. Donenfeld wrote:
> Hi Konstantin,
> 
> Still not a fan at all of the attestation stuff

But look how pretty it looks!
https://mricon.com/misc/attestation.png

Would people like it better if I added emojis? :)
https://mricon.com/misc/attestation-with-feeling.png

(Just kidding. I know that not everyone is interested in patch-level 
signatures, but seeing as we don't have *any* other mechanism of doing 
end-to-end attestation for mailed patches at this time, I wanted to at 
least take a stab at it and offer something usable and behind-the-scenes 
enough not to annoy the heck out of everyone.)

> , but the lore
> integration is a great convenience. Thanks for this great tool. I've
> packaged it up for Gentoo:
> 
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8374a4402f40ad1ff3613e281db5bb897708fcea

Nice. I've been holding off on doing the same for Fedora until b4 sees a 
bit less churn. Perhaps in another week or so once the 0.3.y branch 
stabilizes.

> Will you continue to send release announcements here, or should I set
> up a notifier through other means?

I will always send announcements to tools@linux.kernel.org at least, and 
major feature additions will always be announced on the users list.

> I'm curious to learn: where did this name "b4" come from? Is it like
> the millennial meme phrase "inb4" being a deconstruction of "inbox"
> for a postmodern command line work of art?

Nothing that clever. I'm following the Trek theme started by V'ger:

  The name "b4" was chosen for ease of typing and because B-4 was the
  precursor to Lore and Data in the Star Trek universe.

-K

Re: [kernel.org users] b4 v0.3.4 released

[Rob Herring]

[-- Attachment #1: Type: text/plain, Size: 1624 bytes --]

On Mon, Mar 23, 2020 at 4:20 PM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
>
> Hello:
>
> I pushed v0.3.4 to pypi, so if you're using it from there, you can run
> "pip install --user --upgrade b4" to grab the latest version.
>
> Notable changes compared to 0.3.3:
>
> - deals properly with series versions that are only mentioned in the
>   cover letter
> - adds a b4.sh wrapper that lets you run b4 without needing to install
>   with pip
> - fixes a couple of backtraces due to bugs with patch attestation
> - adds a caching layer to avoid hitting lore.kernel.org on repeating
>   runs (e.g. when rerunning with -t, -s, -l, and similar)
> - properly refuses to install on python version < 3.5
> - uses more descriptive filenames for "b4 am" mboxes

I'm seeing corruption of the subject on patches[1] with a comma in the
subject (and I'd guess only ones longer than 70 something chars). It's
fine on lore web interface, but it gets wrapped in the downloaded
mbox. A log is attached. When it gets applied, a space is inserted
after the comma:

dt-bindings: iio/accel: Drop duplicate adi, adxl345/6 from trivial-devices.yaml

Interestingly, DT patchwork suffers from the same issue.

Also, I got tripped up with missing acks again because dri-devel got
picked and it moderates non-subscribers. A way to set the default
project would be nice. Perhaps rather than a single default, making it
a list of lists to try. Or trying lkml first might work better if
we're just guessing.

Rob

[1] https://lore.kernel.org/linux-devicetree/20200325220542.19189-2-robh@kernel.org/

[-- Attachment #2: log --]
[-- Type: application/octet-stream, Size: 6851 bytes --]

Running git --no-pager config -z --get-regexp b4\..*
Running git --no-pager config -z --get-regexp get-lore-mbox\..*
Running git --no-pager config -z --get-regexp gpg\..*
Looking up https://lore.kernel.org/r/20200325220542.19189-2-robh@kernel.org
Grabbing thread from lore.kernel.org
Saving ./20200325220542.19189-2-robh@kernel.org.t.mbx-loose
Kept in thread: 20200325220542.19189-2-robh@kernel.org
Kept in thread: 20200325220542.19189-4-robh@kernel.org
Kept in thread: 20200325220542.19189-5-robh@kernel.org
Kept in thread: 20200325220749.GF12169@sirena.org.uk
Kept in thread: 20200325233409.GB16767@onstation.org
Kept in thread: 88deb8bb-718c-f8ce-885e-dbc122201f16@baylibre.com
Kept in thread: e60df575-c70d-a194-6c54-32b5ae69a041@baylibre.com
Kept in thread: CA+M3ks7Y9YEUP_pc_Hyffj5EaLFGEA5_91vH0QcPUCbkwz4VYw@mail.gmail.com
Kept in thread: 20200325220542.19189-1-robh@kernel.org
Kept in thread: 20200325220542.19189-3-robh@kernel.org
Analyzing 10 messages in the thread
Looking at: [PATCH 1/4] dt-bindings: iio/accel: Drop duplicate adi, adxl345/6 from trivial-devices.yaml
  adding as patch
Looking at: [PATCH 3/4] dt-bindings: Clean-up schema errors due to missing 'addtionalProperties: false'
  adding as patch
Looking at: [PATCH 4/4] dt-bindings: Add missing 'additionalProperties: false'
  adding as patch
  trailer exact email match
Looking at: Re: [PATCH 4/4] dt-bindings: Add missing 'additionalProperties: false'
  adding to followups
  trailer exact email match
Looking at: Re: [PATCH 2/4] dt-bindings: sram: qcom: Clean-up 'ranges' and child node names
  adding to followups
  trailer exact email match
Looking at: Re: [PATCH 4/4] dt-bindings: Add missing 'additionalProperties: false'
  adding to followups
  trailer exact email match
Looking at: Re: [PATCH 3/4] dt-bindings: Clean-up schema errors due to missing 'addtionalProperties: false'
  adding to followups
Looking at: Re: [PATCH 4/4] dt-bindings: Add missing 'additionalProperties: false'
  adding to followups
Looking at: [PATCH 0/4] dt-bindings: additionalProperties clean-up
  adding as v1 cover letter
Looking at: [PATCH 2/4] dt-bindings: sram: qcom: Clean-up 'ranges' and child node names
  adding as patch
Analyzing follow-up: Re: [PATCH 4/4] dt-bindings: Add missing 'additionalProperties: false' (broonie@kernel.org)
 Parent: [PATCH 4/4] dt-bindings: Add missing 'additionalProperties: false'
 Trailers:
  Acked-by: Mark Brown <broonie@kernel.org>
Analyzing follow-up: Re: [PATCH 2/4] dt-bindings: sram: qcom: Clean-up 'ranges' and child node names (masneyb@onstation.org)
 Parent: [PATCH 2/4] dt-bindings: sram: qcom: Clean-up 'ranges' and child node names
 Trailers:
  Reviewed-by: Brian Masney <masneyb@onstation.org>
Analyzing follow-up: Re: [PATCH 4/4] dt-bindings: Add missing 'additionalProperties: false' (narmstrong@baylibre.com)
 Parent: [PATCH 4/4] dt-bindings: Add missing 'additionalProperties: false'
 Trailers:
  Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
Analyzing follow-up: Re: [PATCH 3/4] dt-bindings: Clean-up schema errors due to missing 'addtionalProperties: false' (narmstrong@baylibre.com)
 Parent: [PATCH 3/4] dt-bindings: Clean-up schema errors due to missing 'addtionalProperties: false'
 Trailers:
  Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
Analyzing follow-up: Re: [PATCH 4/4] dt-bindings: Add missing 'additionalProperties: false' (benjamin.gaignard@linaro.org)
  no trailers found, skipping
---
Writing ./20200325_robh_dt_bindings_additionalproperties_clean_up.mbx
Running git --no-pager config -z --get-regexp user\..*
Running git --no-pager mailinfo --encoding=UTF-8 /tmp/tmp61n85c5l /tmp/tmp4phphh0_
Query URL: https://lore.kernel.org/signatures/?q=32a61b36-0a980876-4602f928&x=A&o=-1
  [PATCH 1/4] dt-bindings: iio/accel: Drop duplicate adi, adxl345/6 from trivial-devices.yaml
    . Signed-off-by: Rob Herring <robh@kernel.org>
    . Cc: Peter Meerwald-Stadler <pmeerw@pmeerw.net>
    . Cc: Lars-Peter Clausen <lars@metafoo.de>
    . Cc: linux-iio@vger.kernel.org
    . Cc: Michael Hennerich <michael.hennerich@analog.com>
    . Cc: Hartmut Knaack <knaack.h@gmx.de>
    . Cc: Jonathan Cameron <jic23@kernel.org>
Running git --no-pager mailinfo --encoding=UTF-8 /tmp/tmpymycy7tf /tmp/tmpp_muuj8x
  [PATCH 2/4] dt-bindings: sram: qcom: Clean-up 'ranges' and child node names
    . Fixes: 957fd69d396b ("dt-bindings: soc: qcom: add On Chip MEMory (OCMEM) bindings")
    . Signed-off-by: Rob Herring <robh@kernel.org>
    + Reviewed-by: Brian Masney <masneyb@onstation.org>
    . Cc: linux-arm-msm@vger.kernel.org
    . Cc: Bjorn Andersson <bjorn.andersson@linaro.org>
    . Cc: Brian Masney <masneyb@onstation.org>
Running git --no-pager mailinfo --encoding=UTF-8 /tmp/tmptgx22k0_ /tmp/tmp9lihsggr
  [PATCH 3/4] dt-bindings: Clean-up schema errors due to missing 'addtionalProperties: false'
    . Signed-off-by: Rob Herring <robh@kernel.org>
    + Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
    . Cc: Linus Walleij <linus.walleij@linaro.org>
    . Cc: Kevin Hilman <khilman@baylibre.com>
    . Cc: Lee Jones <lee.jones@linaro.org>
    . Cc: dri-devel@lists.freedesktop.org
    . Cc: Thomas Gleixner <tglx@linutronix.de>
    . Cc: Bartosz Golaszewski <bgolaszewski@baylibre.com>
    . Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
    . Cc: linux-arm-kernel@lists.infradead.org
    . Cc: Lars-Peter Clausen <lars@metafoo.de>
    . Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
    . Cc: Hartmut Knaack <knaack.h@gmx.de>
    . Cc: Zhang Rui <rui.zhang@intel.com>
    . Cc: netdev@vger.kernel.org
    . Cc: Peter Meerwald-Stadler <pmeerw@pmeerw.net>
    . Cc: Mark Brown <broonie@kernel.org>
    . Cc: linux-amlogic@lists.infradead.org
    . Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
    . Cc: linux-clk@vger.kernel.org
    . Cc: linux-iio@vger.kernel.org
    . Cc: Jonathan Cameron <jic23@kernel.org>
    . Cc: Stephen Boyd <sboyd@kernel.org>
    . Cc: Liam Girdwood <lgirdwood@gmail.com>
    . Cc: linux-gpio@vger.kernel.org
    . Cc: linux-media@vger.kernel.org
    . Cc: "David S. Miller" <davem@davemloft.net>
    . Cc: Guillaume La Roque <glaroque@baylibre.com>
    . Cc: Neil Armstrong <narmstrong@baylibre.com>
    . Cc: linux-pm@vger.kernel.org
Running git --no-pager mailinfo --encoding=UTF-8 /tmp/tmpwpp4_xbx /tmp/tmpizm3fhvt
  [PATCH 4/4] dt-bindings: Add missing 'additionalProperties: false'
    . Signed-off-by: Rob Herring <robh@kernel.org>
    + Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
    + Acked-by: Mark Brown <broonie@kernel.org>
---
Total patches: 4
---
Cover: ./20200325_robh_dt_bindings_additionalproperties_clean_up.cover
 Link: https://lore.kernel.org/r/20200325220542.19189-1-robh@kernel.org
 Base: not found, sorry
       git checkout -b 20200325_robh_kernel_org master
       git am ./20200325_robh_dt_bindings_additionalproperties_clean_up.mbx

Re: [kernel.org users] b4 v0.3.4 released

[Rob Herring]

On Thu, Mar 26, 2020 at 10:04 AM Rob Herring <robh@kernel.org> wrote:
>
> On Mon, Mar 23, 2020 at 4:20 PM Konstantin Ryabitsev
> <konstantin@linuxfoundation.org> wrote:
> >
> > Hello:
> >
> > I pushed v0.3.4 to pypi, so if you're using it from there, you can run
> > "pip install --user --upgrade b4" to grab the latest version.
> >
> > Notable changes compared to 0.3.3:
> >
> > - deals properly with series versions that are only mentioned in the
> >   cover letter
> > - adds a b4.sh wrapper that lets you run b4 without needing to install
> >   with pip
> > - fixes a couple of backtraces due to bugs with patch attestation
> > - adds a caching layer to avoid hitting lore.kernel.org on repeating
> >   runs (e.g. when rerunning with -t, -s, -l, and similar)
> > - properly refuses to install on python version < 3.5
> > - uses more descriptive filenames for "b4 am" mboxes
>
> I'm seeing corruption of the subject on patches[1] with a comma in the
> subject (and I'd guess only ones longer than 70 something chars). It's
> fine on lore web interface, but it gets wrapped in the downloaded
> mbox. A log is attached. When it gets applied, a space is inserted
> after the comma:
>
> dt-bindings: iio/accel: Drop duplicate adi, adxl345/6 from trivial-devices.yaml

Nevermind. It's not b4, but the dri-devel list that is corrupting the subject.

https://lore.kernel.org/dri-devel/20200325220542.19189-2-robh@kernel.org/
https://lore.kernel.org/linux-devicetree/20200325220542.19189-2-robh@kernel.org/

Rob

> Interestingly, DT patchwork suffers from the same issue.
>
> Also, I got tripped up with missing acks again because dri-devel got
> picked and it moderates non-subscribers. A way to set the default
> project would be nice. Perhaps rather than a single default, making it
> a list of lists to try. Or trying lkml first might work better if
> we're just guessing.
>
> Rob
>
> [1] https://lore.kernel.org/linux-devicetree/20200325220542.19189-2-robh@kernel.org/

Re: [kernel.org users] b4 v0.3.4 released

[Konstantin Ryabitsev]

On Thu, Mar 26, 2020 at 10:11:44AM -0600, Rob Herring wrote:
> > I'm seeing corruption of the subject on patches[1] with a comma in 
> > the
> > subject (and I'd guess only ones longer than 70 something chars). It's
> > fine on lore web interface, but it gets wrapped in the downloaded
> > mbox. A log is attached. When it gets applied, a space is inserted
> > after the comma:
> >
> > dt-bindings: iio/accel: Drop duplicate adi, adxl345/6 from trivial-devices.yaml
> 
> Nevermind. It's not b4, but the dri-devel list that is corrupting the subject.
> 
> https://lore.kernel.org/dri-devel/20200325220542.19189-2-robh@kernel.org/
> https://lore.kernel.org/linux-devicetree/20200325220542.19189-2-robh@kernel.org/

I think it's actually Python's fault. I'm trying to isolate the 
operation that causes it. It makes sense that dri-devel also exhibits 
it, because it's a mailman-managed list.

-K

Re: [kernel.org users] b4 v0.3.4 released

[James Bottomley]

On Thu, 2020-03-26 at 12:16 -0400, Konstantin Ryabitsev wrote:
> On Thu, Mar 26, 2020 at 10:11:44AM -0600, Rob Herring wrote:
> > > I'm seeing corruption of the subject on patches[1] with a comma
> > > in the subject (and I'd guess only ones longer than 70 something
> > > chars). It's fine on lore web interface, but it gets wrapped in
> > > the downloaded mbox. A log is attached. When it gets applied, a
> > > space is inserted after the comma:
> > > 
> > > dt-bindings: iio/accel: Drop duplicate adi, adxl345/6 from
> > > trivial-devices.yaml
> > 
> > Nevermind. It's not b4, but the dri-devel list that is corrupting
> > the subject.
> > 
> > https://lore.kernel.org/dri-devel/20200325220542.19189-2-robh@kerne
> > l.org/
> > https://lore.kernel.org/linux-devicetree/20200325220542.19189-2-rob
> > h@kernel.org/
> 
> I think it's actually Python's fault. I'm trying to isolate the 
> operation that causes it. It makes sense that dri-devel also
> exhibits it, because it's a mailman-managed list.

It smells like a mishandling of RFC2822 header folding rules.

https://tools.ietf.org/html/rfc2822#section-2.2.3

Lots of email tools get this wrong as well, so they add an extra space
where the subject header got split by the wrapping because they expect
the header to split at a space and the space to be eaten by the split,
but that's not what the rules say.

James

Re: [kernel.org users] b4 v0.3.4 released

[Geert Uytterhoeven]

On Thu, Mar 26, 2020 at 5:27 PM James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Thu, 2020-03-26 at 12:16 -0400, Konstantin Ryabitsev wrote:
> > On Thu, Mar 26, 2020 at 10:11:44AM -0600, Rob Herring wrote:
> > > > I'm seeing corruption of the subject on patches[1] with a comma
> > > > in the subject (and I'd guess only ones longer than 70 something
> > > > chars). It's fine on lore web interface, but it gets wrapped in
> > > > the downloaded mbox. A log is attached. When it gets applied, a
> > > > space is inserted after the comma:
> > > >
> > > > dt-bindings: iio/accel: Drop duplicate adi, adxl345/6 from
> > > > trivial-devices.yaml
> > >
> > > Nevermind. It's not b4, but the dri-devel list that is corrupting
> > > the subject.
> > >
> > > https://lore.kernel.org/dri-devel/20200325220542.19189-2-robh@kerne
> > > l.org/
> > > https://lore.kernel.org/linux-devicetree/20200325220542.19189-2-rob
> > > h@kernel.org/
> >
> > I think it's actually Python's fault. I'm trying to isolate the
> > operation that causes it. It makes sense that dri-devel also
> > exhibits it, because it's a mailman-managed list.
>
> It smells like a mishandling of RFC2822 header folding rules.
>
> https://tools.ietf.org/html/rfc2822#section-2.2.3
>
> Lots of email tools get this wrong as well, so they add an extra space
> where the subject header got split by the wrapping because they expect
> the header to split at a space and the space to be eaten by the split,
> but that's not what the rules say.

Even formail -c gets it wrong, so my scripts have to postprocess the result.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [kernel.org users] b4 v0.3.4 released

[Konstantin Ryabitsev]

On Thu, Mar 26, 2020 at 12:16:22PM -0400, Konstantin Ryabitsev via Linux.Kernel.Org wrote:
> > > I'm seeing corruption of the subject on patches[1] with a comma in 
> > > the
> > > subject (and I'd guess only ones longer than 70 something chars). It's
> > > fine on lore web interface, but it gets wrapped in the downloaded
> > > mbox. A log is attached. When it gets applied, a space is inserted
> > > after the comma:
> > >
> > > dt-bindings: iio/accel: Drop duplicate adi, adxl345/6 from trivial-devices.yaml
> > 
> > Nevermind. It's not b4, but the dri-devel list that is corrupting the subject.
> > 
> > https://lore.kernel.org/dri-devel/20200325220542.19189-2-robh@kernel.org/
> > https://lore.kernel.org/linux-devicetree/20200325220542.19189-2-robh@kernel.org/
> 
> I think it's actually Python's fault. I'm trying to isolate the 
> operation that causes it. It makes sense that dri-devel also exhibits 
> it, because it's a mailman-managed list.

No, your initial assessment is correct -- this is done by the dri-devel 
list. I was confused by the caching layer that needs to be fixed to make 
sure that we can grab the same thread from multiple projects (I'll fix 
it today).

$ b4 am https://lore.kernel.org/linux-devicetree/20200325220542.19189-2-robh@kernel.org -C
...
  [PATCH 1/4] dt-bindings: iio/accel: Drop duplicate adi,adxl345/6 from trivial-devices.yaml
...

vs.

$ b4 am https://lore.kernel.org/dri-devel/20200325220542.19189-2-robh@kernel.org -C
...
  [PATCH 1/4] dt-bindings: iio/accel: Drop duplicate adi, adxl345/6 from trivial-devices.yaml
...

-K

Re: [kernel.org users] b4 v0.3.4 released

[Jason A. Donenfeld]

Looks like the auto project detection has issues:

zx2c4@thinkpad ~/Projects/linux $ b4 am -c
20200326080104.27286-16-masahiroy@kernel.org
Looking up https://lore.kernel.org/r/20200326080104.27286-16-masahiroy@kernel.org
Grabbing thread from lore.kernel.org
Reduced thread to strict matches only (12->11)
Checking for newer revisions on https://lore.kernel.org/linux-crypto/
Analyzing 11 messages in the thread
---
Writing ./v2_20200326_masahiroy_x86_crypto_remove_always_defined_config_as__and_cosolidate_kconfig_makefiles.mbx
 ERROR: missing [1/16]!
 ERROR: missing [2/16]!
 ERROR: missing [3/16]!
 ERROR: missing [4/16]!
 ERROR: missing [5/16]!
 [PATCH v2 06/16] x86: remove always-defined CONFIG_AS_SSSE3
 [PATCH v2 07/16] x86: remove always-defined CONFIG_AS_AVX
 ERROR: missing [8/16]!
 ERROR: missing [9/16]!
 ERROR: missing [10/16]!
 ERROR: missing [11/16]!
 [PATCH v2 12/16] crypto: x86 - rework configuration based on Kconfig
 ERROR: missing [13/16]!
 ERROR: missing [14/16]!
 [PATCH v2 15/16] x86: update AS_* macros to binutils >=2.23,
supporting ADX and AVX2
 [PATCH v2 16/16] crypto: x86 - clean up poly1305-x86_64-cryptogams.S
by 'make clean'
---
Total patches: 5
---
NOTE: Some trailers were sent to the cover letter:
     Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
     Acked-by: Ingo Molnar <mingo@kernel.org>
NOTE: Rerun with -t to apply them to all patches
---
WARNING: Thread incomplete!
Cover: ./v2_20200326_masahiroy_x86_crypto_remove_always_defined_config_as__and_cosolidate_kconfig_makefiles.cover
Link: https://lore.kernel.org/r/20200326080104.27286-1-masahiroy@kernel.org
Base: not found, sorry
      git checkout -b v2_20200326_masahiroy_kernel_org master
      git am ./v2_20200326_masahiroy_x86_crypto_remove_always_defined_config_as__and_cosolidate_kconfig_makefiles.mbx
zx2c4@thinkpad ~/Projects/linux $ b4 am -p linux-kernel -c
20200326080104.27286-16-masahiroy@kernel.org
Looking up https://lore.kernel.org/r/20200326080104.27286-16-masahiroy@kernel.org
Grabbing thread from lore.kernel.org
Server returned an error: 404
zx2c4@thinkpad ~/Projects/linux $ b4 am -p lkml -c
20200326080104.27286-16-masahiroy@kernel.org
Looking up https://lore.kernel.org/r/20200326080104.27286-16-masahiroy@kernel.org
Grabbing thread from lore.kernel.org
Checking for newer revisions on https://lore.kernel.org/lkml/
Analyzing 27 messages in the thread
---
Writing ./v2_20200326_masahiroy_x86_crypto_remove_always_defined_config_as__and_cosolidate_kconfig_makefiles.mbx
 [PATCH v2 01/16] lib/raid6/test: fix build on distros whose /bin/sh is not bash
 [PATCH v2 02/16] x86: remove unneeded defined(__ASSEMBLY__) check
from asm/dwarf2.h
 [PATCH v2 03/16] x86: remove always-defined CONFIG_AS_CFI
 [PATCH v2 04/16] x86: remove unneeded (CONFIG_AS_)CFI_SIGNAL_FRAME
 [PATCH v2 05/16] x86: remove always-defined CONFIG_AS_CFI_SECTIONS
 [PATCH v2 06/16] x86: remove always-defined CONFIG_AS_SSSE3
 [PATCH v2 07/16] x86: remove always-defined CONFIG_AS_AVX
 [PATCH v2 08/16] x86: replace arch macros from compiler with CONFIG_X86_{32,64}
 [PATCH v2 09/16] drm/i915: remove always-defined CONFIG_AS_MOVNTDQA
 [PATCH v2 10/16] x86: probe assembler capabilities via kconfig
instead of makefile
   + Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
 [PATCH v2 11/16] x86: add comments about the binutils version to
support code in as-instr
   + Acked-by: Nick Desaulniers <ndesaulniers@google.com>
 [PATCH v2 12/16] crypto: x86 - rework configuration based on Kconfig
 [PATCH v2 13/16] crypto: curve25519 - do not pollute dispatcher based
on assembler
 [PATCH v2 14/16] Documentation/changes: Raise minimum supported
binutils version to 2.23
 [PATCH v2 15/16] x86: update AS_* macros to binutils >=2.23,
supporting ADX and AVX2
 [PATCH v2 16/16] crypto: x86 - clean up poly1305-x86_64-cryptogams.S
by 'make clean'
---
Total patches: 16
---
NOTE: Some trailers were sent to the cover letter:
     Acked-by: Ingo Molnar <mingo@kernel.org>
     Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
NOTE: Rerun with -t to apply them to all patches
---
Cover: ./v2_20200326_masahiroy_x86_crypto_remove_always_defined_config_as__and_cosolidate_kconfig_makefiles.cover
Link: https://lore.kernel.org/r/20200326080104.27286-1-masahiroy@kernel.org
Base: not found, sorry
      git checkout -b v2_20200326_masahiroy_kernel_org master
      git am ./v2_20200326_masahiroy_x86_crypto_remove_always_defined_config_as__and_cosolidate_kconfig_makefiles.mbx

(Also: lore still calls linux-kernel "lkml" which is a bit confusing.)

Re: [kernel.org users] b4 v0.3.4 released

[Konstantin Ryabitsev]

On Thu, Mar 26, 2020 at 02:18:26PM -0600, Jason A. Donenfeld wrote:
> Looks like the auto project detection has issues:

It's not really auto-project detection. We use public-inbox itself to 
look for "any list where this message-id shows up". Currently, it does 
no thread completion matching and there is no way to set preference. I 
hope that we'll be able to implement these features in the future.

For now, I plan to use the following approach:

- when a thread is incomplete, look at other mailing lists where it was 
  sent (in to: and cc:)
- grab https://lore.kernel.org/lists.txt
- try to see if we have archives for any of those other lists
- try to backfill missing messages from other lists

This is not great, but it should hopefully help with this problem until 
public-inbox gains ability to return a thread collected from multiple 
lists (it should be able to do that once cross-list search becomes 
available).

> (Also: lore still calls linux-kernel "lkml" which is a bit confusing.)

I know, and I'll have some mechanism in place so both work soon.

-K

Re: [kernel.org users] b4 v0.3.4 released

[Konstantin Ryabitsev]

On Thu, 26 Mar 2020 at 16:18, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>
> Looks like the auto project detection has issues:
>
> zx2c4@thinkpad ~/Projects/linux $ b4 am -c
> 20200326080104.27286-16-masahiroy@kernel.org
> Looking up https://lore.kernel.org/r/20200326080104.27286-16-masahiroy@kernel.org
> Grabbing thread from lore.kernel.org
> Reduced thread to strict matches only (12->11)
> Checking for newer revisions on https://lore.kernel.org/linux-crypto/
> Analyzing 11 messages in the thread
> ---
> Writing ./v2_20200326_masahiroy_x86_crypto_remove_always_defined_config_as__and_cosolidate_kconfig_makefiles.mbx
>  ERROR: missing [1/16]!

The latest master is able to backfill missing patches from other
lore.kernel.org lists, so this should significantly improve this
particular problem. Now, if a patch is missing from the thread,
there's a good chance that it's missing from lore.kernel.org entirely.

I hope to release v0.4.0 some time in the next week or so, but if you
can't wait, you can always run it straight from the checkout.

Regards,
-K

Re: [kernel.org users] b4 v0.3.4 released

[Rob Herring]

On Thu, Mar 26, 2020 at 10:32 AM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
>
> On Thu, Mar 26, 2020 at 12:16:22PM -0400, Konstantin Ryabitsev via Linux.Kernel.Org wrote:
> > > > I'm seeing corruption of the subject on patches[1] with a comma in
> > > > the
> > > > subject (and I'd guess only ones longer than 70 something chars). It's
> > > > fine on lore web interface, but it gets wrapped in the downloaded
> > > > mbox. A log is attached. When it gets applied, a space is inserted
> > > > after the comma:
> > > >
> > > > dt-bindings: iio/accel: Drop duplicate adi, adxl345/6 from trivial-devices.yaml
> > >
> > > Nevermind. It's not b4, but the dri-devel list that is corrupting the subject.
> > >
> > > https://lore.kernel.org/dri-devel/20200325220542.19189-2-robh@kernel.org/
> > > https://lore.kernel.org/linux-devicetree/20200325220542.19189-2-robh@kernel.org/
> >
> > I think it's actually Python's fault. I'm trying to isolate the
> > operation that causes it. It makes sense that dri-devel also exhibits
> > it, because it's a mailman-managed list.
>
> No, your initial assessment is correct -- this is done by the dri-devel
> list. I was confused by the caching layer that needs to be fixed to make
> sure that we can grab the same thread from multiple projects (I'll fix
> it today).

Annoyed by this again, I dug into it some more. We're both right. :)

The problem is the email module in python2 which mailman2 uses.

On 2.7:
>>> h = email.header.Header()
>>> h.append('Subject: test,split on a long header laldkdsf dfsdakfds dskj dskfdj sdaklsdlkdfj dalkj sdlk f')
>>> h.encode()
'Subject: test,\n split on a long header laldkdsf dfsdakfds dskj
dskfdj sdaklsdlkdfj dalkj\n sdlk f'

On 3.8:
...
>>> h.encode()
'Subject: test,split on a long header laldkdsf dfsdakfds dskj dskfdj\n
sdaklsdlkdfj dalkj sdlk f'

Probably fixed in this commit which despite being 9 years ago, isn't
in the 2.7 branch:
https://github.com/python/cpython/commit/01581ee0b7c968adb987a36495af7ce5eb794d0d#diff-7d35ae5e9e22a15ee979f1cba58bc60a

I guess we'll just wait for mailman3 adoption. Maybe python2 EOL will help...

Rob