b4 v0.4.0 available with new features

b4 v0.4.0 available with new features

[Konstantin Ryabitsev]

[-- Attachment #1: Type: text/plain, Size: 3647 bytes --]

Hi, all:

I am happy to release b4 version 0.4.0 that contains a set of 
improvements and new features. To upgrade from pypi, run:

pip install --upgrade --user b4

Or you can clone the repository directly:
git clone https://git.kernel.org/pub/scm/utils/b4/b4.git -b stable-0.4.y b4

You can run ./b4.sh from the repository or set an alias/symlink to that 
wrapper.

# Improvements

- When a series is missing patches, b4 is now able to backfill them from 
  other mailing lists tracked on lore.kernel.org. This feature will be 
  improved when public-inbox is able to collect a thread from across all 
  sources.
- If both patch and metadata are identical between rerolls (v1 -> v2), 
  b4 will automatically carry over trailers from v1 to v2. This is handy 
  if there is an sob on [PATCH v2 7/15] from a maintainer that is 
  missing from an identical [PATCH v3 7/15]. In my observation, this 
  happens very rarely, though.
- Many bugfixes that were also backported into stable-0.3.y.

# New features

## b4 pr

You can now process pull requests in addition to patch series:

b4 pr <msgid>

  - downloads that message
  - parses the pull request
  - checks that the remote tip is where the message says it should be
  - makes sure the base-commit is present in the tree
  - makes sure the tip commit isn't already in one of the branches
  - checks if FETCH_HEAD already is at that commit
  - if the above two checks pass, performs a git fetch
  - runs pgp verify checks on FETCH_HEAD and shows the results

b4 pr --check <msgid>

  Runs all of the checks above, but doesn't perform the actual fetch.
  Useful if you don't remember if you've already processed a pull
  request or not.

b4 pr --explode <msgid>

  Runs basic sanity checks and then:

  - performs a git fetch
  - runs a "git format-patch" for base-commit..FETCH_HEAD
  - adds the same to/from/cc headers as in the pull request
  - properly threads each patch below the pull request
  - saves that into a <msgid>.mbox file

  The above is handy if you want to comment on something in a pull
  request and not have to hunt around for sender/cc information.

b4 pr --help

  - will show you more information about subcommands available

## b4 ty

Allows you to automate a lot of "thanks, applied" and "thanks, merged" 
feedback for series and pull requests that were fetched with "b4 am" and 
"b4 pr".

b4 ty --help

  - will show you information about subcommands available

### For patch series:

  - find all commits in the current branch that were made by you 
    (committer=user.email)
  - see if any of them match patch-id's or patch titles tracked from 
    previous "b4 am" runs
  - create an automatic reply message with the summary of the commit and 
    save it as a .thanks file in the output directory provided

### For pull requests
  - Match the tip commit in the pull request to your current branch
  - create an automatic reply message with the summary of the merge and 
    save it as a .thanks file in the output directory provided

### Sending .thanks files

We do not currently automatically send the .thanks files, though this 
functionality will be added in the future for the cases where we find a 
[sendemail] section in .gitconfig.

For the moment, you can send these using "git send-email", "sendmail", 
or you can use bsync or other tools to automatically upload them to the 
Drafts folder of your mail service provider.

# Acknowledgements

I would like to thank Mark Brown for being a willing guinea pig for 
trying out the "b4 ty" feature.

-K

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: [tools] b4 v0.4.0 available with new features

[Martin K. Petersen]

Hi Konstantin!

> ## b4 ty
>
> Allows you to automate a lot of "thanks, applied" and "thanks, merged"
> feedback for series and pull requests that were fetched with "b4 am"
> and "b4 pr".

I just merged a bunch of one-liners and the "thank you" quote ends up
being:

    > Fix the following coccicheck warning:
    > 
    > drivers/scsi/foo/bar/baz.c
    > 
    > Link: https://lore.kernel.org/r/...
    > Reported-by: ...
    > Reviewed-by: ...
    > Signed-off-by: ...
    > Signed-off-by: ...

It would be nice if the quoting function could ignore the trailers
portion of the mail and not just stuff below "---".

Thanks!

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [tools] b4 v0.4.0 available with new features

[Konstantin Ryabitsev]

On Fri, Apr 24, 2020 at 07:35:41PM -0400, Martin K. Petersen wrote:
> 
> Hi Konstantin!
> 
> > ## b4 ty
> >
> > Allows you to automate a lot of "thanks, applied" and "thanks, merged"
> > feedback for series and pull requests that were fetched with "b4 am"
> > and "b4 pr".
> 
> I just merged a bunch of one-liners and the "thank you" quote ends up
> being:
> 
>     > Fix the following coccicheck warning:
>     > 
>     > drivers/scsi/foo/bar/baz.c
>     > 
>     > Link: https://lore.kernel.org/r/...
>     > Reported-by: ...
>     > Reviewed-by: ...
>     > Signed-off-by: ...
>     > Signed-off-by: ...
> 
> It would be nice if the quoting function could ignore the trailers
> portion of the mail and not just stuff below "---".

Yep, good point. The latest master and stable-0.4.y should do a better 
job at making these quotes. Tested on:

https://lore.kernel.org/kernel-hardening/20200229231120.1147527-1-nivedita@alum.mit.edu/

The reply generated was:

-----
On Sat, 29 Feb 2020 18:11:20 -0500, Arvind Sankar wrote:
> This currently leaks kernel physical addresses into userspace.

Applied to mricon/linux.git master, thanks!
...

-----

(Note: The reply quote is generated during the "b4 am" run, so the 
changes will only be seen on newly fetched patches/series.)

Best,
-K

Re: [tools] b4 v0.4.0 available with new features

[Martin K. Petersen]

Konstantin,

> Yep, good point. The latest master and stable-0.4.y should do a better
> job at making these quotes. Tested on:

Works great, thank you!

Second issue:

I have had a few cases today where my merge edits meant that b4 ty
couldn't match the original patch-id up with a commit in my tree.

I can see the patches in question with b4 ty --list but b4 ty -s N
fails because no match can be found. It would be nice to have a way to
specify a commit hash and have b4 still write a .thanks file even when
the patch-id matching fails. Something like:

$ b4 ty -s 10 -c <SHA>

would produce the thank you note with <SHA> as the ${summary} commit
link.

It would also be great for the matching to be more forgiving (capturing
per-patch message-id and matching with Link: tag, for instance). But I
still think I might end up in situations where I would like to be able
to force a thank you note given a ty id number and a commit hash.

Just an idea...

Thanks!

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [tools] b4 v0.4.0 available with new features

[Konstantin Ryabitsev]

On Mon, Apr 27, 2020 at 10:12:07PM -0400, Martin K. Petersen wrote:
> Second issue:
> 
> I have had a few cases today where my merge edits meant that b4 ty
> couldn't match the original patch-id up with a commit in my tree.
> 
> I can see the patches in question with b4 ty --list but b4 ty -s N
> fails because no match can be found. It would be nice to have a way to
> specify a commit hash and have b4 still write a .thanks file even when
> the patch-id matching fails. Something like:
> 
> $ b4 ty -s 10 -c <SHA>
> 
> would produce the thank you note with <SHA> as the ${summary} commit
> link.
> 
> It would also be great for the matching to be more forgiving (capturing
> per-patch message-id and matching with Link: tag, for instance). But I
> still think I might end up in situations where I would like to be able
> to force a thank you note given a ty id number and a commit hash.

Interesting -- did you modify the subjects of those commits as well?  
We're already falling back to matching by subject when we can't find 
exact patch-id matches.

How about we create a .thanks file even if no match is found (when used 
with -s, not --auto), except the summary will omit the commit-id 
information. You can then add that information manually or just send 
without it. After all, if you know for sure that you've applied that 
patch, then we shouldn't be stopping you from notifying the submitter 
about it.

So:

b4 ty -s 1,2,3

will always generate a .thanks file for those series, and will omit 
comit-id information when not found.

Does that sound okay?

-K

Re: [tools] b4 v0.4.0 available with new features

[Martin K. Petersen]

Konstantin,

> Interesting -- did you modify the subjects of those commits as well?  
> We're already falling back to matching by subject when we can't find 
> exact patch-id matches.

I had a couple of cases where I didn't and still got failures. I tried
tweaking the b4 files in .local to match the updated subject to see if I
could convince it to proceed that way but no such luck.

In any case: I do modify subject and commit descriptions for a
substantial amount of the patches I apply. Lots of typos, missing scsi:
tag, wrong driver tag, excessively long lines, etc. I have a hook script
that does a bunch of sanity checking on each patch and won't let me
apply until all issues are addressed.

> b4 ty -s 1,2,3
>
> will always generate a .thanks file for those series, and will omit 
> comit-id information when not found.
>
> Does that sound okay?

Sure. If you could still emit the %{summary} boilerplate with XXXXXXXX
as commit hash so it's easy to update afterwards, that would be great.

Thanks!

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [kernel.org users] [tools] b4 v0.4.0 available with new features

[Jason Gunthorpe]

On Mon, Apr 27, 2020 at 10:12:07PM -0400, Martin K. Petersen wrote:

> It would also be great for the matching to be more forgiving (capturing
> per-patch message-id and matching with Link: tag, for instance). 

+1 on using the link tag it should be a uuid for the original message
to reply to

Jason

Re: [kernel.org users] b4 v0.4.0 available with new features

[Rob Herring]

On Fri, Apr 24, 2020 at 12:04 PM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
>
> Hi, all:
>
> I am happy to release b4 version 0.4.0 that contains a set of
> improvements and new features. To upgrade from pypi, run:

Thanks Konstantin! A couple of requests below.

> pip install --upgrade --user b4
>
> Or you can clone the repository directly:
> git clone https://git.kernel.org/pub/scm/utils/b4/b4.git -b stable-0.4.y b4
>
> You can run ./b4.sh from the repository or set an alias/symlink to that
> wrapper.
>
> # Improvements
>
> - When a series is missing patches, b4 is now able to backfill them from
>   other mailing lists tracked on lore.kernel.org. This feature will be
>   improved when public-inbox is able to collect a thread from across all
>   sources.
> - If both patch and metadata are identical between rerolls (v1 -> v2),
>   b4 will automatically carry over trailers from v1 to v2. This is handy
>   if there is an sob on [PATCH v2 7/15] from a maintainer that is
>   missing from an identical [PATCH v3 7/15]. In my observation, this
>   happens very rarely, though.

This is pretty common actually. At least common enough that I do a
check for this and have a semi-automated reply for it.

I worry that automatically fixing it silently will create yet another
case of tribal knowledge of maintainer practices/requirements for
submitters. Maintainer A using b4 doesn't care if tags are added, but
Maintainer B does care and gets grumpy. We need the tooling to promote
that submitters should add these tags as that is what works in either
case. It would be useful if b4 provided just the check as a separate
command/option for integrating into maintainers tools or maybe
submitter tools like checkpatch.pl.


More generally, as I've started to integrate b4 into my workflow,
there's a couple of useful pieces of functionality already in b4, but
not exposed directly on their own. It would be nice to have both
porcelain and low-level commands.

It would help for scripting if 'mbox' stdout was just the mbox file
name similar to git-format-patch output and send the rest to stderr.
While I have message-id and can construct the mbox name, there can be
some transformation on it.

It would help if the old versions of a series could optionally be
included in the mbox output.

Can the lore search code in get_newest_series be exposed as a command
taking a search string and outputting an mbox? Essentially, what the
lore web interface does with a search box and 'mbox.gz' button (I
tried and gave up doing that with curl).

Rob

Re: [kernel.org users] b4 v0.4.0 available with new features

[Jason Gunthorpe]

On Mon, May 04, 2020 at 12:09:49PM -0500, Rob Herring wrote:
> On Fri, Apr 24, 2020 at 12:04 PM Konstantin Ryabitsev
> <konstantin@linuxfoundation.org> wrote:
> >
> > Hi, all:
> >
> > I am happy to release b4 version 0.4.0 that contains a set of
> > improvements and new features. To upgrade from pypi, run:
> 
> Thanks Konstantin! A couple of requests below.
> 
> > pip install --upgrade --user b4
> >
> > Or you can clone the repository directly:
> > git clone https://git.kernel.org/pub/scm/utils/b4/b4.git -b stable-0.4.y b4
> >
> > You can run ./b4.sh from the repository or set an alias/symlink to that
> > wrapper.
> >
> > # Improvements
> >
> > - When a series is missing patches, b4 is now able to backfill them from
> >   other mailing lists tracked on lore.kernel.org. This feature will be
> >   improved when public-inbox is able to collect a thread from across all
> >   sources.
> > - If both patch and metadata are identical between rerolls (v1 -> v2),
> >   b4 will automatically carry over trailers from v1 to v2. This is handy
> >   if there is an sob on [PATCH v2 7/15] from a maintainer that is
> >   missing from an identical [PATCH v3 7/15]. In my observation, this
> >   happens very rarely, though.
> 
> This is pretty common actually. At least common enough that I do a
> check for this and have a semi-automated reply for it.
> 
> I worry that automatically fixing it silently will create yet another
> case of tribal knowledge of maintainer practices/requirements for
> submitters. Maintainer A using b4 doesn't care if tags are added, but
> Maintainer B does care and gets grumpy. We need the tooling to promote
> that submitters should add these tags as that is what works in either
> case. It would be useful if b4 provided just the check as a separate
> command/option for integrating into maintainers tools or maybe
> submitter tools like checkpatch.pl.

It would be helpful if b4 had a submitter focused command that would
sync the tags from the mailing list with the current series in git (or
whatever)

Jason

Re: [kernel.org users] b4 v0.4.0 available with new features

[Konstantin Ryabitsev]

On Mon, May 04, 2020 at 02:17:58PM -0300, Jason Gunthorpe wrote:
> > I worry that automatically fixing it silently will create yet 
> > another
> > case of tribal knowledge of maintainer practices/requirements for
> > submitters. Maintainer A using b4 doesn't care if tags are added, but
> > Maintainer B does care and gets grumpy. We need the tooling to promote
> > that submitters should add these tags as that is what works in either
> > case. It would be useful if b4 provided just the check as a separate
> > command/option for integrating into maintainers tools or maybe
> > submitter tools like checkpatch.pl.
> 
> It would be helpful if b4 had a submitter focused command that would
> sync the tags from the mailing list with the current series in git (or
> whatever)

Something like "auto-amend committed series with any newer trailers"?  
Not sure I'd be comfortable with that -- I am currently deliberately not 
performing any operations that directly modify git trees (beyond 
creating new branches). Maintainers are subtle and quick to anger, so 
I'm not sure I'm confident enough with my git skills to start rewriting 
history. :)

I think providing a Link: tag is sufficient in this case anyway. Even if 
some trailers were sent to the list after a series was committed, the 
record of that trailer still exists, just external to the git repo.

-K

Re: [kernel.org users] b4 v0.4.0 available with new features

[Jason Gunthorpe]

On Mon, May 04, 2020 at 04:09:15PM -0400, Konstantin Ryabitsev wrote:
> On Mon, May 04, 2020 at 02:17:58PM -0300, Jason Gunthorpe wrote:
> > > I worry that automatically fixing it silently will create yet 
> > > another
> > > case of tribal knowledge of maintainer practices/requirements for
> > > submitters. Maintainer A using b4 doesn't care if tags are added, but
> > > Maintainer B does care and gets grumpy. We need the tooling to promote
> > > that submitters should add these tags as that is what works in either
> > > case. It would be useful if b4 provided just the check as a separate
> > > command/option for integrating into maintainers tools or maybe
> > > submitter tools like checkpatch.pl.
> > 
> > It would be helpful if b4 had a submitter focused command that would
> > sync the tags from the mailing list with the current series in git (or
> > whatever)
> 
> Something like "auto-amend committed series with any newer
> trailers"?

Yes.

> the net result Maintainers are subtle and quick to anger, so 
> I'm not sure I'm confident enough with my git skills to start rewriting 
> history. :)

Ah! I have something for this:

https://github.com/jgunthorpe/Kernel-Maintainer-Tools/blob/master/gj_tools/cmd_gerrit.py#L98

It directly rewrites history, specifically the commit message ..

> I think providing a Link: tag is sufficient in this case anyway. Even if 
> some trailers were sent to the list after a series was committed, the 
> record of that trailer still exists, just external to the git repo.

Many maintainers will be unhappy if someone submits v2 of a series and
does not carry forwards the tags from v1.

Jason

Re: [kernel.org users] b4 v0.4.0 available with new features

[Konstantin Ryabitsev]

On Mon, May 04, 2020 at 12:09:49PM -0500, Rob Herring wrote:
> >
> > - When a series is missing patches, b4 is now able to backfill them from
> >   other mailing lists tracked on lore.kernel.org. This feature will be
> >   improved when public-inbox is able to collect a thread from across all
> >   sources.
> > - If both patch and metadata are identical between rerolls (v1 -> v2),
> >   b4 will automatically carry over trailers from v1 to v2. This is handy
> >   if there is an sob on [PATCH v2 7/15] from a maintainer that is
> >   missing from an identical [PATCH v3 7/15]. In my observation, this
> >   happens very rarely, though.
> 
> This is pretty common actually. At least common enough that I do a
> check for this and have a semi-automated reply for it.

All three things have to be exactly the same:

- subject
- commit message
- patch contents (to the last bit)

The series must also be within the same thread, so if v2 is posted as a 
separate thread to v1, then this won't trigger.

I ran quite a few tests on series being posted to LKML, and there was 
maybe one instance where the trailer to a single patch ended up carried 
over between revisions.

> I worry that automatically fixing it silently will create yet another
> case of tribal knowledge of maintainer practices/requirements for
> submitters. Maintainer A using b4 doesn't care if tags are added, but
> Maintainer B does care and gets grumpy. We need the tooling to promote
> that submitters should add these tags as that is what works in either
> case. It would be useful if b4 provided just the check as a separate
> command/option for integrating into maintainers tools or maybe
> submitter tools like checkpatch.pl.

I see it as a case of "what you get is what was meant" -- if someone 
sends a trailer to a specific patch, and that patch didn't change at all 
between revisions, then we can safely assume that the author's trailer 
can be carried over to the new series revision. In your example, 
"Maintainer B" shouldn't get grumpy, because they should be doing the 
exact same thing anyway -- the reviewer has already sent their feedback 
about that patch, so it's the maintainer's duty to carry it over across 
revisions. In fact, many submitters already include previously received 
trailers with the new series revision, so I would argue that b4 isn't 
doing anything the maintainer wouldn't be doing in the first place.

> More generally, as I've started to integrate b4 into my workflow,
> there's a couple of useful pieces of functionality already in b4, but
> not exposed directly on their own. It would be nice to have both
> porcelain and low-level commands.
> 
> It would help for scripting if 'mbox' stdout was just the mbox file
> name similar to git-format-patch output and send the rest to stderr.
> While I have message-id and can construct the mbox name, there can be
> some transformation on it.
> 
> It would help if the old versions of a series could optionally be
> included in the mbox output.
> 
> Can the lore search code in get_newest_series be exposed as a command
> taking a search string and outputting an mbox? Essentially, what the
> lore web interface does with a search box and 'mbox.gz' button (I
> tried and gave up doing that with curl).

A lot of search stuff is currently very kludgy in b4 (e.g. we're abusing 
RSS search feeds for it). The situation will be very much improved once 
some of the necessary changes are implemented on the public-inbox side 
of things:

- unified search across all mailing lists
- ability to reconstitute a thread from multiple lists/sources
- proper query and subscription API

Looks like these features will be landing in public-inbox some time in 
the near future and that would vastly expand what we can do with 
search-based queries. I'd rather wait for that to happen instead of 
creating more kludgy RSS abuse.

I'll see what I can do about making b4 more pipe-friendly for some 
operations.

-K

Re: [kernel.org users] b4 v0.4.0 available with new features

[Jason Gunthorpe]

On Mon, May 04, 2020 at 04:03:21PM -0400, Konstantin Ryabitsev wrote:
> On Mon, May 04, 2020 at 12:09:49PM -0500, Rob Herring wrote:
> > >
> > > - When a series is missing patches, b4 is now able to backfill them from
> > >   other mailing lists tracked on lore.kernel.org. This feature will be
> > >   improved when public-inbox is able to collect a thread from across all
> > >   sources.
> > > - If both patch and metadata are identical between rerolls (v1 -> v2),
> > >   b4 will automatically carry over trailers from v1 to v2. This is handy
> > >   if there is an sob on [PATCH v2 7/15] from a maintainer that is
> > >   missing from an identical [PATCH v3 7/15]. In my observation, this
> > >   happens very rarely, though.
> > 
> > This is pretty common actually. At least common enough that I do a
> > check for this and have a semi-automated reply for it.
> 
> All three things have to be exactly the same:
> 
> - subject
> - commit message
> - patch contents (to the last bit)
> 
> The series must also be within the same thread, so if v2 is posted as a 
> separate thread to v1, then this won't trigger.

Are we supposed to thread v2 under v1 ? I though no?

Jason

Re: [kernel.org users] b4 v0.4.0 available with new features

[Rob Herring]

On Mon, May 4, 2020 at 3:03 PM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
>
> On Mon, May 04, 2020 at 12:09:49PM -0500, Rob Herring wrote:
> > >
> > > - When a series is missing patches, b4 is now able to backfill them from
> > >   other mailing lists tracked on lore.kernel.org. This feature will be
> > >   improved when public-inbox is able to collect a thread from across all
> > >   sources.
> > > - If both patch and metadata are identical between rerolls (v1 -> v2),
> > >   b4 will automatically carry over trailers from v1 to v2. This is handy
> > >   if there is an sob on [PATCH v2 7/15] from a maintainer that is
> > >   missing from an identical [PATCH v3 7/15]. In my observation, this
> > >   happens very rarely, though.
> >
> > This is pretty common actually. At least common enough that I do a
> > check for this and have a semi-automated reply for it.
>
> All three things have to be exactly the same:
>
> - subject
> - commit message
> - patch contents (to the last bit)
>
> The series must also be within the same thread, so if v2 is posted as a
> separate thread to v1, then this won't trigger.

Ah, that's why you don't see much. I think that's the exception at
least for what I see. I know I've seen maintainers asking for new
versions to *not* be threaded. I don't recall any asking for
threading. I'd guess a majority don't care. To quote
submitting-patches.rst:

"However, for a multi-patch series, it is generally best to avoid
using In-Reply-To: to link to older versions of the series."

My current check just looks for matching $subject and my tag. Adding a
PW diff hash check would be a good addition because my current
checking is manual.

> I ran quite a few tests on series being posted to LKML, and there was
> maybe one instance where the trailer to a single patch ended up carried
> over between revisions.
>
> > I worry that automatically fixing it silently will create yet another
> > case of tribal knowledge of maintainer practices/requirements for
> > submitters. Maintainer A using b4 doesn't care if tags are added, but
> > Maintainer B does care and gets grumpy. We need the tooling to promote
> > that submitters should add these tags as that is what works in either
> > case. It would be useful if b4 provided just the check as a separate
> > command/option for integrating into maintainers tools or maybe
> > submitter tools like checkpatch.pl.
>
> I see it as a case of "what you get is what was meant" -- if someone
> sends a trailer to a specific patch, and that patch didn't change at all
> between revisions, then we can safely assume that the author's trailer
> can be carried over to the new series revision. In your example,
> "Maintainer B" shouldn't get grumpy, because they should be doing the
> exact same thing anyway -- the reviewer has already sent their feedback
> about that patch, so it's the maintainer's duty to carry it over across
> revisions. In fact, many submitters already include previously received
> trailers with the new series revision, so I would argue that b4 isn't
> doing anything the maintainer wouldn't be doing in the first place.

If the previous version is sitting right in front of me in the same
thread, then yes I might remember tags on prior versions. Otherwise, I
don't remember what happened last week.

The accepted rule is the submitter should apply the tags when
submitting a new version. That's why many times they are already
included, but the question is what to do when not included. The
tooling for submitters should tell and/or help them to do things
correctly, and maintainer tools should verify and educate.

> > More generally, as I've started to integrate b4 into my workflow,
> > there's a couple of useful pieces of functionality already in b4, but
> > not exposed directly on their own. It would be nice to have both
> > porcelain and low-level commands.
> >
> > It would help for scripting if 'mbox' stdout was just the mbox file
> > name similar to git-format-patch output and send the rest to stderr.
> > While I have message-id and can construct the mbox name, there can be
> > some transformation on it.
> >
> > It would help if the old versions of a series could optionally be
> > included in the mbox output.
> >
> > Can the lore search code in get_newest_series be exposed as a command
> > taking a search string and outputting an mbox? Essentially, what the
> > lore web interface does with a search box and 'mbox.gz' button (I
> > tried and gave up doing that with curl).
>
> A lot of search stuff is currently very kludgy in b4 (e.g. we're abusing
> RSS search feeds for it). The situation will be very much improved once
> some of the necessary changes are implemented on the public-inbox side
> of things:
>
> - unified search across all mailing lists
> - ability to reconstitute a thread from multiple lists/sources
> - proper query and subscription API
>
> Looks like these features will be landing in public-inbox some time in
> the near future and that would vastly expand what we can do with
> search-based queries. I'd rather wait for that to happen instead of
> creating more kludgy RSS abuse.

Okay. I can identify with wanting to push features into an upstream
dependency. ;)


Rob

Re: [kernel.org users] b4 v0.4.0 available with new features

[Rob Herring]

On Fri, Apr 24, 2020 at 12:04 PM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
>
> Hi, all:
>
> I am happy to release b4 version 0.4.0 that contains a set of
> improvements and new features. To upgrade from pypi, run:
>
> pip install --upgrade --user b4
>
> Or you can clone the repository directly:
> git clone https://git.kernel.org/pub/scm/utils/b4/b4.git -b stable-0.4.y b4
>
> You can run ./b4.sh from the repository or set an alias/symlink to that
> wrapper.
>
> # Improvements
>
> - When a series is missing patches, b4 is now able to backfill them from
>   other mailing lists tracked on lore.kernel.org. This feature will be
>   improved when public-inbox is able to collect a thread from across all
>   sources.
> - If both patch and metadata are identical between rerolls (v1 -> v2),
>   b4 will automatically carry over trailers from v1 to v2. This is handy
>   if there is an sob on [PATCH v2 7/15] from a maintainer that is
>   missing from an identical [PATCH v3 7/15]. In my observation, this
>   happens very rarely, though.
> - Many bugfixes that were also backported into stable-0.3.y.

I found a minor issue on patches with no 'To' header. They end up with this:

To: unlisted-recipients: no To-header on input <;

However, that appears to be a public-inbox problem as it's on there
too. Here's an example[1].

Rob

[1] https://lore.kernel.org/linux-devicetree/20200506010809.6348-2-i.mikhaylov@yadro.com/

Re: [kernel.org users] b4 v0.4.0 available with new features

[Konstantin Ryabitsev]

On Wed, May 06, 2020 at 02:23:16PM -0500, Rob Herring wrote:
> I found a minor issue on patches with no 'To' header. They end up with 
> this:
> 
> To: unlisted-recipients: no To-header on input <;
> 
> However, that appears to be a public-inbox problem as it's on there
> too. Here's an example[1].

Interesting. I'm not seeing any strange output from that series. "b4 am" 
and "b4 ty" seem to be generating proper things.

Can you try with the latest master/stable-0.4.y and show me the place 
where this is still showing up? When I do it, the .mbx file is just 
missing the "To:" header entirely.

Regards,
-K

Re: [kernel.org users] b4 v0.4.0 available with new features

[Rob Herring]

On Thu, May 7, 2020 at 3:20 PM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
>
> On Wed, May 06, 2020 at 02:23:16PM -0500, Rob Herring wrote:
> > I found a minor issue on patches with no 'To' header. They end up with
> > this:
> >
> > To: unlisted-recipients: no To-header on input <;
> >
> > However, that appears to be a public-inbox problem as it's on there
> > too. Here's an example[1].
>
> Interesting. I'm not seeing any strange output from that series. "b4 am"
> and "b4 ty" seem to be generating proper things.

Yes, 'b4 am' is correct, but 'b4 mbox' is not. That's with 0.4.2.

Rob

Re: [kernel.org users] b4 v0.4.0 available with new features

[Rob Herring]

On Tue, May 12, 2020 at 1:07 PM Rob Herring <robh@kernel.org> wrote:
>
> On Thu, May 7, 2020 at 3:20 PM Konstantin Ryabitsev
> <konstantin@linuxfoundation.org> wrote:
> >
> > On Wed, May 06, 2020 at 02:23:16PM -0500, Rob Herring wrote:
> > > I found a minor issue on patches with no 'To' header. They end up with
> > > this:
> > >
> > > To: unlisted-recipients: no To-header on input <;
> > >
> > > However, that appears to be a public-inbox problem as it's on there
> > > too. Here's an example[1].
> >
> > Interesting. I'm not seeing any strange output from that series. "b4 am"
> > and "b4 ty" seem to be generating proper things.
>
> Yes, 'b4 am' is correct, but 'b4 mbox' is not. That's with 0.4.2.

This is still an issue for me. I'm on current master, commit
ca902b59c3d1 ("Fix mbox naming inconsistencies").

Rob

Re: [kernel.org users] b4 v0.4.0 available with new features

[Konstantin Ryabitsev]

On Tue, Jun 09, 2020 at 04:24:29PM -0600, Rob Herring wrote:
> > > On Wed, May 06, 2020 at 02:23:16PM -0500, Rob Herring wrote:
> > > > I found a minor issue on patches with no 'To' header. They end up with
> > > > this:
> > > >
> > > > To: unlisted-recipients: no To-header on input <;
> > > >
> > > > However, that appears to be a public-inbox problem as it's on there
> > > > too. Here's an example[1].
> > >
> > > Interesting. I'm not seeing any strange output from that series. "b4 am"
> > > and "b4 ty" seem to be generating proper things.
> >
> > Yes, 'b4 am' is correct, but 'b4 mbox' is not. That's with 0.4.2.
> 
> This is still an issue for me. I'm on current master, commit
> ca902b59c3d1 ("Fix mbox naming inconsistencies").

The reason I haven't really done anything about it is because we 
currently don't do anything at all to messages retrieved via "b4 mbox" 
(as opposed to "b4 am"). We just grab them as-is from public-inbox, and 
that's how those messages are in the archive:

https://lore.kernel.org/linux-devicetree/20200506010809.6348-2-i.mikhaylov@yadro.com/raw

As far as I can tell, this is inserted by some MTA even before it gets 
to vger, as it shows up in other public archives that I could find. The 
MUAs should properly deal with this situation, so my current approach is 
to treat this as "broken as designed".

-K