New version of lore available for preview

New version of lore available for preview

[Konstantin Ryabitsev]

[-- Attachment #1: Type: text/plain, Size: 1441 bytes --]

Hi, all:

We will be deploying a newer version of lore.kernel.org soon, with the
following significant improvements:

1. The default view becomes /all/, which displays messages from all lists and
   allows the usual query/retrieval operations across all sources. This
   achieves multiple goals, such as ability to aggregate threads regardless on
   which list the follow-up discussion took place and will automatically
   backfill missing messages if they are present on one list but not on
   another.
2. Instead of a single system, lore.kernel.org will be mirrored across three
   nodes currently serving git.kernel.org. This both reduces single points of
   failure and makes things faster for folks in EU/APAC.
3. The web interface will now respect your browser-preferred color scheme and
   will display dark mode interface if that's how you like it.

You can access the new version at https://x-lore.kernel.org/ and it should be
fully usable for daily operations.

You can use x-lore with b4, though I strongly recommend switching to master
branch in order to better deal with some of the changes happening on the
server-side. Simply add the following to your ~/.gitconfig:

[b4]
  midmask = https://x-lore.kernel.org/r/%s

Please report any problems or hiccups that you discover. There is no specific
ETA for the new version go-live, but if all goes well, the hope is to have it
in place by end of August.

Best regards,
-Konstantin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: New version of lore available for preview

[Jason A. Donenfeld]

Planning on having /linux-kernel/ be canonical, with /lkml/
redirecting to it, for x-lore?

Re: New version of lore available for preview

[Konstantin Ryabitsev]

On Wed, Aug 18, 2021 at 09:09:13PM +0200, Jason A. Donenfeld wrote:
> Planning on having /linux-kernel/ be canonical, with /lkml/
> redirecting to it, for x-lore?

It's not as straightforward as that, unfortunately (it would break existing
mirrors), but we'll now serve the same content from both locations.

-K

Re: New version of lore available for preview

[Junio C Hamano]

https://x-lore.kernel.org/all/_/text/color/ mentions public-inbox;
those who are not familiar with the history of software that powers
lore may find it confusing.

2021年8月18日(水) 12:48 Konstantin Ryabitsev <konstantin@linuxfoundation.org>:
>
> On Wed, Aug 18, 2021 at 09:09:13PM +0200, Jason A. Donenfeld wrote:
> > Planning on having /linux-kernel/ be canonical, with /lkml/
> > redirecting to it, for x-lore?
>
> It's not as straightforward as that, unfortunately (it would break existing
> mirrors), but we'll now serve the same content from both locations.
>
> -K
>

Re: New version of lore available for preview

[Rob Herring]

On Wed, Aug 18, 2021 at 2:07 PM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
>
> Hi, all:
>
> We will be deploying a newer version of lore.kernel.org soon, with the
> following significant improvements:
>
> 1. The default view becomes /all/, which displays messages from all lists and
>    allows the usual query/retrieval operations across all sources. This
>    achieves multiple goals, such as ability to aggregate threads regardless on
>    which list the follow-up discussion took place and will automatically
>    backfill missing messages if they are present on one list but not on
>    another.
> 2. Instead of a single system, lore.kernel.org will be mirrored across three
>    nodes currently serving git.kernel.org. This both reduces single points of
>    failure and makes things faster for folks in EU/APAC.
> 3. The web interface will now respect your browser-preferred color scheme and
>    will display dark mode interface if that's how you like it.
>
> You can access the new version at https://x-lore.kernel.org/ and it should be
> fully usable for daily operations.
>
> You can use x-lore with b4, though I strongly recommend switching to master
> branch in order to better deal with some of the changes happening on the
> server-side. Simply add the following to your ~/.gitconfig:
>
> [b4]
>   midmask = https://x-lore.kernel.org/r/%s
>
> Please report any problems or hiccups that you discover. There is no specific
> ETA for the new version go-live, but if all goes well, the hope is to have it
> in place by end of August.

I've switched over and so far, so good. I'm happy that now I get full
threads regardless of the sender's choice of MLs and no more mangling
of subjects with commas in them (DT patches frequently have commas).

Thanks!,
Rob

Re: New version of lore available for preview

[Konstantin Ryabitsev]

On Wed, Aug 25, 2021 at 11:30:37AM -0500, Rob Herring wrote:
> > You can use x-lore with b4, though I strongly recommend switching to master
> > branch in order to better deal with some of the changes happening on the
> > server-side. Simply add the following to your ~/.gitconfig:
> >
> > [b4]
> >   midmask = https://x-lore.kernel.org/r/%s
> >
> > Please report any problems or hiccups that you discover. There is no specific
> > ETA for the new version go-live, but if all goes well, the hope is to have it
> > in place by end of August.
> 
> I've switched over and so far, so good. I'm happy that now I get full
> threads regardless of the sender's choice of MLs and no more mangling
> of subjects with commas in them (DT patches frequently have commas).

Great to hear, thank you for sharing that. I've been running b4 in a loop
against /all/ as well, and so far so good on my end, too. The plan is to
switch over to the new lore on Monday, August 30, but I need to release b4 0.8
first, which is needed to better work with the upstream public-inbox changes.

Best regards,
-K

Re: New version of lore available for preview

[Josh Triplett]

On Wed, Aug 25, 2021 at 12:44:59PM -0400, Konstantin Ryabitsev wrote:
> Great to hear, thank you for sharing that. I've been running b4 in a loop
> against /all/ as well, and so far so good on my end, too. The plan is to
> switch over to the new lore on Monday, August 30, but I need to release b4 0.8
> first, which is needed to better work with the upstream public-inbox changes.

Would you consider applying the following patch, to improve the mobile layout
on x-lore? Without this change, the mail listing pages are much narrower than
the device screen, making them hard to read. This *should* make the pages size
perfectly to the device.

---- >8 -----
From bebb1f75e90dd25492430004fee916e92c5eec0a Mon Sep 17 00:00:00 2001
Message-Id: <bebb1f75e90dd25492430004fee916e92c5eec0a.1629927361.git.josh@joshtriplett.org>
From: Josh Triplett <josh@joshtriplett.org>
Date: Tue, 24 Aug 2021 13:00:06 -0700
Subject: [PATCH] Display shorter public-inbox.org URL in the footer, to
 improve mobile layout

The long un-wrappable onion URL in the footer causes automatic
width-detection logic in mobile browsers to make the whole page as wide
as that URL, rather than a width suitable for displaying the rest of the
content. This makes the rest of the page substantially narrower than the
device.

Display the shorter public-inbox.org URL, to make page layout look
better on mobile devices.

Signed-off-by: Josh Triplett <josh@joshtriplett.org>
---
 lib/PublicInbox/WwwStream.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/WwwStream.pm b/lib/PublicInbox/WwwStream.pm
index 2f8212d4..474b3504 100644
--- a/lib/PublicInbox/WwwStream.pm
+++ b/lib/PublicInbox/WwwStream.pm
@@ -12,8 +12,8 @@ our @EXPORT_OK = qw(html_oneshot);
 use bytes (); # length
 use PublicInbox::Hval qw(ascii_html prurl ts2str);
 our $TOR_URL = 'https://www.torproject.org/';
-our $CODE_URL = [ qw(http://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/public-inbox.git
-	https://public-inbox.org/public-inbox.git) ];
+our $CODE_URL = [ qw(https://public-inbox.org/public-inbox.git
+	http://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/public-inbox.git) ];
 
 sub base_url ($) {
 	my $ctx = shift;
-- 
2.33.0

Re: New version of lore available for preview

[Konstantin Ryabitsev]

On Wed, Aug 25, 2021 at 02:39:29PM -0700, Josh Triplett wrote:
> On Wed, Aug 25, 2021 at 12:44:59PM -0400, Konstantin Ryabitsev wrote:
> > Great to hear, thank you for sharing that. I've been running b4 in a loop
> > against /all/ as well, and so far so good on my end, too. The plan is to
> > switch over to the new lore on Monday, August 30, but I need to release b4 0.8
> > first, which is needed to better work with the upstream public-inbox changes.
> 
> Would you consider applying the following patch, to improve the mobile layout
> on x-lore? Without this change, the mail listing pages are much narrower than
> the device screen, making them hard to read. This *should* make the pages size
> perfectly to the device.

I don't want to diverge from upstream, but I'm working with them on this
issue. My hope is to move the entire basement view (with mirroring and cloning
instructions) onto a separate page so it doesn't interfere with thread
rendering:

https://public-inbox.org/meta/20210826132747.6gxuwnhftyf7c6hp@nitro.local/

-K

Re: New version of lore available for preview

[Josh Triplett]

On Thu, Aug 26, 2021 at 10:04:27AM -0400, Konstantin Ryabitsev wrote:
> On Wed, Aug 25, 2021 at 02:39:29PM -0700, Josh Triplett wrote:
> > On Wed, Aug 25, 2021 at 12:44:59PM -0400, Konstantin Ryabitsev wrote:
> > > Great to hear, thank you for sharing that. I've been running b4 in a loop
> > > against /all/ as well, and so far so good on my end, too. The plan is to
> > > switch over to the new lore on Monday, August 30, but I need to release b4 0.8
> > > first, which is needed to better work with the upstream public-inbox changes.
> > 
> > Would you consider applying the following patch, to improve the mobile layout
> > on x-lore? Without this change, the mail listing pages are much narrower than
> > the device screen, making them hard to read. This *should* make the pages size
> > perfectly to the device.
> 
> I don't want to diverge from upstream, but I'm working with them on this
> issue. My hope is to move the entire basement view (with mirroring and cloning
> instructions) onto a separate page so it doesn't interfere with thread
> rendering:
> 
> https://public-inbox.org/meta/20210826132747.6gxuwnhftyf7c6hp@nitro.local/

Sounds reasonable; a separate page for that footer information seems
like a good idea anyway.

Re: New version of lore available for preview

[Jason Gunthorpe]

On Wed, Aug 18, 2021 at 03:07:50PM -0400, Konstantin Ryabitsev wrote:

> Please report any problems or hiccups that you discover. There is no specific
> ETA for the new version go-live, but if all goes well, the hope is to have it
> in place by end of August.

Hi Konstantin,

The https://lore.kernel.org/lists.txt URL now gives a 404, is this
expected?

If so does someone have a fix for lorifier.py? It is very unhappy to lose
this URL..

Thanks,
Jason

Re: New version of lore available for preview

[Konstantin Ryabitsev]

On Thu, Sep 02, 2021 at 04:53:32PM -0300, Jason Gunthorpe wrote:
> > Please report any problems or hiccups that you discover. There is no specific
> > ETA for the new version go-live, but if all goes well, the hope is to have it
> > in place by end of August.
> 
> Hi Konstantin,
> 
> The https://lore.kernel.org/lists.txt URL now gives a 404, is this
> expected?
> 
> If so does someone have a fix for lorifier.py? It is very unhappy to lose
> this URL..

I forgot about the lorifier. However, the upside is that you don't really need
the lists.txt URL any more, as you can just blanket aim things at
lore.kernel.org/all/<message-id> -- if it's on lore, it will be there.

I pushed a quick fix here:
https://github.com/mricon/lorifier/

If you really want to only show the link if a message is actually on lore,
it's easy to add a HEAD call to see if we get a 404 or not, so let me know if
it's something you'd need.

-K

Re: New version of lore available for preview

[Masahiro Yamada]

On Fri, Sep 3, 2021 at 5:14 AM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
>
> On Thu, Sep 02, 2021 at 04:53:32PM -0300, Jason Gunthorpe wrote:
> > > Please report any problems or hiccups that you discover. There is no specific
> > > ETA for the new version go-live, but if all goes well, the hope is to have it
> > > in place by end of August.
> >
> > Hi Konstantin,
> >
> > The https://lore.kernel.org/lists.txt URL now gives a 404, is this
> > expected?
> >
> > If so does someone have a fix for lorifier.py? It is very unhappy to lose
> > this URL..
>
> I forgot about the lorifier. However, the upside is that you don't really need
> the lists.txt URL any more, as you can just blanket aim things at
> lore.kernel.org/all/<message-id> -- if it's on lore, it will be there.
>
> I pushed a quick fix here:
> https://github.com/mricon/lorifier/
>
> If you really want to only show the link if a message is actually on lore,
> it's easy to add a HEAD call to see if we get a 404 or not, so let me know if
> it's something you'd need.
>
> -K
>


I used to use
  https://lore.kernel.org/patchwork/project/lkml/list/
to pick up patches sent to LKML, but it is gone now.


I use
  https://patchwork.kernel.org/
to pick up patches per subsystem,
but LKML is missing there.

Re: New version of lore available for preview

[Jason Gunthorpe]

On Thu, Sep 02, 2021 at 04:14:02PM -0400, Konstantin Ryabitsev wrote:
> On Thu, Sep 02, 2021 at 04:53:32PM -0300, Jason Gunthorpe wrote:
> > > Please report any problems or hiccups that you discover. There is no specific
> > > ETA for the new version go-live, but if all goes well, the hope is to have it
> > > in place by end of August.
> > 
> > Hi Konstantin,
> > 
> > The https://lore.kernel.org/lists.txt URL now gives a 404, is this
> > expected?
> > 
> > If so does someone have a fix for lorifier.py? It is very unhappy to lose
> > this URL..
> 
> I forgot about the lorifier. However, the upside is that you don't really need
> the lists.txt URL any more, as you can just blanket aim things at
> lore.kernel.org/all/<message-id> -- if it's on lore, it will be there.

What is the difference between /all/ and /r/ ? They seem to give the
same page now?

> I pushed a quick fix here:
> https://github.com/mricon/lorifier/

Thanks! Without that it is completely broken if you don't already have
a copy of the lists.txt

Jason

Re: New version of lore available for preview

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 486 bytes --]

On Fri, Sep 03, 2021 at 07:44:35AM +0900, Masahiro Yamada wrote:

> I used to use
>   https://lore.kernel.org/patchwork/project/lkml/list/
> to pick up patches sent to LKML, but it is gone now.

> I use
>   https://patchwork.kernel.org/
> to pick up patches per subsystem,
> but LKML is missing there.

I used to do that too but at some point I switched to using b4 which
does the same thing with downloading patches and tags but uses lore as
the backend instead of a patchwork client.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: New version of lore available for preview

[Konstantin Ryabitsev]

On Fri, Sep 03, 2021 at 07:44:35AM +0900, Masahiro Yamada wrote:
> I used to use
>   https://lore.kernel.org/patchwork/project/lkml/list/
> to pick up patches sent to LKML, but it is gone now.

As Mark has suggested, just use b4 to do the same. Running a patchwork
instance just for the purpose of retrieving a few patches was extremely
cost-inefficient, especially alongside the same functionality being offered by
public-inbox and b4.

You can install b4 from pypi (pip install b4), or you can clone the git
repository and use it straight from there:

https://git.kernel.org/pub/scm/utils/b4/b4.git

To retrieve a mbox file ready to be applied to a tree, you just need to know
the message-ID of the message (any message in the tread will do), and then
just run:

b4 am [that-message-id]

The output should tell you what to do next.

-K

Re: New version of lore available for preview

[Lee Jones]

On Fri, 3 Sept 2021 at 16:21, Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
>
> On Fri, Sep 03, 2021 at 07:44:35AM +0900, Masahiro Yamada wrote:
> > I used to use
> >   https://lore.kernel.org/patchwork/project/lkml/list/
> > to pick up patches sent to LKML, but it is gone now.
>
> As Mark has suggested, just use b4 to do the same. Running a patchwork
> instance just for the purpose of retrieving a few patches was extremely
> cost-inefficient, especially alongside the same functionality being offered by
> public-inbox and b4.
>
> You can install b4 from pypi (pip install b4), or you can clone the git
> repository and use it straight from there:
>
> https://git.kernel.org/pub/scm/utils/b4/b4.git
>
> To retrieve a mbox file ready to be applied to a tree, you just need to know
> the message-ID of the message (any message in the tread will do), and then
> just run:
>
> b4 am [that-message-id]

[trying again with HTML disabled]

I'm happy with my current Patchwork based solution and would like to
remain pro-choice.

Is there another solution besides being forced to use b4?

Can we represent LKML on Patchwork as a reasonable alternative?

-- 
Lee Jones [李琼斯]
Linaro Services Senior Technical Lead
Linaro.org │ Open source software for ARM SoCs
Follow Linaro: Facebook | Twitter | Blog

Re: New version of lore available for preview

[Konstantin Ryabitsev]

On Mon, Sep 06, 2021 at 10:59:38AM +0100, Lee Jones wrote:
> > To retrieve a mbox file ready to be applied to a tree, you just need to know
> > the message-ID of the message (any message in the tread will do), and then
> > just run:
> >
> > b4 am [that-message-id]
> 
> [trying again with HTML disabled]
> 
> I'm happy with my current Patchwork based solution and would like to
> remain pro-choice.

I can understand that feeling, but it was really quite expensive to run the
LKML patchwork:

- it is super memory hungry, eating up 18+GB of RAM after just a few days of
  uptime
- its database backend is ginormous due to all the accumulated LKML patches
- it routinely pegs the database due to bot activity

All of that was costing us hundreds of dollars a month just to run it for the
few people who were legitimately using it to download patches.

> Is there another solution besides being forced to use b4?

None that I can think of other than running patchwork locally. I'm not sure I
quite understand the reluctance to switch to b4, seeing as without patchwork's
state tracking features, the functionality between pwclient and b4 is nearly
identical -- you provide a URL and you get a mailbox with patches back.

> Can we represent LKML on Patchwork as a reasonable alternative?

No. That said, what I'm currently working on is a way to provide query-defined
public-inbox sources. If you can define what kind of patches you are
interested in as a lore.kernel.org query, we can save that search and feed
*that* into patchwork. So, instead of feeding all of LKML into patchwork just
for the few patches you're interested in, we can just feed the subset of
patches that you actually want.

I'll be talking more about this at my upcoming plumbers talk:
https://linuxplumbersconf.org/event/11/contributions/983/

-K

Re: New version of lore available for preview

[Steven Rostedt]

On Wed, 8 Sep 2021 14:37:46 -0400
Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:

> > Can we represent LKML on Patchwork as a reasonable alternative?  
> 
> No. That said, what I'm currently working on is a way to provide query-defined
> public-inbox sources. If you can define what kind of patches you are
> interested in as a lore.kernel.org query, we can save that search and feed
> *that* into patchwork. So, instead of feeding all of LKML into patchwork just
> for the few patches you're interested in, we can just feed the subset of
> patches that you actually want.

This is basically what I have done locally. I set up a local instance of
patchwork. Added a procmail filter to copy all my emails with "PATCH" in the
subject into its own folder. Then I have a cron job that runs the following
script to add "List-ID: <rostedt.inbox.com>" to these emails, as my local
patchwork "listens" to the "mailing list" called "rostedt.inbox.com". And
then send them off to my local patchwork instance.

#!/usr/bin/perl -w

my $state = "none";
my $patch = 0;

while (<>) {
        if (/^From /) {
                $state = "head";
                $patch = 0;
        } elsif ($state eq "head") {
                if (/^\s*$/) {
                        print "List-ID: <rostedt.inbox.com>\n" if ($patch);
                        $state = "body";
                } else {
                        if (/list-id/i) {
                                next;
                        } elsif (/^Subject.*patch/i) {
                                $patch = 1;
                        }
                }
        }
        print;
}


This works great for me, but I can imagine if people have their own public
inboxes, where we can just create a patchwork instance that feeds off of
these inboxes, I believe you will get the same functionality that I have.

Note, I also have filters to read patches that have "[for-next]" and
"[for-linus]" which converts the patches in patchwork from "New" to "Under
Review". Then I have a subscription to all commits that go into Linus's
tree, and I process all of them to go through and "Approve" any patch that
it finds.

I still would love a way to download patchwork offline, where it loads
locally all pending patches, and lets you review / apply them locally
without needing to be connected online, as this would be useful during
flights. Then when you land, you can upload your changes.

-- Steve

Re: New version of lore available for preview

[Lee Jones]

On Thu, 09 Sep 2021, Steven Rostedt wrote:
> On Wed, 8 Sep 2021 14:37:46 -0400
> Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> 
> > > Can we represent LKML on Patchwork as a reasonable alternative?  
> > 
> > No. That said, what I'm currently working on is a way to provide query-defined
> > public-inbox sources. If you can define what kind of patches you are
> > interested in as a lore.kernel.org query, we can save that search and feed
> > *that* into patchwork. So, instead of feeding all of LKML into patchwork just
> > for the few patches you're interested in, we can just feed the subset of
> > patches that you actually want.
> 
> This is basically what I have done locally. I set up a local instance of
> patchwork. Added a procmail filter to copy all my emails with "PATCH" in the
> subject into its own folder. Then I have a cron job that runs the following
> script to add "List-ID: <rostedt.inbox.com>" to these emails, as my local
> patchwork "listens" to the "mailing list" called "rostedt.inbox.com". And
> then send them off to my local patchwork instance.
> 
> #!/usr/bin/perl -w
> 
> my $state = "none";
> my $patch = 0;
> 
> while (<>) {
>         if (/^From /) {
>                 $state = "head";
>                 $patch = 0;
>         } elsif ($state eq "head") {
>                 if (/^\s*$/) {
>                         print "List-ID: <rostedt.inbox.com>\n" if ($patch);
>                         $state = "body";
>                 } else {
>                         if (/list-id/i) {
>                                 next;
>                         } elsif (/^Subject.*patch/i) {
>                                 $patch = 1;
>                         }
>                 }
>         }
>         print;
> }
> 
> 
> This works great for me, but I can imagine if people have their own public
> inboxes, where we can just create a patchwork instance that feeds off of
> these inboxes, I believe you will get the same functionality that I have.
> 
> Note, I also have filters to read patches that have "[for-next]" and
> "[for-linus]" which converts the patches in patchwork from "New" to "Under
> Review". Then I have a subscription to all commits that go into Linus's
> tree, and I process all of them to go through and "Approve" any patch that
> it finds.
> 
> I still would love a way to download patchwork offline, where it loads
> locally all pending patches, and lets you review / apply them locally
> without needing to be connected online, as this would be useful during
> flights. Then when you land, you can upload your changes.

Well, as beautifully inefficient as that all sounds, it might be
slightly less time consuming if I just succumb to peer pressure and
switch pwclient out for b4.

Thankfully, my use-case is much simpler than Steven's.

In my current implementation, I have pwclient pull down the patch,
apply my SoB and apply the it, all in one foul swoop:

  pwclient git-am -p ${project} -3 -s ${pwid}

Does b4 provide this functionality too, or do I have to script around
it?  How have other's chosen to automatically apply the downloaded
mboxes?

-- 
Lee Jones [李琼斯]
Senior Technical Lead - Developer Services
Linaro.org │ Open source software for Arm SoCs
Follow Linaro: Facebook | Twitter | Blog

Re: New version of lore available for preview

[Konstantin Ryabitsev]

On Thu, Sep 09, 2021 at 02:32:25PM +0100, Lee Jones wrote:
> In my current implementation, I have pwclient pull down the patch,
> apply my SoB and apply the it, all in one foul swoop:
> 
>   pwclient git-am -p ${project} -3 -s ${pwid}
> 
> Does b4 provide this functionality too, or do I have to script around
> it?  How have other's chosen to automatically apply the downloaded
> mboxes?

b4 am -3 -s -o- msgid | git am

-K

Re: New version of lore available for preview

[Greg KH]

On Thu, Sep 09, 2021 at 02:32:25PM +0100, Lee Jones wrote:
> On Thu, 09 Sep 2021, Steven Rostedt wrote:
> > On Wed, 8 Sep 2021 14:37:46 -0400
> > Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> > 
> > > > Can we represent LKML on Patchwork as a reasonable alternative?  
> > > 
> > > No. That said, what I'm currently working on is a way to provide query-defined
> > > public-inbox sources. If you can define what kind of patches you are
> > > interested in as a lore.kernel.org query, we can save that search and feed
> > > *that* into patchwork. So, instead of feeding all of LKML into patchwork just
> > > for the few patches you're interested in, we can just feed the subset of
> > > patches that you actually want.
> > 
> > This is basically what I have done locally. I set up a local instance of
> > patchwork. Added a procmail filter to copy all my emails with "PATCH" in the
> > subject into its own folder. Then I have a cron job that runs the following
> > script to add "List-ID: <rostedt.inbox.com>" to these emails, as my local
> > patchwork "listens" to the "mailing list" called "rostedt.inbox.com". And
> > then send them off to my local patchwork instance.
> > 
> > #!/usr/bin/perl -w
> > 
> > my $state = "none";
> > my $patch = 0;
> > 
> > while (<>) {
> >         if (/^From /) {
> >                 $state = "head";
> >                 $patch = 0;
> >         } elsif ($state eq "head") {
> >                 if (/^\s*$/) {
> >                         print "List-ID: <rostedt.inbox.com>\n" if ($patch);
> >                         $state = "body";
> >                 } else {
> >                         if (/list-id/i) {
> >                                 next;
> >                         } elsif (/^Subject.*patch/i) {
> >                                 $patch = 1;
> >                         }
> >                 }
> >         }
> >         print;
> > }
> > 
> > 
> > This works great for me, but I can imagine if people have their own public
> > inboxes, where we can just create a patchwork instance that feeds off of
> > these inboxes, I believe you will get the same functionality that I have.
> > 
> > Note, I also have filters to read patches that have "[for-next]" and
> > "[for-linus]" which converts the patches in patchwork from "New" to "Under
> > Review". Then I have a subscription to all commits that go into Linus's
> > tree, and I process all of them to go through and "Approve" any patch that
> > it finds.
> > 
> > I still would love a way to download patchwork offline, where it loads
> > locally all pending patches, and lets you review / apply them locally
> > without needing to be connected online, as this would be useful during
> > flights. Then when you land, you can upload your changes.
> 
> Well, as beautifully inefficient as that all sounds, it might be
> slightly less time consuming if I just succumb to peer pressure and
> switch pwclient out for b4.
> 
> Thankfully, my use-case is much simpler than Steven's.
> 
> In my current implementation, I have pwclient pull down the patch,
> apply my SoB and apply the it, all in one foul swoop:
> 
>   pwclient git-am -p ${project} -3 -s ${pwid}
> 
> Does b4 provide this functionality too, or do I have to script around
> it?  How have other's chosen to automatically apply the downloaded
> mboxes?

I do it from my mail client by piping the message to this:
	b4 am -t -o - | git am -s

which will download the whole series of patches, and acks and
reviewed-by and apply them, and then apply them with git.

I'm sure your email client can do much the same thing.

thanks,

greg k-h

Re: New version of lore available for preview

[Lee Jones]

On Thu, 09 Sep 2021, Konstantin Ryabitsev wrote:

> On Thu, Sep 09, 2021 at 02:32:25PM +0100, Lee Jones wrote:
> > In my current implementation, I have pwclient pull down the patch,
> > apply my SoB and apply the it, all in one foul swoop:
> > 
> >   pwclient git-am -p ${project} -3 -s ${pwid}
> > 
> > Does b4 provide this functionality too, or do I have to script around
> > it?  How have other's chosen to automatically apply the downloaded
> > mboxes?
> 
> b4 am -3 -s -o- msgid | git am

Ah yes, piping to stdout and straight into Git works great.

And with "-P _", it does exactly what I want.

Thanks for your quick response Konstantin.

-- 
Lee Jones [李琼斯]
Senior Technical Lead - Developer Services
Linaro.org │ Open source software for Arm SoCs
Follow Linaro: Facebook | Twitter | Blog

Re: New version of lore available for preview

[Lee Jones]

On Thu, 09 Sep 2021, Greg KH wrote:

> On Thu, Sep 09, 2021 at 02:32:25PM +0100, Lee Jones wrote:
> > On Thu, 09 Sep 2021, Steven Rostedt wrote:
> > > On Wed, 8 Sep 2021 14:37:46 -0400
> > > Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> > > 
> > > > > Can we represent LKML on Patchwork as a reasonable alternative?  
> > > > 
> > > > No. That said, what I'm currently working on is a way to provide query-defined
> > > > public-inbox sources. If you can define what kind of patches you are
> > > > interested in as a lore.kernel.org query, we can save that search and feed
> > > > *that* into patchwork. So, instead of feeding all of LKML into patchwork just
> > > > for the few patches you're interested in, we can just feed the subset of
> > > > patches that you actually want.
> > > 
> > > This is basically what I have done locally. I set up a local instance of
> > > patchwork. Added a procmail filter to copy all my emails with "PATCH" in the
> > > subject into its own folder. Then I have a cron job that runs the following
> > > script to add "List-ID: <rostedt.inbox.com>" to these emails, as my local
> > > patchwork "listens" to the "mailing list" called "rostedt.inbox.com". And
> > > then send them off to my local patchwork instance.
> > > 
> > > #!/usr/bin/perl -w
> > > 
> > > my $state = "none";
> > > my $patch = 0;
> > > 
> > > while (<>) {
> > >         if (/^From /) {
> > >                 $state = "head";
> > >                 $patch = 0;
> > >         } elsif ($state eq "head") {
> > >                 if (/^\s*$/) {
> > >                         print "List-ID: <rostedt.inbox.com>\n" if ($patch);
> > >                         $state = "body";
> > >                 } else {
> > >                         if (/list-id/i) {
> > >                                 next;
> > >                         } elsif (/^Subject.*patch/i) {
> > >                                 $patch = 1;
> > >                         }
> > >                 }
> > >         }
> > >         print;
> > > }
> > > 
> > > 
> > > This works great for me, but I can imagine if people have their own public
> > > inboxes, where we can just create a patchwork instance that feeds off of
> > > these inboxes, I believe you will get the same functionality that I have.
> > > 
> > > Note, I also have filters to read patches that have "[for-next]" and
> > > "[for-linus]" which converts the patches in patchwork from "New" to "Under
> > > Review". Then I have a subscription to all commits that go into Linus's
> > > tree, and I process all of them to go through and "Approve" any patch that
> > > it finds.
> > > 
> > > I still would love a way to download patchwork offline, where it loads
> > > locally all pending patches, and lets you review / apply them locally
> > > without needing to be connected online, as this would be useful during
> > > flights. Then when you land, you can upload your changes.
> > 
> > Well, as beautifully inefficient as that all sounds, it might be
> > slightly less time consuming if I just succumb to peer pressure and
> > switch pwclient out for b4.
> > 
> > Thankfully, my use-case is much simpler than Steven's.
> > 
> > In my current implementation, I have pwclient pull down the patch,
> > apply my SoB and apply the it, all in one foul swoop:
> > 
> >   pwclient git-am -p ${project} -3 -s ${pwid}
> > 
> > Does b4 provide this functionality too, or do I have to script around
> > it?  How have other's chosen to automatically apply the downloaded
> > mboxes?
> 
> I do it from my mail client by piping the message to this:
> 	b4 am -t -o - | git am -s

Not sure what -t does:

  Apply trailers sent to the cover letter to all patches

> which will download the whole series of patches, and acks and
> reviewed-by and apply them, and then apply them with git.
> 
> I'm sure your email client can do much the same thing.

Right, but I don't want it to do that.

I operate using worktrees and have different Mutt keys apply patches
where required.  It's also very unusual for me to apply wholes sets of
patches, thus plucking patches one at a time make more sense as part
of my flow.

-- 
Lee Jones [李琼斯]
Senior Technical Lead - Developer Services
Linaro.org │ Open source software for Arm SoCs
Follow Linaro: Facebook | Twitter | Blog

Re: New version of lore available for preview

[Steven Rostedt]

On Thu, 9 Sep 2021 14:32:25 +0100
Lee Jones <lee.jones@linaro.org> wrote:

> Well, as beautifully inefficient as that all sounds, it might be
> slightly less time consuming if I just succumb to peer pressure and
> switch pwclient out for b4.

The old "Show them an even more painful alternative to make the current
solution seem good" trick.

> 
> Thankfully, my use-case is much simpler than Steven's.

Of course, my solution came before b4 was even thought of (I even mentioned
this solution in the kernel summit discussion that started the path to b4).

The one thing I need that b4 doesn't give me, is a place outside of email
to keep track of the state of patches. I've failed in all my attempts at
using email folders to do this. What I love about patchwork is that I can
keep the patches at various states.

 New - I just received it and need to look at it / test it.

 Under review - They have been tested, just need to be pushed to upstream.

 Accepted - they have been accepted.

I also created other states for topics, like "Real Time", "Scheduler",
"RCU", etc, so that I can go back if I have time (which currently I never
do), and look at these patches.

I also use the "delegate" option to assign patches to myself that are high
in priority.

I tried doing this with email folders, and even a git repo, but because I
use email and git for so much else, it still gets lost in the noise. I like
patchwork because its web based, and I have a window up all the time that
shows me what patches I need to work on next.

The pulling into git is actually one of the trivial parts of the process.

-- Steve

Re: New version of lore available for preview

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 401 bytes --]

On Thu, Sep 09, 2021 at 02:54:02PM +0100, Lee Jones wrote:
> On Thu, 09 Sep 2021, Greg KH wrote:

> > I do it from my mail client by piping the message to this:
> > 	b4 am -t -o - | git am -s

> Not sure what -t does:

>   Apply trailers sent to the cover letter to all patches

If someone sends an Acked-by or whatever in reply to the cover letter b4
will pick that up for all patches in the series.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: New version of lore available for preview

[Lee Jones]

On Thu, 09 Sep 2021, Steven Rostedt wrote:

> On Thu, 9 Sep 2021 14:32:25 +0100
> Lee Jones <lee.jones@linaro.org> wrote:
> 
> > Well, as beautifully inefficient as that all sounds, it might be
> > slightly less time consuming if I just succumb to peer pressure and
> > switch pwclient out for b4.
> 
> The old "Show them an even more painful alternative to make the current
> solution seem good" trick.

Works every time! :)

> > Thankfully, my use-case is much simpler than Steven's.
> 
> Of course, my solution came before b4 was even thought of (I even mentioned
> this solution in the kernel summit discussion that started the path to b4).
> 
> The one thing I need that b4 doesn't give me, is a place outside of email
> to keep track of the state of patches. I've failed in all my attempts at
> using email folders to do this. What I love about patchwork is that I can
> keep the patches at various states.
> 
>  New - I just received it and need to look at it / test it.
> 
>  Under review - They have been tested, just need to be pushed to upstream.
> 
>  Accepted - they have been accepted.
> 
> I also created other states for topics, like "Real Time", "Scheduler",
> "RCU", etc, so that I can go back if I have time (which currently I never
> do), and look at these patches.
> 
> I also use the "delegate" option to assign patches to myself that are high
> in priority.
> 
> I tried doing this with email folders, and even a git repo, but because I
> use email and git for so much else, it still gets lost in the noise. I like
> patchwork because its web based, and I have a window up all the time that
> shows me what patches I need to work on next.
> 
> The pulling into git is actually one of the trivial parts of the process.

I like to keep my setup as KISS as possible.

Fortunately, in terms of upstream reviews I've managed to whittle
patches down into 3 states;

 - unread
 - does not need review
 - needs review

The latter is represented with Mutt's 'flag-message' functionality.

-- 
Lee Jones [李琼斯]
Senior Technical Lead - Developer Services
Linaro.org │ Open source software for Arm SoCs
Follow Linaro: Facebook | Twitter | Blog

Re: New version of lore available for preview

[Lee Jones]

On Thu, 09 Sep 2021, Mark Brown wrote:

> On Thu, Sep 09, 2021 at 02:54:02PM +0100, Lee Jones wrote:
> > On Thu, 09 Sep 2021, Greg KH wrote:
> 
> > > I do it from my mail client by piping the message to this:
> > > 	b4 am -t -o - | git am -s
> 
> > Not sure what -t does:
> 
> >   Apply trailers sent to the cover letter to all patches
> 
> If someone sends an Acked-by or whatever in reply to the cover letter b4
> will pick that up for all patches in the series.

Got it.  Thanks Mark.

-- 
Lee Jones [李琼斯]
Senior Technical Lead - Developer Services
Linaro.org │ Open source software for Arm SoCs
Follow Linaro: Facebook | Twitter | Blog

Re: New version of lore available for preview

[Konstantin Ryabitsev]

On Thu, Sep 09, 2021 at 02:45:53PM +0100, Lee Jones wrote:
> > b4 am -3 -s -o- msgid | git am
> 
> Ah yes, piping to stdout and straight into Git works great.
> 
> And with "-P _", it does exactly what I want.
> 
> Thanks for your quick response Konstantin.

No worries, and like I said -- if you actually need patchwork for tracking
state, then we'll be adding a way to create search-based projects in the near
future.

-K

Re: New version of lore available for preview

[Alexandre Belloni]

On 09/09/2021 14:45:53+0100, Lee Jones wrote:
> On Thu, 09 Sep 2021, Konstantin Ryabitsev wrote:
> 
> > On Thu, Sep 09, 2021 at 02:32:25PM +0100, Lee Jones wrote:
> > > In my current implementation, I have pwclient pull down the patch,
> > > apply my SoB and apply the it, all in one foul swoop:
> > > 
> > >   pwclient git-am -p ${project} -3 -s ${pwid}
> > > 
> > > Does b4 provide this functionality too, or do I have to script around
> > > it?  How have other's chosen to automatically apply the downloaded
> > > mboxes?
> > 
> > b4 am -3 -s -o- msgid | git am
> 
> Ah yes, piping to stdout and straight into Git works great.
> 
> And with "-P _", it does exactly what I want.
> 

I did use the stdout piping to git am until I have seen someone on a
mailing list adding this to .gitconfig

[alias]
	b4am = "!sh -c 'b4 am -ctsl -o- $1 | git am' -"
	b4am1 = "!sh -c 'b4 am -ctsl -o- -P_ $1 | git am' -"

and then you can directly do git b4am <msg-id>

-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: New version of lore available for preview

[Peter Zijlstra]

On Thu, Sep 09, 2021 at 02:32:25PM +0100, Lee Jones wrote:
> Does b4 provide this functionality too, or do I have to script around
> it?  How have other's chosen to automatically apply the downloaded
> mboxes?

My .muttrc has:

macro index B "<tag-prefix><pipe-entry>formail -ds | b4 am -slt -m - -o - | formail -s ~/mutt/mutt-pipe.sh<enter><untag-pattern>~T<enter>"

(which relies on that same muttrc having: set pipe_split=no)

which allows me to tag a (partial) thread using ESC-t (or otherwise) and
apply the resulting lot (with folding of tags and adding signature and
all the goodness) to my quilt series (as done by my mutt-pipe.sh
script).

The initial "formail -ds" is required because mutt pipe output isn't
exactly the mbox format b4 expects and formail basically reformats it,
it's mbox in mbox out. Then b4 takes the mbox, folds tags and generates
a reduced mbox with only patches; formail then splits it and my old and
trusty mutt-pipe.sh frobs it into quilt one at a time.

Re: New version of lore available for preview

[Konstantin Ryabitsev]

On Thu, Sep 09, 2021 at 04:51:14PM +0200, Alexandre Belloni wrote:
> I did use the stdout piping to git am until I have seen someone on a
> mailing list adding this to .gitconfig
> 
> [alias]
> 	b4am = "!sh -c 'b4 am -ctsl -o- $1 | git am' -"
> 	b4am1 = "!sh -c 'b4 am -ctsl -o- -P_ $1 | git am' -"
> 
> and then you can directly do git b4am <msg-id>

I intend to add a command to just apply the series directly if all the
prechecks pass:

- attestation is good
- applies cleanly
- no trailer warnings

The fact that folks are writing such aliases tells me that this is needed and
I should stop avoiding modifying the working tree.

-K

Re: New version of lore available for preview

[James Bottomley]

On Thu, 2021-09-09 at 10:39 -0400, Konstantin Ryabitsev wrote:
> On Thu, Sep 09, 2021 at 02:45:53PM +0100, Lee Jones wrote:
> > > b4 am -3 -s -o- msgid | git am
> > 
> > Ah yes, piping to stdout and straight into Git works great.
> > 
> > And with "-P _", it does exactly what I want.
> > 
> > Thanks for your quick response Konstantin.
> 
> No worries, and like I said -- if you actually need patchwork for
> tracking state, then we'll be adding a way to create search-based
> projects in the near future.

I thought public inbox had a mechanism for tracking local state: the
most pressing being whether you've read the message or not when you use
imap backed by public inbox.  Can't this local tracking be extended to
do other useful things patchwork previously did?

James

Re: New version of lore available for preview

[Konstantin Ryabitsev]

On Thu, Sep 09, 2021 at 08:24:28AM -0700, James Bottomley wrote:
> > No worries, and like I said -- if you actually need patchwork for
> > tracking state, then we'll be adding a way to create search-based
> > projects in the near future.
> 
> I thought public inbox had a mechanism for tracking local state: the
> most pressing being whether you've read the message or not when you use
> imap backed by public inbox.  Can't this local tracking be extended to
> do other useful things patchwork previously did?

Pretty sure that's not currently the case -- public-inbox-imapd provides
readonly access and does no state tracking.

That's not to be confused with the new "lei" command provided by public-inbox
that can collect threads from local and remote sources and make them available
either as local maildirs or upload them to an imap folder. The "lei" interface
is still being finalized, so I don't want to over-advertise it yet until the
dust settles. Once completed, it should allow anyone to "receive" mail based
on custom queries straight from lore or other public-inbox or imap sources.

-K

Re: New version of lore available for preview

[Lee Jones]

On Thu, 09 Sep 2021, Peter Zijlstra wrote:

> On Thu, Sep 09, 2021 at 02:32:25PM +0100, Lee Jones wrote:
> > Does b4 provide this functionality too, or do I have to script around
> > it?  How have other's chosen to automatically apply the downloaded
> > mboxes?
> 
> My .muttrc has:
> 
> macro index B "<tag-prefix><pipe-entry>formail -ds | b4 am -slt -m - -o - | formail -s ~/mutt/mutt-pipe.sh<enter><untag-pattern>~T<enter>"
> 
> (which relies on that same muttrc having: set pipe_split=no)
> 
> which allows me to tag a (partial) thread using ESC-t (or otherwise) and
> apply the resulting lot (with folding of tags and adding signature and
> all the goodness) to my quilt series (as done by my mutt-pipe.sh
> script).
> 
> The initial "formail -ds" is required because mutt pipe output isn't
> exactly the mbox format b4 expects and formail basically reformats it,
> it's mbox in mbox out. Then b4 takes the mbox, folds tags and generates
> a reduced mbox with only patches; formail then splits it and my old and
> trusty mutt-pipe.sh frobs it into quilt one at a time.

This might well be handy.

I'll probably look into this when I have a need i.e. when I come
across my first multi-patch merge where there has been subsequent
reviews (trailers) added to the thread.

Thanks.

-- 
Lee Jones [李琼斯]
Senior Technical Lead - Developer Services
Linaro.org │ Open source software for Arm SoCs
Follow Linaro: Facebook | Twitter | Blog

Re: New version of lore available for preview

[Bjorn Andersson]

On Thu 09 Sep 06:43 PDT 2021, Greg KH wrote:

> On Thu, Sep 09, 2021 at 02:32:25PM +0100, Lee Jones wrote:
> > On Thu, 09 Sep 2021, Steven Rostedt wrote:
> > > On Wed, 8 Sep 2021 14:37:46 -0400
> > > Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> > > 
> > > > > Can we represent LKML on Patchwork as a reasonable alternative?  
> > > > 
> > > > No. That said, what I'm currently working on is a way to provide query-defined
> > > > public-inbox sources. If you can define what kind of patches you are
> > > > interested in as a lore.kernel.org query, we can save that search and feed
> > > > *that* into patchwork. So, instead of feeding all of LKML into patchwork just
> > > > for the few patches you're interested in, we can just feed the subset of
> > > > patches that you actually want.
> > > 
> > > This is basically what I have done locally. I set up a local instance of
> > > patchwork. Added a procmail filter to copy all my emails with "PATCH" in the
> > > subject into its own folder. Then I have a cron job that runs the following
> > > script to add "List-ID: <rostedt.inbox.com>" to these emails, as my local
> > > patchwork "listens" to the "mailing list" called "rostedt.inbox.com". And
> > > then send them off to my local patchwork instance.
> > > 
> > > #!/usr/bin/perl -w
> > > 
> > > my $state = "none";
> > > my $patch = 0;
> > > 
> > > while (<>) {
> > >         if (/^From /) {
> > >                 $state = "head";
> > >                 $patch = 0;
> > >         } elsif ($state eq "head") {
> > >                 if (/^\s*$/) {
> > >                         print "List-ID: <rostedt.inbox.com>\n" if ($patch);
> > >                         $state = "body";
> > >                 } else {
> > >                         if (/list-id/i) {
> > >                                 next;
> > >                         } elsif (/^Subject.*patch/i) {
> > >                                 $patch = 1;
> > >                         }
> > >                 }
> > >         }
> > >         print;
> > > }
> > > 
> > > 
> > > This works great for me, but I can imagine if people have their own public
> > > inboxes, where we can just create a patchwork instance that feeds off of
> > > these inboxes, I believe you will get the same functionality that I have.
> > > 
> > > Note, I also have filters to read patches that have "[for-next]" and
> > > "[for-linus]" which converts the patches in patchwork from "New" to "Under
> > > Review". Then I have a subscription to all commits that go into Linus's
> > > tree, and I process all of them to go through and "Approve" any patch that
> > > it finds.
> > > 
> > > I still would love a way to download patchwork offline, where it loads
> > > locally all pending patches, and lets you review / apply them locally
> > > without needing to be connected online, as this would be useful during
> > > flights. Then when you land, you can upload your changes.
> > 
> > Well, as beautifully inefficient as that all sounds, it might be
> > slightly less time consuming if I just succumb to peer pressure and
> > switch pwclient out for b4.
> > 
> > Thankfully, my use-case is much simpler than Steven's.
> > 
> > In my current implementation, I have pwclient pull down the patch,
> > apply my SoB and apply the it, all in one foul swoop:
> > 
> >   pwclient git-am -p ${project} -3 -s ${pwid}
> > 
> > Does b4 provide this functionality too, or do I have to script around
> > it?  How have other's chosen to automatically apply the downloaded
> > mboxes?
> 
> I do it from my mail client by piping the message to this:
> 	b4 am -t -o - | git am -s
> 

The integration directly in the mail client is a really nice trick.

> which will download the whole series of patches, and acks and
> reviewed-by and apply them, and then apply them with git.
> 

Unfortunately when it comes to hardware platform enablement we do get a
lot of series that spans across multiple subsystems and we've seen
several examples where applying the whole series causes merge conflicts
down the road.

As this is very common in my inbox I've yet to migrate to b4.

Is this (pick a single msg) supported in b4, am I just missing something
in the documentation?

Regards,
Bjorn

Re: New version of lore available for preview

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 365 bytes --]

On Fri, Sep 10, 2021 at 09:07:52AM -0700, Bjorn Andersson wrote:

> Is this (pick a single msg) supported in b4, am I just missing something
> in the documentation?

You can pick a subset of patches but only by specifying the patch
numbers on the command line - I do this outside the mail client as part
of saying which trees and branches the patches are going to.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: New version of lore available for preview

[Peter Zijlstra]

On Fri, Sep 10, 2021 at 05:12:04PM +0100, Mark Brown wrote:
> On Fri, Sep 10, 2021 at 09:07:52AM -0700, Bjorn Andersson wrote:
> 
> > Is this (pick a single msg) supported in b4, am I just missing something
> > in the documentation?
> 
> You can pick a subset of patches but only by specifying the patch
> numbers on the command line - I do this outside the mail client as part
> of saying which trees and branches the patches are going to.

See my earlier email to Lee, you can have mutt pipe in whichever
messages you want to apply (and their replies) and have b4 do it's magic
on them.

I typically start by selecting the whole thread using ESC-t and then
unselecting the patches I want to hold back, but whatever works for you
best...

basically you have to use: "b4 -m -" (in addition to the other flags) in
order for it to accept a local mbox on stdin, instead of scraping the
web.