LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Bjorn Helgaas <bhelgaas@google.com>
To: Eric Wong <e@80x24.org>
Cc: Joey Pabalinas <joeypabalinas@gmail.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	kernelnewbies@kernelnewbies.org,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Konstantin Ryabitsev <konstantin@linuxfoundation.org>,
	Eric Biederman <ebiederm@xmission.com>,
	Jasper Spaans <j@jasper.es>
Subject: Re: [RFC] LKML Archive in Maildir Format
Date: Wed, 6 Mar 2019 14:50:20 -0600
Message-ID: <CAErSpo6QTdABwBx-nvkmJyv7JHb95azapwLtBf++Ved7=UpMWA@mail.gmail.com> (raw)
In-Reply-To: <20190305232600.GA12110@dcvr>

On Tue, Mar 5, 2019 at 5:26 PM Eric Wong <e@80x24.org> wrote:
> Bjorn Helgaas <bhelgaas@google.com> wrote:

> > Any pointers?  I guess there's no mutt backend that can read a
> > public-inbox archive directly?
>
> There's mutt patches to support reading over NNTP, so that
> works:
>
>         mutt -f news://$INBOX_HOST/$INBOX_NEWSGROUP

Neomutt includes NNTP support, so I tried this:

  neomutt -f news://nntp.lore.kernel.org/org.kernel.vger.linux-kernel

which worked OK but (1) I only see the most recent 1000 messages and
(2) obviously isn't reading a *local* archive.  Neomutt took about 45
seconds to start up over my wimpy ISP.

I assume I could probably have a local archive and run a local NNTP
server and point neomutt at that local server.  But I don't know how
full-archive searching would work there.

> I don't think mutt handles mboxrd 100% correctly, but it's close
> enough that you can can download the gzipped mboxrd of a search
> query and open it via "mutt -f /path/to/downloaded/mbox.gz"
>
>   curl -XPOST -OJ "$INBOX_URL/?q=$SEARCH_QUERY&x=m"

I got nothing at all with -XPOST, but this:

  curl -OJ "https://lore.kernel.org/linux-pci/?q=d:20190301..&x=m"

got me the HTML source.  Nothing that looks like mboxrd.  I assume
this is stupid user error on my part, but even with that resolved, it
wouldn't have the nice git fetch properties of the git archive, i.e.,
incremental updates of only new stuff, would it?

I think my ideal solution would be a mutt that could read the git
archive directly, plus a notmuch index.  But AFAIK, mutt can't do
that, and notmuch only works with one message per file, not with the
git archive.

Something that might work would be to use Konstantin's "git archive to
maildir" hint but shard into a bunch of smaller maildirs instead of
one big one, then have notmuch index those, and use mutt or vim with
notmuch queries instead of having it read in a maildir.

But I feel like I must be missing the solution that's obvious to
everybody but me.

Bjorn

  reply index

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-16 19:06 Joey Pabalinas
2018-12-16 19:17 ` Joe Perches
2018-12-16 19:21   ` Joey Pabalinas
2018-12-16 19:55     ` Konstantin Ryabitsev
2018-12-16 21:55       ` Joey Pabalinas
2018-12-18 20:26     ` Jasper Spaans
2018-12-18 22:53       ` Joey Pabalinas
2018-12-16 19:46 ` Konstantin Ryabitsev
2018-12-16 19:53   ` Joey Pabalinas
2019-01-04  1:35     ` Eric Wong
2019-03-05 20:48       ` Bjorn Helgaas
2019-03-05 23:26         ` Eric Wong
2019-03-06 20:50           ` Bjorn Helgaas [this message]
2019-03-07  3:44             ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAErSpo6QTdABwBx-nvkmJyv7JHb95azapwLtBf++Ved7=UpMWA@mail.gmail.com' \
    --to=bhelgaas@google.com \
    --cc=e@80x24.org \
    --cc=ebiederm@xmission.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=j@jasper.es \
    --cc=joeypabalinas@gmail.com \
    --cc=kernelnewbies@kernelnewbies.org \
    --cc=konstantin@linuxfoundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git