From: Dmitry Vyukov <dvyukov@google.com>
To: Daniel Axtens <dja@axtens.net>
Cc: workflows@vger.kernel.org, automated-testing@yoctoproject.org,
Konstantin Ryabitsev <konstantin@linuxfoundation.org>,
Brendan Higgins <brendanhiggins@google.com>,
Han-Wen Nienhuys <hanwen@google.com>,
Kevin Hilman <khilman@baylibre.com>,
Veronika Kabatova <vkabatov@redhat.com>
Subject: Re: Structured feeds
Date: Thu, 7 Nov 2019 11:43:45 +0100 [thread overview]
Message-ID: <CACT4Y+Z1sL_Ag51T3N0nhf35N-aYMQ1pzenVwSRRR0=VUBvMCQ@mail.gmail.com> (raw)
In-Reply-To: <87woccgea3.fsf@dja-thinkpad.axtens.net>
On Thu, Nov 7, 2019 at 11:41 AM Daniel Axtens <dja@axtens.net> wrote:
>
> Dmitry Vyukov <dvyukov@google.com> writes:
>
> > On Wed, Nov 6, 2019 at 4:35 PM Daniel Axtens <dja@axtens.net> wrote:
> >>
> >> > As soon as we have a bridge from plain-text emails into the structured
> >> > form, we can start building everything else in the structured world.
> >> > Such bridge needs to parse new incoming emails, try to make sense out
> >> > of them (new patch, new patch version, comment, etc) and then push the
> >> > information in structured form. Then e.g. CIs can fetch info about
> >>
> >> This is an non-trivial problem, fwiw. Patchwork's email parser clocks in
> >> at almost thirteen hundred lines, and that's with the benefit of the
> >> Python standard library. It also regularly gets patched to handle
> >> changes to email systems (e.g. DMARC), changes to git (git request-pull
> >> format changed subtly in 2.14.3), the bizzare ways people send email,
> >> and so on.
> >>
> >> Patchwork does expose much of this as an API, for example for patches:
> >> https://patchwork.ozlabs.org/api/patches/?order=-id so if you want to
> >> build on that feel free. We can possibly add data to the API if that
> >> would be helpful. (Patches are always welcome too, if you don't want to
> >> wait an indeterminate amount of time.)
> >
> > Hi Daniel,
> >
> > Thanks!
> > Could you provide a link to the code?
> > Do you have a test suite for the parser (set of email samples and what
> > they should be parsed to)?
>
> Sure:
> https://github.com/getpatchwork/patchwork in particular
> https://github.com/getpatchwork/patchwork/blob/master/patchwork/parser.py and
> https://github.com/getpatchwork/patchwork/tree/master/patchwork/tests
Added here for future reference:
https://github.com/dvyukov/kit/blob/master/doc/references.md#patchwork
Thanks!
next prev parent reply other threads:[~2019-11-07 10:44 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-05 10:02 Structured feeds Dmitry Vyukov
2019-11-06 15:35 ` Daniel Axtens
2019-11-06 20:50 ` Konstantin Ryabitsev
2019-11-07 9:08 ` Dmitry Vyukov
2019-11-07 10:57 ` Daniel Axtens
2019-11-07 11:26 ` Veronika Kabatova
2019-11-08 0:24 ` Eric Wong
2019-11-07 11:09 ` Daniel Axtens
2019-11-08 14:18 ` Daniel Axtens
2019-11-09 7:41 ` Johannes Berg
2019-11-12 10:44 ` Daniel Borkmann
[not found] ` <208edf06eb4c56a4f376caf0feced65f09d23f93.camel@that.guru>
2019-11-30 18:16 ` Johannes Berg
2019-11-30 18:36 ` Stephen Finucane
2019-11-07 8:53 ` Dmitry Vyukov
2019-11-07 10:40 ` Daniel Axtens
2019-11-07 10:43 ` Dmitry Vyukov [this message]
2019-11-07 20:43 ` [Automated-testing] " Don Zickus
2019-11-08 7:58 ` Dmitry Vyukov
2019-11-08 15:26 ` Don Zickus
2019-11-08 11:44 ` Daniel Axtens
2019-11-08 14:54 ` Don Zickus
2019-11-06 19:54 ` Han-Wen Nienhuys
2019-11-06 20:31 ` Sean Whitton
2019-11-07 9:04 ` Dmitry Vyukov
2019-11-07 8:48 ` [Automated-testing] " Tim.Bird
2019-11-07 9:13 ` Dmitry Vyukov
2019-11-07 9:20 ` Tim.Bird
2019-11-07 20:53 ` Don Zickus
2019-11-08 8:05 ` Dmitry Vyukov
2019-11-08 14:52 ` Don Zickus
2019-11-11 9:20 ` Dmitry Vyukov
2019-11-11 15:14 ` Don Zickus
2019-11-12 22:54 ` Konstantin Ryabitsev
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CACT4Y+Z1sL_Ag51T3N0nhf35N-aYMQ1pzenVwSRRR0=VUBvMCQ@mail.gmail.com' \
--to=dvyukov@google.com \
--cc=automated-testing@yoctoproject.org \
--cc=brendanhiggins@google.com \
--cc=dja@axtens.net \
--cc=hanwen@google.com \
--cc=khilman@baylibre.com \
--cc=konstantin@linuxfoundation.org \
--cc=vkabatov@redhat.com \
--cc=workflows@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).