* [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded
@ 2018-01-05 20:43 Thomas Petazzoni
2018-01-07 23:33 ` Andrew Donnellan
0 siblings, 1 reply; 21+ messages in thread
From: Thomas Petazzoni @ 2018-01-05 20:43 UTC (permalink / raw)
To: buildroot
Hello,
Just a quick note to let you know that from here, patchwork.ozlabs.org
seems to be down at this time (or awfully slow).
Also, since a few weeks, we have seen that patchwork misses some
patches that are sent on the Buildroot mailing list. The mails are sent
on the mailing list, they are in the mailing list archives, but
patchwork does not record them.
An example is:
http://lists.busybox.net/pipermail/buildroot/2017-December/209885.html
PATCH 3/3 was not recorded by patchwork, while other patches in the
series have been.
This again happened today. The first send of:
http://lists.busybox.net/pipermail/buildroot/2018-January/210797.html
only had a few of its patches recorded by patchwork. It was resent a
few minutes later:
http://lists.busybox.net/pipermail/buildroot/2018-January/210858.html
and then all the patches got recorded.
This is quite annoying. Is there a way to look in the patchwork logs
what has happened with those patches ?
Thanks a lot!
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded
2018-01-05 20:43 [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded Thomas Petazzoni
@ 2018-01-07 23:33 ` Andrew Donnellan
2018-01-08 8:10 ` Thomas Petazzoni
0 siblings, 1 reply; 21+ messages in thread
From: Andrew Donnellan @ 2018-01-07 23:33 UTC (permalink / raw)
To: buildroot
On 06/01/18 07:43, Thomas Petazzoni wrote:
> Hello,
>
> Just a quick note to let you know that from here, patchwork.ozlabs.org
> seems to be down at this time (or awfully slow).
bilbo (i.e. ozlabs.org) required a reboot a couple of days ago for a
kernel upgrade iirc.
ajd at bilbo:~$ uptime
10:27:36 up 2 days, 1:10, 5 users, load average: 1.38, 1.96, 1.64
>
> Also, since a few weeks, we have seen that patchwork misses some
> patches that are sent on the Buildroot mailing list. The mails are sent
> on the mailing list, they are in the mailing list archives, but
> patchwork does not record them.
>
> An example is:
>
> http://lists.busybox.net/pipermail/buildroot/2017-December/209885.html
>
> PATCH 3/3 was not recorded by patchwork, while other patches in the
> series have been.
Hmm, looking at the patch list, I also see that patch 2/3 in that series
(https://patchwork.ozlabs.org/patch/852063/) wasn't correctly identified
in the same series as patch 1
(https://patchwork.ozlabs.org/patch/852062/) which is a bit weird.
>
> This again happened today. The first send of:
>
> http://lists.busybox.net/pipermail/buildroot/2018-January/210797.html
>
> only had a few of its patches recorded by patchwork. It was resent a
> few minutes later:
>
> http://lists.busybox.net/pipermail/buildroot/2018-January/210858.html
>
> and then all the patches got recorded.
>
> This is quite annoying. Is there a way to look in the patchwork logs
> what has happened with those patches ?
I don't have access to the patchwork logs myself - jk, any thoughts?
--
Andrew Donnellan OzLabs, ADL Canberra
andrew.donnellan at au1.ibm.com IBM Australia Limited
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded
2018-01-07 23:33 ` Andrew Donnellan
@ 2018-01-08 8:10 ` Thomas Petazzoni
2018-02-05 21:45 ` Thomas Petazzoni
0 siblings, 1 reply; 21+ messages in thread
From: Thomas Petazzoni @ 2018-01-08 8:10 UTC (permalink / raw)
To: buildroot
Hello,
On Mon, 8 Jan 2018 10:33:21 +1100, Andrew Donnellan wrote:
> > Also, since a few weeks, we have seen that patchwork misses some
> > patches that are sent on the Buildroot mailing list. The mails are sent
> > on the mailing list, they are in the mailing list archives, but
> > patchwork does not record them.
> >
> > An example is:
> >
> > http://lists.busybox.net/pipermail/buildroot/2017-December/209885.html
> >
> > PATCH 3/3 was not recorded by patchwork, while other patches in the
> > series have been.
>
> Hmm, looking at the patch list, I also see that patch 2/3 in that series
> (https://patchwork.ozlabs.org/patch/852063/) wasn't correctly identified
> in the same series as patch 1
> (https://patchwork.ozlabs.org/patch/852062/) which is a bit weird.
Note: this issue happened again yesterday. In this series of 4 patches:
http://lists.busybox.net/pipermail/buildroot/2018-January/210935.html
Only patches 2/4 and 4/4 have been recorded by patchwork:
https://patchwork.ozlabs.org/patch/856422/
https://patchwork.ozlabs.org/patch/856423/
Even though all 4 patches have been received through the mailing list.
Best regards,
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded
2018-01-08 8:10 ` Thomas Petazzoni
@ 2018-02-05 21:45 ` Thomas Petazzoni
2018-02-06 8:41 ` Andrew Donnellan
0 siblings, 1 reply; 21+ messages in thread
From: Thomas Petazzoni @ 2018-02-05 21:45 UTC (permalink / raw)
To: buildroot
Hello,
On Mon, 8 Jan 2018 09:10:44 +0100, Thomas Petazzoni wrote:
> > Hmm, looking at the patch list, I also see that patch 2/3 in that series
> > (https://patchwork.ozlabs.org/patch/852063/) wasn't correctly identified
> > in the same series as patch 1
> > (https://patchwork.ozlabs.org/patch/852062/) which is a bit weird.
>
> Note: this issue happened again yesterday. In this series of 4 patches:
>
> http://lists.busybox.net/pipermail/buildroot/2018-January/210935.html
>
> Only patches 2/4 and 4/4 have been recorded by patchwork:
>
> https://patchwork.ozlabs.org/patch/856422/
> https://patchwork.ozlabs.org/patch/856423/
This is still happening, and pretty badly. Sometimes almost entire
series are skipped. Most recent example is this series:
http://lists.busybox.net/pipermail/buildroot/2018-February/213111.html
It has 15 patches and a cover letter, and all what patchwork recorded
is:
https://patchwork.ozlabs.org/patch/869198/
https://patchwork.ozlabs.org/patch/869197/
Only two patches out of 15 + a cover letter. Not great.
Is there anything that can be done about this ? An example of
Message-Id that was not recorded is:
Message-Id: <8027bae45d8e041c8a1e0bc714ab378ff984ded3.1517820133.git.yann.morin.1998@free.fr>
Thanks a lot,
Thomas
--
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded
2018-02-05 21:45 ` Thomas Petazzoni
@ 2018-02-06 8:41 ` Andrew Donnellan
2018-02-06 9:03 ` Jeremy Kerr
2018-02-13 22:19 ` Thomas Petazzoni
0 siblings, 2 replies; 21+ messages in thread
From: Andrew Donnellan @ 2018-02-06 8:41 UTC (permalink / raw)
To: buildroot
On 06/02/18 08:45, Thomas Petazzoni wrote:
> Is there anything that can be done about this ? An example of
> Message-Id that was not recorded is:
>
> Message-Id: <8027bae45d8e041c8a1e0bc714ab378ff984ded3.1517820133.git.yann.morin.1998@free.fr>
I've scraped the buildroot archives to see if there's anything obviously
wrong with that particular message ID - it appears to parse fine.
jk, any idea whether there's something particular about the ozlabs.org
instance that could be causing it to drop these patches?
--
Andrew Donnellan OzLabs, ADL Canberra
andrew.donnellan at au1.ibm.com IBM Australia Limited
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded
2018-02-06 8:41 ` Andrew Donnellan
@ 2018-02-06 9:03 ` Jeremy Kerr
2018-02-06 9:55 ` Thomas Petazzoni
2018-02-13 22:19 ` Thomas Petazzoni
1 sibling, 1 reply; 21+ messages in thread
From: Jeremy Kerr @ 2018-02-06 9:03 UTC (permalink / raw)
To: buildroot
Hi Andrew,
> jk, any idea whether there's something particular about the ozlabs.org
> instance that could be causing it to drop these patches?
We're running plain upstream with regards to the parsing code; so if it
parses we should be fine there.
Assume the parsing is OK, there are a couple of reasons for potential
drops:
- the database server was down at the time of parse. This is quite
infrequent, but can happen during database upgrades. The last one of
those was at 2018-01-27 05:08:30 UTC, so doesn't look like the case
here
- after the update to 2.x, database load has significantly increased;
there have been occasions where db connections are rejected due to
this. The recent patchwork update should address some issues there,
but I'm not sure whether we're back to optimal db behaviour now...
Cheers,
Jeremy
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded
2018-02-06 9:03 ` Jeremy Kerr
@ 2018-02-06 9:55 ` Thomas Petazzoni
0 siblings, 0 replies; 21+ messages in thread
From: Thomas Petazzoni @ 2018-02-06 9:55 UTC (permalink / raw)
To: buildroot
Hello,
On Tue, 6 Feb 2018 17:03:50 +0800, Jeremy Kerr wrote:
> We're running plain upstream with regards to the parsing code; so if it
> parses we should be fine there.
>
> Assume the parsing is OK, there are a couple of reasons for potential
> drops:
>
> - the database server was down at the time of parse. This is quite
> infrequent, but can happen during database upgrades. The last one of
> those was at 2018-01-27 05:08:30 UTC, so doesn't look like the case
> here
>
> - after the update to 2.x, database load has significantly increased;
> there have been occasions where db connections are rejected due to
> this. The recent patchwork update should address some issues there,
> but I'm not sure whether we're back to optimal db behaviour now...
Could it be that the e-mail hasn't been received? Could you check on
the server the logs, and see if the mail has been received at all?
Perhaps it was dropped due to spam detection or something like that.
It would be useful to understand if those e-mails that made it to the
mailing list were received by the patchwork server. We are really
seeing lots of patches being not recorded by patchwork, which makes it
a lot less usable :-/
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded
2018-02-06 8:41 ` Andrew Donnellan
2018-02-06 9:03 ` Jeremy Kerr
@ 2018-02-13 22:19 ` Thomas Petazzoni
2018-02-14 3:56 ` Andrew Donnellan
1 sibling, 1 reply; 21+ messages in thread
From: Thomas Petazzoni @ 2018-02-13 22:19 UTC (permalink / raw)
To: buildroot
Andrew, Jeremy,
On Tue, 6 Feb 2018 19:41:09 +1100, Andrew Donnellan wrote:
> On 06/02/18 08:45, Thomas Petazzoni wrote:
> > Is there anything that can be done about this ? An example of
> > Message-Id that was not recorded is:
> >
> > Message-Id: <8027bae45d8e041c8a1e0bc714ab378ff984ded3.1517820133.git.yann.morin.1998@free.fr>
>
> I've scraped the buildroot archives to see if there's anything obviously
> wrong with that particular message ID - it appears to parse fine.
>
> jk, any idea whether there's something particular about the ozlabs.org
> instance that could be causing it to drop these patches?
Any further comments ?
Today, patchwork missed the following e-mail:
http://lists.busybox.net/pipermail/buildroot/2018-February/213695.html
Patches 1/3 and 2/3 in this series have been recorded, but not Patch
3/3.
This is really getting annoying for the Buildroot project, and we may
potentially "miss" contributions because of this: we entirely rely on
patchwork as our TODO-list, so if a patch is missing in patchwork, we
will forget about it. When only a few patches within a series are
missing, we obviously notice. But for single patches, when they are not
recorded, we simply miss them entirely.
Can we do something about this ? We're really happy otherwise by the
patchwork instance at ozlabs.org, and we would hate having to run our
own instance :-/
Thanks a lot!
Thomas
--
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded
2018-02-13 22:19 ` Thomas Petazzoni
@ 2018-02-14 3:56 ` Andrew Donnellan
2018-02-14 8:01 ` Thomas Petazzoni
0 siblings, 1 reply; 21+ messages in thread
From: Andrew Donnellan @ 2018-02-14 3:56 UTC (permalink / raw)
To: buildroot
On 14/02/18 09:19, Thomas Petazzoni wrote:
> Patches 1/3 and 2/3 in this series have been recorded, but not Patch
> 3/3.
>
> This is really getting annoying for the Buildroot project, and we may
> potentially "miss" contributions because of this: we entirely rely on
> patchwork as our TODO-list, so if a patch is missing in patchwork, we
> will forget about it. When only a few patches within a series are
> missing, we obviously notice. But for single patches, when they are not
> recorded, we simply miss them entirely.
>
> Can we do something about this ? We're really happy otherwise by the
> patchwork instance at ozlabs.org, and we would hate having to run our
> own instance :-/
We've done some digging...
https://patchwork.ozlabs.org/patch/872845/
https://patchwork.ozlabs.org/patch/872846/
These are the other two patches in the series, and yet patchwork has
assigned them two different series, so evidently something screwy is
happening there.
sfr took a look at the mail logs for us, it looks like:
- all 3 emails were received and processed by our SMTP system
- patch 2 was received first, and patches 1 and 3 were received during
the same second a couple of seconds later
Patch 3 has not hit the database at all, it's possible we've hit some
race condition somewhere that prevented it from being saved at the same
time as patch 1, or perhaps we ran into something completely
different... we're going to add some extra logging to see if we can
capture more info next time this happens.
Many apologies for the inconvenience!
Andrew
--
Andrew Donnellan OzLabs, ADL Canberra
andrew.donnellan at au1.ibm.com IBM Australia Limited
^ permalink raw reply [flat|nested] 21+ messages in thread
* [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded
2018-02-14 3:56 ` Andrew Donnellan
@ 2018-02-14 8:01 ` Thomas Petazzoni
[not found] ` <87lgftau6p.fsf@linkitivity.dja.id.au>
0 siblings, 1 reply; 21+ messages in thread
From: Thomas Petazzoni @ 2018-02-14 8:01 UTC (permalink / raw)
To: buildroot
Hello,
On Wed, 14 Feb 2018 14:56:56 +1100, Andrew Donnellan wrote:
> > Can we do something about this ? We're really happy otherwise by the
> > patchwork instance at ozlabs.org, and we would hate having to run our
> > own instance :-/
>
> We've done some digging...
>
> https://patchwork.ozlabs.org/patch/872845/
> https://patchwork.ozlabs.org/patch/872846/
>
> These are the other two patches in the series, and yet patchwork has
> assigned them two different series, so evidently something screwy is
> happening there.
>
> sfr took a look at the mail logs for us, it looks like:
>
> - all 3 emails were received and processed by our SMTP system
> - patch 2 was received first, and patches 1 and 3 were received during
> the same second a couple of seconds later
>
> Patch 3 has not hit the database at all, it's possible we've hit some
> race condition somewhere that prevented it from being saved at the same
> time as patch 1, or perhaps we ran into something completely
> different... we're going to add some extra logging to see if we can
> capture more info next time this happens.
>
> Many apologies for the inconvenience!
As asked by Jeremy, I've changed the patchwork e-mail address to which
the Buildroot patches are being sent, so that you can add more logging,
but only for Buildroot stuff.
Regarding the patches not properly assigned to series, we have already
seen that many times, even in situations were no patches are lost. Some
patches in the series are properly marked as being part of the series,
and some are not. It feels like if some patches are received before the
cover letter, patchwork gets confused, but that's just a guess. It
doesn't happen very often, but it does happen. However, this is a lot
less annoying that "losing" patches.
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2018-07-08 9:34 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-05 20:43 [Buildroot] patchwork.ozlabs.org down, and e-mails not recorded Thomas Petazzoni
2018-01-07 23:33 ` Andrew Donnellan
2018-01-08 8:10 ` Thomas Petazzoni
2018-02-05 21:45 ` Thomas Petazzoni
2018-02-06 8:41 ` Andrew Donnellan
2018-02-06 9:03 ` Jeremy Kerr
2018-02-06 9:55 ` Thomas Petazzoni
2018-02-13 22:19 ` Thomas Petazzoni
2018-02-14 3:56 ` Andrew Donnellan
2018-02-14 8:01 ` Thomas Petazzoni
[not found] ` <87lgftau6p.fsf@linkitivity.dja.id.au>
2018-02-16 14:49 ` Thomas Petazzoni
2018-02-17 3:29 ` Andrew Donnellan
2018-03-10 11:58 ` Peter Korsgaard
[not found] ` <20180312015055.3dc9789c@canb.auug.org.au>
2018-03-12 4:57 ` Jeremy Kerr
2018-03-20 12:19 ` Jeremy Kerr
2018-03-20 15:08 ` Thomas Petazzoni
2018-03-31 16:27 ` Yann E. MORIN
2018-03-31 16:29 ` Yann E. MORIN
2018-07-08 9:34 ` Yann E. MORIN
2018-03-12 19:53 ` daggs
2018-03-12 21:23 ` Peter Korsgaard
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.