From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com> In-Reply-To: From: Todd Kjos Date: Mon, 21 Jun 2021 12:59:21 -0700 Message-ID: Subject: Re: can't access kernelci results Content-Type: multipart/alternative; boundary="000000000000a77c3705c54c1ca9" List-ID: To: Guillaume Tucker Cc: kernelci@groups.io, Guenter Roeck , Kevin Hilman --000000000000a77c3705c54c1ca9 Content-Type: text/plain; charset="UTF-8" I'm having this issue again (can't access kernelci results at https://linux.kernelci.org/job/android/). Is the mongod service borked again? On Mon, Apr 5, 2021 at 4:14 PM Todd Kjos wrote: > Working again. Thanks Guillaume. > > On Mon, Apr 5, 2021 at 1:58 PM Guillaume Tucker < > guillaume.tucker@collabora.com> wrote: > >> On 05/04/2021 21:39, Guenter Roeck via groups.io wrote: >> > On Mon, Apr 5, 2021 at 12:30 PM Todd Kjos via groups.io >> > wrote: >> >> >> >> I get errors when attempting to access >> https://linux.kernelci.org/job/android/. Is this a known issue? Any ETA >> for a fix? >> >> >> > Looks like the problem is not limited to android. I get error 500 for >> > jobs and build results. >> >> That was another case of the mongod service stopping, it's pretty >> rare but has happened a couple of times before. Things are now >> back to normal, but we've missed a few results while it was down. >> Sorry for the disruption. >> >> We've deployed a monitoring tool on some servers but not yet on >> this one, it should happen soon so this kind of issues would be >> detected much quicker. We've also started looking into using a >> Mongo database in Azure rather than a manual install to reduce >> maintenance work. >> >> Best wishes, >> Guillaume >> > --000000000000a77c3705c54c1ca9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

I'm having this issue again (can't access kernelci= results at=C2=A0https:= //linux.kernelci.org/job/android/). Is the mongod service borked again?=

= On Mon, Apr 5, 2021 at 4:14 PM Todd Kjos <tkjos@google.com> wrote:

Working again. Thanks Guillaume= .

On Mon, Apr 5, 2021 at 1:58 PM Guillaume Tucker <guillaume.tucker@collabora.co= m> wrote:
On 05/04/2021 21:39, Guenter Roeck via groups.io wrote:
> On Mon, Apr 5, 2021 at 12:30 PM Todd Kjos via groups.io
> <tkjos=3D= google.com@groups.io> wrote:
>>
>> I get errors when attempting to access https://linux= .kernelci.org/job/android/. Is this a known issue? Any ETA for a fix? >>
> Looks like the problem is not limited to android. I get error 500 for<= br> > jobs and build results.

That was another case of the mongod service stopping, it's pretty
rare but has happened a couple of times before.=C2=A0 Things are now
back to normal, but we've missed a few results while it was down.
Sorry for the disruption.

We've deployed a monitoring tool on some servers but not yet on
this one, it should happen soon so this kind of issues would be
detected much quicker.=C2=A0 We've also started looking into using a Mongo database in Azure rather than a manual install to reduce
maintenance work.

Best wishes,
Guillaume

--000000000000a77c3705c54c1ca9-- From mboxrd@z Thu Jan 1 00:00:00 1970 Subject: Re: can't access kernelci results References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

From: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= Message-ID: <285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com> Date: Tue, 22 Jun 2021 11:43:36 +0200 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable List-ID: To: kernelci@groups.io, tkjos@google.com, Guillaume Tucker Cc: Guenter Roeck , Kevin Hilman On 21.06.2021 21:59, Todd Kjos via groups.io wrote: > I'm having this issue again (can't access kernelci results > at=C2=A0https://linux.kernelci.org/job/android/ > ). Is the mongod service borked > again? >=20 Yes, there was an issue with mongod service again. It's working now, celery processed pending tasks and everything seems responsive. Thanks, Micha=C5=82 From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 From: Todd Kjos Date: Mon, 5 Apr 2021 12:02:55 -0700 Message-ID: Subject: can't access kernelci results Content-Type: multipart/alternative; boundary="0000000000000f698405bf3e5939" List-ID: To: kernelci@groups.io, Kevin Hilman --0000000000000f698405bf3e5939 Content-Type: text/plain; charset="UTF-8" I get errors when attempting to access https://linux.kernelci.org/job/android/. Is this a known issue? Any ETA for a fix? -Todd --0000000000000f698405bf3e5939 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

I get errors when attempting to access=C2=A0https://linux.kernelci.org/job/androi= d/. Is this a known=C2=A0issue? Any ETA for a fix?

-= Todd

--0000000000000f698405bf3e5939-- From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com> In-Reply-To: <285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com> From: Todd Kjos Date: Tue, 22 Jun 2021 08:07:12 -0700 Message-ID: Subject: Re: can't access kernelci results Content-Type: multipart/alternative; boundary="000000000000d6114805c55c252c" List-ID: To: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= Cc: kernelci@groups.io, Guillaume Tucker , Guenter Roeck , Kevin Hilman --000000000000d6114805c55c252c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hmm, it's still not working for me -- did it break again? On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka wrote: > On 21.06.2021 21:59, Todd Kjos via groups.io wrote: > > I'm having this issue again (can't access kernelci results > > at https://linux.kernelci.org/job/android/ > > ). Is the mongod service borke= d > > again? > > > > Yes, there was an issue with mongod service again. It's working now, > celery processed pending tasks and everything seems responsive. > > Thanks, > Micha=C5=82 > --000000000000d6114805c55c252c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Hmm, it's still=C2=A0not working for me -- did it brea= k again?

On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka <michal.galka@collabora.com> w= rote:

On 21.06.2= 021 21:59, Todd Kjos via groups.io wrote:
> I'm having this issue again (can't access kernelci results
> at=C2=A0https://linux.kernelci.org/job/android/
> <https://linux.kernelci.org/job/android/>). Is= the mongod service borked
> again?
>

Yes, there was an issue with mongod service again. It's working now, celery processed pending tasks and everything seems responsive.

Thanks,
Micha=C5=82

--000000000000d6114805c55c252c-- From mboxrd@z Thu Jan 1 00:00:00 1970 Subject: Re: can't access kernelci results References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com> From: "Guillaume Tucker" Message-ID: Date: Tue, 22 Jun 2021 16:16:56 +0100 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable List-ID: To: Todd Kjos , =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= Cc: kernelci@groups.io, Guenter Roeck , Kevin Hilman On 22/06/2021 16:07, Todd Kjos wrote: > Hmm, it's still=C2=A0not working for me -- did it break again? Yes there was an OOM issue: [4316410.416514] Killed process 20620 (mongod) total-vm:20171340kB, anon-r= ss:18654740kB, file-rss:0kB, shmem-rss:0kB [4316412.060511] oom_reaper: reaped process 20620 (mongod), now anon-rss:0= kB, file-rss:0kB, shmem-rss:0kB We're looking into moving the Mongo DB instance to the cloud in Azure. Hopefully that will save us fixing this kind of maintenance issues. Meanwhile, we should take a look and see if there's some config option to limit the amount of RAM used by Mongo DB to work around the problem. Best wishes, Guillaume > On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka > wrote: >=20 > On 21.06.2021 21:59, Todd Kjos via groups.io wrot= e: > > I'm having this issue again (can't access kernelci results > > at=C2=A0https://linux.kernelci.org/job/android/ > > ). Is the mongod service = borked > > again? > > >=20 > Yes, there was an issue with mongod service again. It's working now, > celery processed pending tasks and everything seems responsive. >=20 > Thanks, > Micha=C5=82 >=20 From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: In-Reply-To: From: "Guenter Roeck" Date: Mon, 5 Apr 2021 13:39:34 -0700 Message-ID: Subject: Re: can't access kernelci results Content-Type: text/plain; charset="UTF-8" List-ID: To: kernelci@groups.io, Todd Kjos Cc: Kevin Hilman On Mon, Apr 5, 2021 at 12:30 PM Todd Kjos via groups.io wrote: > > I get errors when attempting to access https://linux.kernelci.org/job/android/. Is this a known issue? Any ETA for a fix? > Looks like the problem is not limited to android. I get error 500 for jobs and build results. Guenter > -Todd > From mboxrd@z Thu Jan 1 00:00:00 1970 Subject: Re: can't access kernelci results References: From: "Guillaume Tucker" Message-ID: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com> Date: Mon, 5 Apr 2021 21:58:33 +0100 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit List-ID: To: kernelci@groups.io, groeck@google.com, Todd Kjos Cc: Kevin Hilman On 05/04/2021 21:39, Guenter Roeck via groups.io wrote: > On Mon, Apr 5, 2021 at 12:30 PM Todd Kjos via groups.io > wrote: >> >> I get errors when attempting to access https://linux.kernelci.org/job/android/. Is this a known issue? Any ETA for a fix? >> > Looks like the problem is not limited to android. I get error 500 for > jobs and build results. That was another case of the mongod service stopping, it's pretty rare but has happened a couple of times before. Things are now back to normal, but we've missed a few results while it was down. Sorry for the disruption. We've deployed a monitoring tool on some servers but not yet on this one, it should happen soon so this kind of issues would be detected much quicker. We've also started looking into using a Mongo database in Azure rather than a manual install to reduce maintenance work. Best wishes, Guillaume From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com> In-Reply-To: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com> From: Todd Kjos Date: Mon, 5 Apr 2021 16:14:50 -0700 Message-ID: Subject: Re: can't access kernelci results Content-Type: multipart/alternative; boundary="00000000000026774f05bf41debd" List-ID: To: Guillaume Tucker Cc: kernelci@groups.io, Guenter Roeck , Kevin Hilman --00000000000026774f05bf41debd Content-Type: text/plain; charset="UTF-8" Working again. Thanks Guillaume. On Mon, Apr 5, 2021 at 1:58 PM Guillaume Tucker < guillaume.tucker@collabora.com> wrote: > On 05/04/2021 21:39, Guenter Roeck via groups.io wrote: > > On Mon, Apr 5, 2021 at 12:30 PM Todd Kjos via groups.io > > wrote: > >> > >> I get errors when attempting to access > https://linux.kernelci.org/job/android/. Is this a known issue? Any ETA > for a fix? > >> > > Looks like the problem is not limited to android. I get error 500 for > > jobs and build results. > > That was another case of the mongod service stopping, it's pretty > rare but has happened a couple of times before. Things are now > back to normal, but we've missed a few results while it was down. > Sorry for the disruption. > > We've deployed a monitoring tool on some servers but not yet on > this one, it should happen soon so this kind of issues would be > detected much quicker. We've also started looking into using a > Mongo database in Azure rather than a manual install to reduce > maintenance work. > > Best wishes, > Guillaume > --00000000000026774f05bf41debd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Working again. Thanks Guillaume.

On Mon, Apr 5, 2021 = at 1:58 PM Guillaume Tucker <guillaume.tucker@collabora.com> wrote:

On 05/04/2021 21:39, Guenter Roeck via= groups.i= o wrote:
> On Mon, Apr 5, 2021 at 12:30 PM Todd Kjos via groups.io
> <tkjos=3D= google.com@groups.io> wrote:
>>
>> I get errors when attempting to access https://linux= .kernelci.org/job/android/. Is this a known issue? Any ETA for a fix? >>
> Looks like the problem is not limited to android. I get error 500 for<= br> > jobs and build results.

That was another case of the mongod service stopping, it's pretty
rare but has happened a couple of times before.=C2=A0 Things are now
back to normal, but we've missed a few results while it was down.
Sorry for the disruption.

We've deployed a monitoring tool on some servers but not yet on
this one, it should happen soon so this kind of issues would be
detected much quicker.=C2=A0 We've also started looking into using a Mongo database in Azure rather than a manual install to reduce
maintenance work.

Best wishes,
Guillaume

--00000000000026774f05bf41debd-- From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

In-Reply-To: From: Todd Kjos Date: Thu, 2 Sep 2021 08:58:10 -0700 Message-ID: Subject: Re: can't access kernelci results Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: Guillaume Tucker Cc: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , kernelci@groups.io, Guenter Roeck , Kevin Hilman I'm getting errors again when attempting to access https://linux.kernelci.org/job/android/. mongod issues again? "Error while loading data from the server (error code: 500). Please contact the website administrator" On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker wrote: > > On 22/06/2021 16:07, Todd Kjos wrote: > > Hmm, it's still not working for me -- did it break again? > > Yes there was an OOM issue: > > [4316410.416514] Killed process 20620 (mongod) total-vm:20171340kB, anon-= rss:18654740kB, file-rss:0kB, shmem-rss:0kB > [4316412.060511] oom_reaper: reaped process 20620 (mongod), now anon-rss:= 0kB, file-rss:0kB, shmem-rss:0kB > > We're looking into moving the Mongo DB instance to the cloud in > Azure. Hopefully that will save us fixing this kind of > maintenance issues. Meanwhile, we should take a look and see if > there's some config option to limit the amount of RAM used by > Mongo DB to work around the problem. > > Best wishes, > Guillaume > > > > On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka > wrote: > > > > On 21.06.2021 21:59, Todd Kjos via groups.io wro= te: > > > I'm having this issue again (can't access kernelci results > > > at https://linux.kernelci.org/job/android/ > > > >). Is the mongod service borked > > > again? > > > > > > > Yes, there was an issue with mongod service again. It's working now= , > > celery processed pending tasks and everything seems responsive. > > > > Thanks, > > Micha=C5=82 > > > From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

In-Reply-To: From: Todd Kjos Date: Thu, 2 Sep 2021 12:28:28 -0700 Message-ID: Subject: Re: can't access kernelci results Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: Guillaume Tucker Cc: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , kernelci@groups.io, Guenter Roeck , Kevin Hilman It's working again. Thanks. On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos wrote: > > I'm getting errors again when attempting to access > https://linux.kernelci.org/job/android/. mongod issues again? > > "Error while loading data from the server (error code: 500). Please > contact the website administrator" > > On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker > wrote: > > > > On 22/06/2021 16:07, Todd Kjos wrote: > > > Hmm, it's still not working for me -- did it break again? > > > > Yes there was an OOM issue: > > > > [4316410.416514] Killed process 20620 (mongod) total-vm:20171340kB, ano= n-rss:18654740kB, file-rss:0kB, shmem-rss:0kB > > [4316412.060511] oom_reaper: reaped process 20620 (mongod), now anon-rs= s:0kB, file-rss:0kB, shmem-rss:0kB > > > > We're looking into moving the Mongo DB instance to the cloud in > > Azure. Hopefully that will save us fixing this kind of > > maintenance issues. Meanwhile, we should take a look and see if > > there's some config option to limit the amount of RAM used by > > Mongo DB to work around the problem. > > > > Best wishes, > > Guillaume > > > > > > > On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka > wrote: > > > > > > On 21.06.2021 21:59, Todd Kjos via groups.io w= rote: > > > > I'm having this issue again (can't access kernelci results > > > > at https://linux.kernelci.org/job/android/ > > > > >). Is the mongod service borked > > > > again? > > > > > > > > > > Yes, there was an issue with mongod service again. It's working n= ow, > > > celery processed pending tasks and everything seems responsive. > > > > > > Thanks, > > > Micha=C5=82 > > > > > From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

In-Reply-To: From: Todd Kjos Date: Tue, 30 Nov 2021 08:35:02 -0800 Message-ID: Subject: Re: can't access kernelci results Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: Guillaume Tucker Cc: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , kernelci@groups.io, Guenter Roeck , Kevin Hilman I can't load the data from https://linux.kernelci.org/job/android/ ("Error loading build data."). Is mongod "borked" again? On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos wrote: > > It's working again. Thanks. > > On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos wrote: > > > > I'm getting errors again when attempting to access > > https://linux.kernelci.org/job/android/. mongod issues again? > > > > "Error while loading data from the server (error code: 500). Please > > contact the website administrator" > > > > On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker > > wrote: > > > > > > On 22/06/2021 16:07, Todd Kjos wrote: > > > > Hmm, it's still not working for me -- did it break again? > > > > > > Yes there was an OOM issue: > > > > > > [4316410.416514] Killed process 20620 (mongod) total-vm:20171340kB, a= non-rss:18654740kB, file-rss:0kB, shmem-rss:0kB > > > [4316412.060511] oom_reaper: reaped process 20620 (mongod), now anon-= rss:0kB, file-rss:0kB, shmem-rss:0kB > > > > > > We're looking into moving the Mongo DB instance to the cloud in > > > Azure. Hopefully that will save us fixing this kind of > > > maintenance issues. Meanwhile, we should take a look and see if > > > there's some config option to limit the amount of RAM used by > > > Mongo DB to work around the problem. > > > > > > Best wishes, > > > Guillaume > > > > > > > > > > On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka > wrote: > > > > > > > > On 21.06.2021 21:59, Todd Kjos via groups.io = wrote: > > > > > I'm having this issue again (can't access kernelci results > > > > > at https://linux.kernelci.org/job/android/ > > > > > >). Is the mongod service borked > > > > > again? > > > > > > > > > > > > > Yes, there was an issue with mongod service again. It's working= now, > > > > celery processed pending tasks and everything seems responsive. > > > > > > > > Thanks, > > > > Micha=C5=82 > > > > > > > From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

In-Reply-To: From: Todd Kjos Date: Tue, 30 Nov 2021 08:54:51 -0800 Message-ID: Subject: Re: can't access kernelci results Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: Guillaume Tucker Cc: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , kernelci@groups.io, Guenter Roeck , Kevin Hilman ...and now it's working again. Sorry about the noise if the issue was on my side. On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos wrote: > > I can't load the data from https://linux.kernelci.org/job/android/ > ("Error loading build data."). Is mongod "borked" again? > > On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos wrote: > > > > It's working again. Thanks. > > > > On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos wrote: > > > > > > I'm getting errors again when attempting to access > > > https://linux.kernelci.org/job/android/. mongod issues again? > > > > > > "Error while loading data from the server (error code: 500). Please > > > contact the website administrator" > > > > > > On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker > > > wrote: > > > > > > > > On 22/06/2021 16:07, Todd Kjos wrote: > > > > > Hmm, it's still not working for me -- did it break again? > > > > > > > > Yes there was an OOM issue: > > > > > > > > [4316410.416514] Killed process 20620 (mongod) total-vm:20171340kB,= anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB > > > > [4316412.060511] oom_reaper: reaped process 20620 (mongod), now ano= n-rss:0kB, file-rss:0kB, shmem-rss:0kB > > > > > > > > We're looking into moving the Mongo DB instance to the cloud in > > > > Azure. Hopefully that will save us fixing this kind of > > > > maintenance issues. Meanwhile, we should take a look and see if > > > > there's some config option to limit the amount of RAM used by > > > > Mongo DB to work around the problem. > > > > > > > > Best wishes, > > > > Guillaume > > > > > > > > > > > > > On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka > wrote: > > > > > > > > > > On 21.06.2021 21:59, Todd Kjos via groups.io wrote: > > > > > > I'm having this issue again (can't access kernelci results > > > > > > at https://linux.kernelci.org/job/android/ > > > > > > >). Is the mongod service borked > > > > > > again? > > > > > > > > > > > > > > > > Yes, there was an issue with mongod service again. It's worki= ng now, > > > > > celery processed pending tasks and everything seems responsiv= e. > > > > > > > > > > Thanks, > > > > > Micha=C5=82 > > > > > > > > > From mboxrd@z Thu Jan 1 00:00:00 1970 Subject: Re: can't access kernelci results References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

From: "Guillaume Tucker" Message-ID: <803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> Date: Wed, 1 Dec 2021 09:48:58 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable List-ID: To: Todd Kjos Cc: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , kernelci@groups.io, Guenter Roeck , Kevin Hilman Hi Todd, On 30/11/2021 16:54, Todd Kjos wrote: > ...and now it's working again. Sorry about the noise if the issue was > on my side. The MongoDB server did crash but it was restarted a few minutes later. There's some kind of watchdog to do that automatically, maybe you were out of luck and checked the website during that window. Thanks for reporting the issue in any case. Best wishes, Guillaume > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos wrote: >> >> I can't load the data from https://linux.kernelci.org/job/android/ >> ("Error loading build data."). Is mongod "borked" again? >> >> On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos wrote: >>> >>> It's working again. Thanks. >>> >>> On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos wrote: >>>> >>>> I'm getting errors again when attempting to access >>>> https://linux.kernelci.org/job/android/. mongod issues again? >>>> >>>> "Error while loading data from the server (error code: 500). Please >>>> contact the website administrator" >>>> >>>> On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker >>>> wrote: >>>>> >>>>> On 22/06/2021 16:07, Todd Kjos wrote: >>>>>> Hmm, it's still not working for me -- did it break again? >>>>> >>>>> Yes there was an OOM issue: >>>>> >>>>> [4316410.416514] Killed process 20620 (mongod) total-vm:20171340kB, a= non-rss:18654740kB, file-rss:0kB, shmem-rss:0kB >>>>> [4316412.060511] oom_reaper: reaped process 20620 (mongod), now anon-= rss:0kB, file-rss:0kB, shmem-rss:0kB >>>>> >>>>> We're looking into moving the Mongo DB instance to the cloud in >>>>> Azure. Hopefully that will save us fixing this kind of >>>>> maintenance issues. Meanwhile, we should take a look and see if >>>>> there's some config option to limit the amount of RAM used by >>>>> Mongo DB to work around the problem. >>>>> >>>>> Best wishes, >>>>> Guillaume >>>>> >>>>> >>>>>> On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka > wrote: >>>>>> >>>>>> On 21.06.2021 21:59, Todd Kjos via groups.io = wrote: >>>>>> > I'm having this issue again (can't access kernelci results >>>>>> > at https://linux.kernelci.org/job/android/ >>>>>> > ). Is the mongod serv= ice borked >>>>>> > again? >>>>>> > >>>>>> >>>>>> Yes, there was an issue with mongod service again. It's working = now, >>>>>> celery processed pending tasks and everything seems responsive. >>>>>> >>>>>> Thanks, >>>>>> Micha=C5=82 >>>>>> >>>>> From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

<803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> In-Reply-To: <803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> From: Todd Kjos Date: Fri, 18 Mar 2022 08:47:22 -0700 Message-ID: Subject: Re: can't access kernelci results Content-Type: multipart/alternative; boundary="000000000000bfaf5405da801043" List-ID: To: Guillaume Tucker Cc: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , kernelci@groups.io, Guenter Roeck , Kevin Hilman --000000000000bfaf5405da801043 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Is the MongoDB server down again? I don't get results at https://linux.kernelci.org/job/android/. On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tucker < guillaume.tucker@collabora.com> wrote: > Hi Todd, > > On 30/11/2021 16:54, Todd Kjos wrote: > > ...and now it's working again. Sorry about the noise if the issue was > > on my side. > > The MongoDB server did crash but it was restarted a few minutes > later. There's some kind of watchdog to do that automatically, > maybe you were out of luck and checked the website during that > window. Thanks for reporting the issue in any case. > > Best wishes, > Guillaume > > > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos wrote: > >> > >> I can't load the data from https://linux.kernelci.org/job/android/ > >> ("Error loading build data."). Is mongod "borked" again? > >> > >> On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos wrote: > >>> > >>> It's working again. Thanks. > >>> > >>> On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos wrote: > >>>> > >>>> I'm getting errors again when attempting to access > >>>> https://linux.kernelci.org/job/android/. mongod issues again? > >>>> > >>>> "Error while loading data from the server (error code: 500). Please > >>>> contact the website administrator" > >>>> > >>>> On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker > >>>> wrote: > >>>>> > >>>>> On 22/06/2021 16:07, Todd Kjos wrote: > >>>>>> Hmm, it's still not working for me -- did it break again? > >>>>> > >>>>> Yes there was an OOM issue: > >>>>> > >>>>> [4316410.416514] Killed process 20620 (mongod) total-vm:20171340kB, > anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB > >>>>> [4316412.060511] oom_reaper: reaped process 20620 (mongod), now > anon-rss:0kB, file-rss:0kB, shmem-rss:0kB > >>>>> > >>>>> We're looking into moving the Mongo DB instance to the cloud in > >>>>> Azure. Hopefully that will save us fixing this kind of > >>>>> maintenance issues. Meanwhile, we should take a look and see if > >>>>> there's some config option to limit the amount of RAM used by > >>>>> Mongo DB to work around the problem. > >>>>> > >>>>> Best wishes, > >>>>> Guillaume > >>>>> > >>>>> > >>>>>> On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka < > michal.galka@collabora.com > wrote: > >>>>>> > >>>>>> On 21.06.2021 21:59, Todd Kjos via groups.io > wrote: > >>>>>> > I'm having this issue again (can't access kernelci results > >>>>>> > at https://linux.kernelci.org/job/android/ < > https://linux.kernelci.org/job/android/> > >>>>>> > https://linux.kernelci.org/job/android/>>). Is the mongod service borked > >>>>>> > again? > >>>>>> > > >>>>>> > >>>>>> Yes, there was an issue with mongod service again. It's workin= g > now, > >>>>>> celery processed pending tasks and everything seems responsive= . > >>>>>> > >>>>>> Thanks, > >>>>>> Micha=C5=82 > >>>>>> > >>>>> > > --000000000000bfaf5405da801043 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Is the MongoDB server down again? I don't get results = at=C2=A0https://linux.k= ernelci.org/job/android/.

On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tuck= er <guillaume.tucker@c= ollabora.com> wrote:

Hi Todd,

On 30/11/2021 16:54, Todd Kjos wrote:
> ...and now it's working again. Sorry about the noise if the issue = was
> on my side.

The MongoDB server did crash but it was restarted a few minutes
later.=C2=A0 There's some kind of watchdog to do that automatically, maybe you were out of luck and checked the website during that
window.=C2=A0 Thanks for reporting the issue in any case.

Best wishes,
Guillaume

> On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos <tkjos@google.com> wrote:
>>
>> I can't load the data from https://linux.kernelc= i.org/job/android/
>> ("Error loading build data."). Is mongod "borked&qu= ot; again?
>>
>> On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos <tkjos@google.com> wrote:
>>>
>>> It's working again. Thanks.
>>>
>>> On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos <tkjos@google.com> wrote:
>>>>
>>>> I'm getting errors again when attempting to access
>>>> https://linux.kernelci.org/job/android/.= mongod issues again?
>>>>
>>>> "Error while loading data from the server (error code= : 500). Please
>>>> contact the website administrator"
>>>>
>>>> On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker
>>>> <guillaume.tucker@collabora.com> wrote:
>>>>>
>>>>> On 22/06/2021 16:07, Todd Kjos wrote:
>>>>>> Hmm, it's still not working for me -- did it b= reak again?
>>>>>
>>>>> Yes there was an OOM issue:
>>>>>
>>>>> [4316410.416514] Killed process 20620 (mongod) total-v= m:20171340kB, anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB
>>>>> [4316412.060511] oom_reaper: reaped process 20620 (mon= god), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
>>>>>
>>>>> We're looking into moving the Mongo DB instance to= the cloud in
>>>>> Azure.=C2=A0 Hopefully that will save us fixing this k= ind of
>>>>> maintenance issues.=C2=A0 Meanwhile, we should take a = look and see if
>>>>> there's some config option to limit the amount of = RAM used by
>>>>> Mongo DB to work around the problem.
>>>>>
>>>>> Best wishes,
>>>>> Guillaume
>>>>>
>>>>>
>>>>>> On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5= =82ka <m= ichal.galka@collabora.com <mailto:michal.galka@collabora.com>> wrote:=
>>>>>>
>>>>>>=C2=A0 =C2=A0 =C2=A0On 21.06.2021 21:59, Todd Kjos = via group= s.io <http://groups.io> wrote:
>>>>>>=C2=A0 =C2=A0 =C2=A0> I'm having this issue = again (can't access kernelci results
>>>>>>=C2=A0 =C2=A0 =C2=A0> at https://l= inux.kernelci.org/job/android/ <https://linux.kernelci= .org/job/android/>
>>>>>>=C2=A0 =C2=A0 =C2=A0> <https://= linux.kernelci.org/job/android/ <https://linux.kernelc= i.org/job/android/>>). Is the mongod service borked
>>>>>>=C2=A0 =C2=A0 =C2=A0> again?
>>>>>>=C2=A0 =C2=A0 =C2=A0>
>>>>>>
>>>>>>=C2=A0 =C2=A0 =C2=A0Yes, there was an issue with mo= ngod service again. It's working now,
>>>>>>=C2=A0 =C2=A0 =C2=A0celery processed pending tasks = and everything seems responsive.
>>>>>>
>>>>>>=C2=A0 =C2=A0 =C2=A0Thanks,
>>>>>>=C2=A0 =C2=A0 =C2=A0Micha=C5=82
>>>>>>
>>>>>

--000000000000bfaf5405da801043-- From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <2059d7b9-2ae1-af0e-5231-1e8c9803f027@collabora.com> Date: Fri, 18 Mar 2022 17:19:27 +0000 MIME-Version: 1.0 Subject: Re: can't access kernelci results References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

<803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> From: "Guillaume Tucker" In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable List-ID: To: Todd Kjos Cc: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , kernelci@groups.io, Guenter Roeck , Kevin Hilman On 18/03/2022 15:47, Todd Kjos wrote: > Is the MongoDB server down again? I don't get results at=C2=A0https://lin= ux.kernelci.org/job/android/. It's not that, unfortunately there's some persisting issue which seems to h= ave started yesterday. We're investigating now, please bear with us. Thank = you for reporting the problem. Best wishes, Guillaume > > On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tucker wrote: > > Hi Todd, > > On 30/11/2021 16:54, Todd Kjos wrote: > > ...and now it's working again. Sorry about the noise if the issue w= as > > on my side. > > The MongoDB server did crash but it was restarted a few minutes > later.=C2=A0 There's some kind of watchdog to do that automatically, > maybe you were out of luck and checked the website during that > window.=C2=A0 Thanks for reporting the issue in any case. > > Best wishes, > Guillaume > > > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos wrote: > >> > >> I can't load the data from https://linux.kernelci.org/job/android/ > >> ("Error loading build data."). Is mongod "borked" again? > >> > >> On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos wrote= : > >>> > >>> It's working again. Thanks. > >>> > >>> On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos wrote= : > >>>> > >>>> I'm getting errors again when attempting to access > >>>> https://linux.kernelci.org/job/android/. mongod issues again? > >>>> > >>>> "Error while loading data from the server (error code: 500). Ple= ase > >>>> contact the website administrator" > >>>> > >>>> On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker > >>>> wrote: > >>>>> > >>>>> On 22/06/2021 16:07, Todd Kjos wrote: > >>>>>> Hmm, it's still not working for me -- did it break again? > >>>>> > >>>>> Yes there was an OOM issue: > >>>>> > >>>>> [4316410.416514] Killed process 20620 (mongod) total-vm:2017134= 0kB, anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB > >>>>> [4316412.060511] oom_reaper: reaped process 20620 (mongod), now= anon-rss:0kB, file-rss:0kB, shmem-rss:0kB > >>>>> > >>>>> We're looking into moving the Mongo DB instance to the cloud in > >>>>> Azure.=C2=A0 Hopefully that will save us fixing this kind of > >>>>> maintenance issues.=C2=A0 Meanwhile, we should take a look and = see if > >>>>> there's some config option to limit the amount of RAM used by > >>>>> Mongo DB to work around the problem. > >>>>> > >>>>> Best wishes, > >>>>> Guillaume > >>>>> > >>>>> > >>>>>> On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka > wrote: > >>>>>> > >>>>>>=C2=A0 =C2=A0 =C2=A0On 21.06.2021 21:59, Todd Kjos via groups.i= o wrote: > >>>>>>=C2=A0 =C2=A0 =C2=A0> I'm having this issue again (can't access= kernelci results > >>>>>>=C2=A0 =C2=A0 =C2=A0> at https://linux.kernelci.org/job/android= / > >>>>>>=C2=A0 =C2=A0 =C2=A0> = ). Is the mongod service borked > >>>>>>=C2=A0 =C2=A0 =C2=A0> again? > >>>>>>=C2=A0 =C2=A0 =C2=A0> > >>>>>> > >>>>>>=C2=A0 =C2=A0 =C2=A0Yes, there was an issue with mongod service= again. It's working now, > >>>>>>=C2=A0 =C2=A0 =C2=A0celery processed pending tasks and everythi= ng seems responsive. > >>>>>> > >>>>>>=C2=A0 =C2=A0 =C2=A0Thanks, > >>>>>>=C2=A0 =C2=A0 =C2=A0Micha=C5=82 > >>>>>> > >>>>> > From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: Date: Fri, 18 Mar 2022 17:26:46 +0000 MIME-Version: 1.0 Subject: Re: can't access kernelci results From: "Guillaume Tucker" References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

<803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> <2059d7b9-2ae1-af0e-5231-1e8c9803f027@collabora.com> In-Reply-To: <2059d7b9-2ae1-af0e-5231-1e8c9803f027@collabora.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable List-ID: To: Todd Kjos Cc: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , kernelci@groups.io, Guenter Roeck , Kevin Hilman On 18/03/2022 17:19, Guillaume Tucker wrote: > On 18/03/2022 15:47, Todd Kjos wrote: >> Is the MongoDB server down again? I don't get results at=C2=A0https://li= nux.kernelci.org/job/android/. > > It's not that, unfortunately there's some persisting issue which seems to= have started yesterday. We're investigating now, please bear with us. Than= k you for reporting the problem. > Alright, there was a problem with background processing tasks on the server= . That's resolved now, although a number of kernel builds need to be restar= ted which is why so many trees appear to be "in progress" on the web dashbo= ard. Things will start picking up again gradually, it might take a few hour= s to get back to normal. Sorry for the inconvenience. Best wishes, Guillaume >> >> On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tucker wrote: >> >> Hi Todd, >> >> On 30/11/2021 16:54, Todd Kjos wrote: >> > ...and now it's working again. Sorry about the noise if the issue = was >> > on my side. >> >> The MongoDB server did crash but it was restarted a few minutes >> later.=C2=A0 There's some kind of watchdog to do that automatically, >> maybe you were out of luck and checked the website during that >> window.=C2=A0 Thanks for reporting the issue in any case. >> >> Best wishes, >> Guillaume >> >> > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos wrote= : >> >> >> >> I can't load the data from https://linux.kernelci.org/job/android= / >> >> ("Error loading build data."). Is mongod "borked" again? >> >> >> >> On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos wrot= e: >> >>> >> >>> It's working again. Thanks. >> >>> >> >>> On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos wrot= e: >> >>>> >> >>>> I'm getting errors again when attempting to access >> >>>> https://linux.kernelci.org/job/android/. mongod issues again? >> >>>> >> >>>> "Error while loading data from the server (error code: 500). Pl= ease >> >>>> contact the website administrator" >> >>>> >> >>>> On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker >> >>>> wrote: >> >>>>> >> >>>>> On 22/06/2021 16:07, Todd Kjos wrote: >> >>>>>> Hmm, it's still not working for me -- did it break again? >> >>>>> >> >>>>> Yes there was an OOM issue: >> >>>>> >> >>>>> [4316410.416514] Killed process 20620 (mongod) total-vm:201713= 40kB, anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB >> >>>>> [4316412.060511] oom_reaper: reaped process 20620 (mongod), no= w anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >> >>>>> >> >>>>> We're looking into moving the Mongo DB instance to the cloud i= n >> >>>>> Azure.=C2=A0 Hopefully that will save us fixing this kind of >> >>>>> maintenance issues.=C2=A0 Meanwhile, we should take a look and= see if >> >>>>> there's some config option to limit the amount of RAM used by >> >>>>> Mongo DB to work around the problem. >> >>>>> >> >>>>> Best wishes, >> >>>>> Guillaume >> >>>>> >> >>>>> >> >>>>>> On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka > wrote: >> >>>>>> >> >>>>>>=C2=A0 =C2=A0 =C2=A0On 21.06.2021 21:59, Todd Kjos via groups.= io wrote: >> >>>>>>=C2=A0 =C2=A0 =C2=A0> I'm having this issue again (can't acces= s kernelci results >> >>>>>>=C2=A0 =C2=A0 =C2=A0> at https://linux.kernelci.org/job/androi= d/ >> >>>>>>=C2=A0 =C2=A0 =C2=A0> ). Is the mongod service borked >> >>>>>>=C2=A0 =C2=A0 =C2=A0> again? >> >>>>>>=C2=A0 =C2=A0 =C2=A0> >> >>>>>> >> >>>>>>=C2=A0 =C2=A0 =C2=A0Yes, there was an issue with mongod servic= e again. It's working now, >> >>>>>>=C2=A0 =C2=A0 =C2=A0celery processed pending tasks and everyth= ing seems responsive. >> >>>>>> >> >>>>>>=C2=A0 =C2=A0 =C2=A0Thanks, >> >>>>>>=C2=A0 =C2=A0 =C2=A0Micha=C5=82 >> >>>>>> >> >>>>> >> > From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

<803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> <2059d7b9-2ae1-af0e-5231-1e8c9803f027@collabora.com> In-Reply-To: From: Todd Kjos Date: Mon, 26 Sep 2022 08:29:12 -0700 Message-ID: Subject: Re: can't access kernelci results Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: Guillaume Tucker Cc: =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , kernelci@groups.io, Guenter Roeck , Kevin Hilman Having trouble accessing https://linux.kernelci.org/job/android/ today - server issues? On Fri, Mar 18, 2022 at 10:26 AM Guillaume Tucker < guillaume.tucker@collabora.com> wrote: > On 18/03/2022 17:19, Guillaume Tucker wrote: > > On 18/03/2022 15:47, Todd Kjos wrote: > > Is the MongoDB server down again? I don't get results at > https://linux.kernelci.org/job/android/. > > It's not that, unfortunately there's some persisting issue which seems to > have started yesterday. We're investigating now, please bear with us. Tha= nk > you for reporting the problem. > > Alright, there was a problem with background processing tasks on the > server. That's resolved now, although a number of kernel builds need to b= e > restarted which is why so many trees appear to be "in progress" on the we= b > dashboard. Things will start picking up again gradually, it might take a > few hours to get back to normal. Sorry for the inconvenience. > > Best wishes, > Guillaume > > > On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tucker < > guillaume.tucker@collabora.com> wrote: > >> Hi Todd, >> >> On 30/11/2021 16:54, Todd Kjos wrote: >> > ...and now it's working again. Sorry about the noise if the issue was >> > on my side. >> >> The MongoDB server did crash but it was restarted a few minutes >> later. There's some kind of watchdog to do that automatically, >> maybe you were out of luck and checked the website during that >> window. Thanks for reporting the issue in any case. >> >> Best wishes, >> Guillaume >> >> > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos wrote: >> >> >> >> I can't load the data from https://linux.kernelci.org/job/android/ >> >> ("Error loading build data."). Is mongod "borked" again? >> >> >> >> On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos wrote: >> >>> >> >>> It's working again. Thanks. >> >>> >> >>> On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos wrote: >> >>>> >> >>>> I'm getting errors again when attempting to access >> >>>> https://linux.kernelci.org/job/android/. mongod issues again? >> >>>> >> >>>> "Error while loading data from the server (error code: 500). Please >> >>>> contact the website administrator" >> >>>> >> >>>> On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker >> >>>> wrote: >> >>>>> >> >>>>> On 22/06/2021 16:07, Todd Kjos wrote: >> >>>>>> Hmm, it's still not working for me -- did it break again? >> >>>>> >> >>>>> Yes there was an OOM issue: >> >>>>> >> >>>>> [4316410.416514] Killed process 20620 (mongod) total-vm:20171340kB= , >> anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB >> >>>>> [4316412.060511] oom_reaper: reaped process 20620 (mongod), now >> anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >> >>>>> >> >>>>> We're looking into moving the Mongo DB instance to the cloud in >> >>>>> Azure. Hopefully that will save us fixing this kind of >> >>>>> maintenance issues. Meanwhile, we should take a look and see if >> >>>>> there's some config option to limit the amount of RAM used by >> >>>>> Mongo DB to work around the problem. >> >>>>> >> >>>>> Best wishes, >> >>>>> Guillaume >> >>>>> >> >>>>> >> >>>>>> On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka < >> michal.galka@collabora.com > wrote: >> >>>>>> >> >>>>>> On 21.06.2021 21:59, Todd Kjos via groups.io >> wrote: >> >>>>>> > I'm having this issue again (can't access kernelci results >> >>>>>> > at https://linux.kernelci.org/job/android/ < >> https://linux.kernelci.org/job/android/> >> >>>>>> > > https://linux.kernelci.org/job/android/>>). Is the mongod service borked >> >>>>>> > again? >> >>>>>> > >> >>>>>> >> >>>>>> Yes, there was an issue with mongod service again. It's >> working now, >> >>>>>> celery processed pending tasks and everything seems responsiv= e. >> >>>>>> >> >>>>>> Thanks, >> >>>>>> Micha=C5=82 >> >>>>>> >> >>>>> >> >> > > From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <4fa9cc35a9b96c5fd27786493cddb2e961403f63.camel@collabora.com> Subject: Re: can't access kernelci results From: "Denys Fedoryshchenko" Date: Mon, 26 Sep 2022 18:35:11 +0300 In-Reply-To: References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

<803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> <2059d7b9-2ae1-af0e-5231-1e8c9803f027@collabora.com>

MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: kernelci@groups.io, tkjos@google.com, Guillaume Tucker Cc: =?UTF-8?Q?Micha=C5=82_Ga=C5=82ka?= , Guenter Roeck , Kevin Hilman We are investingating right now and trying to solve the issue. On Mon, 2022-09-26 at 08:29 -0700, Todd Kjos via groups.io wrote: > Having trouble accessing > https://linux.kernelci.org/job/android/=C2=A0today - > server issues? >=20 > On Fri, Mar 18, 2022 at 10:26 AM Guillaume Tucker < > guillaume.tucker@collabora.com> wrote: >=20 > > On 18/03/2022 17:19, Guillaume Tucker wrote: > >=20 > > On 18/03/2022 15:47, Todd Kjos wrote: > >=20 > > Is the MongoDB server down again? I don't get results at > > https://linux.kernelci.org/job/android/. > >=20 > > It's not that, unfortunately there's some persisting issue which > > seems to > > have started yesterday. We're investigating now, please bear with > > us. Thank > > you for reporting the problem. > >=20 > > Alright, there was a problem with background processing tasks on > > the > > server. That's resolved now, although a number of kernel builds > > need to be > > restarted which is why so many trees appear to be "in progress" on > > the web > > dashboard. Things will start picking up again gradually, it might > > take a > > few hours to get back to normal. Sorry for the inconvenience. > >=20 > > Best wishes, > > Guillaume > >=20 > >=20 > > On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tucker < > > guillaume.tucker@collabora.com> wrote: > >=20 > > > Hi Todd, > > >=20 > > > On 30/11/2021 16:54, Todd Kjos wrote: > > > > ...and now it's working again. Sorry about the noise if the > > > > issue was > > > > on my side. > > >=20 > > > The MongoDB server did crash but it was restarted a few minutes > > > later.=C2=A0 There's some kind of watchdog to do that automatically, > > > maybe you were out of luck and checked the website during that > > > window.=C2=A0 Thanks for reporting the issue in any case. > > >=20 > > > Best wishes, > > > Guillaume > > >=20 > > > > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos > > > > wrote: > > > > >=20 > > > > > I can't load the data from > > > > > https://linux.kernelci.org/job/android/ > > > > > ("Error loading build data."). Is mongod "borked" again? > > > > >=20 > > > > > On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos > > > > > wrote: > > > > > >=20 > > > > > > It's working again. Thanks. > > > > > >=20 > > > > > > On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos > > > > > > wrote: > > > > > > >=20 > > > > > > > I'm getting errors again when attempting to access > > > > > > > https://linux.kernelci.org/job/android/. mongod issues > > > > > > > again? > > > > > > >=20 > > > > > > > "Error while loading data from the server (error code: > > > > > > > 500). Please > > > > > > > contact the website administrator" > > > > > > >=20 > > > > > > > On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker > > > > > > > wrote: > > > > > > > >=20 > > > > > > > > On 22/06/2021 16:07, Todd Kjos wrote: > > > > > > > > > Hmm, it's still not working for me -- did it break > > > > > > > > > again? > > > > > > > >=20 > > > > > > > > Yes there was an OOM issue: > > > > > > > >=20 > > > > > > > > [4316410.416514] Killed process 20620 (mongod) total- > > > > > > > > vm:20171340kB, > > > anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB > > > > > > > > [4316412.060511] oom_reaper: reaped process 20620 > > > > > > > > (mongod), now > > > anon-rss:0kB, file-rss:0kB, shmem-rss:0kB > > > > > > > >=20 > > > > > > > > We're looking into moving the Mongo DB instance to the > > > > > > > > cloud in > > > > > > > > Azure.=C2=A0 Hopefully that will save us fixing this kind o= f > > > > > > > > maintenance issues.=C2=A0 Meanwhile, we should take a look > > > > > > > > and see if > > > > > > > > there's some config option to limit the amount of RAM > > > > > > > > used by > > > > > > > > Mongo DB to work around the problem. > > > > > > > >=20 > > > > > > > > Best wishes, > > > > > > > > Guillaume > > > > > > > >=20 > > > > > > > >=20 > > > > > > > > > On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka < > > > michal.galka@collabora.com=C2=A0> > > > wrote: > > > > > > > > >=20 > > > > > > > > > =C2=A0=C2=A0=C2=A0 On 21.06.2021 21:59, Todd Kjos via gro= ups.io > > > > > > > > > > > > wrote: > > > > > > > > > =C2=A0=C2=A0=C2=A0 > I'm having this issue again (can't a= ccess > > > > > > > > > kernelci results > > > > > > > > > =C2=A0=C2=A0=C2=A0 > at https://linux.kernelci.org/job/an= droid/=C2=A0< > > > https://linux.kernelci.org/job/android/> > > > > > > > > > =C2=A0=C2=A0=C2=A0 > > > https://linux.kernelci.org/job/android/>>). Is the mongod service > > > borked > > > > > > > > > =C2=A0=C2=A0=C2=A0 > again? > > > > > > > > > =C2=A0=C2=A0=C2=A0 > > > > > > > > > >=20 > > > > > > > > > =C2=A0=C2=A0=C2=A0 Yes, there was an issue with mongod se= rvice > > > > > > > > > again. It's > > > working now, > > > > > > > > > =C2=A0=C2=A0=C2=A0 celery processed pending tasks and eve= rything > > > > > > > > > seems responsive. > > > > > > > > >=20 > > > > > > > > > =C2=A0=C2=A0=C2=A0 Thanks, > > > > > > > > > =C2=A0=C2=A0=C2=A0 Micha=C5=82 > > > > > > > > >=20 > > > > > > > >=20 > > >=20 > > >=20 > >=20 > >=20 >=20 >=20 >=20 >=20 >=20 From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

<803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> <2059d7b9-2ae1-af0e-5231-1e8c9803f027@collabora.com>

<17187305E642C450.18151@groups.io> <4fb452486cc177348cfbe50ae905705aa71f6ada.camel@collabora.com> In-Reply-To: <4fb452486cc177348cfbe50ae905705aa71f6ada.camel@collabora.com> From: Todd Kjos Date: Mon, 26 Sep 2022 09:13:04 -0700 Message-ID: Subject: Re: can't access kernelci results Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: Denys Fedoryshchenko Cc: kernelci@groups.io, Guillaume Tucker , =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , Guenter Roeck , Kevin Hilman Thanks for the quick response! It's working for me now. On Mon, Sep 26, 2022 at 9:06 AM Denys Fedoryshchenko wrote: > Upd: dashboard should be fine now, but we are running update right now, > so some glitches might happen. > > On Mon, 2022-09-26 at 18:35 +0300, Denys Fedoryshchenko wrote: > > We are investingating right now and trying to solve the issue. > > > > On Mon, 2022-09-26 at 08:29 -0700, Todd Kjos via groups.io wrote: > > > Having trouble accessing > > > https://linux.kernelci.org/job/android/ today - > > > server issues? > > > > > > On Fri, Mar 18, 2022 at 10:26 AM Guillaume Tucker < > > > guillaume.tucker@collabora.com> wrote: > > > > > > > On 18/03/2022 17:19, Guillaume Tucker wrote: > > > > > > > > On 18/03/2022 15:47, Todd Kjos wrote: > > > > > > > > Is the MongoDB server down again? I don't get results at > > > > https://linux.kernelci.org/job/android/. > > > > > > > > It's not that, unfortunately there's some persisting issue which > > > > seems to > > > > have started yesterday. We're investigating now, please bear with > > > > us. Thank > > > > you for reporting the problem. > > > > > > > > Alright, there was a problem with background processing tasks on > > > > the > > > > server. That's resolved now, although a number of kernel builds > > > > need to be > > > > restarted which is why so many trees appear to be "in progress" > > > > on > > > > the web > > > > dashboard. Things will start picking up again gradually, it might > > > > take a > > > > few hours to get back to normal. Sorry for the inconvenience. > > > > > > > > Best wishes, > > > > Guillaume > > > > > > > > > > > > On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tucker < > > > > guillaume.tucker@collabora.com> wrote: > > > > > > > > > Hi Todd, > > > > > > > > > > On 30/11/2021 16:54, Todd Kjos wrote: > > > > > > ...and now it's working again. Sorry about the noise if the > > > > > > issue was > > > > > > on my side. > > > > > > > > > > The MongoDB server did crash but it was restarted a few minutes > > > > > later. There's some kind of watchdog to do that automatically, > > > > > maybe you were out of luck and checked the website during that > > > > > window. Thanks for reporting the issue in any case. > > > > > > > > > > Best wishes, > > > > > Guillaume > > > > > > > > > > > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos > > > > > > wrote: > > > > > > > > > > > > > > I can't load the data from > > > > > > > https://linux.kernelci.org/job/android/ > > > > > > > ("Error loading build data."). Is mongod "borked" again? > > > > > > > > > > > > > > On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > It's working again. Thanks. > > > > > > > > > > > > > > > > On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > I'm getting errors again when attempting to access > > > > > > > > > https://linux.kernelci.org/job/android/. mongod issues > > > > > > > > > again? > > > > > > > > > > > > > > > > > > "Error while loading data from the server (error code: > > > > > > > > > 500). Please > > > > > > > > > contact the website administrator" > > > > > > > > > > > > > > > > > > On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > On 22/06/2021 16:07, Todd Kjos wrote: > > > > > > > > > > > Hmm, it's still not working for me -- did it break > > > > > > > > > > > again? > > > > > > > > > > > > > > > > > > > > Yes there was an OOM issue: > > > > > > > > > > > > > > > > > > > > [4316410.416514] Killed process 20620 (mongod) total- > > > > > > > > > > vm:20171340kB, > > > > > anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB > > > > > > > > > > [4316412.060511] oom_reaper: reaped process 20620 > > > > > > > > > > (mongod), now > > > > > anon-rss:0kB, file-rss:0kB, shmem-rss:0kB > > > > > > > > > > > > > > > > > > > > We're looking into moving the Mongo DB instance to > > > > > > > > > > the > > > > > > > > > > cloud in > > > > > > > > > > Azure. Hopefully that will save us fixing this kind > > > > > > > > > > of > > > > > > > > > > maintenance issues. Meanwhile, we should take a look > > > > > > > > > > and see if > > > > > > > > > > there's some config option to limit the amount of RAM > > > > > > > > > > used by > > > > > > > > > > Mongo DB to work around the problem. > > > > > > > > > > > > > > > > > > > > Best wishes, > > > > > > > > > > Guillaume > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82k= a < > > > > > michal.galka@collabora.com > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > On 21.06.2021 21:59, Todd Kjos via groups.io > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > I'm having this issue again (can't access > > > > > > > > > > > kernelci results > > > > > > > > > > > > at https://linux.kernelci.org/job/android/ < > > > > > https://linux.kernelci.org/job/android/> > > > > > > > > > > > > > > > > https://linux.kernelci.org/job/android/>>). Is the mongod > > > > > service > > > > > borked > > > > > > > > > > > > again? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yes, there was an issue with mongod service > > > > > > > > > > > again. It's > > > > > working now, > > > > > > > > > > > celery processed pending tasks and everything > > > > > > > > > > > seems responsive. > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Micha=C5=82 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >=20 > > > > > > > From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <4fb452486cc177348cfbe50ae905705aa71f6ada.camel@collabora.com> Subject: Re: can't access kernelci results From: "Denys Fedoryshchenko" Date: Mon, 26 Sep 2022 19:06:42 +0300 In-Reply-To: <17187305E642C450.18151@groups.io> References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

<803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> <2059d7b9-2ae1-af0e-5231-1e8c9803f027@collabora.com>

<17187305E642C450.18151@groups.io> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: kernelci@groups.io, tkjos@google.com, Guillaume Tucker Cc: =?UTF-8?Q?Micha=C5=82_Ga=C5=82ka?= , Guenter Roeck , Kevin Hilman Upd: dashboard should be fine now, but we are running update right now, so some glitches might happen. On Mon, 2022-09-26 at 18:35 +0300, Denys Fedoryshchenko wrote: > We are investingating right now and trying to solve the issue. >=20 > On Mon, 2022-09-26 at 08:29 -0700, Todd Kjos via groups.io wrote: > > Having trouble accessing > > https://linux.kernelci.org/job/android/=C2=A0today - > > server issues? > >=20 > > On Fri, Mar 18, 2022 at 10:26 AM Guillaume Tucker < > > guillaume.tucker@collabora.com> wrote: > >=20 > > > On 18/03/2022 17:19, Guillaume Tucker wrote: > > >=20 > > > On 18/03/2022 15:47, Todd Kjos wrote: > > >=20 > > > Is the MongoDB server down again? I don't get results at > > > https://linux.kernelci.org/job/android/. > > >=20 > > > It's not that, unfortunately there's some persisting issue which > > > seems to > > > have started yesterday. We're investigating now, please bear with > > > us. Thank > > > you for reporting the problem. > > >=20 > > > Alright, there was a problem with background processing tasks on > > > the > > > server. That's resolved now, although a number of kernel builds > > > need to be > > > restarted which is why so many trees appear to be "in progress" > > > on > > > the web > > > dashboard. Things will start picking up again gradually, it might > > > take a > > > few hours to get back to normal. Sorry for the inconvenience. > > >=20 > > > Best wishes, > > > Guillaume > > >=20 > > >=20 > > > On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tucker < > > > guillaume.tucker@collabora.com> wrote: > > >=20 > > > > Hi Todd, > > > >=20 > > > > On 30/11/2021 16:54, Todd Kjos wrote: > > > > > ...and now it's working again. Sorry about the noise if the > > > > > issue was > > > > > on my side. > > > >=20 > > > > The MongoDB server did crash but it was restarted a few minutes > > > > later.=C2=A0 There's some kind of watchdog to do that automatically= , > > > > maybe you were out of luck and checked the website during that > > > > window.=C2=A0 Thanks for reporting the issue in any case. > > > >=20 > > > > Best wishes, > > > > Guillaume > > > >=20 > > > > > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos > > > > > wrote: > > > > > >=20 > > > > > > I can't load the data from > > > > > > https://linux.kernelci.org/job/android/ > > > > > > ("Error loading build data."). Is mongod "borked" again? > > > > > >=20 > > > > > > On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos > > > > > > > > > > > > wrote: > > > > > > >=20 > > > > > > > It's working again. Thanks. > > > > > > >=20 > > > > > > > On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos > > > > > > > > > > > > > > wrote: > > > > > > > >=20 > > > > > > > > I'm getting errors again when attempting to access > > > > > > > > https://linux.kernelci.org/job/android/. mongod issues > > > > > > > > again? > > > > > > > >=20 > > > > > > > > "Error while loading data from the server (error code: > > > > > > > > 500). Please > > > > > > > > contact the website administrator" > > > > > > > >=20 > > > > > > > > On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker > > > > > > > > wrote: > > > > > > > > >=20 > > > > > > > > > On 22/06/2021 16:07, Todd Kjos wrote: > > > > > > > > > > Hmm, it's still not working for me -- did it break > > > > > > > > > > again? > > > > > > > > >=20 > > > > > > > > > Yes there was an OOM issue: > > > > > > > > >=20 > > > > > > > > > [4316410.416514] Killed process 20620 (mongod) total- > > > > > > > > > vm:20171340kB, > > > > anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB > > > > > > > > > [4316412.060511] oom_reaper: reaped process 20620 > > > > > > > > > (mongod), now > > > > anon-rss:0kB, file-rss:0kB, shmem-rss:0kB > > > > > > > > >=20 > > > > > > > > > We're looking into moving the Mongo DB instance to > > > > > > > > > the > > > > > > > > > cloud in > > > > > > > > > Azure.=C2=A0 Hopefully that will save us fixing this kind > > > > > > > > > of > > > > > > > > > maintenance issues.=C2=A0 Meanwhile, we should take a loo= k > > > > > > > > > and see if > > > > > > > > > there's some config option to limit the amount of RAM > > > > > > > > > used by > > > > > > > > > Mongo DB to work around the problem. > > > > > > > > >=20 > > > > > > > > > Best wishes, > > > > > > > > > Guillaume > > > > > > > > >=20 > > > > > > > > >=20 > > > > > > > > > > On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka = < > > > > michal.galka@collabora.com=C2=A0= > > > > > wrote: > > > > > > > > > >=20 > > > > > > > > > > =C2=A0=C2=A0=C2=A0 On 21.06.2021 21:59, Todd Kjos via g= roups.io > > > > > > > > > > > > > > wrote: > > > > > > > > > > =C2=A0=C2=A0=C2=A0 > I'm having this issue again (can't= access > > > > > > > > > > kernelci results > > > > > > > > > > =C2=A0=C2=A0=C2=A0 > at https://linux.kernelci.org/job/= android/=C2=A0< > > > > https://linux.kernelci.org/job/android/> > > > > > > > > > > =C2=A0=C2=A0=C2=A0 > > > > https://linux.kernelci.org/job/android/>>). Is the mongod > > > > service > > > > borked > > > > > > > > > > =C2=A0=C2=A0=C2=A0 > again? > > > > > > > > > > =C2=A0=C2=A0=C2=A0 > > > > > > > > > > >=20 > > > > > > > > > > =C2=A0=C2=A0=C2=A0 Yes, there was an issue with mongod = service > > > > > > > > > > again. It's > > > > working now, > > > > > > > > > > =C2=A0=C2=A0=C2=A0 celery processed pending tasks and e= verything > > > > > > > > > > seems responsive. > > > > > > > > > >=20 > > > > > > > > > > =C2=A0=C2=A0=C2=A0 Thanks, > > > > > > > > > > =C2=A0=C2=A0=C2=A0 Micha=C5=82 > > > > > > > > > >=20 > > > > > > > > >=20 > > > >=20 > > > >=20 > > >=20 > > >=20 > >=20 > >=20 > >=20 > >=20 > >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20 From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: Subject: Re: can't access kernelci results From: "Denys Fedoryshchenko" Date: Fri, 14 Oct 2022 21:16:59 +0300 In-Reply-To: References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com>

<285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com>

<803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> <2059d7b9-2ae1-af0e-5231-1e8c9803f027@collabora.com>

<17187305E642C450.18151@groups.io> <4fb452486cc177348cfbe50ae905705aa71f6ada.camel@collabora.com>

MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: Todd Kjos Cc: kernelci@groups.io, Guillaume Tucker