From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <4fb452486cc177348cfbe50ae905705aa71f6ada.camel@collabora.com> Subject: Re: can't access kernelci results From: "Denys Fedoryshchenko" Date: Mon, 26 Sep 2022 19:06:42 +0300 In-Reply-To: <17187305E642C450.18151@groups.io> References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com> <285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com> <803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> <2059d7b9-2ae1-af0e-5231-1e8c9803f027@collabora.com> <17187305E642C450.18151@groups.io> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: kernelci@groups.io, tkjos@google.com, Guillaume Tucker Cc: =?UTF-8?Q?Micha=C5=82_Ga=C5=82ka?= , Guenter Roeck , Kevin Hilman Upd: dashboard should be fine now, but we are running update right now, so some glitches might happen. On Mon, 2022-09-26 at 18:35 +0300, Denys Fedoryshchenko wrote: > We are investingating right now and trying to solve the issue. >=20 > On Mon, 2022-09-26 at 08:29 -0700, Todd Kjos via groups.io wrote: > > Having trouble accessing > > https://linux.kernelci.org/job/android/=C2=A0today - > > server issues? > >=20 > > On Fri, Mar 18, 2022 at 10:26 AM Guillaume Tucker < > > guillaume.tucker@collabora.com> wrote: > >=20 > > > On 18/03/2022 17:19, Guillaume Tucker wrote: > > >=20 > > > On 18/03/2022 15:47, Todd Kjos wrote: > > >=20 > > > Is the MongoDB server down again? I don't get results at > > > https://linux.kernelci.org/job/android/. > > >=20 > > > It's not that, unfortunately there's some persisting issue which > > > seems to > > > have started yesterday. We're investigating now, please bear with > > > us. Thank > > > you for reporting the problem. > > >=20 > > > Alright, there was a problem with background processing tasks on > > > the > > > server. That's resolved now, although a number of kernel builds > > > need to be > > > restarted which is why so many trees appear to be "in progress" > > > on > > > the web > > > dashboard. Things will start picking up again gradually, it might > > > take a > > > few hours to get back to normal. Sorry for the inconvenience. > > >=20 > > > Best wishes, > > > Guillaume > > >=20 > > >=20 > > > On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tucker < > > > guillaume.tucker@collabora.com> wrote: > > >=20 > > > > Hi Todd, > > > >=20 > > > > On 30/11/2021 16:54, Todd Kjos wrote: > > > > > ...and now it's working again. Sorry about the noise if the > > > > > issue was > > > > > on my side. > > > >=20 > > > > The MongoDB server did crash but it was restarted a few minutes > > > > later.=C2=A0 There's some kind of watchdog to do that automatically= , > > > > maybe you were out of luck and checked the website during that > > > > window.=C2=A0 Thanks for reporting the issue in any case. > > > >=20 > > > > Best wishes, > > > > Guillaume > > > >=20 > > > > > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos > > > > > wrote: > > > > > >=20 > > > > > > I can't load the data from > > > > > > https://linux.kernelci.org/job/android/ > > > > > > ("Error loading build data."). Is mongod "borked" again? > > > > > >=20 > > > > > > On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos > > > > > > > > > > > > wrote: > > > > > > >=20 > > > > > > > It's working again. Thanks. > > > > > > >=20 > > > > > > > On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos > > > > > > > > > > > > > > wrote: > > > > > > > >=20 > > > > > > > > I'm getting errors again when attempting to access > > > > > > > > https://linux.kernelci.org/job/android/. mongod issues > > > > > > > > again? > > > > > > > >=20 > > > > > > > > "Error while loading data from the server (error code: > > > > > > > > 500). Please > > > > > > > > contact the website administrator" > > > > > > > >=20 > > > > > > > > On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker > > > > > > > > wrote: > > > > > > > > >=20 > > > > > > > > > On 22/06/2021 16:07, Todd Kjos wrote: > > > > > > > > > > Hmm, it's still not working for me -- did it break > > > > > > > > > > again? > > > > > > > > >=20 > > > > > > > > > Yes there was an OOM issue: > > > > > > > > >=20 > > > > > > > > > [4316410.416514] Killed process 20620 (mongod) total- > > > > > > > > > vm:20171340kB, > > > > anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB > > > > > > > > > [4316412.060511] oom_reaper: reaped process 20620 > > > > > > > > > (mongod), now > > > > anon-rss:0kB, file-rss:0kB, shmem-rss:0kB > > > > > > > > >=20 > > > > > > > > > We're looking into moving the Mongo DB instance to > > > > > > > > > the > > > > > > > > > cloud in > > > > > > > > > Azure.=C2=A0 Hopefully that will save us fixing this kind > > > > > > > > > of > > > > > > > > > maintenance issues.=C2=A0 Meanwhile, we should take a loo= k > > > > > > > > > and see if > > > > > > > > > there's some config option to limit the amount of RAM > > > > > > > > > used by > > > > > > > > > Mongo DB to work around the problem. > > > > > > > > >=20 > > > > > > > > > Best wishes, > > > > > > > > > Guillaume > > > > > > > > >=20 > > > > > > > > >=20 > > > > > > > > > > On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82ka = < > > > > michal.galka@collabora.com=C2=A0= > > > > > wrote: > > > > > > > > > >=20 > > > > > > > > > > =C2=A0=C2=A0=C2=A0 On 21.06.2021 21:59, Todd Kjos via g= roups.io > > > > > > > > > > > > > > wrote: > > > > > > > > > > =C2=A0=C2=A0=C2=A0 > I'm having this issue again (can't= access > > > > > > > > > > kernelci results > > > > > > > > > > =C2=A0=C2=A0=C2=A0 > at https://linux.kernelci.org/job/= android/=C2=A0< > > > > https://linux.kernelci.org/job/android/> > > > > > > > > > > =C2=A0=C2=A0=C2=A0 > > > > https://linux.kernelci.org/job/android/>>). Is the mongod > > > > service > > > > borked > > > > > > > > > > =C2=A0=C2=A0=C2=A0 > again? > > > > > > > > > > =C2=A0=C2=A0=C2=A0 > > > > > > > > > > >=20 > > > > > > > > > > =C2=A0=C2=A0=C2=A0 Yes, there was an issue with mongod = service > > > > > > > > > > again. It's > > > > working now, > > > > > > > > > > =C2=A0=C2=A0=C2=A0 celery processed pending tasks and e= verything > > > > > > > > > > seems responsive. > > > > > > > > > >=20 > > > > > > > > > > =C2=A0=C2=A0=C2=A0 Thanks, > > > > > > > > > > =C2=A0=C2=A0=C2=A0 Micha=C5=82 > > > > > > > > > >=20 > > > > > > > > >=20 > > > >=20 > > > >=20 > > >=20 > > >=20 > >=20 > >=20 > >=20 > >=20 > >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20