From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <2a6d7691-b8a5-da92-865c-8cdf5efb673f@collabora.com> <285b11ad-40dc-eb88-427e-f32c4a422c85@collabora.com> <803b89a9-9141-636f-8fd3-f519b0440075@collabora.com> <2059d7b9-2ae1-af0e-5231-1e8c9803f027@collabora.com> <17187305E642C450.18151@groups.io> <4fb452486cc177348cfbe50ae905705aa71f6ada.camel@collabora.com> In-Reply-To: From: Todd Kjos Date: Fri, 14 Oct 2022 11:12:45 -0700 Message-ID: Subject: Re: can't access kernelci results Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable List-ID: To: Denys Fedoryshchenko Cc: kernelci@groups.io, Guillaume Tucker , =?UTF-8?B?TWljaGHFgiBHYcWCa2E=?= , Guenter Roeck , Kevin Hilman The dashboard is broken again. Known issue? "Error while loading data from the server (error code: 500). Please contact the website administrator" On Mon, Sep 26, 2022 at 9:13 AM Todd Kjos wrote: > Thanks for the quick response! It's working for me now. > > On Mon, Sep 26, 2022 at 9:06 AM Denys Fedoryshchenko < > denys.f@collabora.com> wrote: > >> Upd: dashboard should be fine now, but we are running update right now, >> so some glitches might happen. >> >> On Mon, 2022-09-26 at 18:35 +0300, Denys Fedoryshchenko wrote: >> > We are investingating right now and trying to solve the issue. >> > >> > On Mon, 2022-09-26 at 08:29 -0700, Todd Kjos via groups.io wrote: >> > > Having trouble accessing >> > > https://linux.kernelci.org/job/android/ today - >> > > server issues? >> > > >> > > On Fri, Mar 18, 2022 at 10:26 AM Guillaume Tucker < >> > > guillaume.tucker@collabora.com> wrote: >> > > >> > > > On 18/03/2022 17:19, Guillaume Tucker wrote: >> > > > >> > > > On 18/03/2022 15:47, Todd Kjos wrote: >> > > > >> > > > Is the MongoDB server down again? I don't get results at >> > > > https://linux.kernelci.org/job/android/. >> > > > >> > > > It's not that, unfortunately there's some persisting issue which >> > > > seems to >> > > > have started yesterday. We're investigating now, please bear with >> > > > us. Thank >> > > > you for reporting the problem. >> > > > >> > > > Alright, there was a problem with background processing tasks on >> > > > the >> > > > server. That's resolved now, although a number of kernel builds >> > > > need to be >> > > > restarted which is why so many trees appear to be "in progress" >> > > > on >> > > > the web >> > > > dashboard. Things will start picking up again gradually, it might >> > > > take a >> > > > few hours to get back to normal. Sorry for the inconvenience. >> > > > >> > > > Best wishes, >> > > > Guillaume >> > > > >> > > > >> > > > On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tucker < >> > > > guillaume.tucker@collabora.com> wrote: >> > > > >> > > > > Hi Todd, >> > > > > >> > > > > On 30/11/2021 16:54, Todd Kjos wrote: >> > > > > > ...and now it's working again. Sorry about the noise if the >> > > > > > issue was >> > > > > > on my side. >> > > > > >> > > > > The MongoDB server did crash but it was restarted a few minutes >> > > > > later. There's some kind of watchdog to do that automatically, >> > > > > maybe you were out of luck and checked the website during that >> > > > > window. Thanks for reporting the issue in any case. >> > > > > >> > > > > Best wishes, >> > > > > Guillaume >> > > > > >> > > > > > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos >> > > > > > wrote: >> > > > > > > >> > > > > > > I can't load the data from >> > > > > > > https://linux.kernelci.org/job/android/ >> > > > > > > ("Error loading build data."). Is mongod "borked" again? >> > > > > > > >> > > > > > > On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos >> > > > > > > >> > > > > > > wrote: >> > > > > > > > >> > > > > > > > It's working again. Thanks. >> > > > > > > > >> > > > > > > > On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos >> > > > > > > > >> > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > I'm getting errors again when attempting to access >> > > > > > > > > https://linux.kernelci.org/job/android/. mongod issues >> > > > > > > > > again? >> > > > > > > > > >> > > > > > > > > "Error while loading data from the server (error code: >> > > > > > > > > 500). Please >> > > > > > > > > contact the website administrator" >> > > > > > > > > >> > > > > > > > > On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker >> > > > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > On 22/06/2021 16:07, Todd Kjos wrote: >> > > > > > > > > > > Hmm, it's still not working for me -- did it break >> > > > > > > > > > > again? >> > > > > > > > > > >> > > > > > > > > > Yes there was an OOM issue: >> > > > > > > > > > >> > > > > > > > > > [4316410.416514] Killed process 20620 (mongod) total- >> > > > > > > > > > vm:20171340kB, >> > > > > anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB >> > > > > > > > > > [4316412.060511] oom_reaper: reaped process 20620 >> > > > > > > > > > (mongod), now >> > > > > anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >> > > > > > > > > > >> > > > > > > > > > We're looking into moving the Mongo DB instance to >> > > > > > > > > > the >> > > > > > > > > > cloud in >> > > > > > > > > > Azure. Hopefully that will save us fixing this kind >> > > > > > > > > > of >> > > > > > > > > > maintenance issues. Meanwhile, we should take a look >> > > > > > > > > > and see if >> > > > > > > > > > there's some config option to limit the amount of RAM >> > > > > > > > > > used by >> > > > > > > > > > Mongo DB to work around the problem. >> > > > > > > > > > >> > > > > > > > > > Best wishes, >> > > > > > > > > > Guillaume >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > On Tue, Jun 22, 2021 at 2:43 AM Micha=C5=82 Ga=C5=82= ka < >> > > > > michal.galka@collabora.com > >> > > > > wrote: >> > > > > > > > > > > >> > > > > > > > > > > On 21.06.2021 21:59, Todd Kjos via groups.io >> > > > > > > > > > > >> > > > > wrote: >> > > > > > > > > > > > I'm having this issue again (can't access >> > > > > > > > > > > kernelci results >> > > > > > > > > > > > at https://linux.kernelci.org/job/android/ < >> > > > > https://linux.kernelci.org/job/android/> >> > > > > > > > > > > > > > > > > https://linux.kernelci.org/job/android/>>). Is the mongod >> > > > > service >> > > > > borked >> > > > > > > > > > > > again? >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > Yes, there was an issue with mongod service >> > > > > > > > > > > again. It's >> > > > > working now, >> > > > > > > > > > > celery processed pending tasks and everything >> > > > > > > > > > > seems responsive. >> > > > > > > > > > > >> > > > > > > > > > > Thanks, >> > > > > > > > > > > Micha=C5=82 >> > > > > > > > > > > >> > > > > > > > > > >> > > > > >> > > > > >> > > > >> > > > >> > > >> > > >> > > >> > > >> > > >> > >> > >> > >> > >> >=20 >> > >> > >> >> >>