From: "Denys Fedoryshchenko" <denys.f@collabora.com>
To: Todd Kjos <tkjos@google.com>
Cc: kernelci@groups.io,
"Guillaume Tucker" <guillaume.tucker@collabora.com>,
"Michał Gałka" <michal.galka@collabora.com>,
"Guenter Roeck" <groeck@google.com>,
"Kevin Hilman" <khilman@baylibre.com>
Subject: Re: can't access kernelci results
Date: Fri, 14 Oct 2022 21:16:59 +0300 [thread overview]
Message-ID: <b5529b0298af8de1c7d85c784489664305ac309d.camel@collabora.com> (raw)
In-Reply-To: <CAHRSSEzOvvXkGKR6DREvHKMLaRD_SnL3Y5YG-7-t3Ev9=2Upag@mail.gmail.com>
Thanks for info, i am investigating issue and will fix problem shortly.
Hopefully next week we will have monitoring system in place.
On Fri, 2022-10-14 at 11:12 -0700, Todd Kjos wrote:
> The dashboard is broken again. Known issue?
>
> "Error while loading data from the server (error code: 500). Please
> contact the website administrator"
>
> On Mon, Sep 26, 2022 at 9:13 AM Todd Kjos <tkjos@google.com> wrote:
> > Thanks for the quick response! It's working for me now.
> >
> > On Mon, Sep 26, 2022 at 9:06 AM Denys Fedoryshchenko
> > <denys.f@collabora.com> wrote:
> > > Upd: dashboard should be fine now, but we are running update
> > > right now,
> > > so some glitches might happen.
> > >
> > > On Mon, 2022-09-26 at 18:35 +0300, Denys Fedoryshchenko wrote:
> > > > We are investingating right now and trying to solve the issue.
> > > >
> > > > On Mon, 2022-09-26 at 08:29 -0700, Todd Kjos via groups.io
> > > wrote:
> > > > > Having trouble accessing
> > > > > https://linux.kernelci.org/job/android/ today -
> > > > > server issues?
> > > > >
> > > > > On Fri, Mar 18, 2022 at 10:26 AM Guillaume Tucker <
> > > > > guillaume.tucker@collabora.com> wrote:
> > > > >
> > > > > > On 18/03/2022 17:19, Guillaume Tucker wrote:
> > > > > >
> > > > > > On 18/03/2022 15:47, Todd Kjos wrote:
> > > > > >
> > > > > > Is the MongoDB server down again? I don't get results at
> > > > > > https://linux.kernelci.org/job/android/.
> > > > > >
> > > > > > It's not that, unfortunately there's some persisting issue
> > > which
> > > > > > seems to
> > > > > > have started yesterday. We're investigating now, please
> > > bear with
> > > > > > us. Thank
> > > > > > you for reporting the problem.
> > > > > >
> > > > > > Alright, there was a problem with background processing
> > > tasks on
> > > > > > the
> > > > > > server. That's resolved now, although a number of kernel
> > > builds
> > > > > > need to be
> > > > > > restarted which is why so many trees appear to be "in
> > > progress"
> > > > > > on
> > > > > > the web
> > > > > > dashboard. Things will start picking up again gradually, it
> > > might
> > > > > > take a
> > > > > > few hours to get back to normal. Sorry for the
> > > inconvenience.
> > > > > >
> > > > > > Best wishes,
> > > > > > Guillaume
> > > > > >
> > > > > >
> > > > > > On Wed, Dec 1, 2021 at 1:49 AM Guillaume Tucker <
> > > > > > guillaume.tucker@collabora.com> wrote:
> > > > > >
> > > > > > > Hi Todd,
> > > > > > >
> > > > > > > On 30/11/2021 16:54, Todd Kjos wrote:
> > > > > > > > ...and now it's working again. Sorry about the noise if
> > > the
> > > > > > > > issue was
> > > > > > > > on my side.
> > > > > > >
> > > > > > > The MongoDB server did crash but it was restarted a few
> > > minutes
> > > > > > > later. There's some kind of watchdog to do that
> > > automatically,
> > > > > > > maybe you were out of luck and checked the website during
> > > that
> > > > > > > window. Thanks for reporting the issue in any case.
> > > > > > >
> > > > > > > Best wishes,
> > > > > > > Guillaume
> > > > > > >
> > > > > > > > On Tue, Nov 30, 2021 at 8:35 AM Todd Kjos
> > > <tkjos@google.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > I can't load the data from
> > > > > > > > > https://linux.kernelci.org/job/android/
> > > > > > > > > ("Error loading build data."). Is mongod "borked"
> > > again?
> > > > > > > > >
> > > > > > > > > On Thu, Sep 2, 2021 at 12:28 PM Todd Kjos
> > > > > > > > > <tkjos@google.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > It's working again. Thanks.
> > > > > > > > > >
> > > > > > > > > > On Thu, Sep 2, 2021 at 8:58 AM Todd Kjos
> > > > > > > > > > <tkjos@google.com>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > I'm getting errors again when attempting to
> > > access
> > > > > > > > > > > https://linux.kernelci.org/job/android/. mongod
> > > issues
> > > > > > > > > > > again?
> > > > > > > > > > >
> > > > > > > > > > > "Error while loading data from the server (error
> > > code:
> > > > > > > > > > > 500). Please
> > > > > > > > > > > contact the website administrator"
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Jun 22, 2021 at 8:17 AM Guillaume Tucker
> > > > > > > > > > > <guillaume.tucker@collabora.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > On 22/06/2021 16:07, Todd Kjos wrote:
> > > > > > > > > > > > > Hmm, it's still not working for me -- did it
> > > break
> > > > > > > > > > > > > again?
> > > > > > > > > > > >
> > > > > > > > > > > > Yes there was an OOM issue:
> > > > > > > > > > > >
> > > > > > > > > > > > [4316410.416514] Killed process 20620 (mongod)
> > > total-
> > > > > > > > > > > > vm:20171340kB,
> > > > > > > anon-rss:18654740kB, file-rss:0kB, shmem-rss:0kB
> > > > > > > > > > > > [4316412.060511] oom_reaper: reaped process
> > > 20620
> > > > > > > > > > > > (mongod), now
> > > > > > > anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
> > > > > > > > > > > >
> > > > > > > > > > > > We're looking into moving the Mongo DB instance
> > > to
> > > > > > > > > > > > the
> > > > > > > > > > > > cloud in
> > > > > > > > > > > > Azure. Hopefully that will save us fixing this
> > > kind
> > > > > > > > > > > > of
> > > > > > > > > > > > maintenance issues. Meanwhile, we should take
> > > a look
> > > > > > > > > > > > and see if
> > > > > > > > > > > > there's some config option to limit the amount
> > > of RAM
> > > > > > > > > > > > used by
> > > > > > > > > > > > Mongo DB to work around the problem.
> > > > > > > > > > > >
> > > > > > > > > > > > Best wishes,
> > > > > > > > > > > > Guillaume
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Jun 22, 2021 at 2:43 AM Michał Gałka
> > > <
> > > > > > >
> > > michal.galka@collabora.com <mailto:michal.galka@collabora.com>>
> > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > On 21.06.2021 21:59, Todd Kjos via
> > > groups.io
> > > > > > > > > > > > > <http://groups.io>
> > > > > > > wrote:
> > > > > > > > > > > > > > I'm having this issue again (can't
> > > access
> > > > > > > > > > > > > kernelci results
> > > > > > > > > > > > > > at
> > > https://linux.kernelci.org/job/android/ <
> > > > > > > https://linux.kernelci.org/job/android/>
> > > > > > > > > > > > > >
> > > <https://linux.kernelci.org/job/android/ <
> > > > > > > https://linux.kernelci.org/job/android/>>). Is the mongod
> > > > > > > service
> > > > > > > borked
> > > > > > > > > > > > > > again?
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yes, there was an issue with mongod
> > > service
> > > > > > > > > > > > > again. It's
> > > > > > > working now,
> > > > > > > > > > > > > celery processed pending tasks and
> > > everything
> > > > > > > > > > > > > seems responsive.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > Michał
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
next prev parent reply other threads:[~2022-10-14 18:16 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-05 19:02 can't access kernelci results Todd Kjos
2021-04-05 20:39 ` Guenter Roeck
2021-04-05 20:58 ` Guillaume Tucker
2021-04-05 23:14 ` Todd Kjos
2021-06-21 19:59 ` Todd Kjos
2021-06-22 9:43 ` Michał Gałka
2021-06-22 15:07 ` Todd Kjos
2021-06-22 15:16 ` Guillaume Tucker
2021-09-02 15:58 ` Todd Kjos
2021-09-02 19:28 ` Todd Kjos
2021-11-30 16:35 ` Todd Kjos
2021-11-30 16:54 ` Todd Kjos
2021-12-01 9:48 ` Guillaume Tucker
2022-03-18 15:47 ` Todd Kjos
2022-03-18 17:19 ` Guillaume Tucker
2022-03-18 17:26 ` Guillaume Tucker
2022-09-26 15:29 ` Todd Kjos
2022-09-26 15:35 ` Denys Fedoryshchenko
[not found] ` <17187305E642C450.18151@groups.io>
2022-09-26 16:06 ` Denys Fedoryshchenko
2022-09-26 16:13 ` Todd Kjos
2022-10-14 18:12 ` Todd Kjos
2022-10-14 18:16 ` Denys Fedoryshchenko [this message]
2022-10-14 18:39 ` Todd Kjos
2022-10-21 16:25 ` Todd Kjos
2022-10-22 7:00 ` Denys Fedoryshchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b5529b0298af8de1c7d85c784489664305ac309d.camel@collabora.com \
--to=denys.f@collabora.com \
--cc=groeck@google.com \
--cc=guillaume.tucker@collabora.com \
--cc=kernelci@groups.io \
--cc=khilman@baylibre.com \
--cc=michal.galka@collabora.com \
--cc=tkjos@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).