[GSOC] ceph-mgr: Cluster Status Dashboard

[GSOC] ceph-mgr: Cluster Status Dashboard

[saumay agrawal]

Hi Kefu,

I apologize for the late reply.

I have been looking into the PR you mentioned earlier
https://github.com/ceph/ceph/pull/14946 . I have also explored the
Calamari Dashboard in the past few days. I have some queries.

I guess since Calamari Dashboard already does the job of visualizing
the statistics of a Ceph cluster (health, OSDs, MONs, pools, IOPS,
space utilization etc), we would primarily be improving on its UI/UX,
that is to say the visualization portion and some other
functionalities. Please do correct me if I am wrong in this.

Moreover, Calamari Dashboard is made up of Calamari Server (backend)
and Calamari Clients (frontend).

I found out that Calamari Server uses a new REST API instead of being
based upon the CEPH REST API. Will I be working on improving this new
REST API?

Moreover, Calamari Server is written in Django and
django-rest-framework. However in the prototype of the dashboard
mentioned in the PR above, I could find that the dashboard is using
CherryPy too. CherryPy is a minimalist framework which is good for
prototyping and makes it very easy. Meanwhile Django is more of a
complete web application stack. So will we be using a single
technology primarily, or a combination of both?

Last thing I want to ask is for what kinds of things I should be
contacting you and for what kind of things I should be contacting
John? Since John is my mentor on this project and I can see him
working on most of the Dashboard portion on GitHub, I presume for any
dashboard related issue I should contact him. And also to whom should
I send the weekly report of my progress on my project?

Regards,
Saumay.

Re: [GSOC] ceph-mgr: Cluster Status Dashboard

[John Spray]

On Sat, May 13, 2017 at 6:06 AM, saumay agrawal
<saumay.agrawal@gmail.com> wrote:
> Hi Kefu,
>
> I apologize for the late reply.
>
> I have been looking into the PR you mentioned earlier
> https://github.com/ceph/ceph/pull/14946 . I have also explored the
> Calamari Dashboard in the past few days. I have some queries.
>
> I guess since Calamari Dashboard already does the job of visualizing
> the statistics of a Ceph cluster (health, OSDs, MONs, pools, IOPS,
> space utilization etc), we would primarily be improving on its UI/UX,
> that is to say the visualization portion and some other
> functionalities. Please do correct me if I am wrong in this.
>
> Moreover, Calamari Dashboard is made up of Calamari Server (backend)
> and Calamari Clients (frontend).
>
> I found out that Calamari Server uses a new REST API instead of being
> based upon the CEPH REST API. Will I be working on improving this new
> REST API?
>
> Moreover, Calamari Server is written in Django and
> django-rest-framework. However in the prototype of the dashboard
> mentioned in the PR above, I could find that the dashboard is using
> CherryPy too. CherryPy is a minimalist framework which is good for
> prototyping and makes it very easy. Meanwhile Django is more of a
> complete web application stack. So will we be using a single
> technology primarily, or a combination of both?

Welcome!

Don't worry about any of the Django or REST stuff.  Django is being
dropped from everything because the maintainers of downstream
(distribution) packages are highly opposed to supporting it as a
dependency.  The REST part is not needed when the dashboard is built
as a mgr module, because as a mgr module is has direct access to the
Ceph state.

> Last thing I want to ask is for what kinds of things I should be
> contacting you and for what kind of things I should be contacting
> John? Since John is my mentor on this project and I can see him
> working on most of the Dashboard portion on GitHub, I presume for any
> dashboard related issue I should contact him. And also to whom should
> I send the weekly report of my progress on my project?

I think I'm supposed to be a co-mentor (whatever that means :-) ) and
Kefu is your primary point of contact.  I'm always happy to provide
opinions though of course.

John


> Regards,
> Saumay.
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GSOC] ceph-mgr: Cluster Status Dashboard

[saumay agrawal]

On Sat, May 13, 2017 at 1:35 PM, John Spray <jspray@redhat.com> wrote:
> On Sat, May 13, 2017 at 6:06 AM, saumay agrawal
> <saumay.agrawal@gmail.com> wrote:
>> Hi Kefu,
>>
>> I apologize for the late reply.
>>
>> I have been looking into the PR you mentioned earlier
>> https://github.com/ceph/ceph/pull/14946 . I have also explored the
>> Calamari Dashboard in the past few days. I have some queries.
>>
>> I guess since Calamari Dashboard already does the job of visualizing
>> the statistics of a Ceph cluster (health, OSDs, MONs, pools, IOPS,
>> space utilization etc), we would primarily be improving on its UI/UX,
>> that is to say the visualization portion and some other
>> functionalities. Please do correct me if I am wrong in this.
>>
>> Moreover, Calamari Dashboard is made up of Calamari Server (backend)
>> and Calamari Clients (frontend).
>>
>> I found out that Calamari Server uses a new REST API instead of being
>> based upon the CEPH REST API. Will I be working on improving this new
>> REST API?
>>
>> Moreover, Calamari Server is written in Django and
>> django-rest-framework. However in the prototype of the dashboard
>> mentioned in the PR above, I could find that the dashboard is using
>> CherryPy too. CherryPy is a minimalist framework which is good for
>> prototyping and makes it very easy. Meanwhile Django is more of a
>> complete web application stack. So will we be using a single
>> technology primarily, or a combination of both?
>
> Welcome!
>

Thanks, John. :)

> Don't worry about any of the Django or REST stuff.  Django is being
> dropped from everything because the maintainers of downstream
> (distribution) packages are highly opposed to supporting it as a
> dependency.  The REST part is not needed when the dashboard is built
> as a mgr module, because as a mgr module is has direct access to the
> Ceph state.
>

As you say. Can you point out any other areas of Ceph that I should
explore thoroughly for this project? Also, I would be glad if you
could share any kind of technologies that I need to be familiar with
(other than HTML, CSS, JS etc). For example, I'm looking into CherryPy
these days. It will help me to pick up things more easily if I know
the tech you are planning to use for the dashboard.

>> Last thing I want to ask is for what kinds of things I should be
>> contacting you and for what kind of things I should be contacting
>> John? Since John is my mentor on this project and I can see him
>> working on most of the Dashboard portion on GitHub, I presume for any
>> dashboard related issue I should contact him. And also to whom should
>> I send the weekly report of my progress on my project?
>
> I think I'm supposed to be a co-mentor (whatever that means :-) ) and
> Kefu is your primary point of contact.  I'm always happy to provide
> opinions though of course.
>
> John
>
>
>> Regards,
>> Saumay.
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GSOC] ceph-mgr: Cluster Status Dashboard

[Xiaoxi Chen]

FWIW, we have our home-made dashboard  in production including
a) Ceph cluster status  (Healthy, PG, pool, space , IO statistisc by pool)
b) Hardware metrics on ceph nodes(cpu, memory ,disk iostat, network)
c) perf dump via admin socket, most importantly , commit_latency and
apply_latency for OSDs.
d) user facing performance , mostly latency,  collected by periodly
running some test IO on top of the cluster.

The data are come from ceph itself(a) , prometheus agent(b) , hacked
script from "ceph daemonperf" (c), and home mode monitoring code(d).
Dashboard is Grafana on top of prometheus, it is easy to use and easy
to customized.

Wondering if Grafana or whatever other existing dashboard framework
could also be a choice for dashboard? Just ceph-mgr collect all the
informations and dashboard frontend consume data from ceph-mgr via
API?

Xiaoxi

2017-05-13 18:37 GMT+08:00 saumay agrawal <saumay.agrawal@gmail.com>:
> On Sat, May 13, 2017 at 1:35 PM, John Spray <jspray@redhat.com> wrote:
>> On Sat, May 13, 2017 at 6:06 AM, saumay agrawal
>> <saumay.agrawal@gmail.com> wrote:
>>> Hi Kefu,
>>>
>>> I apologize for the late reply.
>>>
>>> I have been looking into the PR you mentioned earlier
>>> https://github.com/ceph/ceph/pull/14946 . I have also explored the
>>> Calamari Dashboard in the past few days. I have some queries.
>>>
>>> I guess since Calamari Dashboard already does the job of visualizing
>>> the statistics of a Ceph cluster (health, OSDs, MONs, pools, IOPS,
>>> space utilization etc), we would primarily be improving on its UI/UX,
>>> that is to say the visualization portion and some other
>>> functionalities. Please do correct me if I am wrong in this.
>>>
>>> Moreover, Calamari Dashboard is made up of Calamari Server (backend)
>>> and Calamari Clients (frontend).
>>>
>>> I found out that Calamari Server uses a new REST API instead of being
>>> based upon the CEPH REST API. Will I be working on improving this new
>>> REST API?
>>>
>>> Moreover, Calamari Server is written in Django and
>>> django-rest-framework. However in the prototype of the dashboard
>>> mentioned in the PR above, I could find that the dashboard is using
>>> CherryPy too. CherryPy is a minimalist framework which is good for
>>> prototyping and makes it very easy. Meanwhile Django is more of a
>>> complete web application stack. So will we be using a single
>>> technology primarily, or a combination of both?
>>
>> Welcome!
>>
>
> Thanks, John. :)
>
>> Don't worry about any of the Django or REST stuff.  Django is being
>> dropped from everything because the maintainers of downstream
>> (distribution) packages are highly opposed to supporting it as a
>> dependency.  The REST part is not needed when the dashboard is built
>> as a mgr module, because as a mgr module is has direct access to the
>> Ceph state.
>>
>
> As you say. Can you point out any other areas of Ceph that I should
> explore thoroughly for this project? Also, I would be glad if you
> could share any kind of technologies that I need to be familiar with
> (other than HTML, CSS, JS etc). For example, I'm looking into CherryPy
> these days. It will help me to pick up things more easily if I know
> the tech you are planning to use for the dashboard.
>
>>> Last thing I want to ask is for what kinds of things I should be
>>> contacting you and for what kind of things I should be contacting
>>> John? Since John is my mentor on this project and I can see him
>>> working on most of the Dashboard portion on GitHub, I presume for any
>>> dashboard related issue I should contact him. And also to whom should
>>> I send the weekly report of my progress on my project?
>>
>> I think I'm supposed to be a co-mentor (whatever that means :-) ) and
>> Kefu is your primary point of contact.  I'm always happy to provide
>> opinions though of course.
>>
>> John
>>
>>
>>> Regards,
>>> Saumay.
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GSOC] ceph-mgr: Cluster Status Dashboard

[Sage Weil]

On Fri, 19 May 2017, Xiaoxi Chen wrote:
> FWIW, we have our home-made dashboard  in production including
> a) Ceph cluster status  (Healthy, PG, pool, space , IO statistisc by pool)
> b) Hardware metrics on ceph nodes(cpu, memory ,disk iostat, network)
> c) perf dump via admin socket, most importantly , commit_latency and
> apply_latency for OSDs.
> d) user facing performance , mostly latency,  collected by periodly
> running some test IO on top of the cluster.
> 
> The data are come from ceph itself(a) , prometheus agent(b) , hacked
> script from "ceph daemonperf" (c), and home mode monitoring code(d).
> Dashboard is Grafana on top of prometheus, it is easy to use and easy
> to customized.

(c) makes the nervous.  You shouldn't need to consume daemonperf output; 
the 'ceph daemon <name> perf dump' provides the same (and more) data in 
json form.

> Wondering if Grafana or whatever other existing dashboard framework
> could also be a choice for dashboard? Just ceph-mgr collect all the
> informations and dashboard frontend consume data from ceph-mgr via
> API?

Yes!  As discussed on an earlier thread, we'd like to have a module or 
feature in ceph-mgr that sends all of its metrics data (which it is 
already collecting) to prometheus (or allows it to be polled by 
prometheus).  I suspect this is a pretty simple task.

I'm also curious how the grafana dashboards are built (I haven't used 
Grafana before).  Is this the sort of thing where the dashboards can be 
developed and managed in source control?  It would be great to have a 
bunch of pre-built and collaboratively developed Ceph dashboards!

sage



> 
> Xiaoxi
> 
> 2017-05-13 18:37 GMT+08:00 saumay agrawal <saumay.agrawal@gmail.com>:
> > On Sat, May 13, 2017 at 1:35 PM, John Spray <jspray@redhat.com> wrote:
> >> On Sat, May 13, 2017 at 6:06 AM, saumay agrawal
> >> <saumay.agrawal@gmail.com> wrote:
> >>> Hi Kefu,
> >>>
> >>> I apologize for the late reply.
> >>>
> >>> I have been looking into the PR you mentioned earlier
> >>> https://github.com/ceph/ceph/pull/14946 . I have also explored the
> >>> Calamari Dashboard in the past few days. I have some queries.
> >>>
> >>> I guess since Calamari Dashboard already does the job of visualizing
> >>> the statistics of a Ceph cluster (health, OSDs, MONs, pools, IOPS,
> >>> space utilization etc), we would primarily be improving on its UI/UX,
> >>> that is to say the visualization portion and some other
> >>> functionalities. Please do correct me if I am wrong in this.
> >>>
> >>> Moreover, Calamari Dashboard is made up of Calamari Server (backend)
> >>> and Calamari Clients (frontend).
> >>>
> >>> I found out that Calamari Server uses a new REST API instead of being
> >>> based upon the CEPH REST API. Will I be working on improving this new
> >>> REST API?
> >>>
> >>> Moreover, Calamari Server is written in Django and
> >>> django-rest-framework. However in the prototype of the dashboard
> >>> mentioned in the PR above, I could find that the dashboard is using
> >>> CherryPy too. CherryPy is a minimalist framework which is good for
> >>> prototyping and makes it very easy. Meanwhile Django is more of a
> >>> complete web application stack. So will we be using a single
> >>> technology primarily, or a combination of both?
> >>
> >> Welcome!
> >>
> >
> > Thanks, John. :)
> >
> >> Don't worry about any of the Django or REST stuff.  Django is being
> >> dropped from everything because the maintainers of downstream
> >> (distribution) packages are highly opposed to supporting it as a
> >> dependency.  The REST part is not needed when the dashboard is built
> >> as a mgr module, because as a mgr module is has direct access to the
> >> Ceph state.
> >>
> >
> > As you say. Can you point out any other areas of Ceph that I should
> > explore thoroughly for this project? Also, I would be glad if you
> > could share any kind of technologies that I need to be familiar with
> > (other than HTML, CSS, JS etc). For example, I'm looking into CherryPy
> > these days. It will help me to pick up things more easily if I know
> > the tech you are planning to use for the dashboard.
> >
> >>> Last thing I want to ask is for what kinds of things I should be
> >>> contacting you and for what kind of things I should be contacting
> >>> John? Since John is my mentor on this project and I can see him
> >>> working on most of the Dashboard portion on GitHub, I presume for any
> >>> dashboard related issue I should contact him. And also to whom should
> >>> I send the weekly report of my progress on my project?
> >>
> >> I think I'm supposed to be a co-mentor (whatever that means :-) ) and
> >> Kefu is your primary point of contact.  I'm always happy to provide
> >> opinions though of course.
> >>
> >> John
> >>
> >>
> >>> Regards,
> >>> Saumay.
> >>> --
> >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> >>> the body of a message to majordomo@vger.kernel.org
> >>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: [GSOC] ceph-mgr: Cluster Status Dashboard

[Dan van der Ster]

On Fri, May 19, 2017 at 2:18 PM, Sage Weil <sage@newdream.net> wrote:
> I'm also curious how the grafana dashboards are built (I haven't used
> Grafana before).  Is this the sort of thing where the dashboards can be
> developed and managed in source control?  It would be great to have a
> bunch of pre-built and collaboratively developed Ceph dashboards!

A grafana dashboard is described by a json file, which can exported
and shared, e.g. at https://grafana.com/dashboards

-- dan

Re: [GSOC] ceph-mgr: Cluster Status Dashboard

[Xiaoxi Chen]

>(c) makes the nervous.  You shouldn't need to consume daemonperf output;
>the 'ceph daemon <name> perf dump' provides the same (and more) data in
>json form.

Yes, we do use "perf dump" but borrow the code pieces from daemonperf
to parse the result, you may need to sub, max, or calc interval
average, based on the counter's type.

Prometheus is a MUST for such use case as it can do histogram and
enable us to show percentile (like 95% commit latency) on dashboard,
it is the only way to read 1000+ osd's metrics.

2017-05-19 20:22 GMT+08:00 Dan van der Ster <dan@vanderster.com>:
> On Fri, May 19, 2017 at 2:18 PM, Sage Weil <sage@newdream.net> wrote:
>> I'm also curious how the grafana dashboards are built (I haven't used
>> Grafana before).  Is this the sort of thing where the dashboards can be
>> developed and managed in source control?  It would be great to have a
>> bunch of pre-built and collaboratively developed Ceph dashboards!
>
> A grafana dashboard is described by a json file, which can exported
> and shared, e.g. at https://grafana.com/dashboards
>
> -- dan

Re: [GSOC] ceph-mgr: Cluster Status Dashboard

[John Spray]

On Fri, May 19, 2017 at 8:18 AM, Xiaoxi Chen <superdebuger@gmail.com> wrote:
> FWIW, we have our home-made dashboard  in production including
> a) Ceph cluster status  (Healthy, PG, pool, space , IO statistisc by pool)
> b) Hardware metrics on ceph nodes(cpu, memory ,disk iostat, network)
> c) perf dump via admin socket, most importantly , commit_latency and
> apply_latency for OSDs.
> d) user facing performance , mostly latency,  collected by periodly
> running some test IO on top of the cluster.
>
> The data are come from ceph itself(a) , prometheus agent(b) , hacked
> script from "ceph daemonperf" (c), and home mode monitoring code(d).
> Dashboard is Grafana on top of prometheus, it is easy to use and easy
> to customized.
>
> Wondering if Grafana or whatever other existing dashboard framework
> could also be a choice for dashboard? Just ceph-mgr collect all the
> informations and dashboard frontend consume data from ceph-mgr via
> API?

Grafana and Prometheus are great, and an effort to put together a
standard configuration that we could ship with Ceph would be a very
positive thing.

"Dashboard" means different things to different people -- the goal of
the mgr module we have here is initially to give people similar
information to "ceph status" but in an easier to consume form.  Tools
for plotting and exploring time series data are kind of a separate
(also important) thing, which is also often called "dashboard" :-)

I say that not to discourage anyone, but because I've found
historically that the larger task of doing an integrated TSDB solution
has sometimes held us back from delivering the bare basics of a web
dashboard that tells you your cluster health, and I'm keen to make
sure that we don't let the complicated parts hold us back from
delivering the simpler parts quickly.

John

>
> Xiaoxi
>
> 2017-05-13 18:37 GMT+08:00 saumay agrawal <saumay.agrawal@gmail.com>:
>> On Sat, May 13, 2017 at 1:35 PM, John Spray <jspray@redhat.com> wrote:
>>> On Sat, May 13, 2017 at 6:06 AM, saumay agrawal
>>> <saumay.agrawal@gmail.com> wrote:
>>>> Hi Kefu,
>>>>
>>>> I apologize for the late reply.
>>>>
>>>> I have been looking into the PR you mentioned earlier
>>>> https://github.com/ceph/ceph/pull/14946 . I have also explored the
>>>> Calamari Dashboard in the past few days. I have some queries.
>>>>
>>>> I guess since Calamari Dashboard already does the job of visualizing
>>>> the statistics of a Ceph cluster (health, OSDs, MONs, pools, IOPS,
>>>> space utilization etc), we would primarily be improving on its UI/UX,
>>>> that is to say the visualization portion and some other
>>>> functionalities. Please do correct me if I am wrong in this.
>>>>
>>>> Moreover, Calamari Dashboard is made up of Calamari Server (backend)
>>>> and Calamari Clients (frontend).
>>>>
>>>> I found out that Calamari Server uses a new REST API instead of being
>>>> based upon the CEPH REST API. Will I be working on improving this new
>>>> REST API?
>>>>
>>>> Moreover, Calamari Server is written in Django and
>>>> django-rest-framework. However in the prototype of the dashboard
>>>> mentioned in the PR above, I could find that the dashboard is using
>>>> CherryPy too. CherryPy is a minimalist framework which is good for
>>>> prototyping and makes it very easy. Meanwhile Django is more of a
>>>> complete web application stack. So will we be using a single
>>>> technology primarily, or a combination of both?
>>>
>>> Welcome!
>>>
>>
>> Thanks, John. :)
>>
>>> Don't worry about any of the Django or REST stuff.  Django is being
>>> dropped from everything because the maintainers of downstream
>>> (distribution) packages are highly opposed to supporting it as a
>>> dependency.  The REST part is not needed when the dashboard is built
>>> as a mgr module, because as a mgr module is has direct access to the
>>> Ceph state.
>>>
>>
>> As you say. Can you point out any other areas of Ceph that I should
>> explore thoroughly for this project? Also, I would be glad if you
>> could share any kind of technologies that I need to be familiar with
>> (other than HTML, CSS, JS etc). For example, I'm looking into CherryPy
>> these days. It will help me to pick up things more easily if I know
>> the tech you are planning to use for the dashboard.
>>
>>>> Last thing I want to ask is for what kinds of things I should be
>>>> contacting you and for what kind of things I should be contacting
>>>> John? Since John is my mentor on this project and I can see him
>>>> working on most of the Dashboard portion on GitHub, I presume for any
>>>> dashboard related issue I should contact him. And also to whom should
>>>> I send the weekly report of my progress on my project?
>>>
>>> I think I'm supposed to be a co-mentor (whatever that means :-) ) and
>>> Kefu is your primary point of contact.  I'm always happy to provide
>>> opinions though of course.
>>>
>>> John
>>>
>>>
>>>> Regards,
>>>> Saumay.
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html