Ceph Manager brag module

Ceph Manager brag module

[Wido den Hollander]

Hi,

After discussing with Sage I started to work [0] on a 'brag' module for 
Ceph Manager.

The intention of this module is to send (after a opt-in!) statistics 
about a Ceph cluster back to the project every few days.

My intention is to include:

- fsid (as identifier)
- number of MONs
- number of OSDs
- FileStore / BlueStore
- number of pools / PGs
- data used in bytes
- data free
- number of objects,
- version(s)

A admin can also say that the report is allowed to be 'public' where you 
can then visit (for example) https://brag.ceph.com/public/<fsid> and 
show off your Ceph cluster.

You could then add a description and we could compile top lists about 
Ceph systems.

How do people think about this? Would this be interesting?

I would try to convince most of my customers to send this information 
back to the project. The more information we get, the better Ceph can 
become.

Wido

[0]: https://github.com/wido/ceph/commits/mgr-brag

Re: Ceph Manager brag module

[Lenz Grimmer]

[-- Attachment #1.1: Type: text/plain, Size: 1809 bytes --]

Hey Wido,

On 12/14/2017 10:24 AM, Wido den Hollander wrote:

> After discussing with Sage I started to work [0] on a 'brag' module for
> Ceph Manager.

Nice!

> The intention of this module is to send (after a opt-in!) statistics
> about a Ceph cluster back to the project every few days.
> 
> My intention is to include:
> 
> - fsid (as identifier)
> - number of MONs
> - number of OSDs
> - FileStore / BlueStore
> - number of pools / PGs
> - data used in bytes
> - data free
> - number of objects,
> - version(s)

Not sure if that could actually be extracted, but how about "Services
enabled", e.g. "RGW", "CephFS" or "RBD", with some additional insight
(e.g. how many RBDs/buckets)?

I think it would be valuable to obtain some (optional) insight into what
that cluster is actually used for.

> A admin can also say that the report is allowed to be 'public' where you
> can then visit (for example) https://brag.ceph.com/public/<fsid> and
> show off your Ceph cluster.
> 
> You could then add a description and we could compile top lists about
> Ceph systems.
> 
> How do people think about this? Would this be interesting?

Absolutely. One challenge I can see here (in addition to users actually
enabling it) is networking - I'd assume that many customers have
restricted their networks, so the MGR cannot easily establish an
outgoing connection without going through a proxy or before the admin
has poked a hole into the firewall. This might require configuration
options or corresponding documentation.

> I would try to convince most of my customers to send this information
> back to the project. The more information we get, the better Ceph can
> become.
> 
> Wido
> 
> [0]: https://github.com/wido/ceph/commits/mgr-brag

Nice work!

Lenz


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 220 bytes --]

Re: Ceph Manager brag module

[Kai Wagner]

[-- Attachment #1.1: Type: text/plain, Size: 2753 bytes --]

Hi Wido,


On 14.12.2017 12:44, Lenz Grimmer wrote:
> Hey Wido,
>
> On 12/14/2017 10:24 AM, Wido den Hollander wrote:
>
>> After discussing with Sage I started to work [0] on a 'brag' module for
>> Ceph Manager.
> Nice!
+1 great idea!
>> The intention of this module is to send (after a opt-in!) statistics
>> about a Ceph cluster back to the project every few days.
>>
>> My intention is to include:
>>
>> - fsid (as identifier)
>> - number of MONs
>> - number of OSDs
>> - FileStore / BlueStore
>> - number of pools / PGs
>> - data used in bytes
>> - data free
>> - number of objects,
>> - version(s)
> Not sure if that could actually be extracted, but how about "Services
> enabled", e.g. "RGW", "CephFS" or "RBD", with some additional insight
> (e.g. how many RBDs/buckets)?
>
> I think it would be valuable to obtain some (optional) insight into what
> that cluster is actually used for.
Two things. It would be really nice if the user could specify which of
those options they're willing to share, so these should be configurable.

Another thing I would like to add here is "performance" of the overall
cluster in general. So iop's/throughput/latency and so on. This way a
user could also check how the performance of their cluster evolved over
time.

Pretty nifty example: After a user upgraded their cluster to the newest
release was their a big performance win or drop maybe caused by this
update? This would be really helpful for everyone.
>
>> A admin can also say that the report is allowed to be 'public' where you
>> can then visit (for example) https://brag.ceph.com/public/<fsid> and
>> show off your Ceph cluster.
>>
>> You could then add a description and we could compile top lists about
>> Ceph systems.
>>
>> How do people think about this? Would this be interesting?
> Absolutely. One challenge I can see here (in addition to users actually
> enabling it) is networking - I'd assume that many customers have
> restricted their networks, so the MGR cannot easily establish an
> outgoing connection without going through a proxy or before the admin
> has poked a hole into the firewall. This might require configuration
> options or corresponding documentation.
That true indeed, but I think customers who are willing to send reports
don't have a problem to route or open this port within their firewall.
>> I would try to convince most of my customers to send this information
>> back to the project. The more information we get, the better Ceph can
>> become.
>>
>> Wido
>>
>> [0]: https://github.com/wido/ceph/commits/mgr-brag
> Nice work!
>
> Lenz
>

-- 
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: Ceph Manager brag module

[Blair Bethwaite]

Nice idea.

I definitely agree that the submission mechanism needs to support HTTP
proxies. I imagine many (most?) ceph-mgr daemons are on private
networks, and not everyone has NAT.

On 14 December 2017 at 20:24, Wido den Hollander <wido@42on.com> wrote:
> Hi,
>
> After discussing with Sage I started to work [0] on a 'brag' module for Ceph
> Manager.
>
> The intention of this module is to send (after a opt-in!) statistics about a
> Ceph cluster back to the project every few days.
>
> My intention is to include:
>
> - fsid (as identifier)
> - number of MONs
> - number of OSDs
> - FileStore / BlueStore
> - number of pools / PGs
> - data used in bytes
> - data free
> - number of objects,
> - version(s)
>
> A admin can also say that the report is allowed to be 'public' where you can
> then visit (for example) https://brag.ceph.com/public/<fsid> and show off
> your Ceph cluster.
>
> You could then add a description and we could compile top lists about Ceph
> systems.
>
> How do people think about this? Would this be interesting?
>
> I would try to convince most of my customers to send this information back
> to the project. The more information we get, the better Ceph can become.
>
> Wido
>
> [0]: https://github.com/wido/ceph/commits/mgr-brag
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Cheers,
~Blairo

Re: Ceph Manager brag module

[Wido den Hollander]

On 12/14/2017 03:08 PM, Blair Bethwaite wrote:
> Nice idea.
> 

Tnx!

> I definitely agree that the submission mechanism needs to support HTTP
> proxies. I imagine many (most?) ceph-mgr daemons are on private
> networks, and not everyone has NAT.

Yes, that's something I'll implement. Should be simple to do with Python 
requests.

Wido

> 
> On 14 December 2017 at 20:24, Wido den Hollander <wido@42on.com> wrote:
>> Hi,
>>
>> After discussing with Sage I started to work [0] on a 'brag' module for Ceph
>> Manager.
>>
>> The intention of this module is to send (after a opt-in!) statistics about a
>> Ceph cluster back to the project every few days.
>>
>> My intention is to include:
>>
>> - fsid (as identifier)
>> - number of MONs
>> - number of OSDs
>> - FileStore / BlueStore
>> - number of pools / PGs
>> - data used in bytes
>> - data free
>> - number of objects,
>> - version(s)
>>
>> A admin can also say that the report is allowed to be 'public' where you can
>> then visit (for example) https://brag.ceph.com/public/<fsid> and show off
>> your Ceph cluster.
>>
>> You could then add a description and we could compile top lists about Ceph
>> systems.
>>
>> How do people think about this? Would this be interesting?
>>
>> I would try to convince most of my customers to send this information back
>> to the project. The more information we get, the better Ceph can become.
>>
>> Wido
>>
>> [0]: https://github.com/wido/ceph/commits/mgr-brag
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 

Re: Ceph Manager brag module

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 895 bytes --]

On Thu, Dec 14, 2017 at 10:24:24AM +0100, Wido den Hollander wrote:
> Hi,
> 
> After discussing with Sage I started to work [0] on a 'brag' module for 
> Ceph Manager.
> 
> The intention of this module is to send (after a opt-in!) statistics 
> about a Ceph cluster back to the project every few days.
I like it, but have one concern.

> My intention is to include:
> - fsid (as identifier)
Please don't expose the raw FSID.
Consider a hash of FSID before submission.

> - number of MONs
> - number of OSDs
> - FileStore / BlueStore
> - number of pools / PGs
> - data used in bytes
> - data free
> - number of objects,
> - version(s)
RGWs?

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Asst. Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]

Re: Ceph Manager brag module

[Sage Weil]

On Thu, 14 Dec 2017, Robin H. Johnson wrote:
> On Thu, Dec 14, 2017 at 10:24:24AM +0100, Wido den Hollander wrote:
> > Hi,
> > 
> > After discussing with Sage I started to work [0] on a 'brag' module for 
> > Ceph Manager.
> > 
> > The intention of this module is to send (after a opt-in!) statistics 
> > about a Ceph cluster back to the project every few days.
> I like it, but have one concern.
> 
> > My intention is to include:
> > - fsid (as identifier)
> Please don't expose the raw FSID.
> Consider a hash of FSID before submission.

+1. I'd actually just generate a new uuid and stash it in mgr/brag/brag-id 
(or similar) in config-key.

> > - number of MONs
> > - number of OSDs
> > - FileStore / BlueStore
> > - number of pools / PGs
> > - data used in bytes
> > - data free
> > - number of objects,
> > - version(s)
> RGWs?

It can summarize everything in servicemap by type so that we also capture 
other daemons too (rbd-mirror and future gateways).

sage

Re: Ceph Manager brag module

[Vasu Kulkarni]

> On Dec 14, 2017, at 1:24 AM, Wido den Hollander <wido@42on.com> wrote:
> 
> Hi,
> 
> After discussing with Sage I started to work [0] on a 'brag' module for Ceph Manager.
> 
> The intention of this module is to send (after a opt-in!) statistics about a Ceph cluster back to the project every few days.
> 
> My intention is to include:
> 
> - fsid (as identifier)
> - number of MONs
> - number of OSDs
> - FileStore / BlueStore
> - number of pools / PGs
> - data used in bytes
> - data free
> - number of objects,
> - version(s)
> 
> A admin can also say that the report is allowed to be 'public' where you can then visit (for example) https://brag.ceph.com/public/<fsid> and show off your Ceph cluster.
> 
> You could then add a description and we could compile top lists about Ceph systems.
> 
> How do people think about this? Would this be interesting?

Great idea, It would be also nice to have some info on hw/osd tree + rados bench results runs for 2 mins with few profiles which could be displayed alongside (could be optional
and admin’s choice)

> 
> I would try to convince most of my customers to send this information back to the project. The more information we get, the better Ceph can become.
> 
> Wido
> 
> [0]: https://github.com/wido/ceph/commits/mgr-brag
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Ceph Manager brag module

[Wido den Hollander]

On 12/15/2017 09:14 AM, Blair Bethwaite wrote:
> I would sign up to have a regular rados bench or similar run at a 
> configurable time on our clusters.

I would have to look into that as I don't know how much the manager can 
access.

At first we just want to gather metadata and send it back so that we get 
a feeling of how Ceph is used and how big/small the systems are.

Thanks for the input!

Wido

> 
> On 15 Dec. 2017 10:44 am, "Vasu Kulkarni" <vakulkar@redhat.com 
> <mailto:vakulkar@redhat.com>> wrote:
> 
> 
>      > On Dec 14, 2017, at 1:24 AM, Wido den Hollander <wido@42on.com
>     <mailto:wido@42on.com>> wrote:
>      >
>      > Hi,
>      >
>      > After discussing with Sage I started to work [0] on a 'brag'
>     module for Ceph Manager.
>      >
>      > The intention of this module is to send (after a opt-in!)
>     statistics about a Ceph cluster back to the project every few days.
>      >
>      > My intention is to include:
>      >
>      > - fsid (as identifier)
>      > - number of MONs
>      > - number of OSDs
>      > - FileStore / BlueStore
>      > - number of pools / PGs
>      > - data used in bytes
>      > - data free
>      > - number of objects,
>      > - version(s)
>      >
>      > A admin can also say that the report is allowed to be 'public'
>     where you can then visit (for example)
>     https://brag.ceph.com/public/<fsid> and show off your Ceph cluster.
>      >
>      > You could then add a description and we could compile top lists
>     about Ceph systems.
>      >
>      > How do people think about this? Would this be interesting?
> 
>     Great idea, It would be also nice to have some info on hw/osd tree +
>     rados bench results runs for 2 mins with few profiles which could be
>     displayed alongside (could be optional
>     and admin’s choice)
> 
>      >
>      > I would try to convince most of my customers to send this
>     information back to the project. The more information we get, the
>     better Ceph can become.
>      >
>      > Wido
>      >
>      > [0]: https://github.com/wido/ceph/commits/mgr-brag
>     <https://github.com/wido/ceph/commits/mgr-brag>
>      > --
>      > To unsubscribe from this list: send the line "unsubscribe
>     ceph-devel" in
>      > the body of a message to majordomo@vger.kernel.org
>     <mailto:majordomo@vger.kernel.org>
>      > More majordomo info at http://vger.kernel.org/majordomo-info.html
>     <http://vger.kernel.org/majordomo-info.html>
> 
>     --
>     To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>     the body of a message to majordomo@vger.kernel.org
>     <mailto:majordomo@vger.kernel.org>
>     More majordomo info at http://vger.kernel.org/majordomo-info.html
>     <http://vger.kernel.org/majordomo-info.html>
>