FW: ceph_monitor - monitor your cluster with parallel python

FW: ceph_monitor - monitor your cluster with parallel python

[Igor.Podoski]

Hey, one more time here, I’ve got reject from mail daemon.

Regards,
Igor.

From: Podoski, Igor 
Sent: Thursday, November 19, 2015 8:53 AM
To: ceph-devel; 'ceph-users@ceph.com'
Subject: ceph_monitor - monitor your cluster with parallel python

Hi Cephers!

I’ve created small tool to help track memory/cpu/io usage. It’s useful for me so I thought I could share with you: https://github.com/aiicore/ceph_monitor

In general this is a python script, that uses parallel python to run a function on remote host. Data is gathered from all hosts and presented on console or added to sqlite database, then can be plotted with e.g. gnuplot. You can define osd ranges, that you want to monitor, or monitor certain process, e.g. osds only from pool that has ssds.

The main concept is that monitor don’t know and don’t care on which host osd’s are running, it treats them as a whole set.

Script uses psutil to get data related to processes (mon/osd/rgw/whatever). In near feature I’d like to add modes that can modify process behavior e.g. psutil has .nice .ionice .cpu_affinity methods, that could be useful in some tests. Basically with parallel python you can run any function remotely, so tuning SO by changing some /proc/* files can be done too.

You can add labels to data to see when what happens.

Sample plot: https://raw.githubusercontent.com/aiicore/ceph_monitor/master/examples/avg_cpu_mem.png 
Simple test: https://github.com/aiicore/ceph_monitor/blob/master/examples/example_test_with_rados.sh 

Short readme:  https://github.com/aiicore/ceph_monitor 
Full readme: https://github.com/aiicore/ceph_monitor/blob/master/readme.txt 

I encourage You to use and develop it, if not just please read the full readme text, maybe you’ll  come up with a better idea based on my this concept and something interesting will happen.

p.s. This currently works with python 2.6 and psutil 0.6.1 on centos 6.6.  If you find any bug – report it on my github as an issue.

!!! Security notice !!!
Parallel python supports SHA authentication  – my version currently runs WITHOUT this so in certain environments it could be dangerous (you could run any function from untrusted client). For now use it only in test/dev isolated clusters.

Regards,
Igor.

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html