* mon config commands
@ 2018-01-11 21:21 Sage Weil
2018-01-12 18:21 ` Alfredo Deza
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Sage Weil @ 2018-01-11 21:21 UTC (permalink / raw)
To: ceph-devel
Hi everyone,
Here's the current state of the mon config commands for mimic. These are
pretty close to being "done" so please review carefully!
config assimilate-conf Assimilate options from a conf, and
return a new, minimal conf file
config dump Show all configuration option(s)
config get <who> {<key>} Show configuration option(s) for an
entity
config help <key> Describe a configuration option
config rm <who> <name> Clear a configuration option for one or
more entities
config set <who> <name> <value> Cet a configuration option for one or
more entities
config show <who> {<key>} Show running configuration
config show-with-defaults <who> Show running configuration (including
compiled-in defaults)
---
First, let's dump everything:
$ bin/ceph config dump
WHO MASK LEVEL OPTION VALUE RO
global advanced mon_pg_warn_min_per_osd 3
global advanced osd_crush_chooseleaf_type 0
global advanced osd_pool_default_min_size 1
global advanced osd_pool_default_size 1
mds advanced debug_mds 20/20
mds advanced debug_mgrc 20/20
mds advanced debug_monc 20/20
mds advanced debug_ms 1/1
mds developer mds_debug_auth_pins true
mds developer mds_debug_frag true
mds developer mds_debug_scatterstat true
mds developer mds_debug_subtrees true
mds developer mds_verify_scatter true
mgr advanced debug_mgr 20/20
mgr advanced debug_mon 20/20
mgr advanced debug_monc 20/20
mgr advanced debug_ms 1/1
mon unknown mon_allow_pool_deletes true *
mon advanced mon_data_avail_crit 1
mon advanced mon_data_avail_warn 2
...
Note the one RO item here is also 'unknown'.. that's because it's not a
valid config option (there's no 's').. it was injected directly into
config-key.
For a single daemon,
gnit:build (wip-config) 02:47 PM $ bin/ceph config get mon.a
WHO MASK LEVEL OPTION VALUE RO
mon unknown mon_allow_pool_deletes true *
mon advanced mon_data_avail_crit 1
mon advanced mon_data_avail_warn 2
mon advanced mon_osd_reporter_subtree_level osd
global advanced mon_pg_warn_min_per_osd 3
global advanced osd_crush_chooseleaf_type 0
global advanced osd_pool_default_min_size 1
global advanced osd_pool_default_size 1
If I try to set something that can't be changed at runtime (say, ms_type),
you get:
gnit:build (wip-config) 02:47 PM $ bin/ceph config set mon.a ms_type
simple
gnit:build (wip-config) 02:48 PM $ bin/ceph config get mon.a
WHO MASK LEVEL OPTION VALUE RO
mon unknown mon_allow_pool_deletes true *
mon advanced mon_data_avail_crit 1
mon advanced mon_data_avail_warn 2
mon advanced mon_osd_reporter_subtree_level osd
global advanced mon_pg_warn_min_per_osd 3
mon.a advanced ms_type simple *
global advanced osd_crush_chooseleaf_type 0
global advanced osd_pool_default_min_size 1
global advanced osd_pool_default_size 1
You can then see this by checking the *running* config (this is what the
daemon has reported as it's actually config to the manager):
gnit:build (wip-config) 02:48 PM $ bin/ceph config show mon.a
NAME VALUE SOURCE OVERRIDES IGNORES
admin_socket /tmp/ceph-asok.GBZTV0/$name.asok file
chdir file
debug_auth 20/20 file
debug_mgrc 20/20 file
debug_mon 20/20 file
debug_ms 1/1 file
debug_paxos 20/20 file
enable_experimental_unrecoverable_data_corrupting_features * file
erasure_code_dir /home/sage/src/ceph6/build/lib file
filestore_fd_cache_size 32 file
heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat file
leveldb_block_size 65536 default
leveldb_cache_size 536870912 default
leveldb_compression 0 default
leveldb_log default
leveldb_write_buffer_size 33554432 default
lockdep 1 file
log_file /home/sage/src/ceph6/build/out/$name.log file
mgr_initial_modules restful status dashboard balancer file
mon_cluster_log_file /home/sage/src/ceph6/build/out/cluster.mon.$id.log file
mon_data /home/sage/src/ceph6/build/dev/mon.a file
mon_data_avail_crit 1 mon
mon_data_avail_warn 2 mon
mon_osd_backfillfull_ratio 0.99 file
mon_osd_full_ratio 0.99 file
mon_osd_nearfull_ratio 0.99 file
mon_osd_reporter_subtree_level osd mon
mon_pg_warn_min_per_osd 3 mon
ms_type async+posix default mon
osd_crush_chooseleaf_type 0 mon
osd_failsafe_full_ratio 0.99 file
osd_pool_default_min_size 1 mon
osd_pool_default_size 1 mon
pid_file /home/sage/src/ceph6/build/out/mon.a.pid cmdline file
plugin_dir /home/sage/src/ceph6/build/lib file
run_dir /home/sage/src/ceph6/build/out file
(sorry for the wide table)
A few things to note: SOURCE can be any of default, file, mon, env,
cmdline, or override. You'll notice that ms_type SOURCE is 'default' and
IGNORES is 'mon', because the daemon is ignoring the mon value--this
option can only be set during startup.
Similarly, if I try to override something else manually,
gnit:build (wip-config) 02:50 PM $ bin/ceph daemon mon.a config set mon_data_avail_warn 3
{
"success": ""
}
gnit:build (wip-config) 02:50 PM $ bin/ceph config show mon.a
NAME VALUE SOURCE OVERRIDES IGNORES
...
mon_data_avail_warn 3 override mon
...
we see that the currently value is an override, and OVERRIDES the mon
value. As far as I can tel this is a pretty complete view of what is
going on. The JSON form has a bit more info, like what the mon value that
was override was:
...
{
"name": "mon_data_avail_warn",
"value": "3",
"source": "override",
"overrides": [
{
"source": "mon",
"value": "2"
}
]
},
...
Anyway, moving on...
You can also 'get' and 'show' specific values, making this useful from a
script:
gnit:build (wip-config) 02:52 PM $ bin/ceph config get mon.a ms_type
simple
gnit:build (wip-config) 02:52 PM $ bin/ceph config show mon.a ms_type
async+posix
This will return valid results for defaults, too:
gnit:build (wip-config) 02:52 PM $ bin/ceph config show mon.a osd_data
/var/lib/ceph/osd/$cluster-$id
...although the result has not has substitutions processed yet (should
it?).
You can get help:
gnit:build (wip-config) 02:54 PM $ bin/ceph config help
bluestore_cache_size
bluestore_cache_size - Cache size (in bytes) for BlueStore
(uint64_t, advanced)
Default: 0
Can update at runtime: true
This includes data and metadata cached by BlueStore as well as memory devoted to rocksdb's cache(s).
I mostly hate how the above looks, BTW--suggestions welcome! In JSON
form,
gnit:build (wip-config) 02:54 PM $ bin/ceph config help bluestore_cache_size -f json-pretty
{
"name": "bluestore_cache_size",
"type": "uint64_t",
"level": "advanced",
"desc": "Cache size (in bytes) for BlueStore",
"long_desc": "This includes data and metadata cached by BlueStore as
well as memory devoted to rocksdb's cache(s).",
"default": 0,
"daemon_default": "",
"tags": [],
"services": [],
"see_also": [],
"min": "",
"max": "",
"can_update_at_runtime": true
}
The 'config set' has basic validation:
gnit:build (wip-config) 03:07 PM $ bin/ceph config set osd.0 foo bar
Error EINVAL: unrecognized config option 'foo'
gnit:build (wip-config) 03:07 PM $ bin/ceph config set osd.0 osd_tier_default_cache_mode bar
Error EINVAL: error parsing value: 'bar' is not one of the permitted values: none, writeback, forward, readonly, readforward, readproxy, proxy
gnit:build (wip-config) 03:08 PM $ bin/ceph config set osd.0
osd_tier_default_cache_mode writeback
and you can clear an option:
gnit:build (wip-config) 03:08 PM $ bin/ceph config get mon.a
WHO MASK LEVEL OPTION VALUE RO
mon unknown mon_allow_pool_deletes true *
mon advanced mon_data_avail_crit 1
mon advanced mon_data_avail_warn 2
mon advanced mon_osd_reporter_subtree_level osd
global advanced mon_pg_warn_min_per_osd 3
mon.a advanced ms_type simple *
global advanced osd_crush_chooseleaf_type 0
global advanced osd_pool_default_min_size 1
global advanced osd_pool_default_size 1
gnit:build (wip-config) 03:08 PM $ bin/ceph config rm mon.a ms_type
gnit:build (wip-config) 03:09 PM $ bin/ceph config get mon.a
WHO MASK LEVEL OPTION VALUE RO
mon unknown mon_allow_pool_deletes true *
mon advanced mon_data_avail_crit 1
mon advanced mon_data_avail_warn 2
mon advanced mon_osd_reporter_subtree_level osd
global advanced mon_pg_warn_min_per_osd 3
global advanced osd_crush_chooseleaf_type 0
global advanced osd_pool_default_min_size 1
global advanced osd_pool_default_size 1
The last one is 'assimilate-conf' and is meant to help transition from
a ceph.conf world to a mon config world. This is a vstart cluster, so all
of the paths are all wonky and there's lots of specialness that you
wouldn't see in the real world. It still imports most stuff:
gnit:build (wip-config) 03:09 PM $ wc -l ceph.conf
115 ceph.conf
gnit:build (wip-config) 03:09 PM $ bin/ceph config assimilate-conf -i ceph.conf -o new.conf
2018-01-11 15:09:55.390 7f5a3cff9700 -1 set_mon_vals failed to set admin_socket = /tmp/ceph-asok.GBZTV0/$name.$pid.asok: Configuration option 'admin_socket' may not be modified at runtime
(That warning appears because the CLI process observed the config change
as it happened, and isn't able to update the admin_socket setting at
runtime.)
What's left is stuff flagged NO_MON_CONFIG or unrecognized as a valid
option:
gnit:build (wip-config) 03:09 PM $ wc -l new.conf
46 new.conf
gnit:build (wip-config) 03:10 PM $ cat new.conf
[client]
keyring = /home/sage/src/ceph6/build/keyring
[client.vstart.sh]
num_mds = 0
num_mgr = 1
num_mon = 1
num_osd = 1
num_rgw = 0
[global]
fsid = 9dbd93f7-635a-4734-bc29-448a2305af7f
lockdep = true
mon_osd_backfillfull_ratio = .99
mon_osd_full_ratio = .99
mon_osd_nearfull_ratio = .99
[mds]
chdir =
mds_data = /home/sage/src/ceph6/build/dev/mds.$id
[mgr]
chdir =
mgr_data = /home/sage/src/ceph6/build/dev/mgr.$id
[mgr.x]
host = gnit
[mon]
chdir =
mgr_initial_modules = restful status dashboard balancer
[mon.a]
host = gnit
mon_addr = 127.0.0.1:40948
mon_data = /home/sage/src/ceph6/build/dev/mon.a
[osd]
chdir =
osd_class_tmp = out
osd_data = /home/sage/src/ceph6/build/dev/osd$id
osd_journal = /home/sage/src/ceph6/build/dev/osd$id/journal
[osd.0]
host = gnit
This command isn't as smart as it maybe could be. It doesn't try to do
anything clever for ceph.conf's with conflicting values from different
hosts... so if host A has debug_ms=1 and host B has debug_ms=2, the mon
will keep whichever was imported last. In general, though, if the config
options are restricted to [type.id] per-daemon sections then things will
work just fine.
We *could* have tools try to look at the result in the mon and simplify or
rearrange, by (for example) moving osd-specific options under [osd] so
that they don't pollute other daemons' configs. I leave this to future
work...
Anyway, please let me know if there is anything ugly or unintuitive or
wrong here. If this looks sane I'll finally write up the docs!
sage
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mon config commands
2018-01-11 21:21 mon config commands Sage Weil
@ 2018-01-12 18:21 ` Alfredo Deza
2018-01-12 18:57 ` Sage Weil
2018-01-12 23:17 ` Gregory Farnum
2018-01-16 16:14 ` Douglas Fuller
2 siblings, 1 reply; 8+ messages in thread
From: Alfredo Deza @ 2018-01-12 18:21 UTC (permalink / raw)
To: Sage Weil; +Cc: ceph-devel
On Thu, Jan 11, 2018 at 4:21 PM, Sage Weil <sage@redhat.com> wrote:
> Hi everyone,
>
> Here's the current state of the mon config commands for mimic. These are
> pretty close to being "done" so please review carefully!
>
> config assimilate-conf Assimilate options from a conf, and
> return a new, minimal conf file
I think this would still be fine as `config assimilate`. The `-conf`
reads a bit redundant, and it wouldn't be terribly obscure to leave it
as plain `assimilate`
> config dump Show all configuration option(s)
> config get <who> {<key>} Show configuration option(s) for an
> entity
Is this output from help? Because "who" is not used anywhere in
configuration, and generally, it is used as "section", "daemon",
"type", or "process".
Using any of the ones we currently describe would be good, "who" is too vague.
> config help <key> Describe a configuration option
> config rm <who> <name> Clear a configuration option for one or
> more entities
> config set <who> <name> <value> Cet a configuration option for one or
> more entities
> config show <who> {<key>} Show running configuration
> config show-with-defaults <who> Show running configuration (including
> compiled-in defaults)
> ---
>
> First, let's dump everything:
>
> $ bin/ceph config dump
> WHO MASK LEVEL OPTION VALUE RO
> global advanced mon_pg_warn_min_per_osd 3
> global advanced osd_crush_chooseleaf_type 0
> global advanced osd_pool_default_min_size 1
> global advanced osd_pool_default_size 1
> mds advanced debug_mds 20/20
> mds advanced debug_mgrc 20/20
> mds advanced debug_monc 20/20
> mds advanced debug_ms 1/1
> mds developer mds_debug_auth_pins true
> mds developer mds_debug_frag true
> mds developer mds_debug_scatterstat true
> mds developer mds_debug_subtrees true
> mds developer mds_verify_scatter true
> mgr advanced debug_mgr 20/20
> mgr advanced debug_mon 20/20
> mgr advanced debug_monc 20/20
> mgr advanced debug_ms 1/1
> mon unknown mon_allow_pool_deletes true *
> mon advanced mon_data_avail_crit 1
> mon advanced mon_data_avail_warn 2
> ...
>
> Note the one RO item here is also 'unknown'.. that's because it's not a
> valid config option (there's no 's').. it was injected directly into
> config-key.
Is there a JSON flag for the above output? How does that look?
The plain output here could be really long, and although one can
probably 'grep' at will, would it be overkill to filter or sort
this so that we can easily pick 'unknown' for example?
For plain (not JSON) the output could be structured in at least two
levels that would make it easier on an admin:
<section>
<level>
<option> <value> <permissions>
<unknown>
<option> <value> <permissions>
That way, it is easier to spot unknown items, or if that is not
desirable, then at least it would be easier to go through them
visually
>
> For a single daemon,
>
> gnit:build (wip-config) 02:47 PM $ bin/ceph config get mon.a
> WHO MASK LEVEL OPTION VALUE RO
> mon unknown mon_allow_pool_deletes true *
> mon advanced mon_data_avail_crit 1
> mon advanced mon_data_avail_warn 2
> mon advanced mon_osd_reporter_subtree_level osd
> global advanced mon_pg_warn_min_per_osd 3
> global advanced osd_crush_chooseleaf_type 0
> global advanced osd_pool_default_min_size 1
> global advanced osd_pool_default_size 1
>
> If I try to set something that can't be changed at runtime (say, ms_type),
> you get:
>
> gnit:build (wip-config) 02:47 PM $ bin/ceph config set mon.a ms_type
> simple
> gnit:build (wip-config) 02:48 PM $ bin/ceph config get mon.a
> WHO MASK LEVEL OPTION VALUE RO
> mon unknown mon_allow_pool_deletes true *
> mon advanced mon_data_avail_crit 1
> mon advanced mon_data_avail_warn 2
> mon advanced mon_osd_reporter_subtree_level osd
> global advanced mon_pg_warn_min_per_osd 3
> mon.a advanced ms_type simple *
> global advanced osd_crush_chooseleaf_type 0
> global advanced osd_pool_default_min_size 1
> global advanced osd_pool_default_size 1
>
> You can then see this by checking the *running* config (this is what the
> daemon has reported as it's actually config to the manager):
>
> gnit:build (wip-config) 02:48 PM $ bin/ceph config show mon.a
> NAME VALUE SOURCE OVERRIDES IGNORES
> admin_socket /tmp/ceph-asok.GBZTV0/$name.asok file
> chdir file
> debug_auth 20/20 file
> debug_mgrc 20/20 file
> debug_mon 20/20 file
> debug_ms 1/1 file
> debug_paxos 20/20 file
> enable_experimental_unrecoverable_data_corrupting_features * file
> erasure_code_dir /home/sage/src/ceph6/build/lib file
> filestore_fd_cache_size 32 file
> heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat file
> leveldb_block_size 65536 default
> leveldb_cache_size 536870912 default
> leveldb_compression 0 default
> leveldb_log default
> leveldb_write_buffer_size 33554432 default
> lockdep 1 file
> log_file /home/sage/src/ceph6/build/out/$name.log file
> mgr_initial_modules restful status dashboard balancer file
> mon_cluster_log_file /home/sage/src/ceph6/build/out/cluster.mon.$id.log file
> mon_data /home/sage/src/ceph6/build/dev/mon.a file
> mon_data_avail_crit 1 mon
> mon_data_avail_warn 2 mon
> mon_osd_backfillfull_ratio 0.99 file
> mon_osd_full_ratio 0.99 file
> mon_osd_nearfull_ratio 0.99 file
> mon_osd_reporter_subtree_level osd mon
> mon_pg_warn_min_per_osd 3 mon
> ms_type async+posix default mon
> osd_crush_chooseleaf_type 0 mon
> osd_failsafe_full_ratio 0.99 file
> osd_pool_default_min_size 1 mon
> osd_pool_default_size 1 mon
> pid_file /home/sage/src/ceph6/build/out/mon.a.pid cmdline file
> plugin_dir /home/sage/src/ceph6/build/lib file
> run_dir /home/sage/src/ceph6/build/out file
>
> (sorry for the wide table)
>
> A few things to note: SOURCE can be any of default, file, mon, env,
> cmdline, or override. You'll notice that ms_type SOURCE is 'default' and
> IGNORES is 'mon', because the daemon is ignoring the mon value--this
> option can only be set during startup.
This is really useful
>
> Similarly, if I try to override something else manually,
>
> gnit:build (wip-config) 02:50 PM $ bin/ceph daemon mon.a config set mon_data_avail_warn 3
> {
> "success": ""
> }
Why is this response JSON? If the response was a success I would
expect a "success": true, but if this is coming directly from the
monitor I guess
we don't have a choice to use it as is?
> gnit:build (wip-config) 02:50 PM $ bin/ceph config show mon.a
> NAME VALUE SOURCE OVERRIDES IGNORES
> ...
> mon_data_avail_warn 3 override mon
> ...
>
> we see that the currently value is an override, and OVERRIDES the mon
> value. As far as I can tel this is a pretty complete view of what is
> going on. The JSON form has a bit more info, like what the mon value that
> was override was:
>
> ...
> {
> "name": "mon_data_avail_warn",
> "value": "3",
> "source": "override",
> "overrides": [
> {
> "source": "mon",
> "value": "2"
> }
> ]
> },
> ...
>
> Anyway, moving on...
>
> You can also 'get' and 'show' specific values, making this useful from a
> script:
>
> gnit:build (wip-config) 02:52 PM $ bin/ceph config get mon.a ms_type
> simple
> gnit:build (wip-config) 02:52 PM $ bin/ceph config show mon.a ms_type
> async+posix
>
> This will return valid results for defaults, too:
>
> gnit:build (wip-config) 02:52 PM $ bin/ceph config show mon.a osd_data
> /var/lib/ceph/osd/$cluster-$id
>
> ...although the result has not has substitutions processed yet (should
> it?).
>
> You can get help:
>
> gnit:build (wip-config) 02:54 PM $ bin/ceph config help
> bluestore_cache_size
> bluestore_cache_size - Cache size (in bytes) for BlueStore
> (uint64_t, advanced)
> Default: 0
> Can update at runtime: true
>
> This includes data and metadata cached by BlueStore as well as memory devoted to rocksdb's cache(s).
>
> I mostly hate how the above looks, BTW--suggestions welcome! In JSON
> form,
>
> gnit:build (wip-config) 02:54 PM $ bin/ceph config help bluestore_cache_size -f json-pretty
>
> {
> "name": "bluestore_cache_size",
> "type": "uint64_t",
> "level": "advanced",
> "desc": "Cache size (in bytes) for BlueStore",
> "long_desc": "This includes data and metadata cached by BlueStore as
> well as memory devoted to rocksdb's cache(s).",
> "default": 0,
> "daemon_default": "",
> "tags": [],
> "services": [],
> "see_also": [],
> "min": "",
> "max": "",
> "can_update_at_runtime": true
> }
>
> The 'config set' has basic validation:
>
> gnit:build (wip-config) 03:07 PM $ bin/ceph config set osd.0 foo bar
> Error EINVAL: unrecognized config option 'foo'
> gnit:build (wip-config) 03:07 PM $ bin/ceph config set osd.0 osd_tier_default_cache_mode bar
> Error EINVAL: error parsing value: 'bar' is not one of the permitted values: none, writeback, forward, readonly, readforward, readproxy, proxy
This type of validation is great. It is not only saying it is not
permitted, but it is giving you a list of what is actually OK. Would
love to see this propagate everywhere
> gnit:build (wip-config) 03:08 PM $ bin/ceph config set osd.0
> osd_tier_default_cache_mode writeback
>
> and you can clear an option:
>
> gnit:build (wip-config) 03:08 PM $ bin/ceph config get mon.a
> WHO MASK LEVEL OPTION VALUE RO
> mon unknown mon_allow_pool_deletes true *
> mon advanced mon_data_avail_crit 1
> mon advanced mon_data_avail_warn 2
> mon advanced mon_osd_reporter_subtree_level osd
> global advanced mon_pg_warn_min_per_osd 3
> mon.a advanced ms_type simple *
> global advanced osd_crush_chooseleaf_type 0
> global advanced osd_pool_default_min_size 1
> global advanced osd_pool_default_size 1
> gnit:build (wip-config) 03:08 PM $ bin/ceph config rm mon.a ms_type
> gnit:build (wip-config) 03:09 PM $ bin/ceph config get mon.a
> WHO MASK LEVEL OPTION VALUE RO
> mon unknown mon_allow_pool_deletes true *
> mon advanced mon_data_avail_crit 1
> mon advanced mon_data_avail_warn 2
> mon advanced mon_osd_reporter_subtree_level osd
> global advanced mon_pg_warn_min_per_osd 3
> global advanced osd_crush_chooseleaf_type 0
> global advanced osd_pool_default_min_size 1
> global advanced osd_pool_default_size 1
>
>
> The last one is 'assimilate-conf' and is meant to help transition from
> a ceph.conf world to a mon config world. This is a vstart cluster, so all
> of the paths are all wonky and there's lots of specialness that you
> wouldn't see in the real world. It still imports most stuff:
>
> gnit:build (wip-config) 03:09 PM $ wc -l ceph.conf
> 115 ceph.conf
> gnit:build (wip-config) 03:09 PM $ bin/ceph config assimilate-conf -i ceph.conf -o new.conf
> 2018-01-11 15:09:55.390 7f5a3cff9700 -1 set_mon_vals failed to set admin_socket = /tmp/ceph-asok.GBZTV0/$name.$pid.asok: Configuration option 'admin_socket' may not be modified at runtime
>
> (That warning appears because the CLI process observed the config change
> as it happened, and isn't able to update the admin_socket setting at
> runtime.)
>
> What's left is stuff flagged NO_MON_CONFIG or unrecognized as a valid
> option:
>
> gnit:build (wip-config) 03:09 PM $ wc -l new.conf
> 46 new.conf
> gnit:build (wip-config) 03:10 PM $ cat new.conf
>
> [client]
> keyring = /home/sage/src/ceph6/build/keyring
>
> [client.vstart.sh]
> num_mds = 0
> num_mgr = 1
> num_mon = 1
> num_osd = 1
> num_rgw = 0
>
> [global]
> fsid = 9dbd93f7-635a-4734-bc29-448a2305af7f
> lockdep = true
> mon_osd_backfillfull_ratio = .99
> mon_osd_full_ratio = .99
> mon_osd_nearfull_ratio = .99
>
> [mds]
> chdir =
> mds_data = /home/sage/src/ceph6/build/dev/mds.$id
>
> [mgr]
> chdir =
> mgr_data = /home/sage/src/ceph6/build/dev/mgr.$id
>
> [mgr.x]
> host = gnit
>
> [mon]
> chdir =
> mgr_initial_modules = restful status dashboard balancer
>
> [mon.a]
> host = gnit
> mon_addr = 127.0.0.1:40948
> mon_data = /home/sage/src/ceph6/build/dev/mon.a
>
> [osd]
> chdir =
> osd_class_tmp = out
> osd_data = /home/sage/src/ceph6/build/dev/osd$id
> osd_journal = /home/sage/src/ceph6/build/dev/osd$id/journal
>
> [osd.0]
> host = gnit
>
> This command isn't as smart as it maybe could be. It doesn't try to do
> anything clever for ceph.conf's with conflicting values from different
> hosts... so if host A has debug_ms=1 and host B has debug_ms=2, the mon
> will keep whichever was imported last. In general, though, if the config
> options are restricted to [type.id] per-daemon sections then things will
> work just fine.
>
> We *could* have tools try to look at the result in the mon and simplify or
> rearrange, by (for example) moving osd-specific options under [osd] so
> that they don't pollute other daemons' configs. I leave this to future
> work...
>
> Anyway, please let me know if there is anything ugly or unintuitive or
> wrong here. If this looks sane I'll finally write up the docs!
>
> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mon config commands
2018-01-12 18:21 ` Alfredo Deza
@ 2018-01-12 18:57 ` Sage Weil
2018-01-15 12:37 ` Alfredo Deza
0 siblings, 1 reply; 8+ messages in thread
From: Sage Weil @ 2018-01-12 18:57 UTC (permalink / raw)
To: Alfredo Deza; +Cc: ceph-devel
On Fri, 12 Jan 2018, Alfredo Deza wrote:
> On Thu, Jan 11, 2018 at 4:21 PM, Sage Weil <sage@redhat.com> wrote:
> > Hi everyone,
> >
> > Here's the current state of the mon config commands for mimic. These are
> > pretty close to being "done" so please review carefully!
> >
> > config assimilate-conf Assimilate options from a conf, and
> > return a new, minimal conf file
>
> I think this would still be fine as `config assimilate`. The `-conf`
> reads a bit redundant, and it wouldn't be terribly obscure to leave it
> as plain `assimilate`
Works for me :)
> > config dump Show all configuration option(s)
> > config get <who> {<key>} Show configuration option(s) for an
> > entity
>
> Is this output from help? Because "who" is not used anywhere in
> configuration, and generally, it is used as "section", "daemon",
> "type", or "process".
>
> Using any of the ones we currently describe would be good, "who" is too
> vague.
It can be any of the above, which is why it's vague... it might be
"global", it might be a daemon type, it might be a daemon name, and it
might be a combination of any of those and a mask (or masks) (e.g.,
"osd/rack:foo/class:ssd").
> > config help <key> Describe a configuration option
> > config rm <who> <name> Clear a configuration option for one or
> > more entities
> > config set <who> <name> <value> Cet a configuration option for one or
> > more entities
> > config show <who> {<key>} Show running configuration
> > config show-with-defaults <who> Show running configuration (including
> > compiled-in defaults)
> > ---
> >
> > First, let's dump everything:
> >
> > $ bin/ceph config dump
> > WHO MASK LEVEL OPTION VALUE RO
> > global advanced mon_pg_warn_min_per_osd 3
> > global advanced osd_crush_chooseleaf_type 0
> > global advanced osd_pool_default_min_size 1
> > global advanced osd_pool_default_size 1
> > mds advanced debug_mds 20/20
> > mds advanced debug_mgrc 20/20
> > mds advanced debug_monc 20/20
> > mds advanced debug_ms 1/1
> > mds developer mds_debug_auth_pins true
> > mds developer mds_debug_frag true
> > mds developer mds_debug_scatterstat true
> > mds developer mds_debug_subtrees true
> > mds developer mds_verify_scatter true
> > mgr advanced debug_mgr 20/20
> > mgr advanced debug_mon 20/20
> > mgr advanced debug_monc 20/20
> > mgr advanced debug_ms 1/1
> > mon unknown mon_allow_pool_deletes true *
> > mon advanced mon_data_avail_crit 1
> > mon advanced mon_data_avail_warn 2
> > ...
> >
> > Note the one RO item here is also 'unknown'.. that's because it's not a
> > valid config option (there's no 's').. it was injected directly into
> > config-key.
>
> Is there a JSON flag for the above output? How does that look?
[
{
"section": "global",
"name": "enable_experimental_unrecoverable_data_corrupting_features",
"value": "*"
},
{
"section": "global",
"name": "erasure_code_dir",
"value": "/home/sage/src/ceph6/build/lib"
},
Hmm, here I call it "section," which is probably not ideal. In the mask
case, though, it's something like
{
"section": "osd.1",
"name": "debug_osd",
"value": "11/11",
"location_type": "rack",
"location_value": "foo",
"device_class": "ssd"
}
> The plain output here could be really long, and although one can
> probably 'grep' at will, would it be overkill to filter or sort
> this so that we can easily pick 'unknown' for example?
>
> For plain (not JSON) the output could be structured in at least two
> levels that would make it easier on an admin:
>
> <section>
> <level>
> <option> <value> <permissions>
> <unknown>
> <option> <value> <permissions>
>
> That way, it is easier to spot unknown items, or if that is not
> desirable, then at least it would be easier to go through them
> visually
We talked about doing indentation before but I wasn't sure I liked it.
Here's the output:
WHO MASK LEVEL OPTION VALUE RO
global advanced enable_experimental_unrecoverable_data_corrupting_features * *
global advanced erasure_code_dir /home/sage/src/ceph6/build/lib *
global developer filestore_fd_cache_size 32
global advanced mon_pg_warn_min_per_osd 3
global advanced osd_crush_chooseleaf_type 0
global advanced osd_failsafe_full_ratio .99
global advanced osd_pool_default_min_size 1
global advanced osd_pool_default_size 1
global advanced plugin_dir /home/sage/src/ceph6/build/lib *
global advanced run_dir /home/sage/src/ceph6/build/out *
client advanced admin_socket /tmp/ceph-asok.GBZTV0/$name.$pid.asok *
client basic log_file /home/sage/src/ceph6/build/out/$name.$pid.log *
client.rgw advanced rgw_crypt_require_ssl false
client.rgw developer rgw_crypt_s3_kms_encryption_keys testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= *
client.rgw basic rgw_frontends civetweb port=8000 *
client.rgw developer rgw_lc_debug_interval 10
mds advanced admin_socket /tmp/ceph-asok.GBZTV0/$name.asok *
mds advanced debug_mds 20/20
mds advanced debug_mgrc 20/20
mds advanced debug_monc 20/20
mds advanced debug_ms 1/1
mds advanced heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat *
mds basic log_file /home/sage/src/ceph6/build/out/$name.log *
mds developer mds_debug_auth_pins true
mds developer mds_debug_frag true
mds developer mds_debug_scatterstat true
mds developer mds_debug_subtrees true
mds advanced mds_root_ino_gid 1031
mds advanced mds_root_ino_uid 1031
mds developer mds_verify_scatter true
mds advanced pid_file /home/sage/src/ceph6/build/out/$name.pid *
mgr advanced admin_socket /tmp/ceph-asok.GBZTV0/$name.asok *
mgr advanced debug_mgr 20/20
mgr advanced debug_mon 20/20
mgr advanced debug_monc 20/20
mgr advanced debug_ms 1/1
mgr advanced heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat *
mgr basic log_file /home/sage/src/ceph6/build/out/$name.log *
mgr advanced mgr_module_path /home/sage/src/ceph6/src/pybind/mgr *
mgr advanced pid_file /home/sage/src/ceph6/build/out/$name.pid *
mgr.x advanced mgr_stats_threshold -2
mon advanced admin_socket /tmp/ceph-asok.GBZTV0/$name.asok *
mon advanced debug_auth 20/20
mon advanced debug_mgrc 20/20
mon advanced debug_mon 20/20
mon advanced debug_ms 1/1
mon advanced debug_paxos 20/20
mon advanced heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat *
mon basic log_file /home/sage/src/ceph6/build/out/$name.log *
mon unknown mon_allow_pool_deletes true *
mon advanced mon_cluster_log_file /home/sage/src/ceph6/build/out/cluster.mon.$id.log *
mon advanced mon_data_avail_crit 1
mon advanced mon_data_avail_warn 2
mon advanced mon_osd_reporter_subtree_level osd
mon advanced pid_file /home/sage/src/ceph6/build/out/$name.pid *
osd advanced admin_socket /tmp/ceph-asok.GBZTV0/$name.asok *
osd developer bluestore_block_create true *
osd developer bluestore_block_db_create true *
osd developer bluestore_block_db_path /home/sage/src/ceph6/build/dev/osd$id/block.db.file *
osd developer bluestore_block_db_size 67108864 *
osd developer bluestore_block_wal_create true *
osd developer bluestore_block_wal_path /home/sage/src/ceph6/build/dev/osd$id/block.wal.file *
osd developer bluestore_block_wal_size 1048576000 *
osd developer bluestore_fsck_on_mount true
osd advanced debug_bdev 20/20
osd advanced debug_bluefs 20/20
osd advanced debug_bluestore 30/30
osd advanced debug_filestore 20/20
osd advanced debug_journal 20/20
osd advanced debug_mgrc 20/20
osd advanced debug_monc 20/20
osd advanced debug_ms 1/1
osd advanced debug_objclass 20/20
osd advanced debug_objecter 20/20
osd advanced debug_osd 25/25
osd advanced debug_reserver 10/10
osd advanced debug_rocksdb 20/20
osd advanced filestore_wbthrottle_btrfs_inodes_hard_limit 30
osd advanced filestore_wbthrottle_btrfs_ios_hard_limit 20
osd advanced filestore_wbthrottle_btrfs_ios_start_flusher 10
osd advanced filestore_wbthrottle_xfs_inodes_hard_limit 30
osd advanced filestore_wbthrottle_xfs_ios_hard_limit 20
osd advanced filestore_wbthrottle_xfs_ios_start_flusher 10
osd advanced heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat *
osd basic log_file /home/sage/src/ceph6/build/out/$name.log *
osd developer osd_check_max_object_name_len_on_startup false
osd advanced osd_class_default_list * *
osd advanced osd_class_dir /home/sage/src/ceph6/build/lib *
osd advanced osd_class_load_list * *
osd advanced osd_copyfrom_max_chunk 524288
osd developer osd_debug_misdirected_ops true
osd developer osd_debug_op_order true
osd advanced osd_journal_size 100
osd advanced osd_objectstore bluestore *
osd advanced osd_scrub_load_threshold 2000
osd advanced pid_file /home/sage/src/ceph6/build/out/$name.pid *
osd.0 advanced osd_tier_default_cache_mode writeback
I'm not sure if your suggestion is to sort by (who,level,option) (which we
could do with the above without too much effort) or to have extra lines in
the output so you'd see something along the lines of
global
advanced
debug_ms 1
ms_type simple
osd
basic
cluster_network 10.0.0.0/8
advanced
debug_osd 10
?
I'm inclined to stick with a table format (both because we do that
elsewhere and we have the handy TextTable class to make it easy). With or
without indentation... what do you think?
> > Similarly, if I try to override something else manually,
> >
> > gnit:build (wip-config) 02:50 PM $ bin/ceph daemon mon.a config set mon_data_avail_warn 3
> > {
> > "success": ""
> > }
>
> Why is this response JSON? If the response was a success I would
> expect a "success": true, but if this is coming directly from the
> monitor I guess
> we don't have a choice to use it as is?
Good question, it's old code I haven't touched. I'll clean it up.
Thanks for the feedback!
sage
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mon config commands
2018-01-11 21:21 mon config commands Sage Weil
2018-01-12 18:21 ` Alfredo Deza
@ 2018-01-12 23:17 ` Gregory Farnum
2018-01-12 23:56 ` Sage Weil
2018-01-16 16:14 ` Douglas Fuller
2 siblings, 1 reply; 8+ messages in thread
From: Gregory Farnum @ 2018-01-12 23:17 UTC (permalink / raw)
To: Sage Weil; +Cc: ceph-devel
The rest looks good, but I don't understand this thing about "RO"
values at all. Based on what's here, it represents options that aren't
live yet, but will be on reboot?
On Thu, Jan 11, 2018 at 1:21 PM, Sage Weil <sage@redhat.com> wrote:
> If I try to set something that can't be changed at runtime (say, ms_type),
> you get:
>
> gnit:build (wip-config) 02:47 PM $ bin/ceph config set mon.a ms_type
> simple
> gnit:build (wip-config) 02:48 PM $ bin/ceph config get mon.a
> WHO MASK LEVEL OPTION VALUE RO
> mon unknown mon_allow_pool_deletes true *
> mon advanced mon_data_avail_crit 1
> mon advanced mon_data_avail_warn 2
> mon advanced mon_osd_reporter_subtree_level osd
> global advanced mon_pg_warn_min_per_osd 3
> mon.a advanced ms_type simple *
> global advanced osd_crush_chooseleaf_type 0
> global advanced osd_pool_default_min_size 1
> global advanced osd_pool_default_size 1
>
> You can then see this by checking the *running* config (this is what the
> daemon has reported as it's actually config to the manager):
>
> gnit:build (wip-config) 02:48 PM $ bin/ceph config show mon.a
> NAME VALUE SOURCE OVERRIDES IGNORES
> admin_socket /tmp/ceph-asok.GBZTV0/$name.asok file
> chdir file
> debug_auth 20/20 file
> debug_mgrc 20/20 file
> debug_mon 20/20 file
> debug_ms 1/1 file
> debug_paxos 20/20 file
> enable_experimental_unrecoverable_data_corrupting_features * file
> erasure_code_dir /home/sage/src/ceph6/build/lib file
> filestore_fd_cache_size 32 file
> heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat file
> leveldb_block_size 65536 default
> leveldb_cache_size 536870912 default
> leveldb_compression 0 default
> leveldb_log default
> leveldb_write_buffer_size 33554432 default
> lockdep 1 file
> log_file /home/sage/src/ceph6/build/out/$name.log file
> mgr_initial_modules restful status dashboard balancer file
> mon_cluster_log_file /home/sage/src/ceph6/build/out/cluster.mon.$id.log file
> mon_data /home/sage/src/ceph6/build/dev/mon.a file
> mon_data_avail_crit 1 mon
> mon_data_avail_warn 2 mon
> mon_osd_backfillfull_ratio 0.99 file
> mon_osd_full_ratio 0.99 file
> mon_osd_nearfull_ratio 0.99 file
> mon_osd_reporter_subtree_level osd mon
> mon_pg_warn_min_per_osd 3 mon
> ms_type async+posix default mon
> osd_crush_chooseleaf_type 0 mon
> osd_failsafe_full_ratio 0.99 file
> osd_pool_default_min_size 1 mon
> osd_pool_default_size 1 mon
> pid_file /home/sage/src/ceph6/build/out/mon.a.pid cmdline file
> plugin_dir /home/sage/src/ceph6/build/lib file
> run_dir /home/sage/src/ceph6/build/out file
>
> (sorry for the wide table)
>
> A few things to note: SOURCE can be any of default, file, mon, env,
> cmdline, or override. You'll notice that ms_type SOURCE is 'default' and
> IGNORES is 'mon', because the daemon is ignoring the mon value--this
> option can only be set during startup.
I also don't get how one mon having a different setting means that all
mons are marked as ignoring it.
Putting them right next to each other also makes it clear in my mind
that people are going to be confused about "config get" versus "config
show". Perhaps it should be "get_stored" and "show_running" or
something? I don't like those names, but they need to be clearer. And
now I've got a new feature request: a command that outputs the
difference between those two. Or perhaps that goes through *all* the
configs and dumps any daemons whose running config differs from their
stored ones.
On Fri, Jan 12, 2018 at 10:57 AM, Sage Weil <sweil@redhat.com> wrote:
> On Fri, 12 Jan 2018, Alfredo Deza wrote:
>> On Thu, Jan 11, 2018 at 4:21 PM, Sage Weil <sage@redhat.com> wrote:
>> > Hi everyone,
>> >
>> > Here's the current state of the mon config commands for mimic. These are
>> > pretty close to being "done" so please review carefully!
>> >
>> > config assimilate-conf Assimilate options from a conf, and
>> > return a new, minimal conf file
>>
>> I think this would still be fine as `config assimilate`. The `-conf`
>> reads a bit redundant, and it wouldn't be terribly obscure to leave it
>> as plain `assimilate`
>
> Works for me :)
Count me as a vote for "assimilate-conf" over "assimilate".
Specificity is easier to search for help on, and easier to understand.
-Greg
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mon config commands
2018-01-12 23:17 ` Gregory Farnum
@ 2018-01-12 23:56 ` Sage Weil
2018-01-13 0:05 ` Vasu Kulkarni
0 siblings, 1 reply; 8+ messages in thread
From: Sage Weil @ 2018-01-12 23:56 UTC (permalink / raw)
To: Gregory Farnum; +Cc: ceph-devel
On Fri, 12 Jan 2018, Gregory Farnum wrote:
> The rest looks good, but I don't understand this thing about "RO"
> values at all. Based on what's here, it represents options that aren't
> live yet, but will be on reboot?
These are options that can't change at runtime, so they need a daemon
restart. (Or, in some cases, they only matter at osd or cluster creation
time.) In reality, we have an enum of (cluster_create, daemon_create,
startup, runtime) for each option for when it can take effect.
Generally the question for the user, though, is "can i set this value
now and have it take effect". If it's flagged as RO, then they can set it
via the mon, but then 'config show' will then show 'mon' under IGNORES
because it's something we can't change at runtime.
> On Thu, Jan 11, 2018 at 1:21 PM, Sage Weil <sage@redhat.com> wrote:
> > gnit:build (wip-config) 02:48 PM $ bin/ceph config show mon.a
> > NAME VALUE SOURCE OVERRIDES IGNORES
...
> > ms_type async+posix default mon
...
> >
> > (sorry for the wide table)
> >
> > A few things to note: SOURCE can be any of default, file, mon, env,
> > cmdline, or override. You'll notice that ms_type SOURCE is 'default' and
> > IGNORES is 'mon', because the daemon is ignoring the mon value--this
> > option can only be set during startup.
>
> I also don't get how one mon having a different setting means that all
> mons are marked as ignoring it.
This is saying that 'mon.a' (who we just 'show'ed) is ignoring the
mon-managed config. If mon.b was restarted and we did a show it would...
well it would also say the same thing since mon's don't fetch the
mon-stored config before startup. (Not sure if we should implement that
or not.)
Is that what you're asking?
> Putting them right next to each other also makes it clear in my mind
> that people are going to be confused about "config get" versus "config
> show". Perhaps it should be "get_stored" and "show_running" or
> something? I don't like those names, but they need to be clearer.
I don't really like 'show' either. 'show-running'? 'running'? I like
'get' though.
> And now I've got a new feature request: a command that outputs the
> difference between those two. Or perhaps that goes through *all* the
> configs and dumps any daemons whose running config differs from their
> stored ones.
'show' already shows you this information via the OVERRIDES and IGNORES
columns.
Oh, you mean that shows *only* the mismatch for the daemon or
cluster...
sage
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mon config commands
2018-01-12 23:56 ` Sage Weil
@ 2018-01-13 0:05 ` Vasu Kulkarni
0 siblings, 0 replies; 8+ messages in thread
From: Vasu Kulkarni @ 2018-01-13 0:05 UTC (permalink / raw)
To: Sage Weil; +Cc: Gregory Farnum, ceph-devel
On Fri, Jan 12, 2018 at 3:56 PM, Sage Weil <sweil@redhat.com> wrote:
> On Fri, 12 Jan 2018, Gregory Farnum wrote:
>> The rest looks good, but I don't understand this thing about "RO"
>> values at all. Based on what's here, it represents options that aren't
>> live yet, but will be on reboot?
>
> These are options that can't change at runtime, so they need a daemon
> restart. (Or, in some cases, they only matter at osd or cluster creation
> time.) In reality, we have an enum of (cluster_create, daemon_create,
> startup, runtime) for each option for when it can take effect.
> Generally the question for the user, though, is "can i set this value
> now and have it take effect". If it's flagged as RO, then they can set it
> via the mon, but then 'config show' will then show 'mon' under IGNORES
> because it's something we can't change at runtime.
It would be nice to throw Error during set command, because it would
let admin/user know it didn't work,
A silent return in my view is still confusing and not many would
understand to rerun show and check
mon.a advanced ms_type simple *
>
>> On Thu, Jan 11, 2018 at 1:21 PM, Sage Weil <sage@redhat.com> wrote:
>> > gnit:build (wip-config) 02:48 PM $ bin/ceph config show mon.a
>> > NAME VALUE SOURCE OVERRIDES IGNORES
> ...
>> > ms_type async+posix default mon
> ...
>> >
>> > (sorry for the wide table)
>> >
>> > A few things to note: SOURCE can be any of default, file, mon, env,
>> > cmdline, or override. You'll notice that ms_type SOURCE is 'default' and
>> > IGNORES is 'mon', because the daemon is ignoring the mon value--this
>> > option can only be set during startup.
>>
>> I also don't get how one mon having a different setting means that all
>> mons are marked as ignoring it.
>
> This is saying that 'mon.a' (who we just 'show'ed) is ignoring the
> mon-managed config. If mon.b was restarted and we did a show it would...
> well it would also say the same thing since mon's don't fetch the
> mon-stored config before startup. (Not sure if we should implement that
> or not.)
>
> Is that what you're asking?
>
>> Putting them right next to each other also makes it clear in my mind
>> that people are going to be confused about "config get" versus "config
>> show". Perhaps it should be "get_stored" and "show_running" or
>> something? I don't like those names, but they need to be clearer.
>
> I don't really like 'show' either. 'show-running'? 'running'? I like
> 'get' though.
>
>> And now I've got a new feature request: a command that outputs the
>> difference between those two. Or perhaps that goes through *all* the
>> configs and dumps any daemons whose running config differs from their
>> stored ones.
>
> 'show' already shows you this information via the OVERRIDES and IGNORES
> columns.
>
> Oh, you mean that shows *only* the mismatch for the daemon or
> cluster...
>
> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mon config commands
2018-01-12 18:57 ` Sage Weil
@ 2018-01-15 12:37 ` Alfredo Deza
0 siblings, 0 replies; 8+ messages in thread
From: Alfredo Deza @ 2018-01-15 12:37 UTC (permalink / raw)
To: Sage Weil; +Cc: ceph-devel
On Fri, Jan 12, 2018 at 1:57 PM, Sage Weil <sweil@redhat.com> wrote:
> On Fri, 12 Jan 2018, Alfredo Deza wrote:
>> On Thu, Jan 11, 2018 at 4:21 PM, Sage Weil <sage@redhat.com> wrote:
>> > Hi everyone,
>> >
>> > Here's the current state of the mon config commands for mimic. These are
>> > pretty close to being "done" so please review carefully!
>> >
>> > config assimilate-conf Assimilate options from a conf, and
>> > return a new, minimal conf file
>>
>> I think this would still be fine as `config assimilate`. The `-conf`
>> reads a bit redundant, and it wouldn't be terribly obscure to leave it
>> as plain `assimilate`
>
> Works for me :)
>
>> > config dump Show all configuration option(s)
>> > config get <who> {<key>} Show configuration option(s) for an
>> > entity
>>
>> Is this output from help? Because "who" is not used anywhere in
>> configuration, and generally, it is used as "section", "daemon",
>> "type", or "process".
>>
>> Using any of the ones we currently describe would be good, "who" is too
>> vague.
>
> It can be any of the above, which is why it's vague... it might be
> "global", it might be a daemon type, it might be a daemon name, and it
> might be a combination of any of those and a mask (or masks) (e.g.,
> "osd/rack:foo/class:ssd").
I see a couple of options here.
1. If we insist on using 'who', then our docs need to be updated to
use this new terminology. There are lots of places where the vagueness
is present and we would need to use this so that
a user can correlate the terminology.
2. Use some other, preexisting term like 'type', which is already
present in multiple places, and is still suitable for
defining a process, daermon, or group
3. Not use it at all, and instead make that item the heading for each
group in the output (just like the JSON output doesn't
have a 'who' key)
>
>> > config help <key> Describe a configuration option
>> > config rm <who> <name> Clear a configuration option for one or
>> > more entities
>> > config set <who> <name> <value> Cet a configuration option for one or
>> > more entities
>> > config show <who> {<key>} Show running configuration
>> > config show-with-defaults <who> Show running configuration (including
>> > compiled-in defaults)
>> > ---
>> >
>> > First, let's dump everything:
>> >
>> > $ bin/ceph config dump
>> > WHO MASK LEVEL OPTION VALUE RO
>> > global advanced mon_pg_warn_min_per_osd 3
>> > global advanced osd_crush_chooseleaf_type 0
>> > global advanced osd_pool_default_min_size 1
>> > global advanced osd_pool_default_size 1
>> > mds advanced debug_mds 20/20
>> > mds advanced debug_mgrc 20/20
>> > mds advanced debug_monc 20/20
>> > mds advanced debug_ms 1/1
>> > mds developer mds_debug_auth_pins true
>> > mds developer mds_debug_frag true
>> > mds developer mds_debug_scatterstat true
>> > mds developer mds_debug_subtrees true
>> > mds developer mds_verify_scatter true
>> > mgr advanced debug_mgr 20/20
>> > mgr advanced debug_mon 20/20
>> > mgr advanced debug_monc 20/20
>> > mgr advanced debug_ms 1/1
>> > mon unknown mon_allow_pool_deletes true *
>> > mon advanced mon_data_avail_crit 1
>> > mon advanced mon_data_avail_warn 2
>> > ...
>> >
>> > Note the one RO item here is also 'unknown'.. that's because it's not a
>> > valid config option (there's no 's').. it was injected directly into
>> > config-key.
>>
>> Is there a JSON flag for the above output? How does that look?
>
> [
> {
> "section": "global",
> "name": "enable_experimental_unrecoverable_data_corrupting_features",
> "value": "*"
> },
> {
> "section": "global",
> "name": "erasure_code_dir",
> "value": "/home/sage/src/ceph6/build/lib"
> },
>
> Hmm, here I call it "section," which is probably not ideal. In the mask
> case, though, it's something like
>
> {
> "section": "osd.1",
> "name": "debug_osd",
> "value": "11/11",
> "location_type": "rack",
> "location_value": "foo",
> "device_class": "ssd"
> }
This is exactly what I am going for. It is going to be tricky to get
'WHO' to stick everywhere and make sense.
On the consumer side, a client would need to understand that human
readable output with WHO might map to 'section' , and if
something else decides this is not a 'section' but a 'daemon', then
'WHO' becomes less meaningful and more confusing.
>
>> The plain output here could be really long, and although one can
>> probably 'grep' at will, would it be overkill to filter or sort
>> this so that we can easily pick 'unknown' for example?
>>
>> For plain (not JSON) the output could be structured in at least two
>> levels that would make it easier on an admin:
>>
>> <section>
>> <level>
>> <option> <value> <permissions>
>> <unknown>
>> <option> <value> <permissions>
>>
>> That way, it is easier to spot unknown items, or if that is not
>> desirable, then at least it would be easier to go through them
>> visually
>
> We talked about doing indentation before but I wasn't sure I liked it.
> Here's the output:
>
> WHO MASK LEVEL OPTION VALUE RO
> global advanced enable_experimental_unrecoverable_data_corrupting_features * *
> global advanced erasure_code_dir /home/sage/src/ceph6/build/lib *
> global developer filestore_fd_cache_size 32
> global advanced mon_pg_warn_min_per_osd 3
> global advanced osd_crush_chooseleaf_type 0
> global advanced osd_failsafe_full_ratio .99
> global advanced osd_pool_default_min_size 1
> global advanced osd_pool_default_size 1
> global advanced plugin_dir /home/sage/src/ceph6/build/lib *
> global advanced run_dir /home/sage/src/ceph6/build/out *
> client advanced admin_socket /tmp/ceph-asok.GBZTV0/$name.$pid.asok *
> client basic log_file /home/sage/src/ceph6/build/out/$name.$pid.log *
> client.rgw advanced rgw_crypt_require_ssl false
> client.rgw developer rgw_crypt_s3_kms_encryption_keys testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= *
> client.rgw basic rgw_frontends civetweb port=8000 *
> client.rgw developer rgw_lc_debug_interval 10
> mds advanced admin_socket /tmp/ceph-asok.GBZTV0/$name.asok *
> mds advanced debug_mds 20/20
> mds advanced debug_mgrc 20/20
> mds advanced debug_monc 20/20
> mds advanced debug_ms 1/1
> mds advanced heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat *
> mds basic log_file /home/sage/src/ceph6/build/out/$name.log *
> mds developer mds_debug_auth_pins true
> mds developer mds_debug_frag true
> mds developer mds_debug_scatterstat true
> mds developer mds_debug_subtrees true
> mds advanced mds_root_ino_gid 1031
> mds advanced mds_root_ino_uid 1031
> mds developer mds_verify_scatter true
> mds advanced pid_file /home/sage/src/ceph6/build/out/$name.pid *
> mgr advanced admin_socket /tmp/ceph-asok.GBZTV0/$name.asok *
> mgr advanced debug_mgr 20/20
> mgr advanced debug_mon 20/20
> mgr advanced debug_monc 20/20
> mgr advanced debug_ms 1/1
> mgr advanced heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat *
> mgr basic log_file /home/sage/src/ceph6/build/out/$name.log *
> mgr advanced mgr_module_path /home/sage/src/ceph6/src/pybind/mgr *
> mgr advanced pid_file /home/sage/src/ceph6/build/out/$name.pid *
> mgr.x advanced mgr_stats_threshold -2
> mon advanced admin_socket /tmp/ceph-asok.GBZTV0/$name.asok *
> mon advanced debug_auth 20/20
> mon advanced debug_mgrc 20/20
> mon advanced debug_mon 20/20
> mon advanced debug_ms 1/1
> mon advanced debug_paxos 20/20
> mon advanced heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat *
> mon basic log_file /home/sage/src/ceph6/build/out/$name.log *
> mon unknown mon_allow_pool_deletes true *
> mon advanced mon_cluster_log_file /home/sage/src/ceph6/build/out/cluster.mon.$id.log *
> mon advanced mon_data_avail_crit 1
> mon advanced mon_data_avail_warn 2
> mon advanced mon_osd_reporter_subtree_level osd
> mon advanced pid_file /home/sage/src/ceph6/build/out/$name.pid *
> osd advanced admin_socket /tmp/ceph-asok.GBZTV0/$name.asok *
> osd developer bluestore_block_create true *
> osd developer bluestore_block_db_create true *
> osd developer bluestore_block_db_path /home/sage/src/ceph6/build/dev/osd$id/block.db.file *
> osd developer bluestore_block_db_size 67108864 *
> osd developer bluestore_block_wal_create true *
> osd developer bluestore_block_wal_path /home/sage/src/ceph6/build/dev/osd$id/block.wal.file *
> osd developer bluestore_block_wal_size 1048576000 *
> osd developer bluestore_fsck_on_mount true
> osd advanced debug_bdev 20/20
> osd advanced debug_bluefs 20/20
> osd advanced debug_bluestore 30/30
> osd advanced debug_filestore 20/20
> osd advanced debug_journal 20/20
> osd advanced debug_mgrc 20/20
> osd advanced debug_monc 20/20
> osd advanced debug_ms 1/1
> osd advanced debug_objclass 20/20
> osd advanced debug_objecter 20/20
> osd advanced debug_osd 25/25
> osd advanced debug_reserver 10/10
> osd advanced debug_rocksdb 20/20
> osd advanced filestore_wbthrottle_btrfs_inodes_hard_limit 30
> osd advanced filestore_wbthrottle_btrfs_ios_hard_limit 20
> osd advanced filestore_wbthrottle_btrfs_ios_start_flusher 10
> osd advanced filestore_wbthrottle_xfs_inodes_hard_limit 30
> osd advanced filestore_wbthrottle_xfs_ios_hard_limit 20
> osd advanced filestore_wbthrottle_xfs_ios_start_flusher 10
> osd advanced heartbeat_file /home/sage/src/ceph6/build/out/$name.heartbeat *
> osd basic log_file /home/sage/src/ceph6/build/out/$name.log *
> osd developer osd_check_max_object_name_len_on_startup false
> osd advanced osd_class_default_list * *
> osd advanced osd_class_dir /home/sage/src/ceph6/build/lib *
> osd advanced osd_class_load_list * *
> osd advanced osd_copyfrom_max_chunk 524288
> osd developer osd_debug_misdirected_ops true
> osd developer osd_debug_op_order true
> osd advanced osd_journal_size 100
> osd advanced osd_objectstore bluestore *
> osd advanced osd_scrub_load_threshold 2000
> osd advanced pid_file /home/sage/src/ceph6/build/out/$name.pid *
> osd.0 advanced osd_tier_default_cache_mode writeback
>
> I'm not sure if your suggestion is to sort by (who,level,option) (which we
> could do with the above without too much effort) or to have extra lines in
> the output so you'd see something along the lines of
Yes to both, they are separate requests. Regardless of output format,
would be nice to say 'give me anything that is RO',
or 'show only advanced and global'.
>
> global
> advanced
> debug_ms 1
> ms_type simple
> osd
> basic
> cluster_network 10.0.0.0/8
> advanced
> debug_osd 10
>
> ?
Yes, by indenting, it makes it easier on the eyes when there is more
than a dozen options with different values. One plus here is that you
could
get rid of the 'WHO' labeling
>
> I'm inclined to stick with a table format (both because we do that
> elsewhere and we have the handy TextTable class to make it easy). With or
> without indentation... what do you think?
The table is fine for short output, it is not going to be nicer on the
user side of things when it is large, and I can see how this can
become
very long. The INI-style ceph.conf files have that indentation, and
when dealing with large configuration files it definitely helps.
I understand though if this would be too much to rework
>
>> > Similarly, if I try to override something else manually,
>> >
>> > gnit:build (wip-config) 02:50 PM $ bin/ceph daemon mon.a config set mon_data_avail_warn 3
>> > {
>> > "success": ""
>> > }
>>
>> Why is this response JSON? If the response was a success I would
>> expect a "success": true, but if this is coming directly from the
>> monitor I guess
>> we don't have a choice to use it as is?
>
> Good question, it's old code I haven't touched. I'll clean it up.
>
>
> Thanks for the feedback!
> sage
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mon config commands
2018-01-11 21:21 mon config commands Sage Weil
2018-01-12 18:21 ` Alfredo Deza
2018-01-12 23:17 ` Gregory Farnum
@ 2018-01-16 16:14 ` Douglas Fuller
2 siblings, 0 replies; 8+ messages in thread
From: Douglas Fuller @ 2018-01-16 16:14 UTC (permalink / raw)
To: Sage Weil; +Cc: ceph-devel
> On Jan 11, 2018, at 2:21 PM, Sage Weil <sage@redhat.com> wrote:
>
> Hi everyone,
>
> Here's the current state of the mon config commands for mimic. These are
> pretty close to being "done" so please review carefully!
Hi Sage,
Could we have a ‘ceph config search’ to do a substring search for a parameter name? Or could we have ‘ceph config show’ dump all substring matches if the argument isn’t an exact match for a parameter?
Cheers,
—Doug
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2018-01-16 16:14 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-11 21:21 mon config commands Sage Weil
2018-01-12 18:21 ` Alfredo Deza
2018-01-12 18:57 ` Sage Weil
2018-01-15 12:37 ` Alfredo Deza
2018-01-12 23:17 ` Gregory Farnum
2018-01-12 23:56 ` Sage Weil
2018-01-13 0:05 ` Vasu Kulkarni
2018-01-16 16:14 ` Douglas Fuller
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.