Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[matthew patton]

LVM and thin in particular is not for noobies or novices. If they get burned then they deserve it for using a technology they didn't bother to study and learn about beforehand. I think the warnings are gratuitous. Just like some of the other output that LVM emits like "create succeeded". Create succeeded is indicated by a 0 return code. Anything else is just noise. If the user wants noise, then need to use the '-v|--verbose' flag. 

I don't think it's a bad idea to have 'LVM_SUPPRESS_WARNINGS=1' in lvm.conf. 

The broader problem is LVM is entirely too chatty and every emitted message needs to be properly examined and assigned to 'verbose', 'debug', and 'normal' and likewise if it should be squelched via 'quiet' or 'double quiet'.

Commands that do listings is IMO the only place where warnings should show up, again subject to 'quiet' and the triggering threshold.

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[Zdenek Kabelac]

Dne 19.9.2017 v 16:34 matthew patton napsal(a):
> LVM and thin in particular is not for noobies or novices. If they get burned then they deserve it for using a technology they didn't bother to study and learn 

Well it's not for novices ;) yet I believe  'lvm2' should not be an easy 
weapon for mass destruction for users data - we are then 'beaten' by rumors 
from the 'other side'

>"create succeeded". Create succeeded is indicated by a 0 return code. Anything else is just noise. If the user wants noise, then need to use the '-v|--verbose' flag.


lvcreate -q  is then likely  wanted  to be 'respected' as normally lvm2 tends 
to be a  bit  'conversational' command.

But I think  '-q'  should be respected and it's not  with this log_warn().

So I think  conversion to  log_print() level is possibly the goal here ??
(covering both side of stories).

Since as it's been said - we emit more lines per command even about successful 
'middle' steps.


Regards

Zdenek

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[Xen]

Gionatan Danti schreef op 19-09-2017 10:44:

> Sure, I was only describing a possible case where the warning is
> "redundant" (ie: because the admin know the snapshot will be
> short-lived).

I would only like to say that reducing the "warning" to a "notice" would 
also reduce the irritation.

Ie. instead "WARNING. Word word word" it could also be

"New volume xx overprovisions thin pool by xxx".

This way you don't give the user the idea that you think he/she is 
stupid.

I know this still implies some verbosity but it doesn't have to be an 
"error". It can simply be a "notice".

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[Gionatan Danti]

Il 19-09-2017 17:30 Zdenek Kabelac ha scritto:
> The main purpose of the Warning is to really warn user there is NOT
> configured auto-extension of thin-pool (so no threshold is setup) - so
> thin-pool is not set-up in 'preferred' way  (so when user counts with
> the fact the thin-pool can grow and auto-extension is enabled - the
> warning is not printed).
> 
> I think it's really useful to give this information to the user -
> since from my experience with users - many of them are simply unaware
> of the fact when they take 3 snapshots they may need  3x more space of
> the origin volume.

Right, but the "skilled admin" need a command line 
parameter/configuration variable to silence this warning...

> So in the case users do want to have 'critical' volumes always 'safe'
> from out-of-space condition - the message tells them when pool can't
> cover all space for all thins.
> 
> Even in this list - people tend to think it is really an easy to just
> drop snapshots like with old-snaps and they even think it will happen
> auto-magically.
> 
> So IMHO we are better to give user really good info about what is going 
> on.
> Once we provide more secure mechanism - we may possibly change the
> time and actual printed message.
> 
> Skilled user just ignores the message - so is the major problem with it 
> ?

... because this auto-trains our eyes to automatically ignore warning 
messages - which is a very bad thing IMHO.
Moreover, as the warning is emitted on STDERR, ignoring it can be 
error-prone, especially in cron scripts (ie: 2>/dev/null, which is 
*really* bad when it destroys/hides important error messages).

> Is it the 'severity' -  so the message should be prefixed with "NOTE:"
> instead of  "WARNING:"    (log_warn() -> log_print_unless_quite())

I second this.

> We have number of similar messages for other cases, so it's relatively
> common in lvm2 to give some guidance messages to users - just this one
> gets some extra tension (i.e. we can open similar discussion about
> handling duplicates cases)

True, and I really appreciate that and the "don't let users destroy its 
data" reasoning. However, when the users feel his was sufficiently 
warned about thin pool fullness, we should provide a mean to silence 
this (expected) error. After all, this is not due to erroneous thin pool 
use. Rather, overprovisioning and snapshots are the main motivation for 
the very existance of thin pool.

Thanks.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[Zdenek Kabelac]

Dne 19.9.2017 v 16:14 David Teigland napsal(a):
> On Tue, Sep 19, 2017 at 01:11:09PM +0200, Zdenek Kabelac wrote:
>> IMHO the most convenient in my eyes is a usage of some sort of 'envvar'
>> LVM_SUPPRESS_POOL_WARNINGS....
> 
> I think we're looking at the wrong thing.  The root problem is what we're
> warning about, not that the warning is being printed.  It doesn't make
> sense to warn about the inherent nature of things.  When peole create a
> linear LV, we don't print a warning that it's not redundant.  By warning
> that the pool is overprovisioned, we also mislead people into thinking
> that this is what they should worry about, when in fact it's free space in
> the pool which is the real thing to worry about.  So I think the message
> should be dropped and replaced with something more useful.
>
The main purpose of the Warning is to really warn user there is NOT configured 
auto-extension of thin-pool (so no threshold is setup) - so thin-pool is not 
set-up in 'preferred' way  (so when user counts with the fact the thin-pool 
can grow and auto-extension is enabled - the warning is not printed).

I think it's really useful to give this information to the user - since from 
my experience with users - many of them are simply unaware of the fact when 
they take 3 snapshots they may need  3x more space of the origin volume.

So in the case users do want to have 'critical' volumes always 'safe' from 
out-of-space condition - the message tells them when pool can't cover all 
space for all thins.

Even in this list - people tend to think it is really an easy to just drop 
snapshots like with old-snaps and they even think it will happen auto-magically.

So IMHO we are better to give user really good info about what is going on.
Once we provide more secure mechanism - we may possibly change the time and 
actual printed message.

Skilled user just ignores the message - so is the major problem with it ?

Is it the 'severity' -  so the message should be prefixed with "NOTE:" instead 
of  "WARNING:"    (log_warn() -> log_print_unless_quite())

We have number of similar messages for other cases, so it's relatively common 
in lvm2 to give some guidance messages to users - just this one gets some 
extra tension (i.e. we can open similar discussion about handling duplicates 
cases)

Regards

Zdenek

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[David Teigland]

On Tue, Sep 19, 2017 at 01:11:09PM +0200, Zdenek Kabelac wrote:
> IMHO the most convenient in my eyes is a usage of some sort of 'envvar'
> LVM_SUPPRESS_POOL_WARNINGS....

I think we're looking at the wrong thing.  The root problem is what we're
warning about, not that the warning is being printed.  It doesn't make
sense to warn about the inherent nature of things.  When peole create a
linear LV, we don't print a warning that it's not redundant.  By warning
that the pool is overprovisioned, we also mislead people into thinking
that this is what they should worry about, when in fact it's free space in
the pool which is the real thing to worry about.  So I think the message
should be dropped and replaced with something more useful.

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[Zdenek Kabelac]

Dne 19.9.2017 v 10:49 Gionatan Danti napsal(a):
> On 18/09/2017 23:10, matthew patton wrote:
>> If the warnings are not being emitted to STDERR then that needs to be fixed 
>> right off the bat.
> 
> The line with WARNINGs are written on STDERR, at least con recent LVM version.
> 
>> 'lvs -q blah' should squash any warnings.
>> 'lvcreate' frankly shouldn't output anything unless invoked with '-v' 
>> anyway. So therefore '-q' should also squash warnings.
> 
> I'm not sure this is the right approach. I do not expect "-q" to remove *all* 
> warnings, because warnings can be quite important.
> 


There are couple troubles - there are always a 'skilled  X  unskilled' users.
and also there are varying distro maintainers..

We are already facing some troubles when some distributions do make a changes 
to default configuration files which are not really ideal for 'default' users 
(in other words opinions about defaults are different - moreover based on 
wrong blog post from users...)

So we can introduce  i.e.  'novice_user = 1' in lvm.conf  - but it might be 
effectively dropped when package maintainer decides this way lvm2 makes less 
annoying messages around some commands.

But in case of thin-pool it's better to warn-ahead  instead of facing troubles 
later  (as full thin-pool is not going to be pleasant experience...)

So we are looking for some solution which cannot be easily 'hidden' for 
everyone - so we do not end with reports about 'hidden over-provisioning' 
causing system malfunctioning.  Yet it should be 'easy' to bypass it for 
skilled admin.


IMHO the most convenient in my eyes is a usage of some sort of 'envvar'
LVM_SUPPRESS_POOL_WARNINGS....


Since we already use similar logic to bypass i.e.  FD close() warnings
LVM_SUPPRESS_FD_WARNINGS
LVM_SUPPRESS_LOCKING_FAILURE_MESSAGES


Regards

Zdenek

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[Gionatan Danti]

On 18/09/2017 23:10, matthew patton wrote:
> If the warnings are not being emitted to STDERR then that needs to be fixed right off the bat.

The line with WARNINGs are written on STDERR, at least con recent LVM 
version.

> 'lvs -q blah' should squash any warnings.
> 'lvcreate' frankly shouldn't output anything unless invoked with '-v' anyway. So therefore '-q' should also squash warnings.

I'm not sure this is the right approach. I do not expect "-q" to remove 
*all* warnings, because warnings can be quite important.

Regards.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[Gionatan Danti]

On 18/09/2017 22:08, Zdenek Kabelac wrote:
> 
> 
> We can possibly print WARNING message only with the 1st. thin LV� which 
> causes overprovisioining of thin-pool.

This should be ok, but it would not cover all use cases. For example, a 
thin snapshot taken, and removed, for backup purpose would repeatedly 
trigger the warning.

So, while more "intelligence" on when printing the warning on lvm side 
is welcomed, I really advocate for a simple command line option (or 
lvm.conf parameter) to skip the warning.

> As� for 'read-only' snapshot - it really doesn't matter - the 
> overprovisioned pool can in the 'worst' case end in out-of-space 
> condition (i.e. fully rewritten origin)

Sure, I was only describing a possible case where the warning is 
"redundant" (ie: because the admin know the snapshot will be short-lived).

Thanks.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[matthew patton]

 > So, any changes to disable the warning at least when taking snapshots?
> https://bugzilla.redhat.com/show_bug.cgi?id=1465974
 
If the warnings are not being emitted to STDERR then that needs to be fixed right off the bat.

'lvs -q blah' should squash any warnings.
'lvcreate' frankly shouldn't output anything unless invoked with '-v' anyway. So therefore '-q' should also squash warnings.

I suspect a wholesale cleanup of the way LVM does logging will suffice to fix the gratuitous and unnecessary output.

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[Zdenek Kabelac]

Dne 18.9.2017 v 21:07 Gionatan Danti napsal(a):
> Il 18-09-2017 20:55 David Teigland ha scritto:
>> It's definitely an irritation, and I described a configurable alternative
>> here that has not yet been implemented:
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=1465974
>>
>> Is this the sort of topic where we should start making use of
>> https://github.com/lvmteam/lvm2/issues ?
> 
> Hi, the proposed solution on BZ entry seems somewhat too "invasive" to me.
> I think it generally is a good idea to warn the users about possibly 
> unexptected behavior (ie: full pool).
> 
> However, for the specific use-case of taking read-only snapshots, something as 
> simple as "--silencepoolsizewarn" flag (or similar configuration variable) 
> would do the trick without distrupting legitimate warnings.
> 


We can possibly print WARNING message only with the 1st. thin LV  which causes 
overprovisioining of thin-pool.

As  for 'read-only' snapshot - it really doesn't matter - the overprovisioned 
pool can in the 'worst' case end in out-of-space condition (i.e. fully 
rewritten origin)

Regards


Zdenek

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[Gionatan Danti]

Il 18-09-2017 20:55 David Teigland ha scritto:
> It's definitely an irritation, and I described a configurable 
> alternative
> here that has not yet been implemented:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1465974
> 
> Is this the sort of topic where we should start making use of
> https://github.com/lvmteam/lvm2/issues ?

Hi, the proposed solution on BZ entry seems somewhat too "invasive" to 
me.
I think it generally is a good idea to warn the users about possibly 
unexptected behavior (ie: full pool).

However, for the specific use-case of taking read-only snapshots, 
something as simple as "--silencepoolsizewarn" flag (or similar 
configuration variable) would do the trick without distrupting 
legitimate warnings.

Regards.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

Re: [linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[David Teigland]

On Mon, Sep 18, 2017 at 07:52:01PM +0200, Gionatan Danti wrote:
> Hi all,
> being an heavy snapshot user, I wonder if (and how) to silence such a
> warning:
> 
> "WARNING: Sum of all thin volume sizes (2.00 GiB) exceeds the size of thin
> pool cl_gdanti-lenovo/thinpool (1.00 GiB)!"
> 
> I fully understand the reasoning for this warning, but snapshots are almost
> guaranteed to raise it. I found that by setting the
> thin_pool_autoextend_threshold the warning disappears, but on some pool I
> would rather like to left autoextend disabled.
> 
> So, any changes to disable the warning at least when taking snapshots?
> Thanks.

It's definitely an irritation, and I described a configurable alternative
here that has not yet been implemented:

https://bugzilla.redhat.com/show_bug.cgi?id=1465974

Is this the sort of topic where we should start making use of
https://github.com/lvmteam/lvm2/issues ?

[linux-lvm] Option to silence "WARNING: Sum of all thin volume sizes exceeds the size of thin pool"

[Gionatan Danti]

Hi all,
being an heavy snapshot user, I wonder if (and how) to silence such a 
warning:

"WARNING: Sum of all thin volume sizes (2.00 GiB) exceeds the size of 
thin pool cl_gdanti-lenovo/thinpool (1.00 GiB)!"

I fully understand the reasoning for this warning, but snapshots are 
almost guaranteed to raise it. I found that by setting the 
thin_pool_autoextend_threshold the warning disappears, but on some pool 
I would rather like to left autoextend disabled.

So, any changes to disable the warning at least when taking snapshots?
Thanks.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8