[linux-lvm] When and why vgs command can change metadata and incur old metadata to be backed up?

[linux-lvm] When and why vgs command can change metadata and incur old metadata to be backed up?

[Eric Ren]

Hi all,

Sometimes, I see the following message in the VG metadata backups under 
/etc/lvm/archive:

"""
contents = "Text Format Volume Group"
version = 1

description = "Created *before* executing 'vgs'"
"""

I'm wondering when and why the new backups will be created by reporting 
command like vgs?

Thanks in advance!
Eric

Re: [linux-lvm] When and why vgs command can change metadata and incur old metadata to be backed up?

[David Teigland]

On Mon, Oct 30, 2017 at 02:06:45PM +0800, Eric Ren wrote:
> Hi all,
> 
> Sometimes, I see the following message in the VG metadata backups under
> /etc/lvm/archive:
> 
> """
> contents = "Text Format Volume Group"
> version = 1
> 
> description = "Created *before* executing 'vgs'"
> """
> 
> I'm wondering when and why the new backups will be created by reporting
> command like vgs?

It's probably a case where lvm sees something wrong after reading the VG
metadata, and automatically tries to fix it, writing a corrected version
of the metadata to disk.  This means that even a command that only reads
and reports lvm information can potentially write to disk.

Right now it's hard to identify the precise instances and locations of
these repairs.  But, I am in the middle of reworking the VG reading code
with the goal of consolidating and clarifying all the cases of repair, at
which point we can improve the way we handle this.  I think we want to try
to make these repairs more limited and controlled, especially for commands
that in theory are only reading and reporting information.  I've also
suggested that whenever repairs are done, lvm should record a persistent
message in the system log with the details, but that idea didn't get a
great reception.

Re: [linux-lvm] When and why vgs command can change metadata and incur old metadata to be backed up?

[Gionatan Danti]

Il 30-10-2017 18:04 David Teigland ha scritto:
> On Mon, Oct 30, 2017 at 02:06:45PM +0800, Eric Ren wrote:
>> Hi all,
>> 
>> Sometimes, I see the following message in the VG metadata backups 
>> under
>> /etc/lvm/archive:
>> 
>> """
>> contents = "Text Format Volume Group"
>> version = 1
>> 
>> description = "Created *before* executing 'vgs'"
>> """
>> 
>> I'm wondering when and why the new backups will be created by 
>> reporting
>> command like vgs?
> 
> It's probably a case where lvm sees something wrong after reading the 
> VG
> metadata, and automatically tries to fix it, writing a corrected 
> version
> of the metadata to disk.  This means that even a command that only 
> reads
> and reports lvm information can potentially write to disk.

Interesting. Eric, can you show the *before* and *after* vgs textual 
metadata (you should find them in /etc/lvm/archive)?

> that in theory are only reading and reporting information.  I've also
> suggested that whenever repairs are done, lvm should record a 
> persistent
> message in the system log with the details, but that idea didn't get a
> great reception.

Just to know: why the idea of syslog reporting was discarded?

Thanks.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

Re: [linux-lvm] When and why vgs command can change metadata and incur old metadata to be backed up?

[David Teigland]

On Mon, Oct 30, 2017 at 07:11:16PM +0100, Gionatan Danti wrote:
> > that in theory are only reading and reporting information.  I've also
> > suggested that whenever repairs are done, lvm should record a persistent
> > message in the system log with the details, but that idea didn't get a
> > great reception.
> 
> Just to know: why the idea of syslog reporting was discarded?

It was so obviously correct to me that I had a hard time arguing for it.
I don't think any of the objections were satisfactory, e.g. nobody has
asked, nobody needs it, the metadata archives are good enough, you can
already configure log messages to go to syslog.

I am still hoping to do something like this either as a part of
centralizing metadata repair (in progress), or as a part of a broader
suggestion that did get a better response but will take more work.  That
was to create a new log file and format in which every lvm disk change is
recorded.  https://bugzilla.redhat.com/show_bug.cgi?id=1455671

Everyone is encouraged to give their feedback about that in that bz, or
create an "issue" on the lvm github.

Re: [linux-lvm] When and why vgs command can change metadata and incur old metadata to be backed up?

[Eric Ren]

Hi all,

> 
> Interesting. Eric, can you show the *before* and *after* vgs textual
> metadata (you should find them in /etc/lvm/archive)?
> 

Ah, I think no need to show the archives now. Alasdair and David have
given us a very good explanation, thanks for them!

Regards,
Eric

Re: [linux-lvm] When and why vgs command can change metadata and incur old metadata to be backed up?

[Alasdair G Kergon]

On Mon, Oct 30, 2017 at 02:06:45PM +0800, Eric Ren wrote:
> """
> contents = "Text Format Volume Group"
> version = 1
>
> description = "Created *before* executing 'vgs'"
> """
> I'm wondering when and why the new backups will be created by reporting  
> command like vgs?

Very simply if the metadata the command has just read in does not match
the last backup stored in the local filesystem and the process is able
and configured to write a new backup.

The command that made the metadata change might not have written a
backup if it crashed, was configured not to write backups, was running
with the filesystem readonly (e.g. booted into a recovery mode), ran on
a different node in a cluster, ran as part of an installer that chose
not to give you any metadata backups, performed metadata recovery etc.
(Plus an old release had a bug where the checking went wrong and it
made a backup every time even though nothing had actually changed.)

Alasdair

Re: [linux-lvm] When and why vgs command can change metadata and incur old metadata to be backed up?

[Eric Ren]

Hi Alasdair,

> Very simply if the metadata the command has just read in does not match
> the last backup stored in the local filesystem and the process is able
> and configured to write a new backup.
>
> The command that made the metadata change might not have written a
> backup if it crashed, was configured not to write backups, was running
> with the filesystem readonly (e.g. booted into a recovery mode), ran on
> a different node in a cluster, ran as part of an installer that chose
> not to give you any metadata backups, performed metadata recovery etc.
> (Plus an old release had a bug where the checking went wrong and it
> made a backup every time even though nothing had actually changed.)

Can you still recall the fix commit for that bug? I recently encountered 
a similar
problem on 2.02.98. Thanks in advance.

Thanks,
Eric