linux-lvm.redhat.com archive mirror
 help / color / mirror / Atom feed
From: Alasdair G Kergon <agk@redhat.com>
To: dm-devel@redhat.com, linux-lvm@redhat.com
Subject: [linux-lvm] storage-logger: Recording changes to the udev database
Date: Thu, 26 Mar 2020 23:31:45 +0000	[thread overview]
Message-ID: <20200326233144.GM17504@agk-dp.fab.redhat.com> (raw)

I'm experimenting with ways of recording changes to the udev database so
you can look back at the history of the storage stack on a particular
machine.  This is still a work-in-progress, but it's reached a point
where I'd like more people to try it out.

I've written a shell script that records data related to storage uevents
in the system journal and a perl script that helps you to interrogate
this data later to create a representation of the storage components.

If you're interested, please try this out and let me know if you think
pursing this approach further would lead to something that you would
use and distributions should ship.

Source code:
  https://github.com/lvmteam/storage-logger
Fedora builds:
  https://copr.fedorainfracloud.org/coprs/agk/storage-logger/build/1320735/
Presentation:
  https://fosdem.org/2020/schedule/event/storage_logger/


Storage-logger
==============
The storage-logger project maintains a record of the storage
configuration of a linux system as it changes over time.
The idea is to provide a quick way to check the state
of a system at times in the past.

Logging
=======
The initial logging implementation is triggered by storage uevents and
consists of two components:

1. A new udev rule file, 99-zzz-storage-logger.rules, which runs after
all the other rules have run and invokes:

2. A script, udev_storage_logger.sh, that captures relevant
information about devices that changed and stores it in the system
journal.

The effect is to log relevant uevents plus some supplementary
information.  It does not yet handle filesystem-related events.

Reporting
=========
Two methods to query the data are offered:

1. journalctl
Reports the raw data using simple filtering.
Data is tagged with the identifier UDEVLOG and retrievable as
key-value pairs.
All the captured data:
  journalctl -t UDEVLOG --output verbose
or as JSON:
  journalctl -t UDEVLOG --output json
Between a time range:
  --since 'YYYY-MM-DD HH:MM:SS' 
  --until 'YYYY-MM-DD HH:MM:SS'
Other filtering features are described in the man page.

2. lsblkj
This wrapper creates a dummy system environment that "looks like" the
system did at a specified earlier time and then runs lsblk against it.
It accepts --since and --until arguments to pass to journalctl to
select the desired data, and passes other arguments controlling
the output format to the real lsblk.  
Use --verbose to watch it setting up the temporary environment .
Use --dry-run to see what it would do without actually doing it.
Use --git to create a git repository recording the changes over time.
 
Alasdair
-- 
agk@redhat.com

             reply	other threads:[~2020-03-26 23:31 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-26 23:31 Alasdair G Kergon [this message]
2020-03-27  0:32 ` Brian McCullough
2020-03-27 19:50 ` Brian McCullough
2020-03-27 20:53   ` Alasdair G Kergon
2020-03-27 20:52     ` Brian McCullough

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200326233144.GM17504@agk-dp.fab.redhat.com \
    --to=agk@redhat.com \
    --cc=dm-devel@redhat.com \
    --cc=linux-lvm@redhat.com \
    --subject='Re: [linux-lvm] storage-logger: Recording changes to the udev database' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).