From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <909d4cae-ddd2-3951-eee8-8dec8faa6f22@redhat.com> <35a99505-b5a9-a398-6ec2-b530733e974c@redhat.com> In-Reply-To: <35a99505-b5a9-a398-6ec2-b530733e974c@redhat.com> From: =?UTF-8?Q?Tomas_Dalebj=C3=B6rk?= Date: Fri, 25 Oct 2019 18:31:25 +0200 Message-ID: Content-Type: multipart/alternative; boundary="00000000000004b7980595beaf8e" Subject: Re: [linux-lvm] exposing snapshot block device Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: Zdenek Kabelac Cc: Gionatan Danti , LVM general discussion and development --00000000000004b7980595beaf8e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Wow! Impressing. This will make history! If this is possible, than we are able to implement a solution, which can do= : - progressive block level incremental forever (always incremental on block level : this already exist) - instant recovery to point in time (using the mentioned methods you just described) For example, lets say that a client wants to restore a file system, or a logical volume to how it looked a like yesterday. Eventhough there are no snapshot, nor any data. Than the client (with some coding); can start from an empty volume, and re-attach a cow device, and convert that using lvconvert --merge, so that the copying can be done in background using the backup server. If you forget about "how we will re-create the cow device"; and just focusing on the LVM ideas of re-attaching a cow device. Do you think that I have understood it correctly? Den tors 24 okt. 2019 kl 18:01 skrev Zdenek Kabelac : > Dne 23. 10. 19 v 13:24 Tomas Dalebj=C3=B6rk napsal(a): > > I have tested FusionIO together with old thick snapshots. > > I created the thick snapshot on a separate old traditional SATA drive, > just to > > check if that could be used as a snapshot target for high performance > disks; > > like a Fusion IO card. > > For those who doesn't know about FusionIO; they can deal with > 150-250,000 IOPS. > > > > And to be honest, I couldn't bottle neck the SATA disk I used as a thic= k > > snapshot target. > > The reason for why is simple: > > - thick snapshots uses sequential write techniques > > > > If I would have been using thin snapshots, than the writes would most > likely > > be more randomized on disk, which would have required more spindles to > coop > > with this. > > > > Anyhow; > > I am still eager to hear how to use an external device to import > snapshots. > > And when I say "import"; I am not talking about copyback, more to use t= o > read > > data from. > > Format of 'on-disk' snapshot metadata for old snapshot is trivial - being > some > header + pairs of dataoffset-TO-FROM - I think googling will reveal coup= le > python tools playing with it. > > You can add pre-created COW image to LV with lvconvert --snapshot > and to avoid 'zeroing' metadata use option -Zn > (BTW in the same way you can detach snapshot from LV with --splitsnapshot > so > you can look how the metadata looks like...) > > Although it's pretty unusual why would anyone create first the COW image > with > all the special layout and then merge it to LV - instead of directly > merging... There is only the 'little' advantage of minimizing 'offline' > time > of such device (and it's the reason why --split exists). > > Regards > > Zdenek > > > --00000000000004b7980595beaf8e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Wow!

Impressing.
T= his will make history!

If this is possible, than w= e are able to implement a solution, which can do:
- progressive b= lock level incremental forever (always incremental on block level : this al= ready exist)
- instant recovery to point in time (using the menti= oned methods you just described)

For example, lets= say that a client wants to restore a file system, or a logical volume to h= ow it looked a like yesterday.
Eventhough there are no snapshot, = nor any data.
Than the client (with some coding); can start from = an empty volume, and re-attach a cow device, and convert that using lvconve= rt --merge, so that the copying can be done in background using the backup = server.

If you forget about "how we will re-c= reate the cow device"; and just focusing on the LVM ideas of re-attach= ing a cow device.
Do you think that I have understood it correctl= y?


Den tors 24 okt. 2019 kl 18:01 skrev Zdenek Kabelac = <zkabelac@redhat.com>:
=
Dne 23. 10. 19 v 13= :24 Tomas Dalebj=C3=B6rk napsal(a):
> I have tested FusionIO together with old thick snapshots.
> I created the thick snapshot on a separate old traditional SATA drive,= just to
> check if that could be used as a snapshot target for high performance = disks;
> like a Fusion IO card.
> For those who doesn't know about FusionIO; they can deal with 150-= 250,000 IOPS.
>
> And to be honest, I couldn't bottle neck the SATA disk I used as a= thick
> snapshot target.
> The reason for why is simple:
> - thick snapshots uses sequential write techniques
>
> If I would have been using thin snapshots, than the writes would most = likely
> be more randomized on disk, which would have required more spindles to= coop
> with this.
>
> Anyhow;
> I am still eager to hear how to use an external device to import snaps= hots.
> And when I say "import"; I am not talking about copyback, mo= re to use to read
> data from.

Format of 'on-disk' snapshot metadata for old snapshot is trivial -= being some
header + pairs of dataoffset-TO-FROM -=C2=A0 I think googling will reveal c= ouple
python tools playing with it.

You can add pre-created COW image to LV=C2=A0 with=C2=A0 lvconvert --snapsh= ot
and to avoid 'zeroing' metadata use option -Zn
(BTW in the same way you can detach snapshot from LV with --splitsnapshot s= o
you can look how the metadata looks like...)

Although it's pretty unusual why would anyone create first the COW imag= e with
all the special layout and then merge it to LV - instead of directly
merging...=C2=A0 =C2=A0There is only the 'little' advantage of mini= mizing 'offline' time
of such device=C2=A0 =C2=A0(and it's the reason why --split exists).
Regards

Zdenek


--00000000000004b7980595beaf8e--