Create subvolume from a directory?

Create subvolume from a directory?

[Alex]

Hi all,

Just a quick question but can't find an obvious answer.

Can I create/convert a existing (btrfs) directory into a subvolume?

It would be very helpful when transferring 'partitions' into btrfs.
I found a similar question way back in google, but that site is
down now generally.

Thanks in advance.

Re: Create subvolume from a directory?

[Chester]

On Tue, Mar 27, 2012 at 12:19 PM, Alex <alex@bpmit.com> wrote:
> Hi all,
>
> Just a quick question but can't find an obvious answer.
>
> Can I create/convert a existing (btrfs) directory into a subvolume?
>
> It would be very helpful when transferring 'partitions' into btrfs.
> I found a similar question way back in google, but that site is
> down now generally.
>
> Thanks in advance.
>
>

I don't think this is possible. The closest thing I can think of is to
take a snapshot of the volume, move the directory to the top of the
subvolume, then delete all other content..

=2E.. That seems like an awful amount of work, but it'll preserve the
contents of the directory without making duplicates.

>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Create subvolume from a directory?

[Matthias G. Eckermann]

Hello Alex and all,

On 2012-03-27 T 17:19 +0000 Alex wrote:

> Just a quick question but can't find an obvious answer.
>=20
> Can I create/convert a existing (btrfs) directory into a
> subvolume?
>=20
> It would be very helpful when transferring 'partitions'
> into btrfs.  I found a similar question way back in
> google, but that site is down now generally.

As far as I am aware, this is not possible directly. My
approach to this would be using copy with reflinks:

------------------------------< snip >------------------------------

## migrate /var/lib/lxc/installserver=20
## from directory to btrfs subvolume

# du -ks /var/lib/lxc/installserver
500332  /var/lib/lxc/installserver

# mv /var/lib/lxc/installserver /var/lib/lxc/installserver_tmp

# btrfs subvol create /var/lib/lxc/installserver
Create subvolume '/var/lib/lxc/installserver'

# time cp -a --reflink /var/lib/lxc/installserver_tmp/rootfs /var/lib/l=
xc/installserver

real    0m1.367s
user    0m0.148s
sys     0m1.108s

## Now remove /var/lib/lxc/installserver_tmp (or not)

------------------------------< snap >------------------------------

Just to compare this with a "mv":

------------------------------< snip >------------------------------

## Go back to former state

# btrfs subvol delete /var/lib/lxc/installserver
Delete subvolume '/var/lib/lxc/installserver'

# btrfs subvol create /var/lib/lxc/installserver
Create subvolume '/var/lib/lxc/installserver'

# time mv /var/lib/lxc/installserver_tmp/rootfs /var/lib/lxc/installser=
ver/

real    0m12.917s
user    0m0.208s=20
sys     0m2.508s=20

------------------------------< snap >------------------------------

While the time measurement might be flawed due to the subvol
actions inbetween, caching etc.: I tried several times, and
"cp --reflinks" always is multiple times faster than "mv" in
my environment.

Or did I misunderstand your question?

so long -
	MgE

--=20
Matthias G. Eckermann     Senior Product Manager   SUSE=C2=AE Linux Ent=
erprise
SUSE LINUX Products GmbH  Maxfeldstra=C3=9Fe 5          90409 N=C3=BCrn=
berg Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imend=C3=B6rffer, HRB 16746 (AG N=C3=
=BCrnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Create subvolume from a directory?

[Fajar A. Nugraha]

On Wed, Mar 28, 2012 at 5:24 AM, Matthias G. Eckermann <mge@suse.com> wrote:
> While the time measurement might be flawed due to the subvol
> actions inbetween, caching etc.: I tried several times, and
> "cp --reflinks" always is multiple times faster than "mv" in
> my environment.

So this is cross-subvolume reflinks? I thought the code for that
wasn't merged yet?

-- 
Fajar

Re: Create subvolume from a directory?

[Liu Bo]

On 03/28/2012 06:24 AM, Matthias G. Eckermann wrote:
> Hello Alex and all,
> 
> On 2012-03-27 T 17:19 +0000 Alex wrote:
> 
>> Just a quick question but can't find an obvious answer.
>>
>> Can I create/convert a existing (btrfs) directory into a
>> subvolume?
>>
>> It would be very helpful when transferring 'partitions'
>> into btrfs.  I found a similar question way back in
>> google, but that site is down now generally.
> 
> As far as I am aware, this is not possible directly. My
> approach to this would be using copy with reflinks:
> 
> ------------------------------< snip >------------------------------
> 
> ## migrate /var/lib/lxc/installserver 
> ## from directory to btrfs subvolume
> 
> # du -ks /var/lib/lxc/installserver
> 500332  /var/lib/lxc/installserver
> 
> # mv /var/lib/lxc/installserver /var/lib/lxc/installserver_tmp
> 
> # btrfs subvol create /var/lib/lxc/installserver
> Create subvolume '/var/lib/lxc/installserver'
> 
> # time cp -a --reflink /var/lib/lxc/installserver_tmp/rootfs /var/lib/lxc/installserver
> 

This is too much weird.

AFAIK, clone between different subvolumes should be forbidden.
So this would get a "Invalid cross-device link", because an individual subvolume can be mounted directly.

thanks,
liubo

> real    0m1.367s
> user    0m0.148s
> sys     0m1.108s
> 
> ## Now remove /var/lib/lxc/installserver_tmp (or not)
> 
> ------------------------------< snap >------------------------------
> 
> Just to compare this with a "mv":
> 
> ------------------------------< snip >------------------------------
> 
> ## Go back to former state
> 
> # btrfs subvol delete /var/lib/lxc/installserver
> Delete subvolume '/var/lib/lxc/installserver'
> 
> # btrfs subvol create /var/lib/lxc/installserver
> Create subvolume '/var/lib/lxc/installserver'
> 
> # time mv /var/lib/lxc/installserver_tmp/rootfs /var/lib/lxc/installserver/
> 
> real    0m12.917s
> user    0m0.208s 
> sys     0m2.508s 
> 
> ------------------------------< snap >------------------------------
> 
> While the time measurement might be flawed due to the subvol
> actions inbetween, caching etc.: I tried several times, and
> "cp --reflinks" always is multiple times faster than "mv" in
> my environment.
> 
> Or did I misunderstand your question?
> 
> so long -
> 	MgE
> 

Re: Create subvolume from a directory?

[David Sterba]

On Wed, Mar 28, 2012 at 08:46:13AM +0700, Fajar A. Nugraha wrote:
> On Wed, Mar 28, 2012 at 5:24 AM, Matthias G. Eckermann <mge@suse.com> wrote:
> > While the time measurement might be flawed due to the subvol
> > actions inbetween, caching etc.: I tried several times, and
> > "cp --reflinks" always is multiple times faster than "mv" in
> > my environment.
> 
> So this is cross-subvolume reflinks? I thought the code for that
> wasn't merged yet?

Matthias was using SLES kernel, the cross-subvolume reflink patch is
present there, he was not aware it's not merged upstream.


david

Re: Create subvolume from a directory?

[David Sterba]

On Tue, Mar 27, 2012 at 05:19:06PM +0000, Alex wrote:
> Can I create/convert a existing (btrfs) directory into a subvolume?

For the reference there's an entry at the wiki:

http://btrfs.ipv5.de/index.php?title=UseCases#Can_I_take_a_snapshot_of_a_directory.3F

as noted in the thread, this relies on the unmerged cross-subvolume
reflink patch.


david

Re: Create subvolume from a directory?

[Alex]

David Sterba <dave <at> jikos.cz> writes:

> 
> On Tue, Mar 27, 2012 at 05:19:06PM +0000, Alex wrote:
> > Can I create/convert a existing (btrfs) directory into a subvolume?
> 
> For the reference there's an entry at the wiki:
> 
>
http://btrfs.ipv5.de/index.php?title=UseCases#Can_I_take_a_snapshot_of_a_directory.3F
> 

Thank you all for your replies.

Kind regards.
(happily experimenting with btrfs in a 3.3 kernel!)

Al.

Re: Create subvolume from a directory?

[Goffredo Baroncelli]

Hello Liu

On 03/28/2012 04:18 AM, Liu Bo wrote:
> On 03/28/2012 06:24 AM, Matthias G. Eckermann wrote:

>> # time cp -a --reflink /var/lib/lxc/installserver_tmp/rootfs /var/lib/lxc/installserver
>>
>
> This is too much weird.
>
> AFAIK, clone between different subvolumes should be forbidden.
> So this would get a "Invalid cross-device link", because an individual subvolume can be mounted directly.

Could you elaborate which would be the issue ?
"cp --reflink"-ing a file is not different than snapshotting a file. In 
any case I could mount a snapshot and not the source subvolume.



>
> thanks,
> liubo
>
>> real    0m1.367s
>> user    0m0.148s
>> sys     0m1.108s
>>
>> ## Now remove /var/lib/lxc/installserver_tmp (or not)
>>
>> ------------------------------<  snap>------------------------------
>>
>> Just to compare this with a "mv":
>>
>> ------------------------------<  snip>------------------------------
>>
>> ## Go back to former state
>>
>> # btrfs subvol delete /var/lib/lxc/installserver
>> Delete subvolume '/var/lib/lxc/installserver'
>>
>> # btrfs subvol create /var/lib/lxc/installserver
>> Create subvolume '/var/lib/lxc/installserver'
>>
>> # time mv /var/lib/lxc/installserver_tmp/rootfs /var/lib/lxc/installserver/
>>
>> real    0m12.917s
>> user    0m0.208s
>> sys     0m2.508s
>>
>> ------------------------------<  snap>------------------------------
>>
>> While the time measurement might be flawed due to the subvol
>> actions inbetween, caching etc.: I tried several times, and
>> "cp --reflinks" always is multiple times faster than "mv" in
>> my environment.
>>
>> Or did I misunderstand your question?
>>
>> so long -
>> 	MgE
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> .
>

Re: Create subvolume from a directory?

[Liu Bo]

On 03/29/2012 12:54 AM, Goffredo Baroncelli wrote:
>>
> 
> Could you elaborate which would be the issue ?
> "cp --reflink"-ing a file is not different than snapshotting a file. In
> any case I could mount a snapshot and not the source subvolume.
> 
> 
> 

We already have a debate about this "cross-link device":
http://comments.gmane.org/gmane.comp.file-systems.btrfs/9864

"cp --reflink" will use clone feature, which can share data among files, but metadata is preserved individually.

My case is that I can mount both a subvolume and a snapshot via "-o subvol=xxx" or "-o subvolid=xxx".

thanks,
liubo

Re: Create subvolume from a directory?

[Hubert Kario]

On Thursday 29 of March 2012 09:24:44 Liu Bo wrote:
> On 03/29/2012 12:54 AM, Goffredo Baroncelli wrote:
> > Could you elaborate which would be the issue ?
> > "cp --reflink"-ing a file is not different than snapshotting a file=
=2E In
> > any case I could mount a snapshot and not the source subvolume.
>=20
> We already have a debate about this "cross-link device":
> http://comments.gmane.org/gmane.comp.file-systems.btrfs/9864
>=20
> "cp --reflink" will use clone feature, which can share data among fil=
es, but
> metadata is preserved individually.
>=20
> My case is that I can mount both a subvolume and a snapshot via "-o
> subvol=3Dxxx" or "-o subvolid=3Dxxx".

And how is this different from regular snapshot of subvolume? In the en=
d you=20
get two files pointing to same data on the disk while having different=20
metadata.

Let me rephrase it:

People don't want to be able to do:

mount /dev/lvm/btrfs /mnt/a -t btrfs -o subvol=3DvolA
mount /dev/lvm/btrfs /mnt/b -t btrfs -o subvol=3DvolB
cp --reflink=3Dalways /mnt/a/file /mnt/b

Just like you can't do hardlinks over `mount --bind` mountpoints, you=20
shouldn't be able to cp reflink over mountpoints. That's expected as th=
is=20
*does* break VFS semantics.

*But* people want to be able to do this:=20

mount /dev/lvm/btrfs /mnt/ -t btrfs
btrfs subvol create /mnt/subvol
big-file-creator > /mnt/subvol/BIG-file
btrfs subvol snapshot /mnt/subvol /mnt/subvol-bak
big-file-editor /mnt/subvol/BIG-file
rm /mnt/subvol-bak/BIG-file
cp --reflink=3Dalways /mnt/subvol/BIG-file /mnt/subvol-bak/BIG-file

This does not cross any VFS boundaries.

Regards,
--=20
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawer=F3w 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Create subvolume from a directory?

[David Sterba]

On Tue, May 01, 2012 at 07:09:21PM +0200, Hubert Kario wrote:
> Let me rephrase it:
> 
> People don't want to be able to do:
> 
> mount /dev/lvm/btrfs /mnt/a -t btrfs -o subvol=volA
> mount /dev/lvm/btrfs /mnt/b -t btrfs -o subvol=volB
> cp --reflink=always /mnt/a/file /mnt/b
> 
> Just like you can't do hardlinks over `mount --bind` mountpoints, you 
> shouldn't be able to cp reflink over mountpoints. That's expected as this 
> *does* break VFS semantics.

Proposed fix (incremental on top of the cross-subvol):

--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -2321,6 +2321,10 @@ static noinline long btrfs_ioctl_clone(struct file *file, unsigned long srcfd,
                goto out_drop_write;
        }

+       ret = -EXDEV;
+       if (src_file->f_path.mnt != file->f_path.mnt)
+               goto out_fput;
+
        src = src_file->f_dentry->d_inode;

        ret = -EINVAL;


david

Re: Create subvolume from a directory?

[Hubert Kario]

[-- Attachment #1: Type: text/plain, Size: 1424 bytes --]

On Wednesday 02 of May 2012 18:33:37 David Sterba wrote:
> On Tue, May 01, 2012 at 07:09:21PM +0200, Hubert Kario wrote:
> > Let me rephrase it:
> > 
> > People don't want to be able to do:
> > 
> > mount /dev/lvm/btrfs /mnt/a -t btrfs -o subvol=volA
> > mount /dev/lvm/btrfs /mnt/b -t btrfs -o subvol=volB
> > cp --reflink=always /mnt/a/file /mnt/b
> > 
> > Just like you can't do hardlinks over `mount --bind` mountpoints, you
> > shouldn't be able to cp reflink over mountpoints. That's expected as this
> > *does* break VFS semantics.
> 
> Proposed fix (incremental on top of the cross-subvol):

I'm a noob as far as kernel development is concerned so take my comments for 
what they're worth.
 
> --- a/fs/btrfs/ioctl.c
> +++ b/fs/btrfs/ioctl.c
> @@ -2321,6 +2321,10 @@ static noinline long btrfs_ioctl_clone(struct file
> *file, unsigned long srcfd, goto out_drop_write;
>         }
> 
> +       ret = -EXDEV;
> +       if (src_file->f_path.mnt != file->f_path.mnt)

I'm not sure about this comparision. Is the f_path struct member used just as 
reference to some general kernel structure?

> +               goto out_fput;
> +
>         src = src_file->f_dentry->d_inode;
> 
>         ret = -EINVAL;

Other that that, looks OK.

Regards,
-- 
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawerów 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 2346 bytes --]

Re: Create subvolume from a directory?

[David Sterba]

On Thu, May 03, 2012 at 03:26:20PM +0200, Hubert Kario wrote:
> > --- a/fs/btrfs/ioctl.c
> > +++ b/fs/btrfs/ioctl.c
> > @@ -2321,6 +2321,10 @@ static noinline long btrfs_ioctl_clone(struct file
> > *file, unsigned long srcfd, goto out_drop_write;
> >         }
> > 
> > +       ret = -EXDEV;
> > +       if (src_file->f_path.mnt != file->f_path.mnt)
> 
> I'm not sure about this comparision. Is the f_path struct member used just as 
> reference to some general kernel structure?

f_path is 'struct path' pointer obtained from a 'struct file' hence the
'f_' prefix, this is a naming scheme used in vfs.

Patch updated in my git repo (branch dev/cross-subvol-clone-v2).

david