btrfs receive to subdirectory

btrfs receive to subdirectory

[Rory Campbell-Lange]

I can send snapshots to <volume>, but not <volume>/<dir>. Please advise
if what I am doing is incorrect.

Rory

Format usb3 disk and mount
	root@orchard:/bkp# mkfs.btrfs /dev/sdb1
	> WARNING! - Btrfs v0.20-rc1-37-g91d9eec IS EXPERIMENTAL
	> WARNING! - see http://btrfs.wiki.kernel.org before using
	> fs created label (null) on /dev/sdb1
	> 	nodesize 4096 leafsize 4096 sectorsize 4096 size 698.64GB
	> Btrfs v0.20-rc1-37-g91d9eec
	mount /dev/sdb1 /mnt


Create snapshots on /bkp share
	root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/snap_081012_1715
	> Create a readonly snapshot of 'subvol' in 'snaps/snap_081012_1715'
	root@orchard:/bkp# mutt -f subvol/INBOX/
    > 1561 kept, 18 deleted.
	root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/snap_081012_1716
    > Create a readonly snapshot of 'subvol' in 'snaps/snap_081012_1716'

Send base backup to /mnt
	root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt
	> At subvol snaps/snap_081012_1715
	> At subvol snap_081012_1715

Send incremental backup to /mnt
	root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
                       snaps/snap_081012_1716 | btrfs receive /mnt
	> At subvol snaps/snap_081012_1716
	> At snapshot snap_081012_1716

	root@orchard:/bkp# ls /mnt
	snap_081012_1715  snap_081012_1716

Results:
	root@orchard:/bkp# btrfs subvolume list /bkp 
	> ID 259 gen 62 top level 5 path subvol
	> ID 278 gen 60 top level 5 path snaps/snap_081012_1715
	> ID 279 gen 62 top level 5 path snaps/snap_081012_1716
	root@orchard:/bkp# btrfs subvolume list /mnt 
	> ID 256 gen 8 top level 5 path snap_081012_1715
	> ID 259 gen 9 top level 5 path snap_081012_1716

Restart:
	root@orchard:/bkp# btrfs subvolume del /mnt/snap_081012_171*
	> Delete subvolume '/mnt/snap_081012_1715'
	> Delete subvolume '/mnt/snap_081012_1716'

Try and snap to /mnt/<subdir>
	root@orchard:/bkp# mkdir /mnt/snaps
	root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt/snaps
	> At subvol snaps/snap_081012_1715
	> At subvol snap_081012_1715
	root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
	                   snaps/snap_081012_1716 | btrfs receive /mnt/snaps
	> At subvol snaps/snap_081012_1716
	> At snapshot snap_081012_1716
	> ERROR: open snaps/snap_081012_1715 failed. No such file or directory
    root@orchard:/bkp# ls /mnt/snaps
    > snap_081012_1715



-- 
Rory Campbell-Lange
rory@campbell-lange.net

Re: btrfs receive to subdirectory

[Arne Jansen]

On 10/08/12 18:30, Rory Campbell-Lange wrote:
> I can send snapshots to <volume>, but not <volume>/<dir>. Please advise
> if what I am doing is incorrect.
> 
> Rory
> 
> Format usb3 disk and mount
> 	root@orchard:/bkp# mkfs.btrfs /dev/sdb1
> 	> WARNING! - Btrfs v0.20-rc1-37-g91d9eec IS EXPERIMENTAL
> 	> WARNING! - see http://btrfs.wiki.kernel.org before using
> 	> fs created label (null) on /dev/sdb1
> 	> 	nodesize 4096 leafsize 4096 sectorsize 4096 size 698.64GB
> 	> Btrfs v0.20-rc1-37-g91d9eec
> 	mount /dev/sdb1 /mnt
> 
> 
> Create snapshots on /bkp share
> 	root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/snap_081012_1715
> 	> Create a readonly snapshot of 'subvol' in 'snaps/snap_081012_1715'
> 	root@orchard:/bkp# mutt -f subvol/INBOX/
>     > 1561 kept, 18 deleted.
> 	root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/snap_081012_1716
>     > Create a readonly snapshot of 'subvol' in 'snaps/snap_081012_1716'
> 
> Send base backup to /mnt
> 	root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt
> 	> At subvol snaps/snap_081012_1715
> 	> At subvol snap_081012_1715
> 
> Send incremental backup to /mnt
> 	root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
>                        snaps/snap_081012_1716 | btrfs receive /mnt
> 	> At subvol snaps/snap_081012_1716
> 	> At snapshot snap_081012_1716
> 
> 	root@orchard:/bkp# ls /mnt
> 	snap_081012_1715  snap_081012_1716
> 
> Results:
> 	root@orchard:/bkp# btrfs subvolume list /bkp 
> 	> ID 259 gen 62 top level 5 path subvol
> 	> ID 278 gen 60 top level 5 path snaps/snap_081012_1715
> 	> ID 279 gen 62 top level 5 path snaps/snap_081012_1716
> 	root@orchard:/bkp# btrfs subvolume list /mnt 
> 	> ID 256 gen 8 top level 5 path snap_081012_1715
> 	> ID 259 gen 9 top level 5 path snap_081012_1716
> 
> Restart:
> 	root@orchard:/bkp# btrfs subvolume del /mnt/snap_081012_171*
> 	> Delete subvolume '/mnt/snap_081012_1715'
> 	> Delete subvolume '/mnt/snap_081012_1716'
> 
> Try and snap to /mnt/<subdir>
> 	root@orchard:/bkp# mkdir /mnt/snaps
> 	root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt/snaps
> 	> At subvol snaps/snap_081012_1715
> 	> At subvol snap_081012_1715
> 	root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
> 	                   snaps/snap_081012_1716 | btrfs receive /mnt/snaps
> 	> At subvol snaps/snap_081012_1716
> 	> At snapshot snap_081012_1716
> 	> ERROR: open snaps/snap_081012_1715 failed. No such file or directory
>     root@orchard:/bkp# ls /mnt/snaps
>     > snap_081012_1715

The target has to be a subvol also. But interestingly enough, it also
fails for a subvol. The base send works, the incremental fails, because
btrfs receive can't find snaps/snap_081012_1715. If you give /mnt/snaps
as the target for the base and just /mnt for the incremental, it works.
There's clearly something broken there...

-arne

> 
> 
> 

Re: btrfs receive to subdirectory

[Rory Campbell-Lange]

On 08/10/12, Arne Jansen (sensille@gmx.net) wrote:
> On 10/08/12 18:30, Rory Campbell-Lange wrote:
> > I can send snapshots to <volume>, but not <volume>/<dir>. Please advise
> > if what I am doing is incorrect.

> > Try and snap to /mnt/<subdir>
> > 	root@orchard:/bkp# mkdir /mnt/snaps
> > 	root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt/snaps
> > 	> At subvol snaps/snap_081012_1715
> > 	> At subvol snap_081012_1715
> > 	root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
> > 	                   snaps/snap_081012_1716 | btrfs receive /mnt/snaps
> > 	> At subvol snaps/snap_081012_1716
> > 	> At snapshot snap_081012_1716
> > 	> ERROR: open snaps/snap_081012_1715 failed. No such file or directory
> >     root@orchard:/bkp# ls /mnt/snaps
> >     > snap_081012_1715
> 
> The target has to be a subvol also. But interestingly enough, it also
> fails for a subvol. The base send works, the incremental fails, because
> btrfs receive can't find snaps/snap_081012_1715. If you give /mnt/snaps
> as the target for the base and just /mnt for the incremental, it works.
> There's clearly something broken there...

Actually, this does work:

    root@orchard:~# mkfs.btrfs /dev/sdb1
        ...snip...
        > Btrfs v0.20-rc1-37-g91d9eec
    root@orchard:/bkp# ls
        > snaps  subvol
    root@orchard:/bkp# mount /dev/sdb1 /mnt
    root@orchard:/bkp# btrfs subvolume create /mnt/subvol
        > Create subvolume '/mnt/subvol'
    root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt/subvol
        > At subvol snaps/snap_081012_1715
        > At subvol snap_081012_1715
    root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
                             snaps/snap_081012_1716 | btrfs receive /mnt/subvol
        > At subvol snaps/snap_081012_1716
        > At snapshot snap_081012_1716

It certainly seems to work if the receive target is a subvolume other
than the implicit subvolume at the root of the mount point.

So one can only btrfs receive to a subvolume, which makes sense, and is
I believe the reason one cannot 'btrfs receive to a subdirectory' as set
out in the subject of my original email.

Are there any other problem cases you can see, Arne?

Regards
Rory

-- 
Rory Campbell-Lange
rory@campbell-lange.net

Re: btrfs receive to subdirectory

[Alex Lyakas]

Hi Rory, Arne,
I think the problem is that currently mnt_fd in struct btrfs_receive
is used both as "mount root" and "directory in which the
subvolume/snapshot needs to be created".
Arne, does the following patch make sense? It uses Jan's
find_mount_root function. With this patch both Rory's tests seem to
work for me.

Thanks,
Alex.



diff --git a/cmds-receive.c b/cmds-receive.c
index a8be6fa..ed53326 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -52,11 +52,13 @@ static int g_verbose = 0;
 struct btrfs_receive
 {
 	int mnt_fd;
+	int dest_dir_fd;

 	int write_fd;
 	char *write_path;

 	char *root_path;
+	char *dest_dir_path; /* relative to root_path */
 	char *full_subvol_path;

 	struct subvol_info *cur_subvol;
@@ -150,8 +152,11 @@ static int process_subvol(const char *path, const
u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;

-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);

 	fprintf(stderr, "At subvol %s\n", path);

@@ -167,7 +172,7 @@ static int process_subvol(const char *path, const
u8 *uuid, u64 ctransid,

 	memset(&args_v1, 0, sizeof(args_v1));
 	strcpy(args_v1.name, path);
-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
 	if (ret < 0) {
 		ret = -errno;
 		fprintf(stderr, "ERROR: creating subvolume %s failed. "
@@ -195,8 +200,11 @@ static int process_snapshot(const char *path,
const u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;

-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);

 	fprintf(stderr, "At snapshot %s\n", path);

@@ -243,7 +251,7 @@ static int process_snapshot(const char *path,
const u8 *uuid, u64 ctransid,
 		goto out;
 	}

-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
 	close(args_v2.fd);
 	if (ret < 0) {
 		ret = -errno;
@@ -790,17 +798,44 @@ struct btrfs_send_ops send_ops = {
 int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd)
 {
 	int ret;
+	char *dest_dir_full_path;
 	int end = 0;

-	r->root_path = strdup(tomnt);
-	r->mnt_fd = open(tomnt, O_RDONLY | O_NOATIME);
+	dest_dir_full_path = realpath(tomnt, NULL);
+	if (!dest_dir_full_path) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: realpath(%s) failed. %s\n", tomnt,
+				strerror(-ret));
+		goto out;
+	}
+	r->dest_dir_fd = open(tomnt, O_RDONLY | O_NOATIME);
+	if (r->dest_dir_fd < 0) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: failed to open destination directory %s.
%s\n", tomnt,
+				strerror(-ret));
+		goto out;
+	}
+	
+	ret = find_mount_root(tomnt, &r->root_path);
+	if (ret < 0) {
+		ret = -EINVAL;
+		fprintf(stderr, "ERROR: failed to determine mount point "
+				"for %s\n", tomnt);
+		goto out;
+	}
+	r->mnt_fd = open(r->root_path, O_RDONLY | O_NOATIME);
 	if (r->mnt_fd < 0) {
 		ret = -errno;
-		fprintf(stderr, "ERROR: failed to open %s. %s\n", tomnt,
+		fprintf(stderr, "ERROR: failed to open %s. %s\n", r->root_path,
 				strerror(-ret));
 		goto out;
 	}

+	/* Find the destination dir relative to mount root */
+	r->dest_dir_path = dest_dir_full_path + strlen(r->root_path);
+	if (r->dest_dir_path[0] == '/')
+		r->dest_dir_path++;
+		
 	ret = subvol_uuid_search_init(r->mnt_fd, &r->sus);
 	if (ret < 0)
 		return ret;
diff --git a/send-utils.h b/send-utils.h
index da407eb..a3e038b 100644
--- a/send-utils.h
+++ b/send-utils.h
@@ -65,5 +65,6 @@ void subvol_uuid_search_add(struct subvol_uuid_search *s,
 char *path_cat(const char *p1, const char *p2);
 char *path_cat3(const char *p1, const char *p2, const char *p3);

+int find_mount_root(const char *path, char **mount_root);

 #endif /* SEND_UTILS_H_ */









On Tue, Oct 9, 2012 at 1:18 PM, Rory Campbell-Lange
<rory@campbell-lange.net> wrote:
> On 08/10/12, Arne Jansen (sensille@gmx.net) wrote:
>> On 10/08/12 18:30, Rory Campbell-Lange wrote:
>> > I can send snapshots to <volume>, but not <volume>/<dir>. Please advise
>> > if what I am doing is incorrect.
>
>> > Try and snap to /mnt/<subdir>
>> >     root@orchard:/bkp# mkdir /mnt/snaps
>> >     root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt/snaps
>> >     > At subvol snaps/snap_081012_1715
>> >     > At subvol snap_081012_1715
>> >     root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
>> >                        snaps/snap_081012_1716 | btrfs receive /mnt/snaps
>> >     > At subvol snaps/snap_081012_1716
>> >     > At snapshot snap_081012_1716
>> >     > ERROR: open snaps/snap_081012_1715 failed. No such file or directory
>> >     root@orchard:/bkp# ls /mnt/snaps
>> >     > snap_081012_1715
>>
>> The target has to be a subvol also. But interestingly enough, it also
>> fails for a subvol. The base send works, the incremental fails, because
>> btrfs receive can't find snaps/snap_081012_1715. If you give /mnt/snaps
>> as the target for the base and just /mnt for the incremental, it works.
>> There's clearly something broken there...
>
> Actually, this does work:
>
>     root@orchard:~# mkfs.btrfs /dev/sdb1
>         ...snip...
>         > Btrfs v0.20-rc1-37-g91d9eec
>     root@orchard:/bkp# ls
>         > snaps  subvol
>     root@orchard:/bkp# mount /dev/sdb1 /mnt
>     root@orchard:/bkp# btrfs subvolume create /mnt/subvol
>         > Create subvolume '/mnt/subvol'
>     root@orchard:/bkp# btrfs send snaps/snap_081012_1715 | btrfs receive /mnt/subvol
>         > At subvol snaps/snap_081012_1715
>         > At subvol snap_081012_1715
>     root@orchard:/bkp# btrfs send -p snaps/snap_081012_1715 \
>                              snaps/snap_081012_1716 | btrfs receive /mnt/subvol
>         > At subvol snaps/snap_081012_1716
>         > At snapshot snap_081012_1716
>
> It certainly seems to work if the receive target is a subvolume other
> than the implicit subvolume at the root of the mount point.
>
> So one can only btrfs receive to a subvolume, which makes sense, and is
> I believe the reason one cannot 'btrfs receive to a subdirectory' as set
> out in the subject of my original email.
>
> Are there any other problem cases you can see, Arne?
>
> Regards
> Rory
>
> --
> Rory Campbell-Lange
> rory@campbell-lange.net
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs receive to subdirectory

[Rory Campbell-Lange]

On 09/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:
> Hi Rory, Arne,
> I think the problem is that currently mnt_fd in struct btrfs_receive
> is used both as "mount root" and "directory in which the
> subvolume/snapshot needs to be created".
> Arne, does the following patch make sense? It uses Jan's
> find_mount_root function. With this patch both Rory's tests seem to
> work for me.

Hi Alex

I haven't had a chance to test this. I'm also on 89fe5b5f of progs
recommended to my by Chris for send/receive testing and your patch
doesn't apply for me. Are you using another revision?

Regards
Rory

-- 
Rory Campbell-Lange
rory@campbell-lange.net

Re: btrfs receive to subdirectory

[Alex Lyakas]

Hi Rory,
thanks for letting me know.
I am using the latest code from Chris's repo (
git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git),
which has been updated recently (latest commit
91d9eec1ff044394f2b98ee7fcb76713dd33b994).
Do you think you can try the following patch on top of this commit? (I
made another small fix).

Basically, the idea is that the path that you give to "btrfs receive"
should be the destination directory, in which the received subvolume
will be created. And this directory should be somewhere under a mount
point, where destination btrfs is mounted (it can also be the same as
the mount point, but not above it). Does this make sense?
Also it should not matter where the sent subvolume resided in the
source fs, and, if this is a diff-send, where you have received the
parent snapshot to (as long as it is on/under the same mount point).

Thanks,
Alex.

diff --git a/cmds-receive.c b/cmds-receive.c
index a8be6fa..6b7cf12 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -52,11 +52,13 @@ static int g_verbose = 0;
 struct btrfs_receive
 {
 	int mnt_fd;
+	int dest_dir_fd;

 	int write_fd;
 	char *write_path;

 	char *root_path;
+	char *dest_dir_path; /* relative to root_path */
 	char *full_subvol_path;

 	struct subvol_info *cur_subvol;
@@ -150,8 +152,11 @@ static int process_subvol(const char *path, const
u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;

-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);

 	fprintf(stderr, "At subvol %s\n", path);

@@ -167,7 +172,7 @@ static int process_subvol(const char *path, const
u8 *uuid, u64 ctransid,

 	memset(&args_v1, 0, sizeof(args_v1));
 	strcpy(args_v1.name, path);
-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
 	if (ret < 0) {
 		ret = -errno;
 		fprintf(stderr, "ERROR: creating subvolume %s failed. "
@@ -195,8 +200,11 @@ static int process_snapshot(const char *path,
const u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;

-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);

 	fprintf(stderr, "At snapshot %s\n", path);

@@ -243,7 +251,7 @@ static int process_snapshot(const char *path,
const u8 *uuid, u64 ctransid,
 		goto out;
 	}

-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
 	close(args_v2.fd);
 	if (ret < 0) {
 		ret = -errno;
@@ -790,17 +798,48 @@ struct btrfs_send_ops send_ops = {
 int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd)
 {
 	int ret;
+	char *dest_dir_full_path;
 	int end = 0;

-	r->root_path = strdup(tomnt);
-	r->mnt_fd = open(tomnt, O_RDONLY | O_NOATIME);
+	dest_dir_full_path = realpath(tomnt, NULL);
+	if (!dest_dir_full_path) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: realpath(%s) failed. %s\n", tomnt,
+				strerror(-ret));
+		goto out;
+	}
+	r->dest_dir_fd = open(dest_dir_full_path, O_RDONLY | O_NOATIME);
+	if (r->dest_dir_fd < 0) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: failed to open destination directory %s. %s\n",
+			    dest_dir_full_path, strerror(-ret));
+		goto out;
+	}
+
+	ret = find_mount_root(dest_dir_full_path, &r->root_path);
+	if (ret < 0) {
+		ret = -EINVAL;
+		fprintf(stderr, "ERROR: failed to determine mount point "
+				"for %s\n", dest_dir_full_path);
+		goto out;
+	}
+	r->mnt_fd = open(r->root_path, O_RDONLY | O_NOATIME);
 	if (r->mnt_fd < 0) {
 		ret = -errno;
-		fprintf(stderr, "ERROR: failed to open %s. %s\n", tomnt,
+		fprintf(stderr, "ERROR: failed to open %s. %s\n", r->root_path,
 				strerror(-ret));
 		goto out;
 	}

+	/*
+	 * find_mount_root returns a root_path that is a subpath of
+	 * dest_dir_full_path. Now get the other part of root_path,
+	 * which is the destination dir relative to root_path.
+	 */
+	r->dest_dir_path = dest_dir_full_path + strlen(r->root_path);
+	if (r->dest_dir_path[0] == '/')
+		r->dest_dir_path++;
+
 	ret = subvol_uuid_search_init(r->mnt_fd, &r->sus);
 	if (ret < 0)
 		return ret;
diff --git a/send-utils.h b/send-utils.h
index da407eb..a3e038b 100644
--- a/send-utils.h
+++ b/send-utils.h
@@ -65,5 +65,6 @@ void subvol_uuid_search_add(struct subvol_uuid_search *s,
 char *path_cat(const char *p1, const char *p2);
 char *path_cat3(const char *p1, const char *p2, const char *p3);

+int find_mount_root(const char *path, char **mount_root);

 #endif /* SEND_UTILS_H_ */




On Thu, Oct 11, 2012 at 12:11 AM, Rory Campbell-Lange
<rory@campbell-lange.net> wrote:
> On 09/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:
>> Hi Rory, Arne,
>> I think the problem is that currently mnt_fd in struct btrfs_receive
>> is used both as "mount root" and "directory in which the
>> subvolume/snapshot needs to be created".
>> Arne, does the following patch make sense? It uses Jan's
>> find_mount_root function. With this patch both Rory's tests seem to
>> work for me.
>
> Hi Alex
>
> I haven't had a chance to test this. I'm also on 89fe5b5f of progs
> recommended to my by Chris for send/receive testing and your patch
> doesn't apply for me. Are you using another revision?
>
> Regards
> Rory
>
> --
> Rory Campbell-Lange
> rory@campbell-lange.net

Re: btrfs receive to subdirectory

[Rory Campbell-Lange]

On 11/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:
> I am using the latest code from Chris's repo (
> git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git),
> which has been updated recently (latest commit
> 91d9eec1ff044394f2b98ee7fcb76713dd33b994).
> Do you think you can try the following patch on top of this commit? (I
> made another small fix).

I had to fix the line wrapping, but your patch applies fine now.

> Basically, the idea is that the path that you give to "btrfs receive"
> should be the destination directory, in which the received subvolume
> will be created. And this directory should be somewhere under a mount
> point, where destination btrfs is mounted (it can also be the same as
> the mount point, but not above it). Does this make sense?
> Also it should not matter where the sent subvolume resided in the
> source fs, and, if this is a diff-send, where you have received the
> parent snapshot to (as long as it is on/under the same mount point).

I don't think the receive pathing works. Please note below the following issues:
    [aa] ERROR: open snapdir/subvol_20121011_1214 failed. No such file or directory
    [bb] ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory
    [cc] ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory

I tested this with the following:

    send mount point /bkp
    receive mount point /mnt

The 'send' mount point has the following structure:

    root@orchard:/bkp# ls
    > snaps  subvol
    (subvol a btrfs subvolume, snaps a directory for holding snapshots)

I then made a directory and two subvolumes in /mnt to receive 'sends':

    root@orchard:/bkp# ls /mnt
    > arbitrary_subvol  snapdir  subvol

I applied your patch to progs 91d9eec1, then did the following rooted in /bkp:

    1. Create snapshots

    root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/subvol_20121011_1214
    > Create a readonly snapshot of 'subvol' in 'snaps/subvol_20121011_1214'

    root@orchard:/bkp# mutt -f subvol/INBOX
    > 1573 kept, 39 deleted.

    root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/subvol_20121011_1216
    > Create a readonly snapshot of 'subvol' in 'snaps/subvol_20121011_1216'

    2. Send base snapshots to /mnt/{snapdir,arbitrary_subvol,subvol}

    root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/snapdir
    > At subvol snaps/subvol_20121011_1214
    > At subvol subvol_20121011_1214

    root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/arbitrary_subvol
    > At subvol snaps/subvol_20121011_1214
    > At subvol subvol_20121011_1214

    root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/subvol
    > At subvol snaps/subvol_20121011_1214
    > At subvol subvol_20121011_1214

    3. Send incremental changes

    root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214 
                       snaps/subvol_20121011_1216/ | btrfs receive /mnt/snapdir
    At subvol snaps/subvol_20121011_1216/
    At snapshot subvol_20121011_1216
    ERROR: open snapdir/subvol_20121011_1214 failed. No such file or directory [aa]

    root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214 
                             snaps/subvol_20121011_1216/ | btrfs receive /mnt/arbitrary_subvol
    At subvol snaps/subvol_20121011_1216/
    At snapshot subvol_20121011_1216
    ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory [bb]

    root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214 
                             snaps/subvol_20121011_1216/ | btrfs receive /mnt/subvol
    At subvol snaps/subvol_20121011_1216/
    At snapshot subvol_20121011_1216
    ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory [cc]


    root@orchard:/bkp# btrfs subvolume list /bkp
    ID 256 gen 9 top level 5 path subvol
    ID 260 gen 7 top level 5 path snaps/subvol_20121011_1214
    ID 261 gen 9 top level 5 path snaps/subvol_20121011_1216

    root@orchard:/bkp# btrfs subvolume list /mnt
    ID 301 gen 66 top level 5 path arbitrary_subvol
    ID 302 gen 69 top level 5 path subvol
    ID 303 gen 66 top level 5 path snapdir/subvol_20121011_1214
    ID 304 gen 69 top level 5 path arbitrary_subvol/subvol_20121011_1214
    ID 305 gen 72 top level 5 path subvol/subvol_20121011_1214



-- 
Rory Campbell-Lange
rory@campbell-lange.net

Re: btrfs receive to subdirectory

[Alex Lyakas]

[-- Attachment #1: Type: text/plain, Size: 5545 bytes --]

Hi Rory,
all your tests work perfectly for me (really, I swear:)), with exact
same mount points and subvol/snap/dir names.
Just to make that the patch some how did not get messed up in the
mail, I attach it.

Can you pls post the output of "cat /proc/mounts" and "mount" (if
you're willing to debug this further).

Other than that, I will probably need to see the contents of your
source subvol (at least file names, not necessarily the content). So
if you can expose it, can you perhaps do something like:
tree -A --inodes --noreport -s /bkp/subvol
and also
tree -A --inodes --noreport -s /mnt
before you start the test (I assume that two subvolumes at /mnt are
empty, right).

However, I doubt the contents will really help, because it seems to
fail very early, even before it starts processing the commands.

Thanks,
Alex.


On Thu, Oct 11, 2012 at 1:33 PM, Rory Campbell-Lange
<rory@campbell-lange.net> wrote:
> On 11/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:
>> I am using the latest code from Chris's repo (
>> git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git),
>> which has been updated recently (latest commit
>> 91d9eec1ff044394f2b98ee7fcb76713dd33b994).
>> Do you think you can try the following patch on top of this commit? (I
>> made another small fix).
>
> I had to fix the line wrapping, but your patch applies fine now.
>
>> Basically, the idea is that the path that you give to "btrfs receive"
>> should be the destination directory, in which the received subvolume
>> will be created. And this directory should be somewhere under a mount
>> point, where destination btrfs is mounted (it can also be the same as
>> the mount point, but not above it). Does this make sense?
>> Also it should not matter where the sent subvolume resided in the
>> source fs, and, if this is a diff-send, where you have received the
>> parent snapshot to (as long as it is on/under the same mount point).
>
> I don't think the receive pathing works. Please note below the following issues:
>     [aa] ERROR: open snapdir/subvol_20121011_1214 failed. No such file or directory
>     [bb] ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory
>     [cc] ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory
>
> I tested this with the following:
>
>     send mount point /bkp
>     receive mount point /mnt
>
> The 'send' mount point has the following structure:
>
>     root@orchard:/bkp# ls
>     > snaps  subvol
>     (subvol a btrfs subvolume, snaps a directory for holding snapshots)
>
> I then made a directory and two subvolumes in /mnt to receive 'sends':
>
>     root@orchard:/bkp# ls /mnt
>     > arbitrary_subvol  snapdir  subvol
>
> I applied your patch to progs 91d9eec1, then did the following rooted in /bkp:
>
>     1. Create snapshots
>
>     root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/subvol_20121011_1214
>     > Create a readonly snapshot of 'subvol' in 'snaps/subvol_20121011_1214'
>
>     root@orchard:/bkp# mutt -f subvol/INBOX
>     > 1573 kept, 39 deleted.
>
>     root@orchard:/bkp# btrfs subvolume snapshot -r subvol snaps/subvol_20121011_1216
>     > Create a readonly snapshot of 'subvol' in 'snaps/subvol_20121011_1216'
>
>     2. Send base snapshots to /mnt/{snapdir,arbitrary_subvol,subvol}
>
>     root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/snapdir
>     > At subvol snaps/subvol_20121011_1214
>     > At subvol subvol_20121011_1214
>
>     root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/arbitrary_subvol
>     > At subvol snaps/subvol_20121011_1214
>     > At subvol subvol_20121011_1214
>
>     root@orchard:/bkp# btrfs send snaps/subvol_20121011_1214 | btrfs receive /mnt/subvol
>     > At subvol snaps/subvol_20121011_1214
>     > At subvol subvol_20121011_1214
>
>     3. Send incremental changes
>
>     root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214
>                        snaps/subvol_20121011_1216/ | btrfs receive /mnt/snapdir
>     At subvol snaps/subvol_20121011_1216/
>     At snapshot subvol_20121011_1216
>     ERROR: open snapdir/subvol_20121011_1214 failed. No such file or directory [aa]
>
>     root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214
>                              snaps/subvol_20121011_1216/ | btrfs receive /mnt/arbitrary_subvol
>     At subvol snaps/subvol_20121011_1216/
>     At snapshot subvol_20121011_1216
>     ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory [bb]
>
>     root@orchard:/bkp# btrfs send -p snaps/subvol_20121011_1214
>                              snaps/subvol_20121011_1216/ | btrfs receive /mnt/subvol
>     At subvol snaps/subvol_20121011_1216/
>     At snapshot subvol_20121011_1216
>     ERROR: open <FS_TREE>/snapdir/subvol_20121011_1214 failed. No such file or directory [cc]
>
>
>     root@orchard:/bkp# btrfs subvolume list /bkp
>     ID 256 gen 9 top level 5 path subvol
>     ID 260 gen 7 top level 5 path snaps/subvol_20121011_1214
>     ID 261 gen 9 top level 5 path snaps/subvol_20121011_1216
>
>     root@orchard:/bkp# btrfs subvolume list /mnt
>     ID 301 gen 66 top level 5 path arbitrary_subvol
>     ID 302 gen 69 top level 5 path subvol
>     ID 303 gen 66 top level 5 path snapdir/subvol_20121011_1214
>     ID 304 gen 69 top level 5 path arbitrary_subvol/subvol_20121011_1214
>     ID 305 gen 72 top level 5 path subvol/subvol_20121011_1214
>
>
>
> --
> Rory Campbell-Lange
> rory@campbell-lange.net

[-- Attachment #2: rory.patch --]
[-- Type: application/octet-stream, Size: 4121 bytes --]

diff --git a/cmds-receive.c b/cmds-receive.c
index a8be6fa..6b7cf12 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -52,11 +52,13 @@ static int g_verbose = 0;
 struct btrfs_receive
 {
 	int mnt_fd;
+	int dest_dir_fd;
 
 	int write_fd;
 	char *write_path;
 
 	char *root_path;
+	char *dest_dir_path; /* relative to root_path */
 	char *full_subvol_path;
 
 	struct subvol_info *cur_subvol;
@@ -150,8 +152,11 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;
 
-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);
 
 	fprintf(stderr, "At subvol %s\n", path);
 
@@ -167,7 +172,7 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid,
 
 	memset(&args_v1, 0, sizeof(args_v1));
 	strcpy(args_v1.name, path);
-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
 	if (ret < 0) {
 		ret = -errno;
 		fprintf(stderr, "ERROR: creating subvolume %s failed. "
@@ -195,8 +200,11 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;
 
-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);
 
 	fprintf(stderr, "At snapshot %s\n", path);
 
@@ -243,7 +251,7 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,
 		goto out;
 	}
 
-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
 	close(args_v2.fd);
 	if (ret < 0) {
 		ret = -errno;
@@ -790,17 +798,48 @@ struct btrfs_send_ops send_ops = {
 int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd)
 {
 	int ret;
+	char *dest_dir_full_path;
 	int end = 0;
 
-	r->root_path = strdup(tomnt);
-	r->mnt_fd = open(tomnt, O_RDONLY | O_NOATIME);
+	dest_dir_full_path = realpath(tomnt, NULL);
+	if (!dest_dir_full_path) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: realpath(%s) failed. %s\n", tomnt,
+				strerror(-ret));
+		goto out;
+	}
+	r->dest_dir_fd = open(dest_dir_full_path, O_RDONLY | O_NOATIME);
+	if (r->dest_dir_fd < 0) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: failed to open destination directory %s. %s\n", 
+			    dest_dir_full_path, strerror(-ret));
+		goto out;
+	}
+
+	ret = find_mount_root(dest_dir_full_path, &r->root_path);
+	if (ret < 0) {
+		ret = -EINVAL;
+		fprintf(stderr, "ERROR: failed to determine mount point "
+				"for %s\n", dest_dir_full_path);
+		goto out;
+	}
+	r->mnt_fd = open(r->root_path, O_RDONLY | O_NOATIME);
 	if (r->mnt_fd < 0) {
 		ret = -errno;
-		fprintf(stderr, "ERROR: failed to open %s. %s\n", tomnt,
+		fprintf(stderr, "ERROR: failed to open %s. %s\n", r->root_path,
 				strerror(-ret));
 		goto out;
 	}
 
+	/*
+	 * find_mount_root returns a root_path that is a subpath of 
+	 * dest_dir_full_path. Now get the other part of root_path,
+	 * which is the destination dir relative to root_path.
+	 */
+	r->dest_dir_path = dest_dir_full_path + strlen(r->root_path);
+	if (r->dest_dir_path[0] == '/')
+		r->dest_dir_path++;
+
 	ret = subvol_uuid_search_init(r->mnt_fd, &r->sus);
 	if (ret < 0)
 		return ret;
diff --git a/send-utils.h b/send-utils.h
index da407eb..a3e038b 100644
--- a/send-utils.h
+++ b/send-utils.h
@@ -65,5 +65,6 @@ void subvol_uuid_search_add(struct subvol_uuid_search *s,
 char *path_cat(const char *p1, const char *p2);
 char *path_cat3(const char *p1, const char *p2, const char *p3);
 
+int find_mount_root(const char *path, char **mount_root);
 
 #endif /* SEND_UTILS_H_ */

Re: btrfs receive to subdirectory

[Rory Campbell-Lange]

On 11/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:
> Hi Rory,
> all your tests work perfectly for me (really, I swear:)), with exact
> same mount points and subvol/snap/dir names.
> Just to make that the patch some how did not get messed up in the
> mail, I attach it.

... happy to try things further. Please confirm your btrfs/kernel
version. I'm on 

    5c4e935f44e1aace28988fdc06f233ef5c184103 refs/heads/send-recv

Cheers
Rory

-- 
Rory Campbell-Lange
rory@campbell-lange.net

Campbell-Lange Workshop
www.campbell-lange.net
0207 6311 555
3 Tottenham Street London W1T 2AF
Registered in England No. 04551928

Re: btrfs receive to subdirectory

[Alex Lyakas]

Hi Rory,
my kernel is slightly different, but the send-receive kernel code that
I use is on the same commit. However, the issue you are hitting is
most probably not a kernel issue. The receive part works totally in
user-space. To confirm that, maybe you can post somewhere the full-
and diff-streams that you generate: instead of piping to "btrfs
receive", pls redirect it into some file (like /tmp/full_stream and
/tmp/diff_stream) and post the files. Not sure how sensitive is the
content of your files or how big the stream is - it might be not
possible for you to do this.

I suspect there is some issue with finding subvolume paths relative to
your mount root.
If possible, pls post "cat /proc/mounts" and also the output of "mount" command.

Finally, it might be useful to post strace outputs of the failing
receives. To do this, please redirect the stream into some file first,
and then receive from this file (and not in one step like you are
doing) under strace.

Next step, I will probably send you a patch with some prints. Hope in
a few iterations we can find the issue (unless you give up:).

Thanks,
Alex.


On Thu, Oct 11, 2012 at 4:56 PM, Rory Campbell-Lange
<rory@campbell-lange.net> wrote:
> On 11/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:
>> Hi Rory,
>> all your tests work perfectly for me (really, I swear:)), with exact
>> same mount points and subvol/snap/dir names.
>> Just to make that the patch some how did not get messed up in the
>> mail, I attach it.
>
> ... happy to try things further. Please confirm your btrfs/kernel
> version. I'm on
>
>     5c4e935f44e1aace28988fdc06f233ef5c184103 refs/heads/send-recv
>
> Cheers
> Rory
>
> --
> Rory Campbell-Lange
> rory@campbell-lange.net
>
> Campbell-Lange Workshop
> www.campbell-lange.net
> 0207 6311 555
> 3 Tottenham Street London W1T 2AF
> Registered in England No. 04551928

Re: btrfs receive to subdirectory

[Alex Lyakas]

[-- Attachment #1: Type: text/plain, Size: 1200 bytes --]

Hi Rory,
can you pls check if you have a file "/etc/mtab" on your machine? If
not or if it's empty or something like, then this probably explains
the issue. In this case, pls apply patch2 that is attached (instead of
previous patches). Otherwise, pls try with patch1 (instead of
previous) and then with patch2 (instead of any previous). Both will do
some amount of prints, pls post them. With both patches your tests
work fine for me.

Thanks,
Alex.



On Thu, Oct 11, 2012 at 4:56 PM, Rory Campbell-Lange
<rory@campbell-lange.net> wrote:
> On 11/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:
>> Hi Rory,
>> all your tests work perfectly for me (really, I swear:)), with exact
>> same mount points and subvol/snap/dir names.
>> Just to make that the patch some how did not get messed up in the
>> mail, I attach it.
>
> ... happy to try things further. Please confirm your btrfs/kernel
> version. I'm on
>
>     5c4e935f44e1aace28988fdc06f233ef5c184103 refs/heads/send-recv
>
> Cheers
> Rory
>
> --
> Rory Campbell-Lange
> rory@campbell-lange.net
>
> Campbell-Lange Workshop
> www.campbell-lange.net
> 0207 6311 555
> 3 Tottenham Street London W1T 2AF
> Registered in England No. 04551928

[-- Attachment #2: r.prints.patch1 --]
[-- Type: application/octet-stream, Size: 5675 bytes --]

diff --git a/cmds-receive.c b/cmds-receive.c
index a8be6fa..ab9483a 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -52,11 +52,13 @@ static int g_verbose = 0;
 struct btrfs_receive
 {
 	int mnt_fd;
+	int dest_dir_fd;
 
 	int write_fd;
 	char *write_path;
 
 	char *root_path;
+	char *dest_dir_path; /* relative to root_path */
 	char *full_subvol_path;
 
 	struct subvol_info *cur_subvol;
@@ -150,8 +152,13 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;
 
-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);
+	fprintf(stderr, "process_subvol: r->cur_subvol->path=%s\n", r->cur_subvol->path);
+	fprintf(stderr, "process_subvol: r->full_subvol_path=%s\n", r->full_subvol_path);
 
 	fprintf(stderr, "At subvol %s\n", path);
 
@@ -167,7 +174,7 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid,
 
 	memset(&args_v1, 0, sizeof(args_v1));
 	strcpy(args_v1.name, path);
-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
 	if (ret < 0) {
 		ret = -errno;
 		fprintf(stderr, "ERROR: creating subvolume %s failed. "
@@ -195,8 +202,13 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;
 
-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);
+	fprintf(stderr, "process_snapshot: r->cur_subvol->path=%s\n", r->cur_subvol->path);
+	fprintf(stderr, "process_snapshot: r->full_subvol_path=%s\n", r->full_subvol_path);
 
 	fprintf(stderr, "At snapshot %s\n", path);
 
@@ -243,7 +255,7 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,
 		goto out;
 	}
 
-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
 	close(args_v2.fd);
 	if (ret < 0) {
 		ret = -errno;
@@ -790,17 +802,52 @@ struct btrfs_send_ops send_ops = {
 int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd)
 {
 	int ret;
+	char *dest_dir_full_path;
 	int end = 0;
 
-	r->root_path = strdup(tomnt);
-	r->mnt_fd = open(tomnt, O_RDONLY | O_NOATIME);
+	fprintf(stderr, "tomnt=%s\n", tomnt);
+	dest_dir_full_path = realpath(tomnt, NULL);
+	if (!dest_dir_full_path) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: realpath(%s) failed. %s\n", tomnt,
+				strerror(-ret));
+		goto out;
+	}
+	fprintf(stderr, "dest_dir_full_path=%s\n", dest_dir_full_path);
+	r->dest_dir_fd = open(dest_dir_full_path, O_RDONLY | O_NOATIME);
+	if (r->dest_dir_fd < 0) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: failed to open destination directory %s. %s\n", 
+			    dest_dir_full_path, strerror(-ret));
+		goto out;
+	}
+
+	ret = find_mount_root(dest_dir_full_path, &r->root_path);
+	if (ret < 0) {
+		ret = -EINVAL;
+		fprintf(stderr, "ERROR: failed to determine mount point "
+				"for %s\n", dest_dir_full_path);
+		goto out;
+	}
+	fprintf(stderr, "r->root_path=%s\n", r->root_path);
+	r->mnt_fd = open(r->root_path, O_RDONLY | O_NOATIME);
 	if (r->mnt_fd < 0) {
 		ret = -errno;
-		fprintf(stderr, "ERROR: failed to open %s. %s\n", tomnt,
+		fprintf(stderr, "ERROR: failed to open %s. %s\n", r->root_path,
 				strerror(-ret));
 		goto out;
 	}
 
+	/*
+	 * find_mount_root returns a root_path that is a subpath of 
+	 * dest_dir_full_path. Now get the other part of root_path,
+	 * which is the destination dir relative to root_path.
+	 */
+	r->dest_dir_path = dest_dir_full_path + strlen(r->root_path);
+	if (r->dest_dir_path[0] == '/')
+		r->dest_dir_path++;
+	fprintf(stderr, "r->dest_dir_path=%s\n", r->dest_dir_path);
+
 	ret = subvol_uuid_search_init(r->mnt_fd, &r->sus);
 	if (ret < 0)
 		return ret;
diff --git a/cmds-send.c b/cmds-send.c
index 9b47e70..69b124a 100644
--- a/cmds-send.c
+++ b/cmds-send.c
@@ -70,13 +70,16 @@ int find_mount_root(const char *path, char **mount_root)
 
 	mnttab = fopen("/etc/mtab", "r");
 	while ((ent = getmntent(mnttab))) {
+		fprintf(stderr, "find_mount_root: looking at ent->mnt_dir=%s\n", ent->mnt_dir);
 		len = strlen(ent->mnt_dir);
 		if (strncmp(ent->mnt_dir, path, len) == 0) {
 			/* match found */
+			fprintf(stderr, "find_mount_root: ent->mnt_dir=%s is a match\n", ent->mnt_dir);
 			if (longest_matchlen < len) {
 				free(longest_match);
 				longest_matchlen = len;
 				longest_match = strdup(ent->mnt_dir);
+				fprintf(stderr, "find_mount_root: longest_match=%s\n", longest_match);
 			}
 		}
 	}
@@ -84,6 +87,8 @@ int find_mount_root(const char *path, char **mount_root)
 	*mount_root = realpath(longest_match, NULL);
 	free(longest_match);
 
+	fprintf(stderr, "find_mount_root: resolved mount_root=%s\n", *mount_root);
+
 	return 0;
 }
 
diff --git a/send-utils.h b/send-utils.h
index da407eb..a3e038b 100644
--- a/send-utils.h
+++ b/send-utils.h
@@ -65,5 +65,6 @@ void subvol_uuid_search_add(struct subvol_uuid_search *s,
 char *path_cat(const char *p1, const char *p2);
 char *path_cat3(const char *p1, const char *p2, const char *p3);
 
+int find_mount_root(const char *path, char **mount_root);
 
 #endif /* SEND_UTILS_H_ */

[-- Attachment #3: r.prints.patch2 --]
[-- Type: application/octet-stream, Size: 5745 bytes --]

diff --git a/cmds-receive.c b/cmds-receive.c
index a8be6fa..ab9483a 100644
--- a/cmds-receive.c
+++ b/cmds-receive.c
@@ -52,11 +52,13 @@ static int g_verbose = 0;
 struct btrfs_receive
 {
 	int mnt_fd;
+	int dest_dir_fd;
 
 	int write_fd;
 	char *write_path;
 
 	char *root_path;
+	char *dest_dir_path; /* relative to root_path */
 	char *full_subvol_path;
 
 	struct subvol_info *cur_subvol;
@@ -150,8 +152,13 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;
 
-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);
+	fprintf(stderr, "process_subvol: r->cur_subvol->path=%s\n", r->cur_subvol->path);
+	fprintf(stderr, "process_subvol: r->full_subvol_path=%s\n", r->full_subvol_path);
 
 	fprintf(stderr, "At subvol %s\n", path);
 
@@ -167,7 +174,7 @@ static int process_subvol(const char *path, const u8 *uuid, u64 ctransid,
 
 	memset(&args_v1, 0, sizeof(args_v1));
 	strcpy(args_v1.name, path);
-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SUBVOL_CREATE, &args_v1);
 	if (ret < 0) {
 		ret = -errno;
 		fprintf(stderr, "ERROR: creating subvolume %s failed. "
@@ -195,8 +202,13 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,
 	r->cur_subvol = calloc(1, sizeof(*r->cur_subvol));
 	r->parent_subvol = NULL;
 
-	r->cur_subvol->path = strdup(path);
-	r->full_subvol_path = path_cat(r->root_path, path);
+	if (strlen(r->dest_dir_path) == 0)
+		r->cur_subvol->path = strdup(path);
+	else
+		r->cur_subvol->path = path_cat(r->dest_dir_path, path);
+	r->full_subvol_path = path_cat3(r->root_path, r->dest_dir_path, path);
+	fprintf(stderr, "process_snapshot: r->cur_subvol->path=%s\n", r->cur_subvol->path);
+	fprintf(stderr, "process_snapshot: r->full_subvol_path=%s\n", r->full_subvol_path);
 
 	fprintf(stderr, "At snapshot %s\n", path);
 
@@ -243,7 +255,7 @@ static int process_snapshot(const char *path, const u8 *uuid, u64 ctransid,
 		goto out;
 	}
 
-	ret = ioctl(r->mnt_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
+	ret = ioctl(r->dest_dir_fd, BTRFS_IOC_SNAP_CREATE_V2, &args_v2);
 	close(args_v2.fd);
 	if (ret < 0) {
 		ret = -errno;
@@ -790,17 +802,52 @@ struct btrfs_send_ops send_ops = {
 int do_receive(struct btrfs_receive *r, const char *tomnt, int r_fd)
 {
 	int ret;
+	char *dest_dir_full_path;
 	int end = 0;
 
-	r->root_path = strdup(tomnt);
-	r->mnt_fd = open(tomnt, O_RDONLY | O_NOATIME);
+	fprintf(stderr, "tomnt=%s\n", tomnt);
+	dest_dir_full_path = realpath(tomnt, NULL);
+	if (!dest_dir_full_path) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: realpath(%s) failed. %s\n", tomnt,
+				strerror(-ret));
+		goto out;
+	}
+	fprintf(stderr, "dest_dir_full_path=%s\n", dest_dir_full_path);
+	r->dest_dir_fd = open(dest_dir_full_path, O_RDONLY | O_NOATIME);
+	if (r->dest_dir_fd < 0) {
+		ret = -errno;
+		fprintf(stderr, "ERROR: failed to open destination directory %s. %s\n", 
+			    dest_dir_full_path, strerror(-ret));
+		goto out;
+	}
+
+	ret = find_mount_root(dest_dir_full_path, &r->root_path);
+	if (ret < 0) {
+		ret = -EINVAL;
+		fprintf(stderr, "ERROR: failed to determine mount point "
+				"for %s\n", dest_dir_full_path);
+		goto out;
+	}
+	fprintf(stderr, "r->root_path=%s\n", r->root_path);
+	r->mnt_fd = open(r->root_path, O_RDONLY | O_NOATIME);
 	if (r->mnt_fd < 0) {
 		ret = -errno;
-		fprintf(stderr, "ERROR: failed to open %s. %s\n", tomnt,
+		fprintf(stderr, "ERROR: failed to open %s. %s\n", r->root_path,
 				strerror(-ret));
 		goto out;
 	}
 
+	/*
+	 * find_mount_root returns a root_path that is a subpath of 
+	 * dest_dir_full_path. Now get the other part of root_path,
+	 * which is the destination dir relative to root_path.
+	 */
+	r->dest_dir_path = dest_dir_full_path + strlen(r->root_path);
+	if (r->dest_dir_path[0] == '/')
+		r->dest_dir_path++;
+	fprintf(stderr, "r->dest_dir_path=%s\n", r->dest_dir_path);
+
 	ret = subvol_uuid_search_init(r->mnt_fd, &r->sus);
 	if (ret < 0)
 		return ret;
diff --git a/cmds-send.c b/cmds-send.c
index 9b47e70..3b89bc8 100644
--- a/cmds-send.c
+++ b/cmds-send.c
@@ -68,15 +68,18 @@ int find_mount_root(const char *path, char **mount_root)
 		return -errno;
 	close(fd);
 
-	mnttab = fopen("/etc/mtab", "r");
+	mnttab = fopen("/proc/mounts", "r");
 	while ((ent = getmntent(mnttab))) {
+		fprintf(stderr, "find_mount_root: looking at ent->mnt_dir=%s\n", ent->mnt_dir);
 		len = strlen(ent->mnt_dir);
 		if (strncmp(ent->mnt_dir, path, len) == 0) {
 			/* match found */
+			fprintf(stderr, "find_mount_root: ent->mnt_dir=%s is a match\n", ent->mnt_dir);
 			if (longest_matchlen < len) {
 				free(longest_match);
 				longest_matchlen = len;
 				longest_match = strdup(ent->mnt_dir);
+				fprintf(stderr, "find_mount_root: longest_match=%s\n", longest_match);
 			}
 		}
 	}
@@ -84,6 +87,8 @@ int find_mount_root(const char *path, char **mount_root)
 	*mount_root = realpath(longest_match, NULL);
 	free(longest_match);
 
+	fprintf(stderr, "find_mount_root: resolved mount_root=%s\n", *mount_root);
+
 	return 0;
 }
 
diff --git a/send-utils.h b/send-utils.h
index da407eb..a3e038b 100644
--- a/send-utils.h
+++ b/send-utils.h
@@ -65,5 +65,6 @@ void subvol_uuid_search_add(struct subvol_uuid_search *s,
 char *path_cat(const char *p1, const char *p2);
 char *path_cat3(const char *p1, const char *p2, const char *p3);
 
+int find_mount_root(const char *path, char **mount_root);
 
 #endif /* SEND_UTILS_H_ */

Re: btrfs receive to subdirectory

[Rory Campbell-Lange]

On 12/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:
> Hi Rory,
> can you pls check if you have a file "/etc/mtab" on your machine? If
> not or if it's empty or something like, then this probably explains
> the issue. In this case, pls apply patch2 that is attached (instead of
> previous patches). Otherwise, pls try with patch1 (instead of
> previous) and then with patch2 (instead of any previous). Both will do
> some amount of prints, pls post them. With both patches your tests
> work fine for me.

Hi Alex

I reset everything and remade the btrfs partitions. I re-pulled 
git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
and applied your first patch.

  (in /bkp)
  mkdir snaps
  btrfs snapshot -r mail snaps/snap1
  btrfs subvolume snapshot -r mail snaps/snap1
  mutt -f mail/INBOX/
  btrfs subvolume snapshot -r mail snaps/snap2

  # targets (dir, subvol with same name, subvol with different name)
  mkdir /mnt/snaps
  btrfs subvolume create /mnt/mail
  btrfs subvolume create /mnt/snapmail
  # base backup
  btrfs send snaps/snap1 | btrfs receive /mnt/snaps 
  btrfs send snaps/snap1 | btrfs receive /mnt/mail
  btrfs send snaps/snap1 | btrfs receive /mnt/snapmail
  # update snapshots
  btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/snaps
  btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/mail
  btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/snapmail

All worked fine. Sorry for the previous errors, which look to be
invalid.

Cheers
Rory


/etc/mtab:

    rootfs / rootfs rw 0 0
    sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
    proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
    udev /dev devtmpfs rw,relatime,size=10240k,nr_inodes=749414,mode=755 0 0
    devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
    tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=600748k,mode=755 0 0
    /dev/disk/by-uuid/6e899e2d-c401-4a92-8adf-a097bcaf2148 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
    tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
    tmpfs /run/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=1396860k 0 0
    /dev/sda1 /boot ext4 rw,relatime,data=ordered 0 0
    rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
    /dev/sdb1 /mnt btrfs rw,relatime,space_cache 0 0
    /dev/sda6 /bkp btrfs rw,relatime,space_cache 0 0

-- 
Rory Campbell-Lange
rory@campbell-lange.net

Re: btrfs receive to subdirectory

[Alex Lyakas]

Rory,
thanks for confirming. I will put together a more formal patch to the list.

Thanks,
Alex.


On Fri, Oct 12, 2012 at 6:10 PM, Rory Campbell-Lange
<rory@campbell-lange.net> wrote:
> On 12/10/12, Alex Lyakas (alex.btrfs@zadarastorage.com) wrote:
>> Hi Rory,
>> can you pls check if you have a file "/etc/mtab" on your machine? If
>> not or if it's empty or something like, then this probably explains
>> the issue. In this case, pls apply patch2 that is attached (instead of
>> previous patches). Otherwise, pls try with patch1 (instead of
>> previous) and then with patch2 (instead of any previous). Both will do
>> some amount of prints, pls post them. With both patches your tests
>> work fine for me.
>
> Hi Alex
>
> I reset everything and remade the btrfs partitions. I re-pulled
> git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
> and applied your first patch.
>
>   (in /bkp)
>   mkdir snaps
>   btrfs snapshot -r mail snaps/snap1
>   btrfs subvolume snapshot -r mail snaps/snap1
>   mutt -f mail/INBOX/
>   btrfs subvolume snapshot -r mail snaps/snap2
>
>   # targets (dir, subvol with same name, subvol with different name)
>   mkdir /mnt/snaps
>   btrfs subvolume create /mnt/mail
>   btrfs subvolume create /mnt/snapmail
>   # base backup
>   btrfs send snaps/snap1 | btrfs receive /mnt/snaps
>   btrfs send snaps/snap1 | btrfs receive /mnt/mail
>   btrfs send snaps/snap1 | btrfs receive /mnt/snapmail
>   # update snapshots
>   btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/snaps
>   btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/mail
>   btrfs send -p snaps/snap1 snaps/snap2 | btrfs receive /mnt/snapmail
>
> All worked fine. Sorry for the previous errors, which look to be
> invalid.
>
> Cheers
> Rory
>
>
> /etc/mtab:
>
>     rootfs / rootfs rw 0 0
>     sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
>     proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
>     udev /dev devtmpfs rw,relatime,size=10240k,nr_inodes=749414,mode=755 0 0
>     devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
>     tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=600748k,mode=755 0 0
>     /dev/disk/by-uuid/6e899e2d-c401-4a92-8adf-a097bcaf2148 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
>     tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
>     tmpfs /run/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=1396860k 0 0
>     /dev/sda1 /boot ext4 rw,relatime,data=ordered 0 0
>     rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
>     /dev/sdb1 /mnt btrfs rw,relatime,space_cache 0 0
>     /dev/sda6 /bkp btrfs rw,relatime,space_cache 0 0
>
> --
> Rory Campbell-Lange
> rory@campbell-lange.net