All of lore.kernel.org
 help / color / mirror / Atom feed
* Set nodatacow per file?
@ 2012-02-13  7:17 Ralf-Peter Rohbeck
  2012-02-13  7:34 ` Chester
                   ` (2 more replies)
  0 siblings, 3 replies; 24+ messages in thread
From: Ralf-Peter Rohbeck @ 2012-02-13  7:17 UTC (permalink / raw)
  To: linux-btrfs

Hello,
is it possible to set nodatacow on a per-file basis? I couldn't find 
anything.
If not, wouldn't that be a great feature to get around the performance 
issues with VM and database storage? Of course cloning should still 
cause COW.

Thanks,
Ralf-Peter

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13  7:17 Set nodatacow per file? Ralf-Peter Rohbeck
@ 2012-02-13  7:34 ` Chester
  2012-02-13  8:10   ` Liu Bo
  2012-02-13  7:40 ` dima
  2012-02-24  5:22 ` dima
  2 siblings, 1 reply; 24+ messages in thread
From: Chester @ 2012-02-13  7:34 UTC (permalink / raw)
  To: Ralf-Peter Rohbeck; +Cc: linux-btrfs

On Mon, Feb 13, 2012 at 1:17 AM, Ralf-Peter Rohbeck <rohbeck@yahoo.com>=
 wrote:
>
> Hello,
> is it possible to set nodatacow on a per-file basis? I couldn't find =
anything.
> If not, wouldn't that be a great feature to get around the performanc=
e issues with VM and database storage? Of course cloning should still c=
ause COW.

IIRC this is already a feature in btrfs. I didn't catch the whole
talk, but Chris mentioned something like this at Scale 10x. I also
remember seeing a patch for it a while back (I think it was from liu
bo) that does this.

>
> Thanks,
> Ralf-Peter
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13  7:17 Set nodatacow per file? Ralf-Peter Rohbeck
  2012-02-13  7:34 ` Chester
@ 2012-02-13  7:40 ` dima
  2012-02-13  8:09   ` Roman Mamedov
  2012-02-13 14:10   ` David Sterba
  2012-02-24  5:22 ` dima
  2 siblings, 2 replies; 24+ messages in thread
From: dima @ 2012-02-13  7:40 UTC (permalink / raw)
  To: linux-btrfs

Hello Ralf-Peter,

Actually it is possible. Check out David's response to my question from 
some time ago:
http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227

The nocow.c script he attached does just the thing you want. The script 
is really working. I needed nocow for different purpose but it did not 
occur to me to try it on VM image and see if the performance would 
improve. Sounds like a great idea. If you get around to try it, pls. 
post your impressions here.

best
~dima



On 02/13/2012 04:17 PM, Ralf-Peter Rohbeck wrote:
> Hello,
> is it possible to set nodatacow on a per-file basis? I couldn't find
> anything.
> If not, wouldn't that be a great feature to get around the performance
> issues with VM and database storage? Of course cloning should still
> cause COW.
>
> Thanks,
> Ralf-Peter
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13  7:40 ` dima
@ 2012-02-13  8:09   ` Roman Mamedov
  2012-02-13  8:40     ` dima
  2012-02-13 13:42     ` dima
  2012-02-13 14:10   ` David Sterba
  1 sibling, 2 replies; 24+ messages in thread
From: Roman Mamedov @ 2012-02-13  8:09 UTC (permalink / raw)
  To: dima; +Cc: linux-btrfs


[-- Attachment #1.1: Type: text/plain, Size: 1104 bytes --]

On Mon, 13 Feb 2012 16:40:03 +0900
dima <dolenin@parallels.com> wrote:

> Hello Ralf-Peter,
> 
> Actually it is possible. Check out David's response to my question from 
> some time ago:
> http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227
> 
> The nocow.c script he attached does just the thing you want. The script 
> is really working. I needed nocow for different purpose but it did not 
> occur to me to try it on VM image and see if the performance would 
> improve. Sounds like a great idea. If you get around to try it, pls. 
> post your impressions here.

Thanks for the link, this is indeed interesting.

I made a couple of small changes, i.e. I wanted a way to unset nocow and to
check that changing flags really worked.

gcc -o /usr/local/bin/nocow nocow.c
ln -sf /usr/local/bin/nocow /usr/local/bin/cow

Perhaps the support for setting this flag should be added to the 'btrfs'
utility.

-- 
With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: nocow.c --]
[-- Type: text/x-csrc, Size: 1622 bytes --]

#include <fcntl.h>
#include <stdlib.h>
#include <sys/ioctl.h>
#include <string.h>

#include <linux/types.h>
#include <stdio.h>
#include <errno.h>

#ifndef FS_IOC_SETFLAGS
#define FS_IOC_SETFLAGS                 _IOW('f', 2, long)
#warning defining SETFLAGS locally
#endif

#ifndef FS_IOC_GETFLAGS
#define FS_IOC_GETFLAGS                 _IOR('f', 1, long)
#warning defining GETFLAGS locally
#endif

#ifndef FS_NOCOW_FL
#define FS_NOCOW_FL                     0x00800000 /* Do not cow file */
#endif

int main(int argc, char **argv)
{
        int fd;
        int r;
  long flags;

  if (argc < 2) {
    printf("usage: %s file\n", argv[0]);
    exit(1);
  }

  fd = open(argv[1], O_RDONLY);
  if (fd == -1) {
    perror("open()");
    return 1;
  }
  printf("GETFLAGS ioctl 0x%x\n", FS_IOC_GETFLAGS);
  printf("SETFLAGS ioctl 0x%x\n", FS_IOC_SETFLAGS);

  r = ioctl(fd, FS_IOC_GETFLAGS, &flags);
  if (r == -1) {
    perror("ioctl(GETFLAGS)");
    return 1;
  } else {
    printf("file flags before: 0x%lx\n", flags);
  }

  if(strcmp(argv[0], "cow")==0) {
    printf("Remove NOCOW flag for %s\n", argv[1]);
    flags ^= FS_NOCOW_FL;
  } else {
    printf("Set NOCOW flag for %s\n", argv[1]);    
    flags |= FS_NOCOW_FL;
  }
  r = ioctl(fd, FS_IOC_SETFLAGS, &flags);
  printf("ioctl returned: %d\n", r);
  if (r == -1) {
    perror("ioctl()");
    return 1;
  }

  r = ioctl(fd, FS_IOC_GETFLAGS, &flags);
  if (r == -1) {
    perror("ioctl(GETFLAGS)");
    return 1;
  } else {
    printf("file flags after:  0x%lx\n", flags);
  }  

  return 0;
}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13  7:34 ` Chester
@ 2012-02-13  8:10   ` Liu Bo
  0 siblings, 0 replies; 24+ messages in thread
From: Liu Bo @ 2012-02-13  8:10 UTC (permalink / raw)
  To: Chester; +Cc: Ralf-Peter Rohbeck, linux-btrfs

On 02/13/2012 03:34 PM, Chester wrote:
> On Mon, Feb 13, 2012 at 1:17 AM, Ralf-Peter Rohbeck <rohbeck@yahoo.com> wrote:
>> Hello,
>> is it possible to set nodatacow on a per-file basis? I couldn't find anything.
>> If not, wouldn't that be a great feature to get around the performance issues with VM and database storage? Of course cloning should still cause COW.
> 
> IIRC this is already a feature in btrfs. I didn't catch the whole
> talk, but Chris mentioned something like this at Scale 10x. I also
> remember seeing a patch for it a while back (I think it was from liu
> bo) that does this.
> 

You're right, and I've made the prog patches which is against linux-ulit(chattr/lsattr):

http://www.spinics.net/lists/linux-btrfs/msg09604.html
http://www.spinics.net/lists/linux-btrfs/msg09605.html

but they are not merged yet.

thanks,
liubo

>> Thanks,
>> Ralf-Peter
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13  8:09   ` Roman Mamedov
@ 2012-02-13  8:40     ` dima
  2012-02-13 13:42     ` dima
  1 sibling, 0 replies; 24+ messages in thread
From: dima @ 2012-02-13  8:40 UTC (permalink / raw)
  To: linux-btrfs

On 02/13/2012 05:09 PM, Roman Mamedov wrote:
> On Mon, 13 Feb 2012 16:40:03 +0900
> dima<dolenin@parallels.com>  wrote:
>
>> Hello Ralf-Peter,
>>
>> Actually it is possible. Check out David's response to my question from
>> some time ago:
>> http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227
>>
>> The nocow.c script he attached does just the thing you want. The script
>> is really working. I needed nocow for different purpose but it did not
>> occur to me to try it on VM image and see if the performance would
>> improve. Sounds like a great idea. If you get around to try it, pls.
>> post your impressions here.
>
> Thanks for the link, this is indeed interesting.
>
> I made a couple of small changes, i.e. I wanted a way to unset nocow and to
> check that changing flags really worked.
>
> gcc -o /usr/local/bin/nocow nocow.c
> ln -sf /usr/local/bin/nocow /usr/local/bin/cow
>
> Perhaps the support for setting this flag should be added to the 'btrfs'
> utility.


Cool. Thanks Roma!
I really wanted the feature to 'unset' the nocow and check the current 
state of flags. Will check it out today. I also think that it should 
definitely be included into the userspace btrfs utilities.
best
~dima

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13  8:09   ` Roman Mamedov
  2012-02-13  8:40     ` dima
@ 2012-02-13 13:42     ` dima
  2012-02-13 13:51       ` Roman Mamedov
  1 sibling, 1 reply; 24+ messages in thread
From: dima @ 2012-02-13 13:42 UTC (permalink / raw)
  To: linux-btrfs

On 02/13/2012 05:09 PM, Roman Mamedov wrote:
> On Mon, 13 Feb 2012 16:40:03 +0900
> dima<dolenin@parallels.com>  wrote:
>
>> Hello Ralf-Peter,
>>
>> Actually it is possible. Check out David's response to my question from
>> some time ago:
>> http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227
>>
>> The nocow.c script he attached does just the thing you want. The script
>> is really working. I needed nocow for different purpose but it did not
>> occur to me to try it on VM image and see if the performance would
>> improve. Sounds like a great idea. If you get around to try it, pls.
>> post your impressions here.
>
> Thanks for the link, this is indeed interesting.
>
> I made a couple of small changes, i.e. I wanted a way to unset nocow and to
> check that changing flags really worked.
>
> gcc -o /usr/local/bin/nocow nocow.c
> ln -sf /usr/local/bin/nocow /usr/local/bin/cow
>
> Perhaps the support for setting this flag should be added to the 'btrfs'
> utility.
>

Hello Roman,
I don't seem to be able to 'unset' the NOCOW flag.  Looking at the code 
I would guess that it is supposed to alternate between 'cow' and 'nocow' 
states, but the condition
     printf("Remove NOCOW flag for %s\n", argv[1]);
never shows for me. What should I do to make it working?
Maybe it would be nice to have a switch to just check if nocow is set on 
file without actually changing the flag.

thanks
~dima

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13 13:42     ` dima
@ 2012-02-13 13:51       ` Roman Mamedov
  2012-02-13 14:31         ` Dmitry Olenin
  0 siblings, 1 reply; 24+ messages in thread
From: Roman Mamedov @ 2012-02-13 13:51 UTC (permalink / raw)
  To: dima; +Cc: linux-btrfs

[-- Attachment #1: Type: text/plain, Size: 1209 bytes --]

On Mon, 13 Feb 2012 22:42:23 +0900
dima <dolenin@parallels.com> wrote:

> > gcc -o /usr/local/bin/nocow nocow.c
> > ln -sf /usr/local/bin/nocow /usr/local/bin/cow

> I don't seem to be able to 'unset' the NOCOW flag.  Looking at the code 
> I would guess that it is supposed to alternate between 'cow' and 'nocow' 
> states, but the condition
>      printf("Remove NOCOW flag for %s\n", argv[1]);
> never shows for me. What should I do to make it working?
> Maybe it would be nice to have a switch to just check if nocow is set on 
> file without actually changing the flag.

If you place it in /usr/local/bin and also make a symlink "nocow -> cow" as
described in the quoted part above, you can then just run: 

# cow ./filename

and the program will instead unset the NOCOW flag.

Of course it remains to be a very basic program, I'm not sure if it's worth
developing it further, or to add this to 'btrfs', as Liu Bo in the adjacent
thread said that there are patches to chattr/lsattr for using the COW
attribute.

-- 
With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13  7:40 ` dima
  2012-02-13  8:09   ` Roman Mamedov
@ 2012-02-13 14:10   ` David Sterba
  2012-02-13 14:21     ` Timo Witte
                       ` (3 more replies)
  1 sibling, 4 replies; 24+ messages in thread
From: David Sterba @ 2012-02-13 14:10 UTC (permalink / raw)
  To: dima; +Cc: linux-btrfs

[-- Attachment #1: Type: text/plain, Size: 548 bytes --]

Hi,

On Mon, Feb 13, 2012 at 04:40:03PM +0900, dima wrote:
> Actually it is possible. Check out David's response to my question from 
> some time ago:
> http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227

this was a quick aid, please see attached file for an updated tool to set
the file flags, now added 'z' for NOCOMPRESS flag, and supports chattr
syntax plus all of the standard file flags.

Setting and unsetting nocow is done like 'fileflags +C file' or -C for
unseting. Without any + or - options it prints current state.


david

[-- Attachment #2: fileflags.c --]
[-- Type: text/plain, Size: 3201 bytes --]

#include <errno.h>
#include <fcntl.h>
#include <inttypes.h>
#include <linux/fs.h>
#include <linux/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>

#ifndef FS_NOCOMP_FL
#define FS_NOCOMP_FL                    0x00000400 /* Don't compress */
#endif

#ifndef FS_NOCOW_FL
#define FS_NOCOW_FL                     0x00800000 /* Do not cow file */
#endif

struct flags_name {
	unsigned long	flag;
	const char	short_name;
	const char	*long_name;
};

static const struct flags_name flags[]={
	/* new */
	{ FS_NOCOW_FL, 'C', "NOCOW" },
	{ FS_NOCOMP_FL, 'z', "Not_Compressed" },
	/* current */
	{ FS_SECRM_FL, 's', "Secure_Deletion" },
	{ FS_UNRM_FL, 'u' , "Undelete" },
	{ FS_SYNC_FL, 'S', "Synchronous_Updates" },
	{ FS_DIRSYNC_FL, 'D', "Synchronous_Directory_Updates" },
	{ FS_IMMUTABLE_FL, 'i', "Immutable" },
	{ FS_APPEND_FL, 'a', "Append_Only" },
	{ FS_NODUMP_FL, 'd', "No_Dump" },
	{ FS_NOATIME_FL, 'A', "No_Atime" },
	{ FS_COMPR_FL, 'c', "Compression_Requested" },
	{ FS_COMPRBLK_FL, 'B', "Compressed_File" },
	{ FS_DIRTY_FL, 'Z', "Compressed_Dirty_File" },
	{ FS_ECOMPR_FL, 'E', "Compression_Error" },
	{ FS_JOURNAL_DATA_FL, 'j', "Journaled_Data" },
	{ FS_INDEX_FL, 'I', "Indexed_directory" },
	{ FS_NOTAIL_FL, 't', "No_Tailmerging" },
	{ FS_TOPDIR_FL, 'T', "Top_of_Directory_Hierarchies" },
	/* { EXT4_EXTENTS_FL, 'e', "Extents" }, */
	/* { EXT4_HUGE_FILE_FL, 'h', "Huge_file" }, */
};

static unsigned long to_set, to_unset;

unsigned long c2val(char c)
{
	int i;

	for(i=0;i<sizeof(flags)/sizeof(flags[0]);i++) {
		if(flags[i].short_name==c)
			return flags[i].flag;
	}

	printf("Warning: flag '%c' not found\n", c);
	return 0;
}
void list_flags(unsigned long fflags)
{
	int i;

	printf("Flags:\n");
	for(i=0;i<sizeof(flags)/sizeof(flags[0]);i++) {
		if(fflags & flags[i].flag) {
			printf(" %c %s\n",
				flags[i].short_name,
				flags[i].long_name);
		}
	}
}

void set_flag(const char* in)
{
	int i;

	for(i=0;i<strlen(in);i++) {
		to_set |= c2val(in[i]);
		printf(" set %c\n", in[i]);
	}
}

void unset_flag(const char* in)
{
	int i;

	for(i=0;i<strlen(in);i++) {
		to_unset |= c2val(in[i]);
		printf(" unset %c\n", in[i]);
	}
}

int main(int argc, char **argv)
{
        int ret;
	int optind;
	int modify=0;

	if (argc < 2) {
		printf("usage: %s [options] [--] file...\n", argv[0]);
		exit(1);
	}
	to_set = 0;
	to_unset = 0;
	for(optind=1;optind<argc;optind++) {
		char *o=argv[optind];
		if(o[0]=='-' && o[1]=='-') {
			optind++;
			break;
		} else if(o[0]=='-') {
			unset_flag(o+1);
			modify=1;
		} else if(o[0]=='+') {
			set_flag(o+1);
			modify=1;
		} else break;
	}

	for(;optind<argc;optind++) {
		unsigned long fflags;
		int fd = -1;

		if(fd!=-1) close(fd);
		fd = open(argv[optind], O_RDONLY);
		if (fd == -1) {
			perror("open()");
			continue;
		}
		ret = ioctl(fd, FS_IOC_GETFLAGS, &fflags);
		if (ret == -1) {
			perror("ioctl(GETFLAGS)");
			continue;
		}
		if(modify) {
			fflags |= to_set;
			fflags &= ~to_unset;
			ret = ioctl(fd, FS_IOC_SETFLAGS, &fflags);
			if (ret == -1) {
				perror("ioctl(SETFLAGS)");
				continue;
			}
		}
		printf("File: %s\n", argv[optind]);
		list_flags(fflags);
		putchar('\n');
	}

        return 0;
}

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13 14:10   ` David Sterba
@ 2012-02-13 14:21     ` Timo Witte
  2012-02-13 15:10     ` dima
                       ` (2 subsequent siblings)
  3 siblings, 0 replies; 24+ messages in thread
From: Timo Witte @ 2012-02-13 14:21 UTC (permalink / raw)
  To: linux-btrfs

[-- Attachment #1: Type: text/plain, Size: 686 bytes --]

The fileflags utility works great!
Thanks!

Am 13.02.2012 15:10, schrieb David Sterba:
> Hi,
> 
> On Mon, Feb 13, 2012 at 04:40:03PM +0900, dima wrote:
>> Actually it is possible. Check out David's response to my question from 
>> some time ago:
>> http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227
> 
> this was a quick aid, please see attached file for an updated tool to set
> the file flags, now added 'z' for NOCOMPRESS flag, and supports chattr
> syntax plus all of the standard file flags.
> 
> Setting and unsetting nocow is done like 'fileflags +C file' or -C for
> unseting. Without any + or - options it prints current state.
> 
> 
> david


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13 13:51       ` Roman Mamedov
@ 2012-02-13 14:31         ` Dmitry Olenin
  0 siblings, 0 replies; 24+ messages in thread
From: Dmitry Olenin @ 2012-02-13 14:31 UTC (permalink / raw)
  To: linux-btrfs

On 02/13/2012 10:51 PM, Roman Mamedov wrote:
> On Mon, 13 Feb 2012 22:42:23 +0900
> dima<dolenin@parallels.com>  wrote:
>
>>> gcc -o /usr/local/bin/nocow nocow.c
>>> ln -sf /usr/local/bin/nocow /usr/local/bin/cow
>
>> I don't seem to be able to 'unset' the NOCOW flag.  Looking at the code
>> I would guess that it is supposed to alternate between 'cow' and 'nocow'
>> states, but the condition
>>       printf("Remove NOCOW flag for %s\n", argv[1]);
>> never shows for me. What should I do to make it working?
>> Maybe it would be nice to have a switch to just check if nocow is set on
>> file without actually changing the flag.
>
> If you place it in /usr/local/bin and also make a symlink "nocow ->  cow" as
> described in the quoted part above, you can then just run:
>
> # cow ./filename
>
> and the program will instead unset the NOCOW flag.
>
> Of course it remains to be a very basic program, I'm not sure if it's worth
> developing it further, or to add this to 'btrfs', as Liu Bo in the adjacent
> thread said that there are patches to chattr/lsattr for using the COW
> attribute.
>

Thanks Roma! I overlooked the symlink. Works!
~dima

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13 14:10   ` David Sterba
  2012-02-13 14:21     ` Timo Witte
@ 2012-02-13 15:10     ` dima
  2012-02-16 13:55     ` NOCOW + compress-force = bug Roman Mamedov
  2012-02-29 15:09     ` Set nodatacow per file? Kyle Gates
  3 siblings, 0 replies; 24+ messages in thread
From: dima @ 2012-02-13 15:10 UTC (permalink / raw)
  To: linux-btrfs

On 02/13/2012 11:10 PM, David Sterba wrote:
> Hi,
>
> On Mon, Feb 13, 2012 at 04:40:03PM +0900, dima wrote:
>> Actually it is possible. Check out David's response to my question from
>> some time ago:
>> http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227
>
> this was a quick aid, please see attached file for an updated tool to set
> the file flags, now added 'z' for NOCOMPRESS flag, and supports chattr
> syntax plus all of the standard file flags.
>
> Setting and unsetting nocow is done like 'fileflags +C file' or -C for
> unseting. Without any + or - options it prints current state.
>
>
> david

Thanks David. Perfect! This would be a great help.
~dima

^ permalink raw reply	[flat|nested] 24+ messages in thread

* NOCOW + compress-force = bug
  2012-02-13 14:10   ` David Sterba
  2012-02-13 14:21     ` Timo Witte
  2012-02-13 15:10     ` dima
@ 2012-02-16 13:55     ` Roman Mamedov
  2012-02-16 14:30       ` David Sterba
  2012-02-16 17:58       ` Chris Mason
  2012-02-29 15:09     ` Set nodatacow per file? Kyle Gates
  3 siblings, 2 replies; 24+ messages in thread
From: Roman Mamedov @ 2012-02-16 13:55 UTC (permalink / raw)
  To: dave; +Cc: dima, linux-btrfs

[-- Attachment #1: Type: text/plain, Size: 1696 bytes --]

Hello,

Please be aware that there seems to be a possible problem with using NOCOW
flag on files situated on a filesystem mounted with compress-force(=lzo, in my
case).

Since experimenting with NOCOW, I started regularly hitting this BUG at
extent-tree.c:5813 

  5813                 BUG_ON(!(flags & BTRFS_BLOCK_FLAG_FULL_BACKREF));

I was unable to make netconsole work over a bridged interface, so can only
post screenshots of this OOPS:
  http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-1.jpg
  http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-2.jpg

This happened four times already, and always on snapshot creation (but not
every case). I have hourly snapshots in crontab, and only one case out of about
ten fails with this problem. Did not try to deliberately reproduce it yet by
manually making snapshots very often, etc.

On Mon, 13 Feb 2012 15:10:40 +0100
David Sterba <dave@jikos.cz> wrote:

> Hi,
> 
> On Mon, Feb 13, 2012 at 04:40:03PM +0900, dima wrote:
> > Actually it is possible. Check out David's response to my question from 
> > some time ago:
> > http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227
> 
> this was a quick aid, please see attached file for an updated tool to set
> the file flags, now added 'z' for NOCOMPRESS flag, and supports chattr
> syntax plus all of the standard file flags.
> 
> Setting and unsetting nocow is done like 'fileflags +C file' or -C for
> unseting. Without any + or - options it prints current state.
> 
> 
> david


-- 
With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: NOCOW + compress-force = bug
  2012-02-16 13:55     ` NOCOW + compress-force = bug Roman Mamedov
@ 2012-02-16 14:30       ` David Sterba
  2012-02-16 17:58       ` Chris Mason
  1 sibling, 0 replies; 24+ messages in thread
From: David Sterba @ 2012-02-16 14:30 UTC (permalink / raw)
  To: Roman Mamedov; +Cc: dave, dima, linux-btrfs

On Thu, Feb 16, 2012 at 07:55:15PM +0600, Roman Mamedov wrote:
>   5813                 BUG_ON(!(flags & BTRFS_BLOCK_FLAG_FULL_BACKREF));
> 
> I was unable to make netconsole work over a bridged interface, so can only
> post screenshots of this OOPS:
>   http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-1.jpg
>   http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-2.jpg
> 
> This happened four times already, and always on snapshot creation (but not
> every case). I have hourly snapshots in crontab, and only one case out of about
> ten fails with this problem. Did not try to deliberately reproduce it yet by
> manually making snapshots very often, etc.

I've hit a different bug with

* a process doing random reads/writes to a file
* looped toggling of NOCOW flag on the file every 10 seconds
* snapshotting of the fs every 10 seconds

and it triggered after a few minutes. It's an ENOSPC at

920         btrfs_i_size_write(parent_inode, parent_inode->i_size +
921                                          dentry->d_name.len * 2);
922         ret = btrfs_update_inode(trans, parent_root, parent_inode);
923         BUG_ON(ret);


[49220.539979] ------------[ cut here ]------------
[49220.543580] kernel BUG at fs/btrfs/transaction.c:923!
[49220.543580] invalid opcode: 0000 [#1] SMP
[49220.543580] CPU 0
[49220.543580] Modules linked in: btrfs aoe [last unloaded: btrfs]
[49220.543580]
[49220.543580] Pid: 12646, comm: btrfs Not tainted 3.2.0-default+ #114 Intel Corporation Santa Rosa platform/Matanzas
[49220.543580] RIP: 0010:[<ffffffffa01086da>]  [<ffffffffa01086da>] create_pending_snapshot+0x4ca/0x4d0 [btrfs]
[49220.543580] RSP: 0018:ffff880000869b48  EFLAGS: 00010286
[49220.543580] RAX: 00000000ffffffe4 RBX: ffff8800191afdf8 RCX: ffff88000e8679d8
[49220.543580] RDX: ffff88006cddd440 RSI: ffffffff8187d42d RDI: 0000000000000246
[49220.543580] RBP: ffff880000869c08 R08: 0000000000000000 R09: 0000000000000000
[49220.543580] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8800347b6740
[49220.543580] R13: ffff880054eb5180 R14: ffff880062b56000 R15: ffff880078f45510
[49220.543580] FS:  00007f6637f5c740(0000) GS:ffff88007dc00000(0000) knlGS:0000000000000000
[49220.543580] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[49220.543580] CR2: 00007f80a26c7000 CR3: 000000001edad000 CR4: 00000000000006f0
[49220.543580] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[49220.543580] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[49220.543580] Process btrfs (pid: 12646, threadinfo ffff880000868000, task ffff88006cddd440)
[49220.543580] Stack:
[49220.543580]  ffff880000000002 0000000000000083 ffff880078394000 ffff8800105d8390
[49220.543580]  ffff88001016bcd0 ffff880062b56000 ffff88007a15e070 ffff8800105d8390
[49220.543580]  ffff880079c14000 ffff8800347b6758 0000000000000175 ffffffffffffff84
[49220.543580] Call Trace:
[49220.543580]  [<ffffffffa00e65fa>] ? btrfs_free_path+0x2a/0x40 [btrfs]
[49220.543580]  [<ffffffffa0108726>] create_pending_snapshots+0x46/0x70 [btrfs]
[49220.543580]  [<ffffffffa0109a57>] btrfs_commit_transaction+0x3e7/0x8d0 [btrfs]
[49220.543580]  [<ffffffff81077aa0>] ? wake_up_bit+0x40/0x40
[49220.543580]  [<ffffffff81357eee>] ? do_raw_spin_unlock+0x5e/0xb0
[49220.543580]  [<ffffffffa0139e48>] btrfs_mksubvol+0x358/0x360 [btrfs]
[49220.543580]  [<ffffffffa0139f50>] btrfs_ioctl_snap_create_transid+0x100/0x160 [btrfs]
[49220.543580]  [<ffffffff81113f73>] ? might_fault+0x53/0xb0
[49220.543580]  [<ffffffffa013a12d>] btrfs_ioctl_snap_create_v2.clone.0+0xfd/0x110 [btrfs]
[49220.543580]  [<ffffffffa013ba47>] btrfs_ioctl+0x4a7/0x1270 [btrfs]
[49220.543580]  [<ffffffff81883400>] ? do_page_fault+0x2d0/0x580
[49220.543580]  [<ffffffff8187f150>] ? _raw_spin_unlock_irq+0x30/0x50
[49220.543580]  [<ffffffff8103ff23>] ? finish_task_switch+0x83/0xf0
[49220.543580]  [<ffffffff8103fee6>] ? finish_task_switch+0x46/0xf0
[49220.543580]  [<ffffffff8114d5e8>] do_vfs_ioctl+0x98/0x560
[49220.543580]  [<ffffffff8108b7a9>] ? trace_hardirqs_off_caller+0x29/0xc0
[49220.543580]  [<ffffffff8187f919>] ? retint_swapgs+0x13/0x1b
[49220.543580]  [<ffffffff8114daff>] sys_ioctl+0x4f/0x80
[49220.543580]  [<ffffffff81887c02>] system_call_fastpath+0x16/0x1b
[49220.543580] Code: 75 fd ff ff 0f 1f 44 00 00 0f b7 16 83 e8 02 48 83 c6 02 66 89 17 48 83 c7 02 e9 64 fd ff ff 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b <0f> 0b 0f 1f 40 00 55 48 89 e5 41 56 41 55 41 54 53 66 66 66 66
[49220.543580] RIP  [<ffffffffa01086da>] create_pending_snapshot+0x4ca/0x4d0 [btrfs]
[49220.543580]  RSP <ffff880000869b48>
[49220.937724] ---[ end trace 8f8b8a074f8e75d4 ]---

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: NOCOW + compress-force = bug
  2012-02-16 13:55     ` NOCOW + compress-force = bug Roman Mamedov
  2012-02-16 14:30       ` David Sterba
@ 2012-02-16 17:58       ` Chris Mason
  2012-03-13 18:11         ` Jeff Mahoney
  1 sibling, 1 reply; 24+ messages in thread
From: Chris Mason @ 2012-02-16 17:58 UTC (permalink / raw)
  To: Roman Mamedov; +Cc: dave, dima, linux-btrfs

On Thu, Feb 16, 2012 at 07:55:15PM +0600, Roman Mamedov wrote:
> Hello,
> 
> Please be aware that there seems to be a possible problem with using NOCOW
> flag on files situated on a filesystem mounted with compress-force(=lzo, in my
> case).
> 
> Since experimenting with NOCOW, I started regularly hitting this BUG at
> extent-tree.c:5813 
> 
>   5813                 BUG_ON(!(flags & BTRFS_BLOCK_FLAG_FULL_BACKREF));
> 
> I was unable to make netconsole work over a bridged interface, so can only
> post screenshots of this OOPS:
>   http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-1.jpg
>   http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-2.jpg
> 
> This happened four times already, and always on snapshot creation (but not
> every case). I have hourly snapshots in crontab, and only one case out of about
> ten fails with this problem. Did not try to deliberately reproduce it yet by
> manually making snapshots very often, etc.

Interesting, NOCOW and compression don't really mix.  We always cow for
compression.  I'll try to reproduce it.

-chris

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-13  7:17 Set nodatacow per file? Ralf-Peter Rohbeck
  2012-02-13  7:34 ` Chester
  2012-02-13  7:40 ` dima
@ 2012-02-24  5:22 ` dima
  2012-02-27 13:54   ` dima
  2 siblings, 1 reply; 24+ messages in thread
From: dima @ 2012-02-24  5:22 UTC (permalink / raw)
  To: linux-btrfs

On 02/13/2012 04:17 PM, Ralf-Peter Rohbeck wrote:
> Hello,
> is it possible to set nodatacow on a per-file basis? I couldn't find
> anything.
> If not, wouldn't that be a great feature to get around the performance
> issues with VM and database storage? Of course cloning should still
> cause COW.

Hello,
Going back to the original question from Ralf I wanted to share my 
experience.

Yesterday I set up KVM+qemu and set -z -C with David's 'fileflags' 
utility for the VM image file.
I was very pleased with results - Redhat 6 Minimal installation was 
installed in 10 minutes whereas it was taking 'forever' the last time I 
tried it some 4 months ago. Writes during installation were very 
moderate. Performance of VM is excellent. Installing some big packages 
with yum inside VM goes very quickly with the speed indistinguishable 
from that of bare metal installs.

I am not quite sure should this improvement be attributed to the nocow 
and nocompress flags or to the overall improvement of btrfs (I am on 
3.3-rc4 kernel) but KVM is definitely more than usable on btrfs now.

I am yet to test the install speed and performance without those flags set.

best
~dima

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-24  5:22 ` dima
@ 2012-02-27 13:54   ` dima
  2012-02-27 22:10     ` Chester
  0 siblings, 1 reply; 24+ messages in thread
From: dima @ 2012-02-27 13:54 UTC (permalink / raw)
  To: linux-btrfs

Hello,

Since several people asked to post the results, here they are.
I tried raw virtio disk with and without -z -C set and also qcow2 virtio 
disk without -z -C set and did not notice any difference in performance 
at all - Redhat 6.2 Minimal installs in 10 minutes in each case. The 
"abysmal" performance as it was some several months ago (like 10 minutes 
just for virtual disk formatting) under the same conditions is no more 
at least on 3.3.0-rc5.

best
~dima


On 02/24/2012 02:22 PM, dima wrote:
> On 02/13/2012 04:17 PM, Ralf-Peter Rohbeck wrote:
>> Hello,
>> is it possible to set nodatacow on a per-file basis? I couldn't find
>> anything.
>> If not, wouldn't that be a great feature to get around the performance
>> issues with VM and database storage? Of course cloning should still
>> cause COW.
>
> Hello,
> Going back to the original question from Ralf I wanted to share my
> experience.
>
> Yesterday I set up KVM+qemu and set -z -C with David's 'fileflags'
> utility for the VM image file.
> I was very pleased with results - Redhat 6 Minimal installation was
> installed in 10 minutes whereas it was taking 'forever' the last time I
> tried it some 4 months ago. Writes during installation were very
> moderate. Performance of VM is excellent. Installing some big packages
> with yum inside VM goes very quickly with the speed indistinguishable
> from that of bare metal installs.
>
> I am not quite sure should this improvement be attributed to the nocow
> and nocompress flags or to the overall improvement of btrfs (I am on
> 3.3-rc4 kernel) but KVM is definitely more than usable on btrfs now.
>
> I am yet to test the install speed and performance without those flags set.
>
> best
> ~dima
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-27 13:54   ` dima
@ 2012-02-27 22:10     ` Chester
  2012-02-28  0:51       ` dima
  0 siblings, 1 reply; 24+ messages in thread
From: Chester @ 2012-02-27 22:10 UTC (permalink / raw)
  To: dima; +Cc: linux-btrfs

On Mon, Feb 27, 2012 at 7:54 AM, dima <dolenin@parallels.com> wrote:
> Hello,
>
> Since several people asked to post the results, here they are.
> I tried raw virtio disk with and without -z -C set and also qcow2 vir=
tio
> disk without -z -C set and did not notice any difference in performan=
ce at
> all - Redhat 6.2 Minimal installs in 10 minutes in each case. The "ab=
ysmal"
> performance as it was some several months ago (like 10 minutes just f=
or
> virtual disk formatting) under the same conditions is no more at leas=
t on
> 3.3.0-rc5.

Just to make sure, this is a _new_ virtual disk right? I can barely
contain my excitement right now. This is amazing progress.

>
> best
> ~dima
>
>
>
> On 02/24/2012 02:22 PM, dima wrote:
>>
>> On 02/13/2012 04:17 PM, Ralf-Peter Rohbeck wrote:
>>>
>>> Hello,
>>> is it possible to set nodatacow on a per-file basis? I couldn't fin=
d
>>> anything.
>>> If not, wouldn't that be a great feature to get around the performa=
nce
>>> issues with VM and database storage? Of course cloning should still
>>> cause COW.
>>
>>
>> Hello,
>> Going back to the original question from Ralf I wanted to share my
>> experience.
>>
>> Yesterday I set up KVM+qemu and set -z -C with David's 'fileflags'
>> utility for the VM image file.
>> I was very pleased with results - Redhat 6 Minimal installation was
>> installed in 10 minutes whereas it was taking 'forever' the last tim=
e I
>> tried it some 4 months ago. Writes during installation were very
>> moderate. Performance of VM is excellent. Installing some big packag=
es
>> with yum inside VM goes very quickly with the speed indistinguishabl=
e
>> from that of bare metal installs.
>>
>> I am not quite sure should this improvement be attributed to the noc=
ow
>> and nocompress flags or to the overall improvement of btrfs (I am on
>> 3.3-rc4 kernel) but KVM is definitely more than usable on btrfs now.
>>
>> I am yet to test the install speed and performance without those fla=
gs
>> set.
>>
>> best
>> ~dima
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrf=
s" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-27 22:10     ` Chester
@ 2012-02-28  0:51       ` dima
  2012-03-02  3:28         ` dima
  0 siblings, 1 reply; 24+ messages in thread
From: dima @ 2012-02-28  0:51 UTC (permalink / raw)
  To: linux-btrfs

On 02/28/2012 07:10 AM, Chester wrote:
> On Mon, Feb 27, 2012 at 7:54 AM, dima<dolenin@parallels.com>  wrote:
>> Hello,
>>
>> Since several people asked to post the results, here they are.
>> I tried raw virtio disk with and without -z -C set and also qcow2 virtio
>> disk without -z -C set and did not notice any difference in performance at
>> all - Redhat 6.2 Minimal installs in 10 minutes in each case. The "abysmal"
>> performance as it was some several months ago (like 10 minutes just for
>> virtual disk formatting) under the same conditions is no more at least on
>> 3.3.0-rc5.
>
> Just to make sure, this is a _new_ virtual disk right? I can barely
> contain my excitement right now. This is amazing progress.

Yes, it is a newly created virtual disk. By the way, one thing that 
slipped out from my message - in case of raw I did pre-allocation of the 
entire image, but in case of qcow2 I unchecked this box in virt-manager 
and the disk was "growing" as the system was installing. Nevertheless I 
did not notice performance degradation during the install.

best
~dima

^ permalink raw reply	[flat|nested] 24+ messages in thread

* RE: Set nodatacow per file?
  2012-02-13 14:10   ` David Sterba
                       ` (2 preceding siblings ...)
  2012-02-16 13:55     ` NOCOW + compress-force = bug Roman Mamedov
@ 2012-02-29 15:09     ` Kyle Gates
  2012-02-29 15:34       ` cwillu
  3 siblings, 1 reply; 24+ messages in thread
From: Kyle Gates @ 2012-02-29 15:09 UTC (permalink / raw)
  To: dave; +Cc: linux-btrfs



> > Actually it is possible. Check out David's response to my question from
> > some time ago:
> > http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227
>
> this was a quick aid, please see attached file for an updated tool to set
> the file flags, now added 'z' for NOCOMPRESS flag, and supports chattr
> syntax plus all of the standard file flags.
>
> Setting and unsetting nocow is done like 'fileflags +C file' or -C for
> unseting. Without any + or - options it prints current state.


I get the following errors when running fileflags on large (>2GB) database files:

open(): No such file or directory

open(): Value too large for defined data type


 		 	   		  

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-29 15:09     ` Set nodatacow per file? Kyle Gates
@ 2012-02-29 15:34       ` cwillu
  0 siblings, 0 replies; 24+ messages in thread
From: cwillu @ 2012-02-29 15:34 UTC (permalink / raw)
  To: Kyle Gates; +Cc: dave, linux-btrfs

> I get the following errors when running fileflags on large (>2GB) database files:
>
> open(): No such file or directory
>
> open(): Value too large for defined data type

http://www.gnu.org/software/coreutils/faq/#Value-too-large-for-defined-data-type

"""The message "Value too large for defined data type" is a system
error message reported when an operation on a large file is attempted
using a non-large file data type. Large files are defined as anything
larger than a signed 32-bit integer, or stated differently, larger
than 2GB.

Many system calls that deal with files return values in a "long int"
data type. On 32-bit hardware a long int is 32-bits and therefore this
imposes a 2GB limit on the size of files. When this was invented that
was HUGE and it was hard to conceive of needing anything that large.
Time has passed and files can be much larger today. On native 64-bit
systems the file size limit is usually 2GB * 2GB. Which we will again
think is huge."""

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Set nodatacow per file?
  2012-02-28  0:51       ` dima
@ 2012-03-02  3:28         ` dima
  0 siblings, 0 replies; 24+ messages in thread
From: dima @ 2012-03-02  3:28 UTC (permalink / raw)
  To: linux-btrfs

On 02/28/2012 09:51 AM, dima wrote:
> On 02/28/2012 07:10 AM, Chester wrote:
>> On Mon, Feb 27, 2012 at 7:54 AM, dima<dolenin@parallels.com> wrote:
>>> Hello,
>>>
>>> Since several people asked to post the results, here they are.
>>> I tried raw virtio disk with and without -z -C set and also qcow2 virtio
>>> disk without -z -C set and did not notice any difference in
>>> performance at
>>> all - Redhat 6.2 Minimal installs in 10 minutes in each case. The
>>> "abysmal"
>>> performance as it was some several months ago (like 10 minutes just for
>>> virtual disk formatting) under the same conditions is no more at
>>> least on
>>> 3.3.0-rc5.
>>
>> Just to make sure, this is a _new_ virtual disk right? I can barely
>> contain my excitement right now. This is amazing progress.
>
> Yes, it is a newly created virtual disk. By the way, one thing that
> slipped out from my message - in case of raw I did pre-allocation of the
> entire image, but in case of qcow2 I unchecked this box in virt-manager
> and the disk was "growing" as the system was installing. Nevertheless I
> did not notice performance degradation during the install.


But still Virtualbox is faster. It installs Redhat 6 Minimal in just 
about 5 minutes. I observe that yum installations work faster.
WindowsXP install under KVM was a hell lasting for about 3 hours and the 
performance is awful with host HD indicator flashing all the time. There 
was no change whatsoever whether the image was with +C +z or not.
On Virtualbox WinXP install completed in about 25 minutes and the 
performance is excellent even w/o guest additions applied.
I wonder why such a big difference.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: NOCOW + compress-force = bug
  2012-02-16 17:58       ` Chris Mason
@ 2012-03-13 18:11         ` Jeff Mahoney
  2012-03-13 18:36           ` Jeff Mahoney
  0 siblings, 1 reply; 24+ messages in thread
From: Jeff Mahoney @ 2012-03-13 18:11 UTC (permalink / raw)
  To: Chris Mason, Roman Mamedov, dave, dima, linux-btrfs

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 02/16/2012 12:58 PM, Chris Mason wrote:
> On Thu, Feb 16, 2012 at 07:55:15PM +0600, Roman Mamedov wrote:
>> Hello,
>> 
>> Please be aware that there seems to be a possible problem with
>> using NOCOW flag on files situated on a filesystem mounted with
>> compress-force(=lzo, in my case).
>> 
>> Since experimenting with NOCOW, I started regularly hitting this
>> BUG at extent-tree.c:5813
>> 
>> 5813                 BUG_ON(!(flags &
>> BTRFS_BLOCK_FLAG_FULL_BACKREF));
>> 
>> I was unable to make netconsole work over a bridged interface, so
>> can only post screenshots of this OOPS: 
>> http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-1.jpg 
>> http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-2.jpg
>> 
>> This happened four times already, and always on snapshot creation
>> (but not every case). I have hourly snapshots in crontab, and
>> only one case out of about ten fails with this problem. Did not
>> try to deliberately reproduce it yet by manually making snapshots
>> very often, etc.
> 
> Interesting, NOCOW and compression don't really mix.  We always cow
> for compression.  I'll try to reproduce it.

I hit this one today without nocow or compression. The only thing
non-default was that I mounted with -ossd. The backing store was a 1GB
non-sparse loopback file on tmpfs.

I had kdump enabled and with 16GB, I wasn't waiting around for the
dump to complete. If it happens again, I'll have a full stack trace.
My test case was filling the disk while making snapshots.

- -Jeff


- -- 
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJPX424AAoJEB57S2MheeWybq0P/1xv09dKpBfuvK/2vpyriebN
3RsPtHVZgAL9y9XeLVw0KPlRKTGM+PIyc+AEloIiYQgULifA625nPa6+DXlNqzCp
0jKVleAE5RLCOtBDC91GX8JO/55fszjTQHSXAHjSRr4vr/4bZEf4tLBTwbW7Nfax
YAUPQ6cu+FQlCDWppXMLswKmLsmnewTnKrZp0YQTOTVGY3dajsvGTR3vR0VyxMwY
FZ13hvGOfP+UXsZC1qijxA3g8CrWwu/dhiU/dnCYwhcCF+0ONiHHHhB3krDAkNP8
yVAZgoSGJsIUdDiqeKXaSm8RDT6vGQwnJicFQSkUqGiHXyi+/fXVUULNSbgj39EC
R2jHLlXtMMaRyoqiM/wOfZUOr5MGQhk0duXB1NjNGAaffAsjvBY0c8y4yvGJVhtR
E2EknQLHr2jBWF8KCpLe0YYLPjcB3Gp3SPUhyGZbg4ATUAv2amMcw13deI8I17gI
v9dJSCWyi+r5c0d26rgtsS9SpfM8qHz5A/EvqVtn99DgS+O49o4+3F8M+2WioDjM
mHndWGXqg4fBIwrpxvo5RtBmQ8OYgZzxZXoZU3XP/eRS4c6z8OJ9U92/pUFP64Id
6tpqU1E9XxEijTWwcIyvkSzu6bjOLQucePBPCKkWLtTy9XrbNMTmvcGozKNCTVKK
km1EWb9RM2m8JPrEc+JA
=ywbL
-----END PGP SIGNATURE-----

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: NOCOW + compress-force = bug
  2012-03-13 18:11         ` Jeff Mahoney
@ 2012-03-13 18:36           ` Jeff Mahoney
  0 siblings, 0 replies; 24+ messages in thread
From: Jeff Mahoney @ 2012-03-13 18:36 UTC (permalink / raw)
  To: Chris Mason, Roman Mamedov, dave, dima, linux-btrfs

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/13/2012 02:11 PM, Jeff Mahoney wrote:
> On 02/16/2012 12:58 PM, Chris Mason wrote:
>> On Thu, Feb 16, 2012 at 07:55:15PM +0600, Roman Mamedov wrote:
>>> Hello,
>>> 
>>> Please be aware that there seems to be a possible problem with 
>>> using NOCOW flag on files situated on a filesystem mounted
>>> with compress-force(=lzo, in my case).
>>> 
>>> Since experimenting with NOCOW, I started regularly hitting
>>> this BUG at extent-tree.c:5813
>>> 
>>> 5813                 BUG_ON(!(flags & 
>>> BTRFS_BLOCK_FLAG_FULL_BACKREF));
>>> 
>>> I was unable to make netconsole work over a bridged interface,
>>> so can only post screenshots of this OOPS: 
>>> http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-1.jpg 
>>> http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-2.jpg
>>> 
>>> This happened four times already, and always on snapshot
>>> creation (but not every case). I have hourly snapshots in
>>> crontab, and only one case out of about ten fails with this
>>> problem. Did not try to deliberately reproduce it yet by
>>> manually making snapshots very often, etc.
> 
>> Interesting, NOCOW and compression don't really mix.  We always
>> cow for compression.  I'll try to reproduce it.
> 
> I hit this one today without nocow or compression. The only thing 
> non-default was that I mounted with -ossd. The backing store was a
> 1GB non-sparse loopback file on tmpfs.
> 
> I had kdump enabled and with 16GB, I wasn't waiting around for the 
> dump to complete. If it happens again, I'll have a full stack
> trace. My test case was filling the disk while making snapshots.

Well that didn't take long.

[  626.100684] ------------[ cut here ]------------
[  626.104053] kernel BUG at
/usr/src/packages/BUILD/kernel-default-3.0.23/linux-3.0/fs/btrfs/extent-tree.c:6091!
[  626.104053] invalid opcode: 0000 [#1] SMP
[  626.104053] CPU 8
[  626.104053] Modules linked in: btrfs zlib_deflate crc32c libcrc32c
autofs4 edd nfs lockd fscache auth_rpcgss nfs_acl sunrpc ipv6 ipv6_lib
af_packet cpufreq_conservative cpufreq_userspace cpufreq_powersave
powernow_k8 mperf microcode fuse loop dm_mod igb i2c_piix4 i2c_core
k10temp sg dca rtc_cmos pcspkr button serio_raw ext3 jbd mbcache
ohci_hcd ehci_hcd usbcore sd_mod usb_common crc_t10dif processor
thermal_sys hwmon ata_generic ahci libahci pata_atiixp libata scsi_mod
[  626.104053] Supported: Yes
[  626.104053]
[  626.104053] Pid: 14214, comm: btrfs Not tainted
3.0.23-0.0.0.0.4dd40bc-default #1 HP ProLiant DL165 G7
[  626.104053] RIP: 0010:[<ffffffffa03cd623>]  [<ffffffffa03cd623>]
alloc_reserved_tree_block+0x1e3/0x1f0 [btrfs]
[  626.104053] RSP: 0018:ffff88020766fae8  EFLAGS: 00010246
[  626.104053] RAX: ffff880434b76000 RBX: 0000000000000e2c RCX:
0000000000000000
[  626.104053] RDX: ffff880000000000 RSI: 0000000000000000 RDI:
ffff880404690678
[  626.104053] RBP: ffff880436e7f670 R08: ffff88020766faa8 R09:
0000000000001000
[  626.104053] R10: 0000000000000e2b R11: b000000000000000 R12:
ffff880404690678
[  626.104053] R13: 0000000000000d9d R14: ffff880432ac15a0 R15:
ffff880437a01c80
[  626.104053] FS:  00007fabeaf13740(0000) GS:ffff88043fc00000(0000)
knlGS:0000000000000000
[  626.104053] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  626.104053] CR2: 00007fabea61bb50 CR3: 0000000402dc1000 CR4:
00000000000006e0
[  626.104053] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  626.104053] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[  626.104053] Process btrfs (pid: 14214, threadinfo ffff88020766e000,
task ffff8802389260c0)
[  626.104053] Stack:
[  626.104053]  ffff88020766fb28 0000000000000000 0000000000000000
0000000004d3e000
[  626.104053]  ffff8800b220c000 ffff8804357bd000 0000003300000000
ffff88043977c540
[  626.104053]  ffff880437a01c80 ffff880432ac15a0 ffff8800b220c000
0000000000000001
[  626.104053] Call Trace:
[  626.104053]  [<ffffffffa03cdecc>] run_delayed_tree_ref+0xfc/0x150
[btrfs]
[  626.104053]  [<ffffffffa03ce21e>] run_clustered_refs+0xce/0x310 [btrfs]
[  626.104053]  [<ffffffffa03ce599>]
btrfs_run_delayed_refs+0x139/0x2e0 [btrfs]
[  626.104053]  [<ffffffffa03de553>]
btrfs_commit_transaction+0x433/0x8a0 [btrfs]
[  626.104053]  [<ffffffffa040a361>] create_snapshot+0x1a1/0x1c0 [btrfs]
[  626.104053]  [<ffffffffa040aa60>] btrfs_mksubvol+0x150/0x1e0 [btrfs]
[  626.104053]  [<ffffffffa040ac5b>]
btrfs_ioctl_snap_create_transid+0x16b/0x1a0 [btrfs]
[  626.104053]  [<ffffffffa040ad98>]
btrfs_ioctl_snap_create_v2+0x108/0x110 [btrfs]
[  626.104053]  [<ffffffffa040fbb7>] btrfs_ioctl+0x697/0x7d0 [btrfs]
[  626.104053]  [<ffffffff81162b7b>] do_vfs_ioctl+0x8b/0x3b0
[  626.104053]  [<ffffffff81162f41>] sys_ioctl+0xa1/0xb0
[  626.104053]  [<ffffffff8144ad12>] system_call_fastpath+0x16/0x1b
[  626.104053] DWARF2 unwinder stuck at system_call_fastpath+0x16/0x1b
[  626.104053]
[  626.104053] Leftover inexact backtrace:
[  626.104053]
[  626.104053] Code: 4c 89 e7 e8 50 76 02 00 e9 6b ff ff ff 48 8b 44
24 78 48 c7 c7 30 94 43 a0 48 8b 50 09 48 8b 30 31 c0 e8 47 2b 07 e1
0f 0b eb fe <0f> 0b eb fe 66 0f 1f 84 00 00 00 00 00 41 57 41 56 41 55
41 54
[  626.104053] RIP  [<ffffffffa03cd623>]
alloc_reserved_tree_block+0x1e3/0x1f0 [btrfs]
[  626.104053]  RSP <ffff88020766fae8>


- -- 
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJPX5OQAAoJEB57S2MheeWyxisP/37vZAl8f9N9OWwXKHiqybla
6nKCLN2DN0XaNNP/2hmjXh/V8AdU1ZQOR4apS1qrLIHXCZOoF3kJlTR5vrEwUxtW
GbkWr6t5TfSpdit+eSceADbGM9Yy7pi6AY830D7qj059dliLfbKHIw7k4gCbWQXa
STnh2ukeFYO75a9w7RbxxlK4I2Lt7Nf5Qmd8XZkyAJyvNqYrMhu/SXkpwz8rym3U
S0VH44zlKsLKSMNvn0dQYwKDZ5z1TbNgna9Wg1CpRYihVfAIjva58YXobkXHgZAb
eIuOoBuPEvdw+ZNSfKlB1a/TblcHUvgSteJYfcJu/EzgGIBKlzzIrR+TQZDegVnR
b4I/Jad/2Xm+1L9OZQHhJAaRetiY15aDtJ+R6+KewsQgYYILNXlIOMjA9+X646Wm
ypWQ6q6XjJ4XMEEGzXGEBfrJpUXeuWEXpQkRfxI7xvGmrXT8/r4yiEqrUVbWOoUa
EKMeN5u61HTXcUCdB0HtjAppO2mqFnuAVH/A4CkO3gRxVaOjPTFKIZszuxE/aJT5
Rs1VZMUoJtbLPu+npQpg2OLMUI8tdyt3RshSWEdynZaAk0utlH0lZa37shkniwe1
cK+tZwZJLQL8YLnBpF+BV9xn9dJTM82acjnOA5akSqyCXgrJcU5jrHYbiyqe44L6
I5rH1gzrESJKc9qU59Pz
=SpJv
-----END PGP SIGNATURE-----

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2012-03-13 18:36 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-02-13  7:17 Set nodatacow per file? Ralf-Peter Rohbeck
2012-02-13  7:34 ` Chester
2012-02-13  8:10   ` Liu Bo
2012-02-13  7:40 ` dima
2012-02-13  8:09   ` Roman Mamedov
2012-02-13  8:40     ` dima
2012-02-13 13:42     ` dima
2012-02-13 13:51       ` Roman Mamedov
2012-02-13 14:31         ` Dmitry Olenin
2012-02-13 14:10   ` David Sterba
2012-02-13 14:21     ` Timo Witte
2012-02-13 15:10     ` dima
2012-02-16 13:55     ` NOCOW + compress-force = bug Roman Mamedov
2012-02-16 14:30       ` David Sterba
2012-02-16 17:58       ` Chris Mason
2012-03-13 18:11         ` Jeff Mahoney
2012-03-13 18:36           ` Jeff Mahoney
2012-02-29 15:09     ` Set nodatacow per file? Kyle Gates
2012-02-29 15:34       ` cwillu
2012-02-24  5:22 ` dima
2012-02-27 13:54   ` dima
2012-02-27 22:10     ` Chester
2012-02-28  0:51       ` dima
2012-03-02  3:28         ` dima

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.