Samba strict allocate = yes stops btrfs compression working

Samba strict allocate = yes stops btrfs compression working

[Mark Ridley]

Hi,

If i set strict allocate = yes in samba to speed up the transfer
of a mssql database dump,
then btrfs does not compress the file. 
I have tried it also by just copying a small file in Windows to the 
samba share and the same.
I have tried btrfs mount options autodefrag and then
 btrfs fi defrag -c and the file still does not get compressed.

I have tried kernels 3.6.11, 3.8 and 3.10.7 on FC16 and FC18.

Any help, much appreciated. 

Thanks.

Mark

Re: Samba strict allocate = yes stops btrfs compression working

[Josef Bacik]

On Thu, Aug 22, 2013 at 09:57:24AM +0000, Mark Ridley wrote:
> Hi,
> 
> If i set strict allocate = yes in samba to speed up the transfer
> of a mssql database dump,
> then btrfs does not compress the file. 
> I have tried it also by just copying a small file in Windows to the 
> samba share and the same.
> I have tried btrfs mount options autodefrag and then
>  btrfs fi defrag -c and the file still does not get compressed.
> 
> I have tried kernels 3.6.11, 3.8 and 3.10.7 on FC16 and FC18.
> 
> Any help, much appreciated. 
>

Not sure what strict allocate = yes does, but I assume it probably does
fallocate() in which case yeah we aren't going to compress, we'll just write
into the preallocated space.  We don't support compressed writes into
preallocated space ATM, and I'm not sure we ever will.  Thanks,

Josef 

Re: Samba strict allocate = yes stops btrfs compression working

[Kai Krakow]

Josef Bacik <jbacik@fusionio.com> schrieb:

> Not sure what strict allocate = yes does, but I assume it probably does
> fallocate() in which case yeah we aren't going to compress, we'll just
> write
> into the preallocated space.  We don't support compressed writes into
> preallocated space ATM, and I'm not sure we ever will.  Thanks,

Good to know, this renders btrfs as efficient storage backend for Windows 
file shares pretty useless. Does this also happen with compress-force?

As a work-around one could write a cronjob that regularly defrags all files 
changed since the last run with -c option...

Thanks,
Kai

Re: Samba strict allocate = yes stops btrfs compression working

[Josef Bacik]

On Thu, Aug 22, 2013 at 08:29:26PM +0200, Kai Krakow wrote:
> Josef Bacik <jbacik@fusionio.com> schrieb:
> 
> > Not sure what strict allocate = yes does, but I assume it probably does
> > fallocate() in which case yeah we aren't going to compress, we'll just
> > write
> > into the preallocated space.  We don't support compressed writes into
> > preallocated space ATM, and I'm not sure we ever will.  Thanks,
> 
> Good to know, this renders btrfs as efficient storage backend for Windows 
> file shares pretty useless. Does this also happen with compress-force?
> 

Yeah the check is done in the same function, basically it goes

if (prealloc)
	do prealloc
else if (compress && (!inode->no_compress || compress_force))
	do compress
else
	do normal cow

> As a work-around one could write a cronjob that regularly defrags all files 
> changed since the last run with -c option...
> 

Yeah basically.  The other option is to just not do strict allocate = yes.  Now
if it's _not_ doing prealloc then there's probably a problem somewhere, but I
assume that is what its doing.  Thanks,

Josef

Re: Samba strict allocate = yes stops btrfs compression working

[Roger Binns]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 22/08/13 07:07, Josef Bacik wrote:
> Not sure what strict allocate = yes does,

I've worked on SMB servers before and can answer that.  Historically the
way Windows apps (right back into the 16 bit days) have made sure there is
space for a file about to be written is to ask the OS to allocate all the
space for it.  (Unix by default leaves holes making a sparse file.)

For example if a 10MB file is going to be written then an allocation will
be done of 10MB.  (The exact underlying protocol commands vary, but
originally were similar to the Unix seek to end and write.)  After that
seeks and writes are done.  Because the allocation succeeded the app knows
that it won't get an out of space error.

Separately from that, it turns out that some filesystems do benefit from
preallocating the file to the expected size, and then writing the contents
in dribs and drabs into the allocated space.

Consequently Samba gives you the option of really allocating all the file,
either for Windows semantics compatibility, or because it results in
improved performance on the Unix filesystem.

However I can't see it being of any benefit on a COW filesystem like btrfs.

Roger



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlIXFtsACgkQmOOfHg372QR7cwCggRyQxtxj9E7dNKV94M/Tv5o6
LC0AoN9icJNVxzkV0kDQSgf3Vt0N3g3V
=wBHz
-----END PGP SIGNATURE-----

Re: Samba strict allocate = yes stops btrfs compression working

[Mark Ridley]

The main reason I started using strict allocate = yes on samba was out of
desperation/exasperation with BTRFS.

BTRFS stalls from time to time causing SAMBA and/or MSSQL to give up on
the dump of a database.

>From what I have noticed, if for example you dump a 50GB database to samba
without strict allocate, then sometimes the file size gets set to 15GB
(for example), MSSQL then fills it in and then the file size jumps up to a
next amount and gets filled in.

But a lot of times, especially on a system that has been going for about 6
months and may be it is fragmented, but the next allocation of size just
stalls for a while, you cannot even ls the directory.

Either Samba or MSSQL gets fed up of this and the dump fails with a write
error.

I am seeing this on 3.3.2, 3.6.11 and even worse on 3.9 and 3.10.

I don't want to try nodatacow (which would probably fix the issue), but
you lose compression on the whole filesystem, autodefrag doesn't fix it
either.

Any suggestions?

Many thanks.

On 23/08/2013 09:01, "Roger Binns" <rogerb@rogerbinns.com> wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>On 22/08/13 07:07, Josef Bacik wrote:
>> Not sure what strict allocate = yes does,
>
>I've worked on SMB servers before and can answer that.  Historically the
>way Windows apps (right back into the 16 bit days) have made sure there is
>space for a file about to be written is to ask the OS to allocate all the
>space for it.  (Unix by default leaves holes making a sparse file.)
>
>For example if a 10MB file is going to be written then an allocation will
>be done of 10MB.  (The exact underlying protocol commands vary, but
>originally were similar to the Unix seek to end and write.)  After that
>seeks and writes are done.  Because the allocation succeeded the app knows
>that it won't get an out of space error.
>
>Separately from that, it turns out that some filesystems do benefit from
>preallocating the file to the expected size, and then writing the contents
>in dribs and drabs into the allocated space.
>
>Consequently Samba gives you the option of really allocating all the file,
>either for Windows semantics compatibility, or because it results in
>improved performance on the Unix filesystem.
>
>However I can't see it being of any benefit on a COW filesystem like
>btrfs.
>
>Roger
>
>
>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.12 (GNU/Linux)
>
>iEYEARECAAYFAlIXFtsACgkQmOOfHg372QR7cwCggRyQxtxj9E7dNKV94M/Tv5o6
>LC0AoN9icJNVxzkV0kDQSgf3Vt0N3g3V
>=wBHz
>-----END PGP SIGNATURE-----
>
>--
>To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Samba strict allocate = yes stops btrfs compression working

[Mark Ridley]

I tried defrag -c and it does nothing to files that have come in with
strict allocate = yes.

On 22/08/2013 19:29, "Kai Krakow" <hurikhan77+btrfs@gmail.com> wrote:

>Josef Bacik <jbacik@fusionio.com> schrieb:
>
>> Not sure what strict allocate = yes does, but I assume it probably does
>> fallocate() in which case yeah we aren't going to compress, we'll just
>> write
>> into the preallocated space.  We don't support compressed writes into
>> preallocated space ATM, and I'm not sure we ever will.  Thanks,
>
>Good to know, this renders btrfs as efficient storage backend for Windows
>file shares pretty useless. Does this also happen with compress-force?
>
>As a work-around one could write a cronjob that regularly defrags all
>files 
>changed since the last run with -c option...
>
>Thanks,
>Kai
>
>--
>To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Samba strict allocate = yes stops btrfs compression working

[Duncan]

Mark Ridley posted on Fri, 23 Aug 2013 09:20:04 +0100 as excerpted:

> I don't want to try nodatacow (which would probably fix the issue), but
> you lose compression on the whole filesystem, autodefrag doesn't fix it
> either.

I don't do servantware (in the context of my sig) and thus don't do samba 
here, so I'll just black-box that side of things entirely.

But from the btrfs side, are you considering nodatacow at the filesystem 
or individual file level?  It seems doing it at the individual file level 
might be what you need.

Note that nodatacow must be set on the file at creation in ordered to 
work, which is impractical to do directly.  However, if you can arrange 
for the files in question to appear in a particular directory, you can 
set the nodatacow attribute on the directory, and files created within it 
will inherit that.

It seems to me that should do what you need, PROVIDED that you can 
arrange for the files to appear in a particular dir/dir-tree, not more or 
less randomly written throughout the entire (btrfs) filesystem.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Samba strict allocate = yes stops btrfs compression working

[Mark Ridley]

The speed improvement for dumping large databases through samba with
strict allocate = yes to BTRFS was amazing.  It reduced a 1 hour dump down
to 20 minutes.

On 23/08/2013 09:01, "Roger Binns" <rogerb@rogerbinns.com> wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>On 22/08/13 07:07, Josef Bacik wrote:
>> Not sure what strict allocate = yes does,
>
>I've worked on SMB servers before and can answer that.  Historically the
>way Windows apps (right back into the 16 bit days) have made sure there is
>space for a file about to be written is to ask the OS to allocate all the
>space for it.  (Unix by default leaves holes making a sparse file.)
>
>For example if a 10MB file is going to be written then an allocation will
>be done of 10MB.  (The exact underlying protocol commands vary, but
>originally were similar to the Unix seek to end and write.)  After that
>seeks and writes are done.  Because the allocation succeeded the app knows
>that it won't get an out of space error.
>
>Separately from that, it turns out that some filesystems do benefit from
>preallocating the file to the expected size, and then writing the contents
>in dribs and drabs into the allocated space.
>
>Consequently Samba gives you the option of really allocating all the file,
>either for Windows semantics compatibility, or because it results in
>improved performance on the Unix filesystem.
>
>However I can't see it being of any benefit on a COW filesystem like
>btrfs.
>
>Roger
>
>
>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.12 (GNU/Linux)
>
>iEYEARECAAYFAlIXFtsACgkQmOOfHg372QR7cwCggRyQxtxj9E7dNKV94M/Tv5o6
>LC0AoN9icJNVxzkV0kDQSgf3Vt0N3g3V
>=wBHz
>-----END PGP SIGNATURE-----
>
>--
>To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Samba strict allocate = yes stops btrfs compression working

[Mark Ridley]

That would be fine, but nodatacow (according to the btrfs wiki) stops
compression, so I might as well get the speed benefits of 'strict allocate
= yes' which also disables compression.

If you want to use BTRFS to store backups then compression has be turned
on.

Database files like MSSQL usually compress down by 7-10 times, so its a
shame to not get this benefit.


On 23/08/2013 10:08, "Duncan" <1i5t5.duncan@cox.net> wrote:

>Mark Ridley posted on Fri, 23 Aug 2013 09:20:04 +0100 as excerpted:
>
>> I don't want to try nodatacow (which would probably fix the issue), but
>> you lose compression on the whole filesystem, autodefrag doesn't fix it
>> either.
>
>I don't do servantware (in the context of my sig) and thus don't do samba
>here, so I'll just black-box that side of things entirely.
>
>But from the btrfs side, are you considering nodatacow at the filesystem
>or individual file level?  It seems doing it at the individual file level
>might be what you need.
>
>Note that nodatacow must be set on the file at creation in ordered to
>work, which is impractical to do directly.  However, if you can arrange
>for the files in question to appear in a particular directory, you can
>set the nodatacow attribute on the directory, and files created within it
>will inherit that.
>
>It seems to me that should do what you need, PROVIDED that you can
>arrange for the files to appear in a particular dir/dir-tree, not more or
>less randomly written throughout the entire (btrfs) filesystem.
>
>-- 
>Duncan - List replies preferred.   No HTML msgs.
>"Every nonfree program has a lord, a master --
>and if you use the program, he is your master."  Richard Stallman
>
>--
>To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Samba strict allocate = yes stops btrfs compression working

[Clemens Eisserer]

Hi,

> The speed improvement for dumping large databases through samba with
> strict allocate = yes to BTRFS was amazing.  It reduced a 1 hour dump down
> to 20 minutes.

What you want btrfs to do is to allocate a file of fixed-size on disk
in advance, without knowing how large the file will be after
compression (which is the only thing of interest to btrfs). So the
whole idea doesn't make a lot of sence.

Sure, you could generate a 50gb file filled with zeros, but once you
put real data in it, it will fragment a lot - probably even more than
the file which was created without strict allocate.

Regards

Re: Samba strict allocate = yes stops btrfs compression working

[Roger Binns]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 23/08/13 01:20, Mark Ridley wrote:
> The main reason I started using strict allocate = yes on samba was out
> of desperation/exasperation with BTRFS.

The most effective performance option is to turn oplocks on.
Opportunistic locks are granted to a client when it is the only one with a
file open.  That lets it do caching and not even tell the server about
record locking.  Once another client opens the same file then the first
client has to flush all outstanding writes, its caches, record locking etc.

I don't know what Samba currently uses as the default value, but it is
traditionally set to off because there are scenarios under which it isn't
sufficiently robust (eg access on the Unix server side, a network break
when there is outstanding data).

http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/locking.html#id2616903

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlIXuX0ACgkQmOOfHg372QSRwgCgwwwoo7QqRql8+CS5xggRBVRt
krAAn1wFiIIliZJ7WbWh/oh8crEWLgfR
=HZG9
-----END PGP SIGNATURE-----