[RFC] Smart fibration plugin ext_4321

[RFC] Smart fibration plugin ext_4321

[Dušan Čolić]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

Fibration is a great way to decrease fragmentation and increase throughput.
Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
and they all have their upsides and downsides.

Proposed fibration plugin combines them all so that it combines files
with same extensions for 1, 2. 3 and 4 character extension  in groups
and sorts them in same fiber group.

With this fibration plugin all eg. xvid files would be in same group
in folder on disk sorted alphabetically so that we will avoid putting
small files between them and in that way reduce fragmentation. That
group (xvid 4 character extensions) would be among last groups under
one directory so that all small files would be written before it.

Problem with the attached patch is that currently every fibre value is
defined as u64  (eg. static __u64 fibre_ext_3) but if I understood
correctly comments in kassign.c and fibration.c fibration part of the
key is only 7 bits long.
If that is true how did fibre_ext_3 worked?

Thanks

Dushan

[-- Attachment #2: fibration_ext_4321.patch --]
[-- Type: application/octet-stream, Size: 2032 bytes --]

Signed-off-by:Dushan Tcholich <dusanc@gmail.com>

--- linux-4.8.15-gentoo/fs/reiser4/plugin/fibration.c	2016-12-24 18:22:09.679645971 +0100
+++ linux/fs/reiser4/plugin/fibration.c	2016-12-25 01:31:43.700383624 +0100
@@ -81,6 +81,25 @@ static __u64 fibre_ext_3(const struct in
 		return FIBRE_NO(0);
 }
 
+/*
+* ext.4321 fibration: try to separate files with different 1, 2, 3 and 4-character
+* extensions from each other.
+*/
+
+static __u64 fibre_ext_4321(const struct inode *dir, const char *name, int len)
+{
+	if (len > 5 && name[len - 5] == '.')
+		return FIBRE_NO(name[len - 4] + name[len - 3] + name[len - 2] + name[len - 1] + 768);
+	else if (len > 4 && name[len - 4] == '.')
+		return FIBRE_NO(name[len - 3] + name[len - 2] + name[len - 1] + 384);
+	else if (len > 3 && name[len - 3] == '.')
+		return FIBRE_NO(name[len - 2] + name[len - 1] + 128);
+	else if (len > 2 && name[len - 2] == '.')
+		return FIBRE_NO(name[len - 1]);
+	else
+		return FIBRE_NO(0);
+} 
+
 static int change_fibration(struct inode *inode,
 			    reiser4_plugin * plugin,
 			    pset_member memb)
@@ -161,7 +180,18 @@ fibration_plugin fibration_plugins[LAST_
 			.linkage = {NULL, NULL}
 		},
 		.fibre = fibre_ext_3
-	}
+	},
+        [FIBRATION_EXT_4321] = {
+                .h = {
+                        .type_id = REISER4_FIBRATION_PLUGIN_TYPE,
+                        .id = FIBRATION_EXT_4321,
+                        .pops = &fibration_plugin_ops,
+                        .label = "ext-4321",
+                        .desc = "fibrate file by 4, 3, 2 and 1 character extension",
+                        .linkage = {NULL, NULL}
+                },
+                .fibre = fibre_ext_4321
+        }
 };
 
 /*
--- linux-4.8.15-gentoo/fs/reiser4/plugin/fibration.h	2016-12-24 18:22:09.683646147 +0100
+++ linux/fs/reiser4/plugin/fibration.h	2016-12-25 01:19:12.975390151 +0100
@@ -20,6 +20,7 @@ typedef enum {
 	FIBRATION_DOT_O,
 	FIBRATION_EXT_1,
 	FIBRATION_EXT_3,
+	FIBRATION_EXT_4321,
 	LAST_FIBRATION_ID
 } reiser4_fibration_id;
 

Re: [RFC] Smart fibration plugin ext_4321

[Dušan Čolić]

[-- Attachment #1: Type: text/plain, Size: 2020 bytes --]

On Sun, Dec 25, 2016 at 1:59 AM, Dušan Čolić <dusanc@gmail.com> wrote:
> Fibration is a great way to decrease fragmentation and increase throughput.
> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
> and they all have their upsides and downsides.
>
> Proposed fibration plugin combines them all so that it combines files
> with same extensions for 1, 2. 3 and 4 character extension  in groups
> and sorts them in same fiber group.
>
> With this fibration plugin all eg. xvid files would be in same group
> in folder on disk sorted alphabetically so that we will avoid putting
> small files between them and in that way reduce fragmentation. That
> group (xvid 4 character extensions) would be among last groups under
> one directory so that all small files would be written before it.
>
> Problem with the attached patch is that currently every fibre value is
> defined as u64  (eg. static __u64 fibre_ext_3) but if I understood
> correctly comments in kassign.c and fibration.c fibration part of the
> key is only 7 bits long.
> If that is true how did fibre_ext_3 worked?
>
Patch v2 addresses the problem that we have only 127 fibration groups
available with 7 bits in the key so I made following groups:

1. Fibration value 0-31 - files with 1 character extension except .o files
2. Fibration value 32-60 - files with 2 character extension
3. Fibration value 61-90 - files with 3 character extension
4. Fibration value 91-120 - files with 4 character extension
5. Fibration value 121-125 - Free for some special files
6. Fibration value 126 - rest of files
7. Fibration value 127 - .o files

Overlap from 128/4=32 and we use offset 30 is because control
characters (those we do not use in file names) are in 0-31 range so we
will never use those values.

I also changed name of fibration plugin to ext_1234 to better reflect
the real function of plugin.

This is all still RFC as I made no changes to reiser4progs yet.

Dushan
> Thanks
>
> Dushan

[-- Attachment #2: fibration_ext_1234.2.patch --]
[-- Type: application/octet-stream, Size: 2303 bytes --]

Signed-off-by:Dushan Tcholich <dusanc@gmail.com>

--- linux-4.8.15-gentoo/fs/reiser4/plugin/fibration.h	2016-12-24 18:22:09.683646147 +0100
+++ linux/fs/reiser4/plugin/fibration.h	2016-12-25 09:30:16.585153093 +0100
@@ -20,6 +20,7 @@ typedef enum {
 	FIBRATION_DOT_O,
 	FIBRATION_EXT_1,
 	FIBRATION_EXT_3,
+	FIBRATION_EXT_1234,
 	LAST_FIBRATION_ID
 } reiser4_fibration_id;
 
--- linux-4.8.15-gentoo/fs/reiser4/plugin/fibration.c	2016-12-24 18:22:09.679645971 +0100
+++ linux/fs/reiser4/plugin/fibration.c	2016-12-25 09:29:58.092340355 +0100
@@ -81,6 +81,31 @@ static __u64 fibre_ext_3(const struct in
 		return FIBRE_NO(0);
 }
 
+/*
+* ext.1234 fibration: try to separate files with different 1, 2, 3 and 4-character
+* extensions from each other in groups of 30 each and put everything else after
+* them (group 126) with .o files at the end (group 127).
+*/
+
+static __u64 fibre_ext_1234(const struct inode *dir, const char *name, int len)
+{
+	if (len > 5 && name[len - 5] == '.')
+		return FIBRE_NO(name[len - 4] + name[len - 3] + name[len - 2] +
+				name[len - 1]) / 16 + 90);
+	else if (len > 4 && name[len - 4] == '.')
+		return FIBRE_NO((name[len - 3] + name[len - 2] +
+				name[len - 1]) / 12 + 60);
+	else if (len > 3 && name[len - 3] == '.')
+		return FIBRE_NO((name[len - 2] + name[len - 1]) / 8 + 30);
+	/* special treatment for .*\.o */
+	else if (len > 2 && name[len - 1] == 'o' && name[len - 2] == '.')
+		return FIBRE_NO(127);
+	else if (len > 2 && name[len - 2] == '.')
+		return FIBRE_NO((name[len - 1]) / 4);
+	else
+		return FIBRE_NO(126);
+} 
+
 static int change_fibration(struct inode *inode,
 			    reiser4_plugin * plugin,
 			    pset_member memb)
@@ -161,7 +186,18 @@ fibration_plugin fibration_plugins[LAST_
 			.linkage = {NULL, NULL}
 		},
 		.fibre = fibre_ext_3
-	}
+	},
+        [FIBRATION_EXT_1234] = {
+                .h = {
+                        .type_id = REISER4_FIBRATION_PLUGIN_TYPE,
+                        .id = FIBRATION_EXT_1234,
+                        .pops = &fibration_plugin_ops,
+                        .label = "ext-1234",
+                        .desc = "fibrate file by 1, 2, 3 and 4 character extension",
+                        .linkage = {NULL, NULL}
+                },
+                .fibre = fibre_ext_1234
+        }
 };
 
 /*

Re: [RFC] Smart fibration plugin ext_4321

[Edward Shishkin]

On 12/25/2016 02:59 AM, Dušan Čolić wrote:
> Fibration is a great way to decrease fragmentation and increase throughput.
> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
> and they all have their upsides and downsides.
>
> Proposed fibration plugin combines them all so that it combines files
> with same extensions for 1, 2. 3 and 4 character extension  in groups
> and sorts them in same fiber group.
>
> With this fibration plugin all eg. xvid files would be in same group
> in folder on disk sorted alphabetically


What application wants all xvid files to be in the same group?
Do you have any benchmark numbers which show advantages
of the new plugin?

Thanks,
Edward.

>   so that we will avoid putting
> small files between them and in that way reduce fragmentation. That
> group (xvid 4 character extensions) would be among last groups under
> one directory so that all small files would be written before it.
>
> Problem with the attached patch is that currently every fibre value is
> defined as u64  (eg. static __u64 fibre_ext_3) but if I understood
> correctly comments in kassign.c and fibration.c fibration part of the
> key is only 7 bits long.
> If that is true how did fibre_ext_3 worked?
>
> Thanks
>
> Dushan

Re: [RFC] Smart fibration plugin ext_4321

[Dušan Čolić]

On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
>
>
>
> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>
>> Fibration is a great way to decrease fragmentation and increase throughput.
>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
>> and they all have their upsides and downsides.
>>
>> Proposed fibration plugin combines them all so that it combines files
>> with same extensions for 1, 2. 3 and 4 character extension  in groups
>> and sorts them in same fiber group.
>>
>> With this fibration plugin all eg. xvid files would be in same group
>> in folder on disk sorted alphabetically
>
>
>
> What application wants all xvid files to be in the same group?
> Do you have any benchmark numbers which show advantages
> of the new plugin?
>

Xvid files are just an example.
ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3, ext_4
and dot_o in one.

In currently default plugin (dot_o) we sort all files by name from the
start except .o files which we put at the end.
So if we had a source directory with .c .h and .o files in it files by
extension would be sorted like: chchchchchchchchoooooooooooooo
I presumed that in some use cases it is better to have files be sorted
ccccccccccchhhhhhhhhhhhhhoooooooooooo

Hypothesis is to use the premise that files of same extension are in
same order of size to reduce fragmentation.

If we group files of same extension in groups in one directory, when
we write files of same extension after deletion of some files of one
extension  their group would be in same order as the deleted file so
they would be written in similar place and occupy the 'hole' of
similar size.
Ofc I am not talking about files of few kB size where Reiser4 is great
at packing but about files from few MB to few GB.

Eg. directory with mp3 and xvid files. mp3s are on the order of MB and
xvid on the order of GB. If we sort them just by name order of xvid
and mp3 files in one directory would be random so when deleting the
smaller ones we would make random holes (like from
mxmxmxxmmmxxxxmxxmmmx to mx xmxx  mx  xmx mmmx).
With grouping of writing where all mp3s would be written first and all
xvid after them after some deletions we would have smaller holes
grouped first  and larger last (like from mmmmmmmmmmmmxxxxxxxxxx to mm
m   mmm mmxx xxx xxx) but the main thing that after writing we would
write mp3s in mp3 holes and xvid in xvid holes ergo. reduce
fragmentation  (like from mm m   mmm mmxx xxx xxx to
mmMmMMMmmmXmmxxXxxx xxx) that we would create if we would try to write
xvid over mp3 holes.

One obvious use case where I hypothesize that this type of fibration
is better long term would be directories with content similar to usual
Downloads directory, a lot of different types (and siyes) of files
that get written and deleted a lot.

ext_1234 fibration is the same as dot_o for directories with only one
or one and .o file extension.

Ofc this is just a hypothesis that I would like to prove with some
fragmentation benchmarks but I wanted to hear your thoughts.

And while I was looking through the code I found a part that I
comprehended, elegant and easy to understand so I wanted to make
something so I could learn more.


> Thanks,
> Edward.
>

Thank you for your time and effort

Dushan


>
>
>>   so that we will avoid putting
>> small files between them and in that way reduce fragmentation. That
>> group (xvid 4 character extensions) would be among last groups under
>> one directory so that all small files would be written before it.
>>
>> Problem with the attached patch is that currently every fibre value is
>> defined as u64  (eg. static __u64 fibre_ext_3) but if I understood
>> correctly comments in kassign.c and fibration.c fibration part of the
>> key is only 7 bits long.
>> If that is true how did fibre_ext_3 worked?
>>
>> Thanks
>>
>> Dushan
>
>

Re: [RFC] Smart fibration plugin ext_4321

[Edward Shishkin]

On 12/26/2016 11:13 PM, Dušan Čolić wrote:
> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>>
>>
>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>> Fibration is a great way to decrease fragmentation and increase throughput.
>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
>>> and they all have their upsides and downsides.
>>>
>>> Proposed fibration plugin combines them all so that it combines files
>>> with same extensions for 1, 2. 3 and 4 character extension  in groups
>>> and sorts them in same fiber group.
>>>
>>> With this fibration plugin all eg. xvid files would be in same group
>>> in folder on disk sorted alphabetically
>>
>>
>> What application wants all xvid files to be in the same group?
>> Do you have any benchmark numbers which show advantages
>> of the new plugin?
>>
> Xvid files are just an example.
> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3, ext_4
> and dot_o in one.
>
> In currently default plugin (dot_o) we sort all files by name from the
> start except .o files which we put at the end.
> So if we had a source directory with .c .h and .o files in it files by
> extension would be sorted like: chchchchchchchchoooooooooooooo
> I presumed that in some use cases it is better to have files be sorted
> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>
> Hypothesis is to use the premise that files of same extension are in
> same order of size to reduce fragmentation.


What kind of fragmentation you are talking about?
Internal (which results in "dead" disk space), or
external (which results in a lot of "extents")?

Edward.


>
> If we group files of same extension in groups in one directory, when
> we write files of same extension after deletion of some files of one
> extension  their group would be in same order as the deleted file so
> they would be written in similar place and occupy the 'hole' of
> similar size.
> Ofc I am not talking about files of few kB size where Reiser4 is great
> at packing but about files from few MB to few GB.
>
> Eg. directory with mp3 and xvid files. mp3s are on the order of MB and
> xvid on the order of GB. If we sort them just by name order of xvid
> and mp3 files in one directory would be random so when deleting the
> smaller ones we would make random holes (like from
> mxmxmxxmmmxxxxmxxmmmx to mx xmxx  mx  xmx mmmx).
> With grouping of writing where all mp3s would be written first and all
> xvid after them after some deletions we would have smaller holes
> grouped first  and larger last (like from mmmmmmmmmmmmxxxxxxxxxx to mm
> m   mmm mmxx xxx xxx) but the main thing that after writing we would
> write mp3s in mp3 holes and xvid in xvid holes ergo. reduce
> fragmentation  (like from mm m   mmm mmxx xxx xxx to
> mmMmMMMmmmXmmxxXxxx xxx) that we would create if we would try to write
> xvid over mp3 holes.
>
> One obvious use case where I hypothesize that this type of fibration
> is better long term would be directories with content similar to usual
> Downloads directory, a lot of different types (and siyes) of files
> that get written and deleted a lot.
>
> ext_1234 fibration is the same as dot_o for directories with only one
> or one and .o file extension.
>
> Ofc this is just a hypothesis that I would like to prove with some
> fragmentation benchmarks but I wanted to hear your thoughts.
>
> And while I was looking through the code I found a part that I
> comprehended, elegant and easy to understand so I wanted to make
> something so I could learn more.
>
>
>> Thanks,
>> Edward.
>>
> Thank you for your time and effort
>
> Dushan
>
>
>>
>>>    so that we will avoid putting
>>> small files between them and in that way reduce fragmentation. That
>>> group (xvid 4 character extensions) would be among last groups under
>>> one directory so that all small files would be written before it.
>>>
>>> Problem with the attached patch is that currently every fibre value is
>>> defined as u64  (eg. static __u64 fibre_ext_3) but if I understood
>>> correctly comments in kassign.c and fibration.c fibration part of the
>>> key is only 7 bits long.
>>> If that is true how did fibre_ext_3 worked?
>>>
>>> Thanks
>>>
>>> Dushan
>>
> --
> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC] Smart fibration plugin ext_4321

[Dušan Čolić]

On Fri, Jan 6, 2017 at 2:44 PM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
> On 12/26/2016 11:13 PM, Dušan Čolić wrote:
>>
>> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>>
>>>
>>>
>>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>>>
>>>> Fibration is a great way to decrease fragmentation and increase
>>>> throughput.
>>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
>>>> and they all have their upsides and downsides.
>>>>
>>>> Proposed fibration plugin combines them all so that it combines files
>>>> with same extensions for 1, 2. 3 and 4 character extension  in groups
>>>> and sorts them in same fiber group.
>>>>
>>>> With this fibration plugin all eg. xvid files would be in same group
>>>> in folder on disk sorted alphabetically
>>>
>>>
>>>
>>> What application wants all xvid files to be in the same group?
>>> Do you have any benchmark numbers which show advantages
>>> of the new plugin?
>>>
>> Xvid files are just an example.
>> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3, ext_4
>> and dot_o in one.
>>
>> In currently default plugin (dot_o) we sort all files by name from the
>> start except .o files which we put at the end.
>> So if we had a source directory with .c .h and .o files in it files by
>> extension would be sorted like: chchchchchchchchoooooooooooooo
>> I presumed that in some use cases it is better to have files be sorted
>> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>>
>> Hypothesis is to use the premise that files of same extension are in
>> same order of size to reduce fragmentation.
>
>
>
> What kind of fragmentation you are talking about?
> Internal (which results in "dead" disk space), or
> external (which results in a lot of "extents")?
>
External

> Edward.
>
>
>>
>> If we group files of same extension in groups in one directory, when
>> we write files of same extension after deletion of some files of one
>> extension  their group would be in same order as the deleted file so
>> they would be written in similar place and occupy the 'hole' of
>> similar size.
>> Ofc I am not talking about files of few kB size where Reiser4 is great
>> at packing but about files from few MB to few GB.
>>
>> Eg. directory with mp3 and xvid files. mp3s are on the order of MB and
>> xvid on the order of GB. If we sort them just by name order of xvid
>> and mp3 files in one directory would be random so when deleting the
>> smaller ones we would make random holes (like from
>> mxmxmxxmmmxxxxmxxmmmx to mx xmxx  mx  xmx mmmx).
>> With grouping of writing where all mp3s would be written first and all
>> xvid after them after some deletions we would have smaller holes
>> grouped first  and larger last (like from mmmmmmmmmmmmxxxxxxxxxx to mm
>> m   mmm mmxx xxx xxx) but the main thing that after writing we would
>> write mp3s in mp3 holes and xvid in xvid holes ergo. reduce
>> fragmentation  (like from mm m   mmm mmxx xxx xxx to
>> mmMmMMMmmmXmmxxXxxx xxx) that we would create if we would try to write
>> xvid over mp3 holes.
>>
>> One obvious use case where I hypothesize that this type of fibration
>> is better long term would be directories with content similar to usual
>> Downloads directory, a lot of different types (and siyes) of files
>> that get written and deleted a lot.
>>
>> ext_1234 fibration is the same as dot_o for directories with only one
>> or one and .o file extension.
>>
>> Ofc this is just a hypothesis that I would like to prove with some
>> fragmentation benchmarks but I wanted to hear your thoughts.
>>
>> And while I was looking through the code I found a part that I
>> comprehended, elegant and easy to understand so I wanted to make
>> something so I could learn more.
>>
>>
>>> Thanks,
>>> Edward.
>>>
>> Thank you for your time and effort
>>
>> Dushan
>>
>>
>>>
>>>>    so that we will avoid putting
>>>> small files between them and in that way reduce fragmentation. That
>>>> group (xvid 4 character extensions) would be among last groups under
>>>> one directory so that all small files would be written before it.
>>>>
>>>> Problem with the attached patch is that currently every fibre value is
>>>> defined as u64  (eg. static __u64 fibre_ext_3) but if I understood
>>>> correctly comments in kassign.c and fibration.c fibration part of the
>>>> key is only 7 bits long.
>>>> If that is true how did fibre_ext_3 worked?
>>>>
>>>> Thanks
>>>>
>>>> Dushan
>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel"
>> in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>

Re: [RFC] Smart fibration plugin ext_4321

[Edward Shishkin]

On 01/06/2017 05:34 PM, Dušan Čolić wrote:
> On Fri, Jan 6, 2017 at 2:44 PM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>> On 12/26/2016 11:13 PM, Dušan Čolić wrote:
>>> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
>>> <edward.shishkin@gmail.com> wrote:
>>>>
>>>>
>>>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>>>> Fibration is a great way to decrease fragmentation and increase
>>>>> throughput.
>>>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
>>>>> and they all have their upsides and downsides.
>>>>>
>>>>> Proposed fibration plugin combines them all so that it combines files
>>>>> with same extensions for 1, 2. 3 and 4 character extension  in groups
>>>>> and sorts them in same fiber group.
>>>>>
>>>>> With this fibration plugin all eg. xvid files would be in same group
>>>>> in folder on disk sorted alphabetically
>>>>
>>>>
>>>> What application wants all xvid files to be in the same group?
>>>> Do you have any benchmark numbers which show advantages
>>>> of the new plugin?
>>>>
>>> Xvid files are just an example.
>>> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3, ext_4
>>> and dot_o in one.
>>>
>>> In currently default plugin (dot_o) we sort all files by name from the
>>> start except .o files which we put at the end.
>>> So if we had a source directory with .c .h and .o files in it files by
>>> extension would be sorted like: chchchchchchchchoooooooooooooo
>>> I presumed that in some use cases it is better to have files be sorted
>>> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>>>
>>> Hypothesis is to use the premise that files of same extension are in
>>> same order of size to reduce fragmentation.
>>
>>
>> What kind of fragmentation you are talking about?
>> Internal (which results in "dead" disk space), or
>> external (which results in a lot of "extents")?
>>
> External
>
>> Edward.
>>
>>
>>> If we group files of same extension in groups in one directory, when
>>> we write files of same extension after deletion of some files of one
>>> extension  their group would be in same order as the deleted file so
>>> they would be written in similar place and occupy the 'hole' of
>>> similar size.


So "similar" means the same order, that is file sizes can differ in 2 times?
TBH, I don't see what can be deduced from this assumption ;)
It can happen that new file either doesn't fit to that hole, or occupies too
small place, so that next file won't fit to the rest of the hole..

Edward.


>>> Ofc I am not talking about files of few kB size where Reiser4 is great
>>> at packing but about files from few MB to few GB.
>>>
>>> Eg. directory with mp3 and xvid files. mp3s are on the order of MB and
>>> xvid on the order of GB. If we sort them just by name order of xvid
>>> and mp3 files in one directory would be random so when deleting the
>>> smaller ones we would make random holes (like from
>>> mxmxmxxmmmxxxxmxxmmmx to mx xmxx  mx  xmx mmmx).
>>> With grouping of writing where all mp3s would be written first and all
>>> xvid after them after some deletions we would have smaller holes
>>> grouped first  and larger last (like from mmmmmmmmmmmmxxxxxxxxxx to mm
>>> m   mmm mmxx xxx xxx) but the main thing that after writing we would
>>> write mp3s in mp3 holes and xvid in xvid holes ergo. reduce
>>> fragmentation  (like from mm m   mmm mmxx xxx xxx to
>>> mmMmMMMmmmXmmxxXxxx xxx) that we would create if we would try to write
>>> xvid over mp3 holes.
>>>
>>> One obvious use case where I hypothesize that this type of fibration
>>> is better long term would be directories with content similar to usual
>>> Downloads directory, a lot of different types (and siyes) of files
>>> that get written and deleted a lot.
>>>
>>> ext_1234 fibration is the same as dot_o for directories with only one
>>> or one and .o file extension.
>>>
>>> Ofc this is just a hypothesis that I would like to prove with some
>>> fragmentation benchmarks but I wanted to hear your thoughts.
>>>
>>> And while I was looking through the code I found a part that I
>>> comprehended, elegant and easy to understand so I wanted to make
>>> something so I could learn more.
>>>
>>>
>>>> Thanks,
>>>> Edward.
>>>>
>>> Thank you for your time and effort
>>>
>>> Dushan
>>>
>>>
>>>>>     so that we will avoid putting
>>>>> small files between them and in that way reduce fragmentation. That
>>>>> group (xvid 4 character extensions) would be among last groups under
>>>>> one directory so that all small files would be written before it.
>>>>>
>>>>> Problem with the attached patch is that currently every fibre value is
>>>>> defined as u64  (eg. static __u64 fibre_ext_3) but if I understood
>>>>> correctly comments in kassign.c and fibration.c fibration part of the
>>>>> key is only 7 bits long.
>>>>> If that is true how did fibre_ext_3 worked?
>>>>>
>>>>> Thanks
>>>>>
>>>>> Dushan
>>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel"
>>> in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>

Re: [RFC] Smart fibration plugin ext_4321

[Edward Shishkin]

On 01/07/2017 01:09 AM, Dušan Čolić wrote:
> On Fri, Jan 6, 2017 at 8:58 PM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>>
>> On 01/06/2017 05:34 PM, Dušan Čolić wrote:
>>> On Fri, Jan 6, 2017 at 2:44 PM, Edward Shishkin
>>> <edward.shishkin@gmail.com> wrote:
>>>> On 12/26/2016 11:13 PM, Dušan Čolić wrote:
>>>>> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>>>>>> Fibration is a great way to decrease fragmentation and increase
>>>>>>> throughput.
>>>>>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
>>>>>>> and they all have their upsides and downsides.
>>>>>>>
>>>>>>> Proposed fibration plugin combines them all so that it combines files
>>>>>>> with same extensions for 1, 2. 3 and 4 character extension  in groups
>>>>>>> and sorts them in same fiber group.
>>>>>>>
>>>>>>> With this fibration plugin all eg. xvid files would be in same group
>>>>>>> in folder on disk sorted alphabetically
>>>>>>
>>>>>>
>>>>>> What application wants all xvid files to be in the same group?
>>>>>> Do you have any benchmark numbers which show advantages
>>>>>> of the new plugin?
>>>>>>
>>>>> Xvid files are just an example.
>>>>> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3, ext_4
>>>>> and dot_o in one.
>>>>>
>>>>> In currently default plugin (dot_o) we sort all files by name from the
>>>>> start except .o files which we put at the end.
>>>>> So if we had a source directory with .c .h and .o files in it files by
>>>>> extension would be sorted like: chchchchchchchchoooooooooooooo
>>>>> I presumed that in some use cases it is better to have files be sorted
>>>>> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>>>>>
>>>>> Hypothesis is to use the premise that files of same extension are in
>>>>> same order of size to reduce fragmentation.
>>>>
>>>>
>>>> What kind of fragmentation you are talking about?
>>>> Internal (which results in "dead" disk space), or
>>>> external (which results in a lot of "extents")?
>>>>
>>> External
>>>
>>>> Edward.
>>>>
>>>>
>>>>> If we group files of same extension in groups in one directory, when
>>>>> we write files of same extension after deletion of some files of one
>>>>> extension  their group would be in same order as the deleted file so
>>>>> they would be written in similar place and occupy the 'hole' of
>>>>> similar size.
>>
>>
>> So "similar" means the same order, that is file sizes can differ in 2 times?
>> TBH, I don't see what can be deduced from this assumption ;)
>> It can happen that new file either doesn't fit to that hole, or occupies too
>> small place, so that next file won't fit to the rest of the hole..
>>
> OFC we can never guarantee that the new file completely fits the hole
> (especially as we go through compression in next layer) but for both
> smaller and larger file than a hole we would have higher probability
> for less extents for situations with 2 or more types of files in a
> directory. For one type of file in a directory behavior would be the
> same as dot_o and ext_1 plugin.


I should upset you: fibration plugins are about mapping of a semantic
tree to the storage tree. Simply speaking, they manage mapping
object-> key, which has nothing common with real locations on disk.

This is a block allocator, who assigns disk addresses to nodes of the
storage tree (right before writing them to disk at flush time).
And I am sure that block allocator doesn't care about fibration groups.

I strongly not recommend you to experiment with block allocator.
Simply because I know how many people killed a lot of time without
results.

Edward.

Re: [RFC] Smart fibration plugin ext_4321

[Dušan Čolić]

On Fri, Jan 6, 2017 at 8:58 PM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
>
>
> On 01/06/2017 05:34 PM, Dušan Čolić wrote:
>>
>> On Fri, Jan 6, 2017 at 2:44 PM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>>
>>> On 12/26/2016 11:13 PM, Dušan Čolić wrote:
>>>>
>>>> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
>>>> <edward.shishkin@gmail.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>>>>>
>>>>>> Fibration is a great way to decrease fragmentation and increase
>>>>>> throughput.
>>>>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
>>>>>> and they all have their upsides and downsides.
>>>>>>
>>>>>> Proposed fibration plugin combines them all so that it combines files
>>>>>> with same extensions for 1, 2. 3 and 4 character extension  in groups
>>>>>> and sorts them in same fiber group.
>>>>>>
>>>>>> With this fibration plugin all eg. xvid files would be in same group
>>>>>> in folder on disk sorted alphabetically
>>>>>
>>>>>
>>>>>
>>>>> What application wants all xvid files to be in the same group?
>>>>> Do you have any benchmark numbers which show advantages
>>>>> of the new plugin?
>>>>>
>>>> Xvid files are just an example.
>>>> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3, ext_4
>>>> and dot_o in one.
>>>>
>>>> In currently default plugin (dot_o) we sort all files by name from the
>>>> start except .o files which we put at the end.
>>>> So if we had a source directory with .c .h and .o files in it files by
>>>> extension would be sorted like: chchchchchchchchoooooooooooooo
>>>> I presumed that in some use cases it is better to have files be sorted
>>>> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>>>>
>>>> Hypothesis is to use the premise that files of same extension are in
>>>> same order of size to reduce fragmentation.
>>>
>>>
>>>
>>> What kind of fragmentation you are talking about?
>>> Internal (which results in "dead" disk space), or
>>> external (which results in a lot of "extents")?
>>>
>> External
>>
>>> Edward.
>>>
>>>
>>>> If we group files of same extension in groups in one directory, when
>>>> we write files of same extension after deletion of some files of one
>>>> extension  their group would be in same order as the deleted file so
>>>> they would be written in similar place and occupy the 'hole' of
>>>> similar size.
>
>
>
> So "similar" means the same order, that is file sizes can differ in 2 times?
> TBH, I don't see what can be deduced from this assumption ;)
> It can happen that new file either doesn't fit to that hole, or occupies too
> small place, so that next file won't fit to the rest of the hole..
>
OFC we can never guarantee that the new file completely fits the hole
(especially as we go through compression in next layer) but for both
smaller and larger file than a hole we would have higher probability
for less extents for situations with 2 or more types of files in a
directory. For one type of file in a directory behavior would be the
same as dot_o and ext_1 plugin.


> Edward.
>
>
>
>>>> Ofc I am not talking about files of few kB size where Reiser4 is great
>>>> at packing but about files from few MB to few GB.
>>>>
>>>> Eg. directory with mp3 and xvid files. mp3s are on the order of MB and
>>>> xvid on the order of GB. If we sort them just by name order of xvid
>>>> and mp3 files in one directory would be random so when deleting the
>>>> smaller ones we would make random holes (like from
>>>> mxmxmxxmmmxxxxmxxmmmx to mx xmxx  mx  xmx mmmx).
>>>> With grouping of writing where all mp3s would be written first and all
>>>> xvid after them after some deletions we would have smaller holes
>>>> grouped first  and larger last (like from mmmmmmmmmmmmxxxxxxxxxx to mm
>>>> m   mmm mmxx xxx xxx) but the main thing that after writing we would
>>>> write mp3s in mp3 holes and xvid in xvid holes ergo. reduce
>>>> fragmentation  (like from mm m   mmm mmxx xxx xxx to
>>>> mmMmMMMmmmXmmxxXxxx xxx) that we would create if we would try to write
>>>> xvid over mp3 holes.
>>>>
>>>> One obvious use case where I hypothesize that this type of fibration
>>>> is better long term would be directories with content similar to usual
>>>> Downloads directory, a lot of different types (and siyes) of files
>>>> that get written and deleted a lot.
>>>>
>>>> ext_1234 fibration is the same as dot_o for directories with only one
>>>> or one and .o file extension.
>>>>
>>>> Ofc this is just a hypothesis that I would like to prove with some
>>>> fragmentation benchmarks but I wanted to hear your thoughts.
>>>>
>>>> And while I was looking through the code I found a part that I
>>>> comprehended, elegant and easy to understand so I wanted to make
>>>> something so I could learn more.
>>>>
>>>>
>>>>> Thanks,
>>>>> Edward.
>>>>>
>>>> Thank you for your time and effort
>>>>
>>>> Dushan
>>>>
>>>>
>>>>>>     so that we will avoid putting
>>>>>> small files between them and in that way reduce fragmentation. That
>>>>>> group (xvid 4 character extensions) would be among last groups under
>>>>>> one directory so that all small files would be written before it.
>>>>>>
>>>>>> Problem with the attached patch is that currently every fibre value is
>>>>>> defined as u64  (eg. static __u64 fibre_ext_3) but if I understood
>>>>>> correctly comments in kassign.c and fibration.c fibration part of the
>>>>>> key is only 7 bits long.
>>>>>> If that is true how did fibre_ext_3 worked?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Dushan
>>>>>
>>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe
>>>> reiserfs-devel"
>>>> in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>>
>

Re: [RFC] Smart fibration plugin ext_4321

[Edward Shishkin]

On 01/07/2017 10:15 AM, Dušan Čolić wrote:
> On Sat, Jan 7, 2017 at 12:05 AM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>> On 01/07/2017 01:09 AM, Dušan Čolić wrote:
>>> On Fri, Jan 6, 2017 at 8:58 PM, Edward Shishkin
>>> <edward.shishkin@gmail.com> wrote:
>>>>
>>>> On 01/06/2017 05:34 PM, Dušan Čolić wrote:
>>>>> On Fri, Jan 6, 2017 at 2:44 PM, Edward Shishkin
>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>> On 12/26/2016 11:13 PM, Dušan Čolić wrote:
>>>>>>> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
>>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>>>>>>>> Fibration is a great way to decrease fragmentation and increase
>>>>>>>>> throughput.
>>>>>>>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
>>>>>>>>> and they all have their upsides and downsides.
>>>>>>>>>
>>>>>>>>> Proposed fibration plugin combines them all so that it combines
>>>>>>>>> files
>>>>>>>>> with same extensions for 1, 2. 3 and 4 character extension  in
>>>>>>>>> groups
>>>>>>>>> and sorts them in same fiber group.
>>>>>>>>>
>>>>>>>>> With this fibration plugin all eg. xvid files would be in same group
>>>>>>>>> in folder on disk sorted alphabetically
>>>>>>>>
>>>>>>>>
>>>>>>>> What application wants all xvid files to be in the same group?
>>>>>>>> Do you have any benchmark numbers which show advantages
>>>>>>>> of the new plugin?
>>>>>>>>
>>>>>>> Xvid files are just an example.
>>>>>>> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3, ext_4
>>>>>>> and dot_o in one.
>>>>>>>
>>>>>>> In currently default plugin (dot_o) we sort all files by name from the
>>>>>>> start except .o files which we put at the end.
>>>>>>> So if we had a source directory with .c .h and .o files in it files by
>>>>>>> extension would be sorted like: chchchchchchchchoooooooooooooo
>>>>>>> I presumed that in some use cases it is better to have files be sorted
>>>>>>> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>>>>>>>
>>>>>>> Hypothesis is to use the premise that files of same extension are in
>>>>>>> same order of size to reduce fragmentation.
>>>>>>
>>>>>>
>>>>>> What kind of fragmentation you are talking about?
>>>>>> Internal (which results in "dead" disk space), or
>>>>>> external (which results in a lot of "extents")?
>>>>>>
>>>>> External
>>>>>
>>>>>> Edward.
>>>>>>
>>>>>>
>>>>>>> If we group files of same extension in groups in one directory, when
>>>>>>> we write files of same extension after deletion of some files of one
>>>>>>> extension  their group would be in same order as the deleted file so
>>>>>>> they would be written in similar place and occupy the 'hole' of
>>>>>>> similar size.
>>>>
>>>>
>>>> So "similar" means the same order, that is file sizes can differ in 2
>>>> times?
>>>> TBH, I don't see what can be deduced from this assumption ;)
>>>> It can happen that new file either doesn't fit to that hole, or occupies
>>>> too
>>>> small place, so that next file won't fit to the rest of the hole..
>>>>
>>> OFC we can never guarantee that the new file completely fits the hole
>>> (especially as we go through compression in next layer) but for both
>>> smaller and larger file than a hole we would have higher probability
>>> for less extents for situations with 2 or more types of files in a
>>> directory. For one type of file in a directory behavior would be the
>>> same as dot_o and ext_1 plugin.
>>
>>
>> I should upset you: fibration plugins are about mapping of a semantic
>> tree to the storage tree. Simply speaking, they manage mapping
>> object-> key, which has nothing common with real locations on diТак ты уже не ищешь? sk.
>>
>> This is a block allocator, who assigns disk addresses to nodes of the
>> storage tree (right before writing them to disk at flush time).
>> And I am sure that block allocator doesn't care about fibration groups.
>>
>> I strongly not recommend you to experiment with block allocator.
>> Simply because I know how many people killed a lot of time without
>> results.
> Then what is this comment in the beginning of kassign.c about:
>
>
> * In reiser4 every piece of file system data and meta-data has a key. Keys
> * are used to store information in and retrieve it from reiser4 internal
> * tree. In addition to this, keys define _ordering_ of all file system
> * information: things having close keys are placed into the same or
> * neighboring (in the tree order) nodes of the tree. As our block allocator
> * tries to respect tree order (see flush.c), keys also define order in which
> * things are laid out on the disk, and hence, affect performance directly.

I can not find where in the code block allocator respects key ordering.
Once you find it, then let me know..

Thanks,
Edward.

Re: [RFC] Smart fibration plugin ext_4321

[Dušan Čolić]

On Sat, Jan 7, 2017 at 12:05 AM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
> On 01/07/2017 01:09 AM, Dušan Čolić wrote:
>>
>> On Fri, Jan 6, 2017 at 8:58 PM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>>
>>>
>>> On 01/06/2017 05:34 PM, Dušan Čolić wrote:
>>>>
>>>> On Fri, Jan 6, 2017 at 2:44 PM, Edward Shishkin
>>>> <edward.shishkin@gmail.com> wrote:
>>>>>
>>>>> On 12/26/2016 11:13 PM, Dušan Čolić wrote:
>>>>>>
>>>>>> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>>>>>>>
>>>>>>>> Fibration is a great way to decrease fragmentation and increase
>>>>>>>> throughput.
>>>>>>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3
>>>>>>>> and they all have their upsides and downsides.
>>>>>>>>
>>>>>>>> Proposed fibration plugin combines them all so that it combines
>>>>>>>> files
>>>>>>>> with same extensions for 1, 2. 3 and 4 character extension  in
>>>>>>>> groups
>>>>>>>> and sorts them in same fiber group.
>>>>>>>>
>>>>>>>> With this fibration plugin all eg. xvid files would be in same group
>>>>>>>> in folder on disk sorted alphabetically
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> What application wants all xvid files to be in the same group?
>>>>>>> Do you have any benchmark numbers which show advantages
>>>>>>> of the new plugin?
>>>>>>>
>>>>>> Xvid files are just an example.
>>>>>> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3, ext_4
>>>>>> and dot_o in one.
>>>>>>
>>>>>> In currently default plugin (dot_o) we sort all files by name from the
>>>>>> start except .o files which we put at the end.
>>>>>> So if we had a source directory with .c .h and .o files in it files by
>>>>>> extension would be sorted like: chchchchchchchchoooooooooooooo
>>>>>> I presumed that in some use cases it is better to have files be sorted
>>>>>> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>>>>>>
>>>>>> Hypothesis is to use the premise that files of same extension are in
>>>>>> same order of size to reduce fragmentation.
>>>>>
>>>>>
>>>>>
>>>>> What kind of fragmentation you are talking about?
>>>>> Internal (which results in "dead" disk space), or
>>>>> external (which results in a lot of "extents")?
>>>>>
>>>> External
>>>>
>>>>> Edward.
>>>>>
>>>>>
>>>>>> If we group files of same extension in groups in one directory, when
>>>>>> we write files of same extension after deletion of some files of one
>>>>>> extension  their group would be in same order as the deleted file so
>>>>>> they would be written in similar place and occupy the 'hole' of
>>>>>> similar size.
>>>
>>>
>>>
>>> So "similar" means the same order, that is file sizes can differ in 2
>>> times?
>>> TBH, I don't see what can be deduced from this assumption ;)
>>> It can happen that new file either doesn't fit to that hole, or occupies
>>> too
>>> small place, so that next file won't fit to the rest of the hole..
>>>
>> OFC we can never guarantee that the new file completely fits the hole
>> (especially as we go through compression in next layer) but for both
>> smaller and larger file than a hole we would have higher probability
>> for less extents for situations with 2 or more types of files in a
>> directory. For one type of file in a directory behavior would be the
>> same as dot_o and ext_1 plugin.
>
>
>
> I should upset you: fibration plugins are about mapping of a semantic
> tree to the storage tree. Simply speaking, they manage mapping
> object-> key, which has nothing common with real locations on disk.
>
> This is a block allocator, who assigns disk addresses to nodes of the
> storage tree (right before writing them to disk at flush time).
> And I am sure that block allocator doesn't care about fibration groups.
>
> I strongly not recommend you to experiment with block allocator.
> Simply because I know how many people killed a lot of time without
> results.
Then what is this comment in the beginning of kassign.c about:


* In reiser4 every piece of file system data and meta-data has a key. Keys
* are used to store information in and retrieve it from reiser4 internal
* tree. In addition to this, keys define _ordering_ of all file system
* information: things having close keys are placed into the same or
* neighboring (in the tree order) nodes of the tree. As our block allocator
* tries to respect tree order (see flush.c), keys also define order in which
* things are laid out on the disk, and hence, affect performance directly.



>
> Edward.
>

Re: [RFC] Smart fibration plugin ext_4321

[Dušan Čolić]

On Sat, Jan 7, 2017 at 8:58 AM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
>
>
> On 01/07/2017 10:15 AM, Dušan Čolić wrote:
>>
>> On Sat, Jan 7, 2017 at 12:05 AM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>>
>>> On 01/07/2017 01:09 AM, Dušan Čolić wrote:
>>>>
>>>> On Fri, Jan 6, 2017 at 8:58 PM, Edward Shishkin
>>>> <edward.shishkin@gmail.com> wrote:
>>>>>
>>>>>
>>>>> On 01/06/2017 05:34 PM, Dušan Čolić wrote:
>>>>>>
>>>>>> On Fri, Jan 6, 2017 at 2:44 PM, Edward Shishkin
>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>
>>>>>>> On 12/26/2016 11:13 PM, Dušan Čolić wrote:
>>>>>>>>
>>>>>>>> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
>>>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>>>>>>>>>
>>>>>>>>>> Fibration is a great way to decrease fragmentation and increase
>>>>>>>>>> throughput.
>>>>>>>>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and
>>>>>>>>>> ext_3
>>>>>>>>>> and they all have their upsides and downsides.
>>>>>>>>>>
>>>>>>>>>> Proposed fibration plugin combines them all so that it combines
>>>>>>>>>> files
>>>>>>>>>> with same extensions for 1, 2. 3 and 4 character extension  in
>>>>>>>>>> groups
>>>>>>>>>> and sorts them in same fiber group.
>>>>>>>>>>
>>>>>>>>>> With this fibration plugin all eg. xvid files would be in same
>>>>>>>>>> group
>>>>>>>>>> in folder on disk sorted alphabetically
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What application wants all xvid files to be in the same group?
>>>>>>>>> Do you have any benchmark numbers which show advantages
>>>>>>>>> of the new plugin?
>>>>>>>>>
>>>>>>>> Xvid files are just an example.
>>>>>>>> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3,
>>>>>>>> ext_4
>>>>>>>> and dot_o in one.
>>>>>>>>
>>>>>>>> In currently default plugin (dot_o) we sort all files by name from
>>>>>>>> the
>>>>>>>> start except .o files which we put at the end.
>>>>>>>> So if we had a source directory with .c .h and .o files in it files
>>>>>>>> by
>>>>>>>> extension would be sorted like: chchchchchchchchoooooooooooooo
>>>>>>>> I presumed that in some use cases it is better to have files be
>>>>>>>> sorted
>>>>>>>> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>>>>>>>>
>>>>>>>> Hypothesis is to use the premise that files of same extension are in
>>>>>>>> same order of size to reduce fragmentation.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> What kind of fragmentation you are talking about?
>>>>>>> Internal (which results in "dead" disk space), or
>>>>>>> external (which results in a lot of "extents")?
>>>>>>>
>>>>>> External
>>>>>>
>>>>>>> Edward.
>>>>>>>
>>>>>>>
>>>>>>>> If we group files of same extension in groups in one directory, when
>>>>>>>> we write files of same extension after deletion of some files of one
>>>>>>>> extension  their group would be in same order as the deleted file so
>>>>>>>> they would be written in similar place and occupy the 'hole' of
>>>>>>>> similar size.
>>>>>
>>>>>
>>>>>
>>>>> So "similar" means the same order, that is file sizes can differ in 2
>>>>> times?
>>>>> TBH, I don't see what can be deduced from this assumption ;)
>>>>> It can happen that new file either doesn't fit to that hole, or
>>>>> occupies
>>>>> too
>>>>> small place, so that next file won't fit to the rest of the hole..
>>>>>
>>>> OFC we can never guarantee that the new file completely fits the hole
>>>> (especially as we go through compression in next layer) but for both
>>>> smaller and larger file than a hole we would have higher probability
>>>> for less extents for situations with 2 or more types of files in a
>>>> directory. For one type of file in a directory behavior would be the
>>>> same as dot_o and ext_1 plugin.
>>>
>>>
>>>
>>> I should upset you: fibration plugins are about mapping of a semantic
>>> tree to the storage tree. Simply speaking, they manage mapping
>>> object-> key, which has nothing common with real locations on diТак ты
>>> уже не ищешь? sk.
>>>
>>> This is a block allocator, who assigns disk addresses to nodes of the
>>> storage tree (right before writing them to disk at flush time).
>>> And I am sure that block allocator doesn't care about fibration groups.
>>>
>>> I strongly not recommend you to experiment with block allocator.
>>> Simply because I know how many people killed a lot of time without
>>> results.
>>
>> Then what is this comment in the beginning of kassign.c about:
>>
>>
>> * In reiser4 every piece of file system data and meta-data has a key. Keys
>> * are used to store information in and retrieve it from reiser4 internal
>> * tree. In addition to this, keys define _ordering_ of all file system
>> * information: things having close keys are placed into the same or
>> * neighboring (in the tree order) nodes of the tree. As our block
>> allocator
>> * tries to respect tree order (see flush.c), keys also define order in
>> which
>> * things are laid out on the disk, and hence, affect performance directly.
>
>
> I can not find where in the code block allocator respects key ordering.
> Once you find it, then let me know..
>
Let me prove it in a way that is possible for me without investing
years in learning R4 internals ;)

krshina3 test # mkfs.reiser4 -o create=reg40,fibration=ext_1_fibre /dev/md123

krshina3 test # ls -la /home/dusan/test/
total 261
drwxr-xr-x  2 root  root      6 Jan  7 17:51 .
drwxr-xr-x 46 dusan dusan    73 Jan  7 17:48 ..
-rw-r--r--  1 root  root   4096 Jan  7 17:49 1.a
-rw-r--r--  1 root  root   8192 Jan  7 17:49 2.b
-rw-r--r--  1 root  root  12288 Jan  7 17:50 3.a
-rw-r--r--  1 root  root  16384 Jan  7 17:51 4.b

krshina3 test # mount /dev/md123 /mnt/test

krshina3 test # cp /home/dusan/test/* /mnt/test

krshina3 test # umount /dev/md123

krshina3 test # debugfs.reiser4 -t /dev/md123
debugfs.reiser4 1.1.0
Format release: 4.0.1
Copyright (C) 2001-2005 by Hans Reiser, licensing governed by
reiser4progs/COPYING.

NODE (25) LEVEL=2 ITEMS=11 SPACE=3562 MKFS ID=0x51a268e2 FLUSH=0x0
#0  NPTR (nodeptr40): [29:1(SD):0:2a:0] OFF=28, LEN=8, flags=0x0 [24]
------------------------------------------------------------------------------
#1  NPTR (nodeptr40): [2a:4(FB):c2312e6100000000:10000:c6e] OFF=36,
LEN=8, flags=0x0 [26]
------------------------------------------------------------------------------
#2  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:c06] OFF=44,
LEN=8, flags=0x0 [27]
------------------------------------------------------------------------------
#3  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:1bc4] OFF=52,
LEN=8, flags=0x0 [28]
------------------------------------------------------------------------------
#4  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:2b82] OFF=60,
LEN=8, flags=0x0 [29]
------------------------------------------------------------------------------
#5  NPTR (nodeptr40): [2a:4(FB):c4322e6200000000:10001:b1a] OFF=68,
LEN=8, flags=0x0 [30]
------------------------------------------------------------------------------
#6  NPTR (nodeptr40): [2a:4(FB):c4322e6200000000:10001:1ad8] OFF=76,
LEN=8, flags=0x0 [31]
------------------------------------------------------------------------------
#7  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:a70] OFF=84,
LEN=8, flags=0x0 [32]
------------------------------------------------------------------------------
#8  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:1a2e] OFF=92,
LEN=8, flags=0x0 [33]
------------------------------------------------------------------------------
#9  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:29ec] OFF=100,
LEN=8, flags=0x0 [34]
------------------------------------------------------------------------------
#10  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:39aa] OFF=108,
LEN=8, flags=0x0 [35]
==============================================================================
NODE (24) LEVEL=1 ITEMS=7 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
#0  SD (stat40): [29:1(SD):0:2a:0] OFF=28, LEN=94, flags=0x0
exts:        3
mask:        0x13
plugin:        sdext_lw
offset:        2
len:        14
mode:        drwxr-xr-x
nlink:        3
size:        6
plugin:        sdext_unix
offset:        16
len:        28
uid:        0
gid:        0
atime:        Sat Jan  7 17:59:37 2017
mtime:        Sat Jan  7 18:03:59 2017
ctime:        Sat Jan  7 18:03:59 2017
rdev:        300
bytes:        300
plugin:        sdext_plugin_set
offset:        44
len:        50
Pset count:     12
        permission : id = 0
        formatting : id = 2 (smart)
              hash : id = 1 (r5_hash)
         fibration : id = 2 (ext_1_fibre)
          statdata : id = 0 (stat40)
           diritem : id = 2 (cde40)
            crypto : id = 0
            digest : id = 0
          compress : id = 0 (lzo1)
      compressMode : id = 4 (conv)
           cluster : id = 0 (64K)
            create : id = 0 (reg40)
------------------------------------------------------------------------------
#1  DIRITEM (cde40): [2a:0(NAME):0:0:0] OFF=122, LEN=302, flags=0x0
NR(6)  NAME                                  OFFSET HASH
               SDKEY
  0 .                                        158
0000000000000000:0000000000000000 0000291:000002a
  1 ..                                       182
0000000000000000:0000000000000000 0000291:000002a
  2 1.a                                      206
0000000000000000:0000000000000000 00002a1:0010000
  3 3.a                                      230
0000000000000000:0000000000000000 00002a1:0010002
  4 2.b                                      254
0000000000000000:0000000000000000 00002a1:0010001
  5 4.b                                      278
0000000000000000:0000000000000000 00002a1:0010003
------------------------------------------------------------------------------
#2  SD (stat40): [2a:1(SD):c2312e6100000000:10000:0] OFF=424, LEN=56, flags=0x0
exts:        3
mask:        0x7
plugin:        sdext_lw
offset:        2
len:        14
mode:        -rw-r--r--
nlink:        1
size:        4096
plugin:        sdext_unix
offset:        16
len:        28
uid:        0
gid:        0
atime:        Sat Jan  7 18:03:59 2017
mtime:        Sat Jan  7 18:03:59 2017
ctime:        Sat Jan  7 18:03:59 2017
rdev:        4096
bytes:        4096
plugin:        sdext_lt
offset:        44
len:        12
atime:        652286102
mtime:        652286102
ctime:        652286102
------------------------------------------------------------------------------
#3  SD (stat40): [2a:1(SD):c2332e6100000000:10002:0] OFF=480, LEN=56, flags=0x0
exts:        3
mask:        0x7
plugin:        sdext_lw
offset:        2
len:        14
mode:        -rw-r--r--
nlink:        1
size:        12288
plugin:        sdext_unix
offset:        16
len:        28
uid:        0
gid:        0
atime:        Sat Jan  7 18:03:59 2017
mtime:        Sat Jan  7 18:03:59 2017
ctime:        Sat Jan  7 18:03:59 2017
rdev:        12288
bytes:        12288
plugin:        sdext_lt
offset:        44
len:        12
atime:        656286256
mtime:        656286256
ctime:        656286256
------------------------------------------------------------------------------
#4  SD (stat40): [2a:1(SD):c4322e6200000000:10001:0] OFF=536, LEN=56, flags=0x0
exts:        3
mask:        0x7
plugin:        sdext_lw
offset:        2
len:        14
mode:        -rw-r--r--
nlink:        1
size:        8192
plugin:        sdext_unix
offset:        16
len:        28
uid:        0
gid:        0
atime:        Sat Jan  7 18:03:59 2017
mtime:        Sat Jan  7 18:03:59 2017
ctime:        Sat Jan  7 18:03:59 2017
rdev:        8192
bytes:        8192
plugin:        sdext_lt
offset:        44
len:        12
atime:        652286102
mtime:        656286256
ctime:        656286256
------------------------------------------------------------------------------
#5  SD (stat40): [2a:1(SD):c4342e6200000000:10003:0] OFF=592, LEN=56, flags=0x0
exts:        3
mask:        0x7
plugin:        sdext_lw
offset:        2
len:        14
mode:        -rw-r--r--
nlink:        1
size:        16384
plugin:        sdext_unix
offset:        16
len:        28
uid:        0
gid:        0
atime:        Sat Jan  7 18:03:59 2017
mtime:        Sat Jan  7 18:03:59 2017
ctime:        Sat Jan  7 18:03:59 2017
rdev:        16384
bytes:        16384
plugin:        sdext_lt
offset:        44
len:        12
atime:        656286256
mtime:        656286256
ctime:        656286256
------------------------------------------------------------------------------
#6  TAIL (plain40): [2a:4(FB):c2312e6100000000:10000:0] OFF=648,
LEN=3182, flags=0x0
==============================================================================
NODE (26) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):c2312e6100000000:10000:c6e] OFF=28,
LEN=914, flags=0x0
------------------------------------------------------------------------------
#1  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:0] OFF=942,
LEN=3078, flags=0x0
==============================================================================
NODE (27) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:c06] OFF=28,
LEN=4030, flags=0x0
==============================================================================
NODE (28) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:1bc4] OFF=28,
LEN=4030, flags=0x50
==============================================================================
NODE (29) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:2b82] OFF=28,
LEN=1150, flags=0x54
------------------------------------------------------------------------------
#1  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:0] OFF=1178,
LEN=2842, flags=0x0
==============================================================================
NODE (30) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:b1a] OFF=28,
LEN=4030, flags=0x0
==============================================================================
NODE (31) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:1ad8] OFF=28,
LEN=1320, flags=0x0
------------------------------------------------------------------------------
#1  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:0] OFF=1348,
LEN=2672, flags=0x0
==============================================================================
NODE (32) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:a70] OFF=28,
LEN=4030, flags=0x0
==============================================================================
NODE (33) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:1a2e] OFF=28,
LEN=4030, flags=0x63
==============================================================================
NODE (34) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:29ec] OFF=28,
LEN=4030, flags=0xbf
==============================================================================
NODE (35) LEVEL=1 ITEMS=1 SPACE=2408 MKFS ID=0x51a268e2 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:39aa] OFF=28,
LEN=1622, flags=0x1b7
==============================================================================

krshina3 test # mkfs.reiser4 -o create=reg40,fibration=dot_o_fibre /dev/md123

krshina3 test # mount /dev/md123 /mnt/test

krshina3 test # cp /home/dusan/test/* /mnt/test

krshina3 test # umount /dev/md123

krshina3 test # debugfs.reiser4 -t /dev/md123
debugfs.reiser4 1.1.0
Format release: 4.0.1
Copyright (C) 2001-2005 by Hans Reiser, licensing governed by
reiser4progs/COPYING.

NODE (25) LEVEL=2 ITEMS=11 SPACE=3562 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  NPTR (nodeptr40): [29:1(SD):0:2a:0] OFF=28, LEN=8, flags=0x0 [24]
------------------------------------------------------------------------------
#1  NPTR (nodeptr40): [2a:4(FB):312e6100000000:10000:c6e] OFF=36,
LEN=8, flags=0x0 [26]
------------------------------------------------------------------------------
#2  NPTR (nodeptr40): [2a:4(FB):322e6200000000:10001:c06] OFF=44,
LEN=8, flags=0x0 [27]
------------------------------------------------------------------------------
#3  NPTR (nodeptr40): [2a:4(FB):322e6200000000:10001:1bc4] OFF=52,
LEN=8, flags=0x0 [28]
------------------------------------------------------------------------------
#4  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:b5c] OFF=60,
LEN=8, flags=0x0 [29]
------------------------------------------------------------------------------
#5  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:1b1a] OFF=68,
LEN=8, flags=0x0 [30]
------------------------------------------------------------------------------
#6  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:2ad8] OFF=76,
LEN=8, flags=0x0 [31]
------------------------------------------------------------------------------
#7  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:a70] OFF=84,
LEN=8, flags=0x0 [32]
------------------------------------------------------------------------------
#8  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:1a2e] OFF=92,
LEN=8, flags=0x0 [33]
------------------------------------------------------------------------------
#9  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:29ec] OFF=100,
LEN=8, flags=0x0 [34]
------------------------------------------------------------------------------
#10  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:39aa] OFF=108,
LEN=8, flags=0x0 [35]
==============================================================================
NODE (24) LEVEL=1 ITEMS=7 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  SD (stat40): [29:1(SD):0:2a:0] OFF=28, LEN=94, flags=0x0
exts:        3
mask:        0x13
plugin:        sdext_lw
offset:        2
len:        14
mode:        drwxr-xr-x
nlink:        3
size:        6
plugin:        sdext_unix
offset:        16
len:        28
uid:        0
gid:        0
atime:        Sat Jan  7 18:05:26 2017
mtime:        Sat Jan  7 18:06:31 2017
ctime:        Sat Jan  7 18:06:31 2017
rdev:        300
bytes:        300
plugin:        sdext_plugin_set
offset:        44
len:        50
Pset count:     12
        permission : id = 0
        formatting : id = 2 (smart)
              hash : id = 1 (r5_hash)
         fibration : id = 1 (dot_o_fibre)
          statdata : id = 0 (stat40)
           diritem : id = 2 (cde40)
            crypto : id = 0
            digest : id = 0
          compress : id = 0 (lzo1)
      compressMode : id = 4 (conv)
           cluster : id = 0 (64K)
            create : id = 0 (reg40)
------------------------------------------------------------------------------
#1  DIRITEM (cde40): [2a:0(NAME):0:0:0] OFF=122, LEN=302, flags=0x0
NR(6)  NAME                                  OFFSET HASH
               SDKEY
  0 .                                        158
0000000000000000:0000000000000000 0000291:000002a
  1 ..                                       182
0000000000000000:0000000000000000 0000291:000002a
  2 1.a                                      206
0000000000000000:0000000000000000 00002a1:0010000
  3 2.b                                      230
0000000000000000:0000000000000000 00002a1:0010001
  4 3.a                                      254
0000000000000000:0000000000000000 00002a1:0010002
  5 4.b                                      278
0000000000000000:0000000000000000 00002a1:0010003
------------------------------------------------------------------------------
#2  SD (stat40): [2a:1(SD):312e6100000000:10000:0] OFF=424, LEN=56, flags=0x0
exts:        3
mask:        0x7
plugin:        sdext_lw
offset:        2
len:        14
mode:        -rw-r--r--
nlink:        1
size:        4096
plugin:        sdext_unix
offset:        16
len:        28
uid:        0
gid:        0
atime:        Sat Jan  7 18:06:31 2017
mtime:        Sat Jan  7 18:06:31 2017
ctime:        Sat Jan  7 18:06:31 2017
rdev:        4096
bytes:        4096
plugin:        sdext_lt
offset:        44
len:        12
atime:        778212799
mtime:        778212799
ctime:        778212799
------------------------------------------------------------------------------
#3  SD (stat40): [2a:1(SD):322e6200000000:10001:0] OFF=480, LEN=56, flags=0x0
exts:        3
mask:        0x7
plugin:        sdext_lw
offset:        2
len:        14
mode:        -rw-r--r--
nlink:        1
size:        8192
plugin:        sdext_unix
offset:        16
len:        28
uid:        0
gid:        0
atime:        Sat Jan  7 18:06:31 2017
mtime:        Sat Jan  7 18:06:31 2017
ctime:        Sat Jan  7 18:06:31 2017
rdev:        8192
bytes:        8192
plugin:        sdext_lt
offset:        44
len:        12
atime:        778212799
mtime:        778212799
ctime:        778212799
------------------------------------------------------------------------------
#4  SD (stat40): [2a:1(SD):332e6100000000:10002:0] OFF=536, LEN=56, flags=0x0
exts:        3
mask:        0x7
plugin:        sdext_lw
offset:        2
len:        14
mode:        -rw-r--r--
nlink:        1
size:        12288
plugin:        sdext_unix
offset:        16
len:        28
uid:        0
gid:        0
atime:        Sat Jan  7 18:06:31 2017
mtime:        Sat Jan  7 18:06:31 2017
ctime:        Sat Jan  7 18:06:31 2017
rdev:        12288
bytes:        12288
plugin:        sdext_lt
offset:        44
len:        12
atime:        782212955
mtime:        782212955
ctime:        782212955
------------------------------------------------------------------------------
#5  SD (stat40): [2a:1(SD):342e6200000000:10003:0] OFF=592, LEN=56, flags=0x0
exts:        3
mask:        0x7
plugin:        sdext_lw
offset:        2
len:        14
mode:        -rw-r--r--
nlink:        1
size:        16384
plugin:        sdext_unix
offset:        16
len:        28
uid:        0
gid:        0
atime:        Sat Jan  7 18:06:31 2017
mtime:        Sat Jan  7 18:06:31 2017
ctime:        Sat Jan  7 18:06:31 2017
rdev:        16384
bytes:        16384
plugin:        sdext_lt
offset:        44
len:        12
atime:        782212955
mtime:        782212955
ctime:        782212955
------------------------------------------------------------------------------
#6  TAIL (plain40): [2a:4(FB):312e6100000000:10000:0] OFF=648,
LEN=3182, flags=0x0
==============================================================================
NODE (26) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):312e6100000000:10000:c6e] OFF=28,
LEN=914, flags=0x0
------------------------------------------------------------------------------
#1  TAIL (plain40): [2a:4(FB):322e6200000000:10001:0] OFF=942,
LEN=3078, flags=0x0
==============================================================================
NODE (27) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):322e6200000000:10001:c06] OFF=28,
LEN=4030, flags=0x0
==============================================================================
NODE (28) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):322e6200000000:10001:1bc4] OFF=28,
LEN=1084, flags=0x0
------------------------------------------------------------------------------
#1  TAIL (plain40): [2a:4(FB):332e6100000000:10002:0] OFF=1112,
LEN=2908, flags=0x0
==============================================================================
NODE (29) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:b5c] OFF=28,
LEN=4030, flags=0x0
==============================================================================
NODE (30) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:1b1a] OFF=28,
LEN=4030, flags=0xc
==============================================================================
NODE (31) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:2ad8] OFF=28,
LEN=1320, flags=0x1cf
------------------------------------------------------------------------------
#1  TAIL (plain40): [2a:4(FB):342e6200000000:10003:0] OFF=1348,
LEN=2672, flags=0x0
==============================================================================
NODE (32) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:a70] OFF=28,
LEN=4030, flags=0x0
==============================================================================
NODE (33) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:1a2e] OFF=28,
LEN=4030, flags=0x21
==============================================================================
NODE (34) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:29ec] OFF=28,
LEN=4030, flags=0x1cf
==============================================================================
NODE (35) LEVEL=1 ITEMS=1 SPACE=2408 MKFS ID=0x1bb9bd27 FLUSH=0x0
#0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:39aa] OFF=28,
LEN=1622, flags=0x16
==============================================================================


As we can see with ext_1_fibre files were sorted:
1.a
3.a
2.b
4.b

With dot_o_fibre:

1.a
2.b
3.a
4.b


> Thanks,
> Edward.

Re: [RFC] Smart fibration plugin ext_4321

[Edward Shishkin]

On 01/07/2017 07:10 PM, Dušan Čolić wrote:
> On Sat, Jan 7, 2017 at 8:58 AM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>>
>> On 01/07/2017 10:15 AM, Dušan Čolić wrote:
>>> On Sat, Jan 7, 2017 at 12:05 AM, Edward Shishkin
>>> <edward.shishkin@gmail.com> wrote:
>>>> On 01/07/2017 01:09 AM, Dušan Čolić wrote:
>>>>> On Fri, Jan 6, 2017 at 8:58 PM, Edward Shishkin
>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>
>>>>>> On 01/06/2017 05:34 PM, Dušan Čolić wrote:
>>>>>>> On Fri, Jan 6, 2017 at 2:44 PM, Edward Shishkin
>>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>> On 12/26/2016 11:13 PM, Dušan Čolić wrote:
>>>>>>>>> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
>>>>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>>>>>>>>>> Fibration is a great way to decrease fragmentation and increase
>>>>>>>>>>> throughput.
>>>>>>>>>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and
>>>>>>>>>>> ext_3
>>>>>>>>>>> and they all have their upsides and downsides.
>>>>>>>>>>>
>>>>>>>>>>> Proposed fibration plugin combines them all so that it combines
>>>>>>>>>>> files
>>>>>>>>>>> with same extensions for 1, 2. 3 and 4 character extension  in
>>>>>>>>>>> groups
>>>>>>>>>>> and sorts them in same fiber group.
>>>>>>>>>>>
>>>>>>>>>>> With this fibration plugin all eg. xvid files would be in same
>>>>>>>>>>> group
>>>>>>>>>>> in folder on disk sorted alphabetically
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> What application wants all xvid files to be in the same group?
>>>>>>>>>> Do you have any benchmark numbers which show advantages
>>>>>>>>>> of the new plugin?
>>>>>>>>>>
>>>>>>>>> Xvid files are just an example.
>>>>>>>>> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3,
>>>>>>>>> ext_4
>>>>>>>>> and dot_o in one.
>>>>>>>>>
>>>>>>>>> In currently default plugin (dot_o) we sort all files by name from
>>>>>>>>> the
>>>>>>>>> start except .o files which we put at the end.
>>>>>>>>> So if we had a source directory with .c .h and .o files in it files
>>>>>>>>> by
>>>>>>>>> extension would be sorted like: chchchchchchchchoooooooooooooo
>>>>>>>>> I presumed that in some use cases it is better to have files be
>>>>>>>>> sorted
>>>>>>>>> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>>>>>>>>>
>>>>>>>>> Hypothesis is to use the premise that files of same extension are in
>>>>>>>>> same order of size to reduce fragmentation.
>>>>>>>>
>>>>>>>>
>>>>>>>> What kind of fragmentation you are talking about?
>>>>>>>> Internal (which results in "dead" disk space), or
>>>>>>>> external (which results in a lot of "extents")?
>>>>>>>>
>>>>>>> External
>>>>>>>
>>>>>>>> Edward.
>>>>>>>>
>>>>>>>>
>>>>>>>>> If we group files of same extension in groups in one directory, when
>>>>>>>>> we write files of same extension after deletion of some files of one
>>>>>>>>> extension  their group would be in same order as the deleted file so
>>>>>>>>> they would be written in similar place and occupy the 'hole' of
>>>>>>>>> similar size.
>>>>>>
>>>>>>
>>>>>> So "similar" means the same order, that is file sizes can differ in 2
>>>>>> times?
>>>>>> TBH, I don't see what can be deduced from this assumption ;)
>>>>>> It can happen that new file either doesn't fit to that hole, or
>>>>>> occupies
>>>>>> too
>>>>>> small place, so that next file won't fit to the rest of the hole..
>>>>>>
>>>>> OFC we can never guarantee that the new file completely fits the hole
>>>>> (especially as we go through compression in next layer) but for both
>>>>> smaller and larger file than a hole we would have higher probability
>>>>> for less extents for situations with 2 or more types of files in a
>>>>> directory. For one type of file in a directory behavior would be the
>>>>> same as dot_o and ext_1 plugin.
>>>>
>>>>
>>>> I should upset you: fibration plugins are about mapping of a semantic
>>>> tree to the storage tree. Simply speaking, they manage mapping
>>>> object-> key, which has nothing common with real locations on diТак ты
>>>> уже не ищешь? sk.
>>>>
>>>> This is a block allocator, who assigns disk addresses to nodes of the
>>>> storage tree (right before writing them to disk at flush time).
>>>> And I am sure that block allocator doesn't care about fibration groups.
>>>>
>>>> I strongly not recommend you to experiment with block allocator.
>>>> Simply because I know how many people killed a lot of time without
>>>> results.
>>> Then what is this comment in the beginning of kassign.c about:
>>>
>>>
>>> * In reiser4 every piece of file system data and meta-data has a key. Keys
>>> * are used to store information in and retrieve it from reiser4 internal
>>> * tree. In addition to this, keys define _ordering_ of all file system
>>> * information: things having close keys are placed into the same or
>>> * neighboring (in the tree order) nodes of the tree. As our block
>>> allocator
>>> * tries to respect tree order (see flush.c), keys also define order in
>>> which
>>> * things are laid out on the disk, and hence, affect performance directly.
>>
>> I can not find where in the code block allocator respects key ordering.
>> Once you find it, then let me know..
>>
> Let me prove it in a way that is possible for me without investing
> years in learning R4 internals ;)
>
> krshina3 test # mkfs.reiser4 -o create=reg40,fibration=ext_1_fibre /dev/md123
>
> krshina3 test # ls -la /home/dusan/test/
> total 261
> drwxr-xr-x  2 root  root      6 Jan  7 17:51 .
> drwxr-xr-x 46 dusan dusan    73 Jan  7 17:48 ..
> -rw-r--r--  1 root  root   4096 Jan  7 17:49 1.a
> -rw-r--r--  1 root  root   8192 Jan  7 17:49 2.b
> -rw-r--r--  1 root  root  12288 Jan  7 17:50 3.a
> -rw-r--r--  1 root  root  16384 Jan  7 17:51 4.b
>
> krshina3 test # mount /dev/md123 /mnt/test
>
> krshina3 test # cp /home/dusan/test/* /mnt/test
>
> krshina3 test # umount /dev/md123
>
> krshina3 test # debugfs.reiser4 -t /dev/md123
> debugfs.reiser4 1.1.0
> Format release: 4.0.1
> Copyright (C) 2001-2005 by Hans Reiser, licensing governed by
> reiser4progs/COPYING.
>
> NODE (25) LEVEL=2 ITEMS=11 SPACE=3562 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  NPTR (nodeptr40): [29:1(SD):0:2a:0] OFF=28, LEN=8, flags=0x0 [24]
> ------------------------------------------------------------------------------
> #1  NPTR (nodeptr40): [2a:4(FB):c2312e6100000000:10000:c6e] OFF=36,
> LEN=8, flags=0x0 [26]
> ------------------------------------------------------------------------------
> #2  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:c06] OFF=44,
> LEN=8, flags=0x0 [27]
> ------------------------------------------------------------------------------
> #3  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:1bc4] OFF=52,
> LEN=8, flags=0x0 [28]
> ------------------------------------------------------------------------------
> #4  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:2b82] OFF=60,
> LEN=8, flags=0x0 [29]
> ------------------------------------------------------------------------------
> #5  NPTR (nodeptr40): [2a:4(FB):c4322e6200000000:10001:b1a] OFF=68,
> LEN=8, flags=0x0 [30]
> ------------------------------------------------------------------------------
> #6  NPTR (nodeptr40): [2a:4(FB):c4322e6200000000:10001:1ad8] OFF=76,
> LEN=8, flags=0x0 [31]
> ------------------------------------------------------------------------------
> #7  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:a70] OFF=84,
> LEN=8, flags=0x0 [32]
> ------------------------------------------------------------------------------
> #8  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:1a2e] OFF=92,
> LEN=8, flags=0x0 [33]
> ------------------------------------------------------------------------------
> #9  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:29ec] OFF=100,
> LEN=8, flags=0x0 [34]
> ------------------------------------------------------------------------------
> #10  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:39aa] OFF=108,
> LEN=8, flags=0x0 [35]
> ==============================================================================
> NODE (24) LEVEL=1 ITEMS=7 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  SD (stat40): [29:1(SD):0:2a:0] OFF=28, LEN=94, flags=0x0
> exts:        3
> mask:        0x13
> plugin:        sdext_lw
> offset:        2
> len:        14
> mode:        drwxr-xr-x
> nlink:        3
> size:        6
> plugin:        sdext_unix
> offset:        16
> len:        28
> uid:        0
> gid:        0
> atime:        Sat Jan  7 17:59:37 2017
> mtime:        Sat Jan  7 18:03:59 2017
> ctime:        Sat Jan  7 18:03:59 2017
> rdev:        300
> bytes:        300
> plugin:        sdext_plugin_set
> offset:        44
> len:        50
> Pset count:     12
>          permission : id = 0
>          formatting : id = 2 (smart)
>                hash : id = 1 (r5_hash)
>           fibration : id = 2 (ext_1_fibre)
>            statdata : id = 0 (stat40)
>             diritem : id = 2 (cde40)
>              crypto : id = 0
>              digest : id = 0
>            compress : id = 0 (lzo1)
>        compressMode : id = 4 (conv)
>             cluster : id = 0 (64K)
>              create : id = 0 (reg40)
> ------------------------------------------------------------------------------
> #1  DIRITEM (cde40): [2a:0(NAME):0:0:0] OFF=122, LEN=302, flags=0x0
> NR(6)  NAME                                  OFFSET HASH
>                 SDKEY
>    0 .                                        158
> 0000000000000000:0000000000000000 0000291:000002a
>    1 ..                                       182
> 0000000000000000:0000000000000000 0000291:000002a
>    2 1.a                                      206
> 0000000000000000:0000000000000000 00002a1:0010000
>    3 3.a                                      230
> 0000000000000000:0000000000000000 00002a1:0010002
>    4 2.b                                      254
> 0000000000000000:0000000000000000 00002a1:0010001
>    5 4.b                                      278
> 0000000000000000:0000000000000000 00002a1:0010003
> ------------------------------------------------------------------------------
> #2  SD (stat40): [2a:1(SD):c2312e6100000000:10000:0] OFF=424, LEN=56, flags=0x0
> exts:        3
> mask:        0x7
> plugin:        sdext_lw
> offset:        2
> len:        14
> mode:        -rw-r--r--
> nlink:        1
> size:        4096
> plugin:        sdext_unix
> offset:        16
> len:        28
> uid:        0
> gid:        0
> atime:        Sat Jan  7 18:03:59 2017
> mtime:        Sat Jan  7 18:03:59 2017
> ctime:        Sat Jan  7 18:03:59 2017
> rdev:        4096
> bytes:        4096
> plugin:        sdext_lt
> offset:        44
> len:        12
> atime:        652286102
> mtime:        652286102
> ctime:        652286102
> ------------------------------------------------------------------------------
> #3  SD (stat40): [2a:1(SD):c2332e6100000000:10002:0] OFF=480, LEN=56, flags=0x0
> exts:        3
> mask:        0x7
> plugin:        sdext_lw
> offset:        2
> len:        14
> mode:        -rw-r--r--
> nlink:        1
> size:        12288
> plugin:        sdext_unix
> offset:        16
> len:        28
> uid:        0
> gid:        0
> atime:        Sat Jan  7 18:03:59 2017
> mtime:        Sat Jan  7 18:03:59 2017
> ctime:        Sat Jan  7 18:03:59 2017
> rdev:        12288
> bytes:        12288
> plugin:        sdext_lt
> offset:        44
> len:        12
> atime:        656286256
> mtime:        656286256
> ctime:        656286256
> ------------------------------------------------------------------------------
> #4  SD (stat40): [2a:1(SD):c4322e6200000000:10001:0] OFF=536, LEN=56, flags=0x0
> exts:        3
> mask:        0x7
> plugin:        sdext_lw
> offset:        2
> len:        14
> mode:        -rw-r--r--
> nlink:        1
> size:        8192
> plugin:        sdext_unix
> offset:        16
> len:        28
> uid:        0
> gid:        0
> atime:        Sat Jan  7 18:03:59 2017
> mtime:        Sat Jan  7 18:03:59 2017
> ctime:        Sat Jan  7 18:03:59 2017
> rdev:        8192
> bytes:        8192
> plugin:        sdext_lt
> offset:        44
> len:        12
> atime:        652286102
> mtime:        656286256
> ctime:        656286256
> ------------------------------------------------------------------------------
> #5  SD (stat40): [2a:1(SD):c4342e6200000000:10003:0] OFF=592, LEN=56, flags=0x0
> exts:        3
> mask:        0x7
> plugin:        sdext_lw
> offset:        2
> len:        14
> mode:        -rw-r--r--
> nlink:        1
> size:        16384
> plugin:        sdext_unix
> offset:        16
> len:        28
> uid:        0
> gid:        0
> atime:        Sat Jan  7 18:03:59 2017
> mtime:        Sat Jan  7 18:03:59 2017
> ctime:        Sat Jan  7 18:03:59 2017
> rdev:        16384
> bytes:        16384
> plugin:        sdext_lt
> offset:        44
> len:        12
> atime:        656286256
> mtime:        656286256
> ctime:        656286256
> ------------------------------------------------------------------------------
> #6  TAIL (plain40): [2a:4(FB):c2312e6100000000:10000:0] OFF=648,
> LEN=3182, flags=0x0
> ==============================================================================
> NODE (26) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):c2312e6100000000:10000:c6e] OFF=28,
> LEN=914, flags=0x0
> ------------------------------------------------------------------------------
> #1  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:0] OFF=942,
> LEN=3078, flags=0x0
> ==============================================================================
> NODE (27) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:c06] OFF=28,
> LEN=4030, flags=0x0
> ==============================================================================
> NODE (28) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:1bc4] OFF=28,
> LEN=4030, flags=0x50
> ==============================================================================
> NODE (29) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:2b82] OFF=28,
> LEN=1150, flags=0x54
> ------------------------------------------------------------------------------
> #1  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:0] OFF=1178,
> LEN=2842, flags=0x0
> ==============================================================================
> NODE (30) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:b1a] OFF=28,
> LEN=4030, flags=0x0
> ==============================================================================
> NODE (31) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:1ad8] OFF=28,
> LEN=1320, flags=0x0
> ------------------------------------------------------------------------------
> #1  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:0] OFF=1348,
> LEN=2672, flags=0x0
> ==============================================================================
> NODE (32) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:a70] OFF=28,
> LEN=4030, flags=0x0
> ==============================================================================
> NODE (33) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:1a2e] OFF=28,
> LEN=4030, flags=0x63
> ==============================================================================
> NODE (34) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:29ec] OFF=28,
> LEN=4030, flags=0xbf
> ==============================================================================
> NODE (35) LEVEL=1 ITEMS=1 SPACE=2408 MKFS ID=0x51a268e2 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:39aa] OFF=28,
> LEN=1622, flags=0x1b7
> ==============================================================================
>
> krshina3 test # mkfs.reiser4 -o create=reg40,fibration=dot_o_fibre /dev/md123
>
> krshina3 test # mount /dev/md123 /mnt/test
>
> krshina3 test # cp /home/dusan/test/* /mnt/test
>
> krshina3 test # umount /dev/md123
>
> krshina3 test # debugfs.reiser4 -t /dev/md123
> debugfs.reiser4 1.1.0
> Format release: 4.0.1
> Copyright (C) 2001-2005 by Hans Reiser, licensing governed by
> reiser4progs/COPYING.
>
> NODE (25) LEVEL=2 ITEMS=11 SPACE=3562 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  NPTR (nodeptr40): [29:1(SD):0:2a:0] OFF=28, LEN=8, flags=0x0 [24]
> ------------------------------------------------------------------------------
> #1  NPTR (nodeptr40): [2a:4(FB):312e6100000000:10000:c6e] OFF=36,
> LEN=8, flags=0x0 [26]
> ------------------------------------------------------------------------------
> #2  NPTR (nodeptr40): [2a:4(FB):322e6200000000:10001:c06] OFF=44,
> LEN=8, flags=0x0 [27]
> ------------------------------------------------------------------------------
> #3  NPTR (nodeptr40): [2a:4(FB):322e6200000000:10001:1bc4] OFF=52,
> LEN=8, flags=0x0 [28]
> ------------------------------------------------------------------------------
> #4  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:b5c] OFF=60,
> LEN=8, flags=0x0 [29]
> ------------------------------------------------------------------------------
> #5  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:1b1a] OFF=68,
> LEN=8, flags=0x0 [30]
> ------------------------------------------------------------------------------
> #6  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:2ad8] OFF=76,
> LEN=8, flags=0x0 [31]
> ------------------------------------------------------------------------------
> #7  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:a70] OFF=84,
> LEN=8, flags=0x0 [32]
> ------------------------------------------------------------------------------
> #8  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:1a2e] OFF=92,
> LEN=8, flags=0x0 [33]
> ------------------------------------------------------------------------------
> #9  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:29ec] OFF=100,
> LEN=8, flags=0x0 [34]
> ------------------------------------------------------------------------------
> #10  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:39aa] OFF=108,
> LEN=8, flags=0x0 [35]
> ==============================================================================
> NODE (24) LEVEL=1 ITEMS=7 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  SD (stat40): [29:1(SD):0:2a:0] OFF=28, LEN=94, flags=0x0
> exts:        3
> mask:        0x13
> plugin:        sdext_lw
> offset:        2
> len:        14
> mode:        drwxr-xr-x
> nlink:        3
> size:        6
> plugin:        sdext_unix
> offset:        16
> len:        28
> uid:        0
> gid:        0
> atime:        Sat Jan  7 18:05:26 2017
> mtime:        Sat Jan  7 18:06:31 2017
> ctime:        Sat Jan  7 18:06:31 2017
> rdev:        300
> bytes:        300
> plugin:        sdext_plugin_set
> offset:        44
> len:        50
> Pset count:     12
>          permission : id = 0
>          formatting : id = 2 (smart)
>                hash : id = 1 (r5_hash)
>           fibration : id = 1 (dot_o_fibre)
>            statdata : id = 0 (stat40)
>             diritem : id = 2 (cde40)
>              crypto : id = 0
>              digest : id = 0
>            compress : id = 0 (lzo1)
>        compressMode : id = 4 (conv)
>             cluster : id = 0 (64K)
>              create : id = 0 (reg40)
> ------------------------------------------------------------------------------
> #1  DIRITEM (cde40): [2a:0(NAME):0:0:0] OFF=122, LEN=302, flags=0x0
> NR(6)  NAME                                  OFFSET HASH
>                 SDKEY
>    0 .                                        158
> 0000000000000000:0000000000000000 0000291:000002a
>    1 ..                                       182
> 0000000000000000:0000000000000000 0000291:000002a
>    2 1.a                                      206
> 0000000000000000:0000000000000000 00002a1:0010000
>    3 2.b                                      230
> 0000000000000000:0000000000000000 00002a1:0010001
>    4 3.a                                      254
> 0000000000000000:0000000000000000 00002a1:0010002
>    5 4.b                                      278
> 0000000000000000:0000000000000000 00002a1:0010003
> ------------------------------------------------------------------------------
> #2  SD (stat40): [2a:1(SD):312e6100000000:10000:0] OFF=424, LEN=56, flags=0x0
> exts:        3
> mask:        0x7
> plugin:        sdext_lw
> offset:        2
> len:        14
> mode:        -rw-r--r--
> nlink:        1
> size:        4096
> plugin:        sdext_unix
> offset:        16
> len:        28
> uid:        0
> gid:        0
> atime:        Sat Jan  7 18:06:31 2017
> mtime:        Sat Jan  7 18:06:31 2017
> ctime:        Sat Jan  7 18:06:31 2017
> rdev:        4096
> bytes:        4096
> plugin:        sdext_lt
> offset:        44
> len:        12
> atime:        778212799
> mtime:        778212799
> ctime:        778212799
> ------------------------------------------------------------------------------
> #3  SD (stat40): [2a:1(SD):322e6200000000:10001:0] OFF=480, LEN=56, flags=0x0
> exts:        3
> mask:        0x7
> plugin:        sdext_lw
> offset:        2
> len:        14
> mode:        -rw-r--r--
> nlink:        1
> size:        8192
> plugin:        sdext_unix
> offset:        16
> len:        28
> uid:        0
> gid:        0
> atime:        Sat Jan  7 18:06:31 2017
> mtime:        Sat Jan  7 18:06:31 2017
> ctime:        Sat Jan  7 18:06:31 2017
> rdev:        8192
> bytes:        8192
> plugin:        sdext_lt
> offset:        44
> len:        12
> atime:        778212799
> mtime:        778212799
> ctime:        778212799
> ------------------------------------------------------------------------------
> #4  SD (stat40): [2a:1(SD):332e6100000000:10002:0] OFF=536, LEN=56, flags=0x0
> exts:        3
> mask:        0x7
> plugin:        sdext_lw
> offset:        2
> len:        14
> mode:        -rw-r--r--
> nlink:        1
> size:        12288
> plugin:        sdext_unix
> offset:        16
> len:        28
> uid:        0
> gid:        0
> atime:        Sat Jan  7 18:06:31 2017
> mtime:        Sat Jan  7 18:06:31 2017
> ctime:        Sat Jan  7 18:06:31 2017
> rdev:        12288
> bytes:        12288
> plugin:        sdext_lt
> offset:        44
> len:        12
> atime:        782212955
> mtime:        782212955
> ctime:        782212955
> ------------------------------------------------------------------------------
> #5  SD (stat40): [2a:1(SD):342e6200000000:10003:0] OFF=592, LEN=56, flags=0x0
> exts:        3
> mask:        0x7
> plugin:        sdext_lw
> offset:        2
> len:        14
> mode:        -rw-r--r--
> nlink:        1
> size:        16384
> plugin:        sdext_unix
> offset:        16
> len:        28
> uid:        0
> gid:        0
> atime:        Sat Jan  7 18:06:31 2017
> mtime:        Sat Jan  7 18:06:31 2017
> ctime:        Sat Jan  7 18:06:31 2017
> rdev:        16384
> bytes:        16384
> plugin:        sdext_lt
> offset:        44
> len:        12
> atime:        782212955
> mtime:        782212955
> ctime:        782212955
> ------------------------------------------------------------------------------
> #6  TAIL (plain40): [2a:4(FB):312e6100000000:10000:0] OFF=648,
> LEN=3182, flags=0x0
> ==============================================================================
> NODE (26) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):312e6100000000:10000:c6e] OFF=28,
> LEN=914, flags=0x0
> ------------------------------------------------------------------------------
> #1  TAIL (plain40): [2a:4(FB):322e6200000000:10001:0] OFF=942,
> LEN=3078, flags=0x0
> ==============================================================================
> NODE (27) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):322e6200000000:10001:c06] OFF=28,
> LEN=4030, flags=0x0
> ==============================================================================
> NODE (28) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):322e6200000000:10001:1bc4] OFF=28,
> LEN=1084, flags=0x0
> ------------------------------------------------------------------------------
> #1  TAIL (plain40): [2a:4(FB):332e6100000000:10002:0] OFF=1112,
> LEN=2908, flags=0x0
> ==============================================================================
> NODE (29) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:b5c] OFF=28,
> LEN=4030, flags=0x0
> ==============================================================================
> NODE (30) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:1b1a] OFF=28,
> LEN=4030, flags=0xc
> ==============================================================================
> NODE (31) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:2ad8] OFF=28,
> LEN=1320, flags=0x1cf
> ------------------------------------------------------------------------------
> #1  TAIL (plain40): [2a:4(FB):342e6200000000:10003:0] OFF=1348,
> LEN=2672, flags=0x0
> ==============================================================================
> NODE (32) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:a70] OFF=28,
> LEN=4030, flags=0x0
> ==============================================================================
> NODE (33) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:1a2e] OFF=28,
> LEN=4030, flags=0x21
> ==============================================================================
> NODE (34) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:29ec] OFF=28,
> LEN=4030, flags=0x1cf
> ==============================================================================
> NODE (35) LEVEL=1 ITEMS=1 SPACE=2408 MKFS ID=0x1bb9bd27 FLUSH=0x0
> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:39aa] OFF=28,
> LEN=1622, flags=0x16
> ==============================================================================
>
>
> As we can see with ext_1_fibre files were sorted:
> 1.a
> 3.a
> 2.b
> 4.b
>
> With dot_o_fibre:
>
> 1.a
> 2.b
> 3.a
> 4.b
>

Thus, respect for fibration groups is also under a big question.
TBH, I prefer to not kill time for projects basing on questionable
assumptions and deductions. Just because there is a lot of
interesting TODOs with tangible results.

Thanks,
Edward.

Re: [RFC] Smart fibration plugin ext_4321

[Edward Shishkin]

On 01/07/2017 09:26 PM, Dušan Čolić wrote:
> On Sat, Jan 7, 2017 at 6:56 PM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>>
>> On 01/07/2017 07:10 PM, Dušan Čolić wrote:
>>> On Sat, Jan 7, 2017 at 8:58 AM, Edward Shishkin
>>> <edward.shishkin@gmail.com> wrote:
>>>>
>>>> On 01/07/2017 10:15 AM, Dušan Čolić wrote:
>>>>> On Sat, Jan 7, 2017 at 12:05 AM, Edward Shishkin
>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>> On 01/07/2017 01:09 AM, Dušan Čolić wrote:
>>>>>>> On Fri, Jan 6, 2017 at 8:58 PM, Edward Shishkin
>>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>>
>>>>>>>> On 01/06/2017 05:34 PM, Dušan Čolić wrote:
>>>>>>>>> On Fri, Jan 6, 2017 at 2:44 PM, Edward Shishkin
>>>>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>>>> On 12/26/2016 11:13 PM, Dušan Čolić wrote:
>>>>>>>>>>> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
>>>>>>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>>>>>>>>>>>> Fibration is a great way to decrease fragmentation and increase
>>>>>>>>>>>>> throughput.
>>>>>>>>>>>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and
>>>>>>>>>>>>> ext_3
>>>>>>>>>>>>> and they all have their upsides and downsides.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Proposed fibration plugin combines them all so that it combines
>>>>>>>>>>>>> files
>>>>>>>>>>>>> with same extensions for 1, 2. 3 and 4 character extension  in
>>>>>>>>>>>>> groups
>>>>>>>>>>>>> and sorts them in same fiber group.
>>>>>>>>>>>>>
>>>>>>>>>>>>> With this fibration plugin all eg. xvid files would be in same
>>>>>>>>>>>>> group
>>>>>>>>>>>>> in folder on disk sorted alphabetically
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> What application wants all xvid files to be in the same group?
>>>>>>>>>>>> Do you have any benchmark numbers which show advantages
>>>>>>>>>>>> of the new plugin?
>>>>>>>>>>>>
>>>>>>>>>>> Xvid files are just an example.
>>>>>>>>>>> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3,
>>>>>>>>>>> ext_4
>>>>>>>>>>> and dot_o in one.
>>>>>>>>>>>
>>>>>>>>>>> In currently default plugin (dot_o) we sort all files by name from
>>>>>>>>>>> the
>>>>>>>>>>> start except .o files which we put at the end.
>>>>>>>>>>> So if we had a source directory with .c .h and .o files in it
>>>>>>>>>>> files
>>>>>>>>>>> by
>>>>>>>>>>> extension would be sorted like: chchchchchchchchoooooooooooooo
>>>>>>>>>>> I presumed that in some use cases it is better to have files be
>>>>>>>>>>> sorted
>>>>>>>>>>> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>>>>>>>>>>>
>>>>>>>>>>> Hypothesis is to use the premise that files of same extension are
>>>>>>>>>>> in
>>>>>>>>>>> same order of size to reduce fragmentation.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> What kind of fragmentation you are talking about?
>>>>>>>>>> Internal (which results in "dead" disk space), or
>>>>>>>>>> external (which results in a lot of "extents")?
>>>>>>>>>>
>>>>>>>>> External
>>>>>>>>>
>>>>>>>>>> Edward.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> If we group files of same extension in groups in one directory,
>>>>>>>>>>> when
>>>>>>>>>>> we write files of same extension after deletion of some files of
>>>>>>>>>>> one
>>>>>>>>>>> extension  their group would be in same order as the deleted file
>>>>>>>>>>> so
>>>>>>>>>>> they would be written in similar place and occupy the 'hole' of
>>>>>>>>>>> similar size.
>>>>>>>>
>>>>>>>>
>>>>>>>> So "similar" means the same order, that is file sizes can differ in 2
>>>>>>>> times?
>>>>>>>> TBH, I don't see what can be deduced from this assumption ;)
>>>>>>>> It can happen that new file either doesn't fit to that hole, or
>>>>>>>> occupies
>>>>>>>> too
>>>>>>>> small place, so that next file won't fit to the rest of the hole..
>>>>>>>>
>>>>>>> OFC we can never guarantee that the new file completely fits the hole
>>>>>>> (especially as we go through compression in next layer) but for both
>>>>>>> smaller and larger file than a hole we would have higher probability
>>>>>>> for less extents for situations with 2 or more types of files in a
>>>>>>> directory. For one type of file in a directory behavior would be the
>>>>>>> same as dot_o and ext_1 plugin.
>>>>>>
>>>>>>
>>>>>> I should upset you: fibration plugins are about mapping of a semantic
>>>>>> tree to the storage tree. Simply speaking, they manage mapping
>>>>>> object-> key, which has nothing common with real locations on diТак ты
>>>>>> уже не ищешь? sk.
>>>>>>
>>>>>> This is a block allocator, who assigns disk addresses to nodes of the
>>>>>> storage tree (right before writing them to disk at flush time).
>>>>>> And I am sure that block allocator doesn't care about fibration groups.
>>>>>>
>>>>>> I strongly not recommend you to experiment with block allocator.
>>>>>> Simply because I know how many people killed a lot of time without
>>>>>> results.
>>>>> Then what is this comment in the beginning of kassign.c about:
>>>>>
>>>>>
>>>>> * In reiser4 every piece of file system data and meta-data has a key.
>>>>> Keys
>>>>> * are used to store information in and retrieve it from reiser4 internal
>>>>> * tree. In addition to this, keys define _ordering_ of all file system
>>>>> * information: things having close keys are placed into the same or
>>>>> * neighboring (in the tree order) nodes of the tree. As our block
>>>>> allocator
>>>>> * tries to respect tree order (see flush.c), keys also define order in
>>>>> which
>>>>> * things are laid out on the disk, and hence, affect performance
>>>>> directly.
>>>>
>>>> I can not find where in the code block allocator respects key ordering.
>>>> Once you find it, then let me know..
>>>>
>>> Let me prove it in a way that is possible for me without investing
>>> years in learning R4 internals ;)
>>>
>>> krshina3 test # mkfs.reiser4 -o create=reg40,fibration=ext_1_fibre
>>> /dev/md123
>>>
>>> krshina3 test # ls -la /home/dusan/test/
>>> total 261
>>> drwxr-xr-x  2 root  root      6 Jan  7 17:51 .
>>> drwxr-xr-x 46 dusan dusan    73 Jan  7 17:48 ..
>>> -rw-r--r--  1 root  root   4096 Jan  7 17:49 1.a
>>> -rw-r--r--  1 root  root   8192 Jan  7 17:49 2.b
>>> -rw-r--r--  1 root  root  12288 Jan  7 17:50 3.a
>>> -rw-r--r--  1 root  root  16384 Jan  7 17:51 4.b
>>>
>>> krshina3 test # mount /dev/md123 /mnt/test
>>>
>>> krshina3 test # cp /home/dusan/test/* /mnt/test
>>>
>>> krshina3 test # umount /dev/md123
>>>
>>> krshina3 test # debugfs.reiser4 -t /dev/md123
>>> debugfs.reiser4 1.1.0
>>> Format release: 4.0.1
>>> Copyright (C) 2001-2005 by Hans Reiser, licensing governed by
>>> reiser4progs/COPYING.
>>>
>>> NODE (25) LEVEL=2 ITEMS=11 SPACE=3562 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  NPTR (nodeptr40): [29:1(SD):0:2a:0] OFF=28, LEN=8, flags=0x0 [24]
>>>
>>> ------------------------------------------------------------------------------
>>> #1  NPTR (nodeptr40): [2a:4(FB):c2312e6100000000:10000:c6e] OFF=36,
>>> LEN=8, flags=0x0 [26]
>>>
>>> ------------------------------------------------------------------------------
>>> #2  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:c06] OFF=44,
>>> LEN=8, flags=0x0 [27]
>>>
>>> ------------------------------------------------------------------------------
>>> #3  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:1bc4] OFF=52,
>>> LEN=8, flags=0x0 [28]
>>>
>>> ------------------------------------------------------------------------------
>>> #4  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:2b82] OFF=60,
>>> LEN=8, flags=0x0 [29]
>>>
>>> ------------------------------------------------------------------------------
>>> #5  NPTR (nodeptr40): [2a:4(FB):c4322e6200000000:10001:b1a] OFF=68,
>>> LEN=8, flags=0x0 [30]
>>>
>>> ------------------------------------------------------------------------------
>>> #6  NPTR (nodeptr40): [2a:4(FB):c4322e6200000000:10001:1ad8] OFF=76,
>>> LEN=8, flags=0x0 [31]
>>>
>>> ------------------------------------------------------------------------------
>>> #7  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:a70] OFF=84,
>>> LEN=8, flags=0x0 [32]
>>>
>>> ------------------------------------------------------------------------------
>>> #8  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:1a2e] OFF=92,
>>> LEN=8, flags=0x0 [33]
>>>
>>> ------------------------------------------------------------------------------
>>> #9  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:29ec] OFF=100,
>>> LEN=8, flags=0x0 [34]
>>>
>>> ------------------------------------------------------------------------------
>>> #10  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:39aa] OFF=108,
>>> LEN=8, flags=0x0 [35]
>>>
>>> ==============================================================================
>>> NODE (24) LEVEL=1 ITEMS=7 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  SD (stat40): [29:1(SD):0:2a:0] OFF=28, LEN=94, flags=0x0
>>> exts:        3
>>> mask:        0x13
>>> plugin:        sdext_lw
>>> offset:        2
>>> len:        14
>>> mode:        drwxr-xr-x
>>> nlink:        3
>>> size:        6
>>> plugin:        sdext_unix
>>> offset:        16
>>> len:        28
>>> uid:        0
>>> gid:        0
>>> atime:        Sat Jan  7 17:59:37 2017
>>> mtime:        Sat Jan  7 18:03:59 2017
>>> ctime:        Sat Jan  7 18:03:59 2017
>>> rdev:        300
>>> bytes:        300
>>> plugin:        sdext_plugin_set
>>> offset:        44
>>> len:        50
>>> Pset count:     12
>>>           permission : id = 0
>>>           formatting : id = 2 (smart)
>>>                 hash : id = 1 (r5_hash)
>>>            fibration : id = 2 (ext_1_fibre)
>>>             statdata : id = 0 (stat40)
>>>              diritem : id = 2 (cde40)
>>>               crypto : id = 0
>>>               digest : id = 0
>>>             compress : id = 0 (lzo1)
>>>         compressMode : id = 4 (conv)
>>>              cluster : id = 0 (64K)
>>>               create : id = 0 (reg40)
>>>
>>> ------------------------------------------------------------------------------
>>> #1  DIRITEM (cde40): [2a:0(NAME):0:0:0] OFF=122, LEN=302, flags=0x0
>>> NR(6)  NAME                                  OFFSET HASH
>>>                  SDKEY
>>>     0 .                                        158
>>> 0000000000000000:0000000000000000 0000291:000002a
>>>     1 ..                                       182
>>> 0000000000000000:0000000000000000 0000291:000002a
>>>     2 1.a                                      206
>>> 0000000000000000:0000000000000000 00002a1:0010000
>>>     3 3.a                                      230
>>> 0000000000000000:0000000000000000 00002a1:0010002
>>>     4 2.b                                      254
>>> 0000000000000000:0000000000000000 00002a1:0010001
>>>     5 4.b                                      278
>>> 0000000000000000:0000000000000000 00002a1:0010003
>>>
>>> ------------------------------------------------------------------------------
>>> #2  SD (stat40): [2a:1(SD):c2312e6100000000:10000:0] OFF=424, LEN=56,
>>> flags=0x0
>>> exts:        3
>>> mask:        0x7
>>> plugin:        sdext_lw
>>> offset:        2
>>> len:        14
>>> mode:        -rw-r--r--
>>> nlink:        1
>>> size:        4096
>>> plugin:        sdext_unix
>>> offset:        16
>>> len:        28
>>> uid:        0
>>> gid:        0
>>> atime:        Sat Jan  7 18:03:59 2017
>>> mtime:        Sat Jan  7 18:03:59 2017
>>> ctime:        Sat Jan  7 18:03:59 2017
>>> rdev:        4096
>>> bytes:        4096
>>> plugin:        sdext_lt
>>> offset:        44
>>> len:        12
>>> atime:        652286102
>>> mtime:        652286102
>>> ctime:        652286102
>>>
>>> ------------------------------------------------------------------------------
>>> #3  SD (stat40): [2a:1(SD):c2332e6100000000:10002:0] OFF=480, LEN=56,
>>> flags=0x0
>>> exts:        3
>>> mask:        0x7
>>> plugin:        sdext_lw
>>> offset:        2
>>> len:        14
>>> mode:        -rw-r--r--
>>> nlink:        1
>>> size:        12288
>>> plugin:        sdext_unix
>>> offset:        16
>>> len:        28
>>> uid:        0
>>> gid:        0
>>> atime:        Sat Jan  7 18:03:59 2017
>>> mtime:        Sat Jan  7 18:03:59 2017
>>> ctime:        Sat Jan  7 18:03:59 2017
>>> rdev:        12288
>>> bytes:        12288
>>> plugin:        sdext_lt
>>> offset:        44
>>> len:        12
>>> atime:        656286256
>>> mtime:        656286256
>>> ctime:        656286256
>>>
>>> ------------------------------------------------------------------------------
>>> #4  SD (stat40): [2a:1(SD):c4322e6200000000:10001:0] OFF=536, LEN=56,
>>> flags=0x0
>>> exts:        3
>>> mask:        0x7
>>> plugin:        sdext_lw
>>> offset:        2
>>> len:        14
>>> mode:        -rw-r--r--
>>> nlink:        1
>>> size:        8192
>>> plugin:        sdext_unix
>>> offset:        16
>>> len:        28
>>> uid:        0
>>> gid:        0
>>> atime:        Sat Jan  7 18:03:59 2017
>>> mtime:        Sat Jan  7 18:03:59 2017
>>> ctime:        Sat Jan  7 18:03:59 2017
>>> rdev:        8192
>>> bytes:        8192
>>> plugin:        sdext_lt
>>> offset:        44
>>> len:        12
>>> atime:        652286102
>>> mtime:        656286256
>>> ctime:        656286256
>>>
>>> ------------------------------------------------------------------------------
>>> #5  SD (stat40): [2a:1(SD):c4342e6200000000:10003:0] OFF=592, LEN=56,
>>> flags=0x0
>>> exts:        3
>>> mask:        0x7
>>> plugin:        sdext_lw
>>> offset:        2
>>> len:        14
>>> mode:        -rw-r--r--
>>> nlink:        1
>>> size:        16384
>>> plugin:        sdext_unix
>>> offset:        16
>>> len:        28
>>> uid:        0
>>> gid:        0
>>> atime:        Sat Jan  7 18:03:59 2017
>>> mtime:        Sat Jan  7 18:03:59 2017
>>> ctime:        Sat Jan  7 18:03:59 2017
>>> rdev:        16384
>>> bytes:        16384
>>> plugin:        sdext_lt
>>> offset:        44
>>> len:        12
>>> atime:        656286256
>>> mtime:        656286256
>>> ctime:        656286256
>>>
>>> ------------------------------------------------------------------------------
>>> #6  TAIL (plain40): [2a:4(FB):c2312e6100000000:10000:0] OFF=648,
>>> LEN=3182, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (26) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):c2312e6100000000:10000:c6e] OFF=28,
>>> LEN=914, flags=0x0
>>>
>>> ------------------------------------------------------------------------------
>>> #1  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:0] OFF=942,
>>> LEN=3078, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (27) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:c06] OFF=28,
>>> LEN=4030, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (28) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:1bc4] OFF=28,
>>> LEN=4030, flags=0x50
>>>
>>> ==============================================================================
>>> NODE (29) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:2b82] OFF=28,
>>> LEN=1150, flags=0x54
>>>
>>> ------------------------------------------------------------------------------
>>> #1  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:0] OFF=1178,
>>> LEN=2842, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (30) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:b1a] OFF=28,
>>> LEN=4030, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (31) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:1ad8] OFF=28,
>>> LEN=1320, flags=0x0
>>>
>>> ------------------------------------------------------------------------------
>>> #1  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:0] OFF=1348,
>>> LEN=2672, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (32) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:a70] OFF=28,
>>> LEN=4030, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (33) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:1a2e] OFF=28,
>>> LEN=4030, flags=0x63
>>>
>>> ==============================================================================
>>> NODE (34) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:29ec] OFF=28,
>>> LEN=4030, flags=0xbf
>>>
>>> ==============================================================================
>>> NODE (35) LEVEL=1 ITEMS=1 SPACE=2408 MKFS ID=0x51a268e2 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:39aa] OFF=28,
>>> LEN=1622, flags=0x1b7
>>>
>>> ==============================================================================
>>>
>>> krshina3 test # mkfs.reiser4 -o create=reg40,fibration=dot_o_fibre
>>> /dev/md123
>>>
>>> krshina3 test # mount /dev/md123 /mnt/test
>>>
>>> krshina3 test # cp /home/dusan/test/* /mnt/test
>>>
>>> krshina3 test # umount /dev/md123
>>>
>>> krshina3 test # debugfs.reiser4 -t /dev/md123
>>> debugfs.reiser4 1.1.0
>>> Format release: 4.0.1
>>> Copyright (C) 2001-2005 by Hans Reiser, licensing governed by
>>> reiser4progs/COPYING.
>>>
>>> NODE (25) LEVEL=2 ITEMS=11 SPACE=3562 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  NPTR (nodeptr40): [29:1(SD):0:2a:0] OFF=28, LEN=8, flags=0x0 [24]
>>>
>>> ------------------------------------------------------------------------------
>>> #1  NPTR (nodeptr40): [2a:4(FB):312e6100000000:10000:c6e] OFF=36,
>>> LEN=8, flags=0x0 [26]
>>>
>>> ------------------------------------------------------------------------------
>>> #2  NPTR (nodeptr40): [2a:4(FB):322e6200000000:10001:c06] OFF=44,
>>> LEN=8, flags=0x0 [27]
>>>
>>> ------------------------------------------------------------------------------
>>> #3  NPTR (nodeptr40): [2a:4(FB):322e6200000000:10001:1bc4] OFF=52,
>>> LEN=8, flags=0x0 [28]
>>>
>>> ------------------------------------------------------------------------------
>>> #4  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:b5c] OFF=60,
>>> LEN=8, flags=0x0 [29]
>>>
>>> ------------------------------------------------------------------------------
>>> #5  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:1b1a] OFF=68,
>>> LEN=8, flags=0x0 [30]
>>>
>>> ------------------------------------------------------------------------------
>>> #6  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:2ad8] OFF=76,
>>> LEN=8, flags=0x0 [31]
>>>
>>> ------------------------------------------------------------------------------
>>> #7  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:a70] OFF=84,
>>> LEN=8, flags=0x0 [32]
>>>
>>> ------------------------------------------------------------------------------
>>> #8  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:1a2e] OFF=92,
>>> LEN=8, flags=0x0 [33]
>>>
>>> ------------------------------------------------------------------------------
>>> #9  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:29ec] OFF=100,
>>> LEN=8, flags=0x0 [34]
>>>
>>> ------------------------------------------------------------------------------
>>> #10  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:39aa] OFF=108,
>>> LEN=8, flags=0x0 [35]
>>>
>>> ==============================================================================
>>> NODE (24) LEVEL=1 ITEMS=7 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  SD (stat40): [29:1(SD):0:2a:0] OFF=28, LEN=94, flags=0x0
>>> exts:        3
>>> mask:        0x13
>>> plugin:        sdext_lw
>>> offset:        2
>>> len:        14
>>> mode:        drwxr-xr-x
>>> nlink:        3
>>> size:        6
>>> plugin:        sdext_unix
>>> offset:        16
>>> len:        28
>>> uid:        0
>>> gid:        0
>>> atime:        Sat Jan  7 18:05:26 2017
>>> mtime:        Sat Jan  7 18:06:31 2017
>>> ctime:        Sat Jan  7 18:06:31 2017
>>> rdev:        300
>>> bytes:        300
>>> plugin:        sdext_plugin_set
>>> offset:        44
>>> len:        50
>>> Pset count:     12
>>>           permission : id = 0
>>>           formatting : id = 2 (smart)
>>>                 hash : id = 1 (r5_hash)
>>>            fibration : id = 1 (dot_o_fibre)
>>>             statdata : id = 0 (stat40)
>>>              diritem : id = 2 (cde40)
>>>               crypto : id = 0
>>>               digest : id = 0
>>>             compress : id = 0 (lzo1)
>>>         compressMode : id = 4 (conv)
>>>              cluster : id = 0 (64K)
>>>               create : id = 0 (reg40)
>>>
>>> ------------------------------------------------------------------------------
>>> #1  DIRITEM (cde40): [2a:0(NAME):0:0:0] OFF=122, LEN=302, flags=0x0
>>> NR(6)  NAME                                  OFFSET HASH
>>>                  SDKEY
>>>     0 .                                        158
>>> 0000000000000000:0000000000000000 0000291:000002a
>>>     1 ..                                       182
>>> 0000000000000000:0000000000000000 0000291:000002a
>>>     2 1.a                                      206
>>> 0000000000000000:0000000000000000 00002a1:0010000
>>>     3 2.b                                      230
>>> 0000000000000000:0000000000000000 00002a1:0010001
>>>     4 3.a                                      254
>>> 0000000000000000:0000000000000000 00002a1:0010002
>>>     5 4.b                                      278
>>> 0000000000000000:0000000000000000 00002a1:0010003
>>>
>>> ------------------------------------------------------------------------------
>>> #2  SD (stat40): [2a:1(SD):312e6100000000:10000:0] OFF=424, LEN=56,
>>> flags=0x0
>>> exts:        3
>>> mask:        0x7
>>> plugin:        sdext_lw
>>> offset:        2
>>> len:        14
>>> mode:        -rw-r--r--
>>> nlink:        1
>>> size:        4096
>>> plugin:        sdext_unix
>>> offset:        16
>>> len:        28
>>> uid:        0
>>> gid:        0
>>> atime:        Sat Jan  7 18:06:31 2017
>>> mtime:        Sat Jan  7 18:06:31 2017
>>> ctime:        Sat Jan  7 18:06:31 2017
>>> rdev:        4096
>>> bytes:        4096
>>> plugin:        sdext_lt
>>> offset:        44
>>> len:        12
>>> atime:        778212799
>>> mtime:        778212799
>>> ctime:        778212799
>>>
>>> ------------------------------------------------------------------------------
>>> #3  SD (stat40): [2a:1(SD):322e6200000000:10001:0] OFF=480, LEN=56,
>>> flags=0x0
>>> exts:        3
>>> mask:        0x7
>>> plugin:        sdext_lw
>>> offset:        2
>>> len:        14
>>> mode:        -rw-r--r--
>>> nlink:        1
>>> size:        8192
>>> plugin:        sdext_unix
>>> offset:        16
>>> len:        28
>>> uid:        0
>>> gid:        0
>>> atime:        Sat Jan  7 18:06:31 2017
>>> mtime:        Sat Jan  7 18:06:31 2017
>>> ctime:        Sat Jan  7 18:06:31 2017
>>> rdev:        8192
>>> bytes:        8192
>>> plugin:        sdext_lt
>>> offset:        44
>>> len:        12
>>> atime:        778212799
>>> mtime:        778212799
>>> ctime:        778212799
>>>
>>> ------------------------------------------------------------------------------
>>> #4  SD (stat40): [2a:1(SD):332e6100000000:10002:0] OFF=536, LEN=56,
>>> flags=0x0
>>> exts:        3
>>> mask:        0x7
>>> plugin:        sdext_lw
>>> offset:        2
>>> len:        14
>>> mode:        -rw-r--r--
>>> nlink:        1
>>> size:        12288
>>> plugin:        sdext_unix
>>> offset:        16
>>> len:        28
>>> uid:        0
>>> gid:        0
>>> atime:        Sat Jan  7 18:06:31 2017
>>> mtime:        Sat Jan  7 18:06:31 2017
>>> ctime:        Sat Jan  7 18:06:31 2017
>>> rdev:        12288
>>> bytes:        12288 interesting TODOs with tangible results.
>>> plugin:        sdext_lt
>>> offset:        44
>>> len:        12
>>> atime:        782212955
>>> mtime:        782212955
>>> ctime:        782212955
>>>
>>> ------------------------------------------------------------------------------
>>> #5  SD (stat40): [2a:1(SD):342e6200000000:10003:0] OFF=592, LEN=56,
>>> flags=0x0
>>> exts:        3
>>> mask:        0x7
>>> plugin:        sdext_lw
>>> offset:        2
>>> len:        14
>>> mode:        -rw-r--r--
>>> nlink:        1
>>> size:        16384
>>> plugin:        sdext_unix
>>> offset:        16
>>> len:        28
>>> uid:        0
>>> gid:        0
>>> atime:        Sat Jan  7 18:06:31 2017
>>> mtime:        Sat Jan  7 18:06:31 2017
>>> ctime:        Sat Jan  7 18:06:31 2017
>>> rdev:        16384
>>> bytes:        16384
>>> plugin:        sdext_lt
>>> offset:        44
>>> len:        12
>>> atime:        782212955
>>> mtime:        782212955
>>> ctime:        782212955
>>>
>>> ------------------------------------------------------------------------------
>>> #6  TAIL (plain40): [2a:4(FB):312e6100000000:10000:0] OFF=648,
>>> LEN=3182, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (26) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):312e6100000000:10000:c6e] OFF=28,
>>> LEN=914, flags=0x0
>>>
>>> ------------------------------------------------------------------------------
>>> #1  TAIL (plain40): [2a:4(FB):322e6200000000:10001:0] OFF=942,
>>> LEN=3078, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (27) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):322e6200000000:10001:c06] OFF=28,
>>> LEN=4030, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (28) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):322e6200000000:10001:1bc4] OFF=28,
>>> LEN=1084, flags=0x0
>>>
>>> ------------------------------------------------------------------------------
>>> #1  TAIL (plain40): [2a:4(FB):332e6100000000:10002:0] OFF=1112,
>>> LEN=2908, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (29) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:b5c] OFF=28,
>>> LEN=4030, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (30) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:1b1a] OFF=28,
>>> LEN=4030, flags=0xc
>>>
>>> ==============================================================================
>>> NODE (31) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:2ad8] OFF=28,
>>> LEN=1320, flags=0x1cf
>>>
>>> ------------------------------------------------------------------------------
>>> #1  TAIL (plain40): [2a:4(FB):342e6200000000:10003:0] OFF=1348,
>>> LEN=2672, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (32) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:a70] OFF=28,
>>> LEN=4030, flags=0x0
>>>
>>> ==============================================================================
>>> NODE (33) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:1a2e] OFF=28,
>>> LEN=4030, flags=0x21
>>>
>>> ==============================================================================
>>> NODE (34) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:29ec] OFF=28,
>>> LEN=4030, flags=0x1cf
>>>
>>> ==============================================================================
>>> NODE (35) LEVEL=1 ITEMS=1 SPACE=2408 MKFS ID=0x1bb9bd27 FLUSH=0x0
>>> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:39aa] OFF=28,
>>> LEN=1622, flags=0x16
>>>
>>> ==============================================================================
>>>
>>>
>>> As we can see with ext_1_fibre files were sorted:
>>> 1.a
>>> 3.a
>>> 2.b
>>> 4.b
>>>
>>> With dot_o_fibre:
>>>
>>> 1.a
>>> 2.b
>>> 3.a
>>> 4.b
>>>
>> Thus, respect for fibration groups is also under a big question.
>> TBH, I prefer to not kill time for projects basing on questionable
>> assumptions and deductions. Just because there is a lot of
>> interesting TODOs with tangible results.
>>
> OK
> I'll get back to automatic R4 git tree lkp-tests project.
>
> I planned to make two testing steps:
> 1. Build, functional and stress testing under VM with R4 partition on RAM disk

Sounds good.

> 2. If it passes step 1 then some performance testing under VM with R4
> on real partition

Performance testing under VM (or containers) is perversion.
Either use a real machine, or don't test at all.

>
> I'll probably use Ubuntu for VM, although I'm not familiar with Debian
> clones, but automatic testing suite supports it.
> I'm more for Gentoo but lkp-tests doesn't support it.

Use Fedora if possible, so that we'll be able to synchronize things..

Thanks,
Edward.

>
> I'll probably run 4.5 to 4.9 patches first to see what I get.

Re: [RFC] Smart fibration plugin ext_4321

[Dušan Čolić]

On Sat, Jan 7, 2017 at 6:56 PM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
>
>
> On 01/07/2017 07:10 PM, Dušan Čolić wrote:
>>
>> On Sat, Jan 7, 2017 at 8:58 AM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>>
>>>
>>> On 01/07/2017 10:15 AM, Dušan Čolić wrote:
>>>>
>>>> On Sat, Jan 7, 2017 at 12:05 AM, Edward Shishkin
>>>> <edward.shishkin@gmail.com> wrote:
>>>>>
>>>>> On 01/07/2017 01:09 AM, Dušan Čolić wrote:
>>>>>>
>>>>>> On Fri, Jan 6, 2017 at 8:58 PM, Edward Shishkin
>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 01/06/2017 05:34 PM, Dušan Čolić wrote:
>>>>>>>>
>>>>>>>> On Fri, Jan 6, 2017 at 2:44 PM, Edward Shishkin
>>>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> On 12/26/2016 11:13 PM, Dušan Čolić wrote:
>>>>>>>>>>
>>>>>>>>>> On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin
>>>>>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 12/25/2016 02:59 AM, Dušan Čolić wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Fibration is a great way to decrease fragmentation and increase
>>>>>>>>>>>> throughput.
>>>>>>>>>>>> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and
>>>>>>>>>>>> ext_3
>>>>>>>>>>>> and they all have their upsides and downsides.
>>>>>>>>>>>>
>>>>>>>>>>>> Proposed fibration plugin combines them all so that it combines
>>>>>>>>>>>> files
>>>>>>>>>>>> with same extensions for 1, 2. 3 and 4 character extension  in
>>>>>>>>>>>> groups
>>>>>>>>>>>> and sorts them in same fiber group.
>>>>>>>>>>>>
>>>>>>>>>>>> With this fibration plugin all eg. xvid files would be in same
>>>>>>>>>>>> group
>>>>>>>>>>>> in folder on disk sorted alphabetically
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> What application wants all xvid files to be in the same group?
>>>>>>>>>>> Do you have any benchmark numbers which show advantages
>>>>>>>>>>> of the new plugin?
>>>>>>>>>>>
>>>>>>>>>> Xvid files are just an example.
>>>>>>>>>> ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3,
>>>>>>>>>> ext_4
>>>>>>>>>> and dot_o in one.
>>>>>>>>>>
>>>>>>>>>> In currently default plugin (dot_o) we sort all files by name from
>>>>>>>>>> the
>>>>>>>>>> start except .o files which we put at the end.
>>>>>>>>>> So if we had a source directory with .c .h and .o files in it
>>>>>>>>>> files
>>>>>>>>>> by
>>>>>>>>>> extension would be sorted like: chchchchchchchchoooooooooooooo
>>>>>>>>>> I presumed that in some use cases it is better to have files be
>>>>>>>>>> sorted
>>>>>>>>>> ccccccccccchhhhhhhhhhhhhhoooooooooooo
>>>>>>>>>>
>>>>>>>>>> Hypothesis is to use the premise that files of same extension are
>>>>>>>>>> in
>>>>>>>>>> same order of size to reduce fragmentation.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What kind of fragmentation you are talking about?
>>>>>>>>> Internal (which results in "dead" disk space), or
>>>>>>>>> external (which results in a lot of "extents")?
>>>>>>>>>
>>>>>>>> External
>>>>>>>>
>>>>>>>>> Edward.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> If we group files of same extension in groups in one directory,
>>>>>>>>>> when
>>>>>>>>>> we write files of same extension after deletion of some files of
>>>>>>>>>> one
>>>>>>>>>> extension  their group would be in same order as the deleted file
>>>>>>>>>> so
>>>>>>>>>> they would be written in similar place and occupy the 'hole' of
>>>>>>>>>> similar size.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> So "similar" means the same order, that is file sizes can differ in 2
>>>>>>> times?
>>>>>>> TBH, I don't see what can be deduced from this assumption ;)
>>>>>>> It can happen that new file either doesn't fit to that hole, or
>>>>>>> occupies
>>>>>>> too
>>>>>>> small place, so that next file won't fit to the rest of the hole..
>>>>>>>
>>>>>> OFC we can never guarantee that the new file completely fits the hole
>>>>>> (especially as we go through compression in next layer) but for both
>>>>>> smaller and larger file than a hole we would have higher probability
>>>>>> for less extents for situations with 2 or more types of files in a
>>>>>> directory. For one type of file in a directory behavior would be the
>>>>>> same as dot_o and ext_1 plugin.
>>>>>
>>>>>
>>>>>
>>>>> I should upset you: fibration plugins are about mapping of a semantic
>>>>> tree to the storage tree. Simply speaking, they manage mapping
>>>>> object-> key, which has nothing common with real locations on diТак ты
>>>>> уже не ищешь? sk.
>>>>>
>>>>> This is a block allocator, who assigns disk addresses to nodes of the
>>>>> storage tree (right before writing them to disk at flush time).
>>>>> And I am sure that block allocator doesn't care about fibration groups.
>>>>>
>>>>> I strongly not recommend you to experiment with block allocator.
>>>>> Simply because I know how many people killed a lot of time without
>>>>> results.
>>>>
>>>> Then what is this comment in the beginning of kassign.c about:
>>>>
>>>>
>>>> * In reiser4 every piece of file system data and meta-data has a key.
>>>> Keys
>>>> * are used to store information in and retrieve it from reiser4 internal
>>>> * tree. In addition to this, keys define _ordering_ of all file system
>>>> * information: things having close keys are placed into the same or
>>>> * neighboring (in the tree order) nodes of the tree. As our block
>>>> allocator
>>>> * tries to respect tree order (see flush.c), keys also define order in
>>>> which
>>>> * things are laid out on the disk, and hence, affect performance
>>>> directly.
>>>
>>>
>>> I can not find where in the code block allocator respects key ordering.
>>> Once you find it, then let me know..
>>>
>> Let me prove it in a way that is possible for me without investing
>> years in learning R4 internals ;)
>>
>> krshina3 test # mkfs.reiser4 -o create=reg40,fibration=ext_1_fibre
>> /dev/md123
>>
>> krshina3 test # ls -la /home/dusan/test/
>> total 261
>> drwxr-xr-x  2 root  root      6 Jan  7 17:51 .
>> drwxr-xr-x 46 dusan dusan    73 Jan  7 17:48 ..
>> -rw-r--r--  1 root  root   4096 Jan  7 17:49 1.a
>> -rw-r--r--  1 root  root   8192 Jan  7 17:49 2.b
>> -rw-r--r--  1 root  root  12288 Jan  7 17:50 3.a
>> -rw-r--r--  1 root  root  16384 Jan  7 17:51 4.b
>>
>> krshina3 test # mount /dev/md123 /mnt/test
>>
>> krshina3 test # cp /home/dusan/test/* /mnt/test
>>
>> krshina3 test # umount /dev/md123
>>
>> krshina3 test # debugfs.reiser4 -t /dev/md123
>> debugfs.reiser4 1.1.0
>> Format release: 4.0.1
>> Copyright (C) 2001-2005 by Hans Reiser, licensing governed by
>> reiser4progs/COPYING.
>>
>> NODE (25) LEVEL=2 ITEMS=11 SPACE=3562 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  NPTR (nodeptr40): [29:1(SD):0:2a:0] OFF=28, LEN=8, flags=0x0 [24]
>>
>> ------------------------------------------------------------------------------
>> #1  NPTR (nodeptr40): [2a:4(FB):c2312e6100000000:10000:c6e] OFF=36,
>> LEN=8, flags=0x0 [26]
>>
>> ------------------------------------------------------------------------------
>> #2  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:c06] OFF=44,
>> LEN=8, flags=0x0 [27]
>>
>> ------------------------------------------------------------------------------
>> #3  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:1bc4] OFF=52,
>> LEN=8, flags=0x0 [28]
>>
>> ------------------------------------------------------------------------------
>> #4  NPTR (nodeptr40): [2a:4(FB):c2332e6100000000:10002:2b82] OFF=60,
>> LEN=8, flags=0x0 [29]
>>
>> ------------------------------------------------------------------------------
>> #5  NPTR (nodeptr40): [2a:4(FB):c4322e6200000000:10001:b1a] OFF=68,
>> LEN=8, flags=0x0 [30]
>>
>> ------------------------------------------------------------------------------
>> #6  NPTR (nodeptr40): [2a:4(FB):c4322e6200000000:10001:1ad8] OFF=76,
>> LEN=8, flags=0x0 [31]
>>
>> ------------------------------------------------------------------------------
>> #7  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:a70] OFF=84,
>> LEN=8, flags=0x0 [32]
>>
>> ------------------------------------------------------------------------------
>> #8  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:1a2e] OFF=92,
>> LEN=8, flags=0x0 [33]
>>
>> ------------------------------------------------------------------------------
>> #9  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:29ec] OFF=100,
>> LEN=8, flags=0x0 [34]
>>
>> ------------------------------------------------------------------------------
>> #10  NPTR (nodeptr40): [2a:4(FB):c4342e6200000000:10003:39aa] OFF=108,
>> LEN=8, flags=0x0 [35]
>>
>> ==============================================================================
>> NODE (24) LEVEL=1 ITEMS=7 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  SD (stat40): [29:1(SD):0:2a:0] OFF=28, LEN=94, flags=0x0
>> exts:        3
>> mask:        0x13
>> plugin:        sdext_lw
>> offset:        2
>> len:        14
>> mode:        drwxr-xr-x
>> nlink:        3
>> size:        6
>> plugin:        sdext_unix
>> offset:        16
>> len:        28
>> uid:        0
>> gid:        0
>> atime:        Sat Jan  7 17:59:37 2017
>> mtime:        Sat Jan  7 18:03:59 2017
>> ctime:        Sat Jan  7 18:03:59 2017
>> rdev:        300
>> bytes:        300
>> plugin:        sdext_plugin_set
>> offset:        44
>> len:        50
>> Pset count:     12
>>          permission : id = 0
>>          formatting : id = 2 (smart)
>>                hash : id = 1 (r5_hash)
>>           fibration : id = 2 (ext_1_fibre)
>>            statdata : id = 0 (stat40)
>>             diritem : id = 2 (cde40)
>>              crypto : id = 0
>>              digest : id = 0
>>            compress : id = 0 (lzo1)
>>        compressMode : id = 4 (conv)
>>             cluster : id = 0 (64K)
>>              create : id = 0 (reg40)
>>
>> ------------------------------------------------------------------------------
>> #1  DIRITEM (cde40): [2a:0(NAME):0:0:0] OFF=122, LEN=302, flags=0x0
>> NR(6)  NAME                                  OFFSET HASH
>>                 SDKEY
>>    0 .                                        158
>> 0000000000000000:0000000000000000 0000291:000002a
>>    1 ..                                       182
>> 0000000000000000:0000000000000000 0000291:000002a
>>    2 1.a                                      206
>> 0000000000000000:0000000000000000 00002a1:0010000
>>    3 3.a                                      230
>> 0000000000000000:0000000000000000 00002a1:0010002
>>    4 2.b                                      254
>> 0000000000000000:0000000000000000 00002a1:0010001
>>    5 4.b                                      278
>> 0000000000000000:0000000000000000 00002a1:0010003
>>
>> ------------------------------------------------------------------------------
>> #2  SD (stat40): [2a:1(SD):c2312e6100000000:10000:0] OFF=424, LEN=56,
>> flags=0x0
>> exts:        3
>> mask:        0x7
>> plugin:        sdext_lw
>> offset:        2
>> len:        14
>> mode:        -rw-r--r--
>> nlink:        1
>> size:        4096
>> plugin:        sdext_unix
>> offset:        16
>> len:        28
>> uid:        0
>> gid:        0
>> atime:        Sat Jan  7 18:03:59 2017
>> mtime:        Sat Jan  7 18:03:59 2017
>> ctime:        Sat Jan  7 18:03:59 2017
>> rdev:        4096
>> bytes:        4096
>> plugin:        sdext_lt
>> offset:        44
>> len:        12
>> atime:        652286102
>> mtime:        652286102
>> ctime:        652286102
>>
>> ------------------------------------------------------------------------------
>> #3  SD (stat40): [2a:1(SD):c2332e6100000000:10002:0] OFF=480, LEN=56,
>> flags=0x0
>> exts:        3
>> mask:        0x7
>> plugin:        sdext_lw
>> offset:        2
>> len:        14
>> mode:        -rw-r--r--
>> nlink:        1
>> size:        12288
>> plugin:        sdext_unix
>> offset:        16
>> len:        28
>> uid:        0
>> gid:        0
>> atime:        Sat Jan  7 18:03:59 2017
>> mtime:        Sat Jan  7 18:03:59 2017
>> ctime:        Sat Jan  7 18:03:59 2017
>> rdev:        12288
>> bytes:        12288
>> plugin:        sdext_lt
>> offset:        44
>> len:        12
>> atime:        656286256
>> mtime:        656286256
>> ctime:        656286256
>>
>> ------------------------------------------------------------------------------
>> #4  SD (stat40): [2a:1(SD):c4322e6200000000:10001:0] OFF=536, LEN=56,
>> flags=0x0
>> exts:        3
>> mask:        0x7
>> plugin:        sdext_lw
>> offset:        2
>> len:        14
>> mode:        -rw-r--r--
>> nlink:        1
>> size:        8192
>> plugin:        sdext_unix
>> offset:        16
>> len:        28
>> uid:        0
>> gid:        0
>> atime:        Sat Jan  7 18:03:59 2017
>> mtime:        Sat Jan  7 18:03:59 2017
>> ctime:        Sat Jan  7 18:03:59 2017
>> rdev:        8192
>> bytes:        8192
>> plugin:        sdext_lt
>> offset:        44
>> len:        12
>> atime:        652286102
>> mtime:        656286256
>> ctime:        656286256
>>
>> ------------------------------------------------------------------------------
>> #5  SD (stat40): [2a:1(SD):c4342e6200000000:10003:0] OFF=592, LEN=56,
>> flags=0x0
>> exts:        3
>> mask:        0x7
>> plugin:        sdext_lw
>> offset:        2
>> len:        14
>> mode:        -rw-r--r--
>> nlink:        1
>> size:        16384
>> plugin:        sdext_unix
>> offset:        16
>> len:        28
>> uid:        0
>> gid:        0
>> atime:        Sat Jan  7 18:03:59 2017
>> mtime:        Sat Jan  7 18:03:59 2017
>> ctime:        Sat Jan  7 18:03:59 2017
>> rdev:        16384
>> bytes:        16384
>> plugin:        sdext_lt
>> offset:        44
>> len:        12
>> atime:        656286256
>> mtime:        656286256
>> ctime:        656286256
>>
>> ------------------------------------------------------------------------------
>> #6  TAIL (plain40): [2a:4(FB):c2312e6100000000:10000:0] OFF=648,
>> LEN=3182, flags=0x0
>>
>> ==============================================================================
>> NODE (26) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):c2312e6100000000:10000:c6e] OFF=28,
>> LEN=914, flags=0x0
>>
>> ------------------------------------------------------------------------------
>> #1  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:0] OFF=942,
>> LEN=3078, flags=0x0
>>
>> ==============================================================================
>> NODE (27) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:c06] OFF=28,
>> LEN=4030, flags=0x0
>>
>> ==============================================================================
>> NODE (28) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:1bc4] OFF=28,
>> LEN=4030, flags=0x50
>>
>> ==============================================================================
>> NODE (29) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):c2332e6100000000:10002:2b82] OFF=28,
>> LEN=1150, flags=0x54
>>
>> ------------------------------------------------------------------------------
>> #1  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:0] OFF=1178,
>> LEN=2842, flags=0x0
>>
>> ==============================================================================
>> NODE (30) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:b1a] OFF=28,
>> LEN=4030, flags=0x0
>>
>> ==============================================================================
>> NODE (31) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):c4322e6200000000:10001:1ad8] OFF=28,
>> LEN=1320, flags=0x0
>>
>> ------------------------------------------------------------------------------
>> #1  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:0] OFF=1348,
>> LEN=2672, flags=0x0
>>
>> ==============================================================================
>> NODE (32) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:a70] OFF=28,
>> LEN=4030, flags=0x0
>>
>> ==============================================================================
>> NODE (33) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:1a2e] OFF=28,
>> LEN=4030, flags=0x63
>>
>> ==============================================================================
>> NODE (34) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:29ec] OFF=28,
>> LEN=4030, flags=0xbf
>>
>> ==============================================================================
>> NODE (35) LEVEL=1 ITEMS=1 SPACE=2408 MKFS ID=0x51a268e2 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):c4342e6200000000:10003:39aa] OFF=28,
>> LEN=1622, flags=0x1b7
>>
>> ==============================================================================
>>
>> krshina3 test # mkfs.reiser4 -o create=reg40,fibration=dot_o_fibre
>> /dev/md123
>>
>> krshina3 test # mount /dev/md123 /mnt/test
>>
>> krshina3 test # cp /home/dusan/test/* /mnt/test
>>
>> krshina3 test # umount /dev/md123
>>
>> krshina3 test # debugfs.reiser4 -t /dev/md123
>> debugfs.reiser4 1.1.0
>> Format release: 4.0.1
>> Copyright (C) 2001-2005 by Hans Reiser, licensing governed by
>> reiser4progs/COPYING.
>>
>> NODE (25) LEVEL=2 ITEMS=11 SPACE=3562 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  NPTR (nodeptr40): [29:1(SD):0:2a:0] OFF=28, LEN=8, flags=0x0 [24]
>>
>> ------------------------------------------------------------------------------
>> #1  NPTR (nodeptr40): [2a:4(FB):312e6100000000:10000:c6e] OFF=36,
>> LEN=8, flags=0x0 [26]
>>
>> ------------------------------------------------------------------------------
>> #2  NPTR (nodeptr40): [2a:4(FB):322e6200000000:10001:c06] OFF=44,
>> LEN=8, flags=0x0 [27]
>>
>> ------------------------------------------------------------------------------
>> #3  NPTR (nodeptr40): [2a:4(FB):322e6200000000:10001:1bc4] OFF=52,
>> LEN=8, flags=0x0 [28]
>>
>> ------------------------------------------------------------------------------
>> #4  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:b5c] OFF=60,
>> LEN=8, flags=0x0 [29]
>>
>> ------------------------------------------------------------------------------
>> #5  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:1b1a] OFF=68,
>> LEN=8, flags=0x0 [30]
>>
>> ------------------------------------------------------------------------------
>> #6  NPTR (nodeptr40): [2a:4(FB):332e6100000000:10002:2ad8] OFF=76,
>> LEN=8, flags=0x0 [31]
>>
>> ------------------------------------------------------------------------------
>> #7  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:a70] OFF=84,
>> LEN=8, flags=0x0 [32]
>>
>> ------------------------------------------------------------------------------
>> #8  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:1a2e] OFF=92,
>> LEN=8, flags=0x0 [33]
>>
>> ------------------------------------------------------------------------------
>> #9  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:29ec] OFF=100,
>> LEN=8, flags=0x0 [34]
>>
>> ------------------------------------------------------------------------------
>> #10  NPTR (nodeptr40): [2a:4(FB):342e6200000000:10003:39aa] OFF=108,
>> LEN=8, flags=0x0 [35]
>>
>> ==============================================================================
>> NODE (24) LEVEL=1 ITEMS=7 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  SD (stat40): [29:1(SD):0:2a:0] OFF=28, LEN=94, flags=0x0
>> exts:        3
>> mask:        0x13
>> plugin:        sdext_lw
>> offset:        2
>> len:        14
>> mode:        drwxr-xr-x
>> nlink:        3
>> size:        6
>> plugin:        sdext_unix
>> offset:        16
>> len:        28
>> uid:        0
>> gid:        0
>> atime:        Sat Jan  7 18:05:26 2017
>> mtime:        Sat Jan  7 18:06:31 2017
>> ctime:        Sat Jan  7 18:06:31 2017
>> rdev:        300
>> bytes:        300
>> plugin:        sdext_plugin_set
>> offset:        44
>> len:        50
>> Pset count:     12
>>          permission : id = 0
>>          formatting : id = 2 (smart)
>>                hash : id = 1 (r5_hash)
>>           fibration : id = 1 (dot_o_fibre)
>>            statdata : id = 0 (stat40)
>>             diritem : id = 2 (cde40)
>>              crypto : id = 0
>>              digest : id = 0
>>            compress : id = 0 (lzo1)
>>        compressMode : id = 4 (conv)
>>             cluster : id = 0 (64K)
>>              create : id = 0 (reg40)
>>
>> ------------------------------------------------------------------------------
>> #1  DIRITEM (cde40): [2a:0(NAME):0:0:0] OFF=122, LEN=302, flags=0x0
>> NR(6)  NAME                                  OFFSET HASH
>>                 SDKEY
>>    0 .                                        158
>> 0000000000000000:0000000000000000 0000291:000002a
>>    1 ..                                       182
>> 0000000000000000:0000000000000000 0000291:000002a
>>    2 1.a                                      206
>> 0000000000000000:0000000000000000 00002a1:0010000
>>    3 2.b                                      230
>> 0000000000000000:0000000000000000 00002a1:0010001
>>    4 3.a                                      254
>> 0000000000000000:0000000000000000 00002a1:0010002
>>    5 4.b                                      278
>> 0000000000000000:0000000000000000 00002a1:0010003
>>
>> ------------------------------------------------------------------------------
>> #2  SD (stat40): [2a:1(SD):312e6100000000:10000:0] OFF=424, LEN=56,
>> flags=0x0
>> exts:        3
>> mask:        0x7
>> plugin:        sdext_lw
>> offset:        2
>> len:        14
>> mode:        -rw-r--r--
>> nlink:        1
>> size:        4096
>> plugin:        sdext_unix
>> offset:        16
>> len:        28
>> uid:        0
>> gid:        0
>> atime:        Sat Jan  7 18:06:31 2017
>> mtime:        Sat Jan  7 18:06:31 2017
>> ctime:        Sat Jan  7 18:06:31 2017
>> rdev:        4096
>> bytes:        4096
>> plugin:        sdext_lt
>> offset:        44
>> len:        12
>> atime:        778212799
>> mtime:        778212799
>> ctime:        778212799
>>
>> ------------------------------------------------------------------------------
>> #3  SD (stat40): [2a:1(SD):322e6200000000:10001:0] OFF=480, LEN=56,
>> flags=0x0
>> exts:        3
>> mask:        0x7
>> plugin:        sdext_lw
>> offset:        2
>> len:        14
>> mode:        -rw-r--r--
>> nlink:        1
>> size:        8192
>> plugin:        sdext_unix
>> offset:        16
>> len:        28
>> uid:        0
>> gid:        0
>> atime:        Sat Jan  7 18:06:31 2017
>> mtime:        Sat Jan  7 18:06:31 2017
>> ctime:        Sat Jan  7 18:06:31 2017
>> rdev:        8192
>> bytes:        8192
>> plugin:        sdext_lt
>> offset:        44
>> len:        12
>> atime:        778212799
>> mtime:        778212799
>> ctime:        778212799
>>
>> ------------------------------------------------------------------------------
>> #4  SD (stat40): [2a:1(SD):332e6100000000:10002:0] OFF=536, LEN=56,
>> flags=0x0
>> exts:        3
>> mask:        0x7
>> plugin:        sdext_lw
>> offset:        2
>> len:        14
>> mode:        -rw-r--r--
>> nlink:        1
>> size:        12288
>> plugin:        sdext_unix
>> offset:        16
>> len:        28
>> uid:        0
>> gid:        0
>> atime:        Sat Jan  7 18:06:31 2017
>> mtime:        Sat Jan  7 18:06:31 2017
>> ctime:        Sat Jan  7 18:06:31 2017
>> rdev:        12288
>> bytes:        12288 interesting TODOs with tangible results.
>> plugin:        sdext_lt
>> offset:        44
>> len:        12
>> atime:        782212955
>> mtime:        782212955
>> ctime:        782212955
>>
>> ------------------------------------------------------------------------------
>> #5  SD (stat40): [2a:1(SD):342e6200000000:10003:0] OFF=592, LEN=56,
>> flags=0x0
>> exts:        3
>> mask:        0x7
>> plugin:        sdext_lw
>> offset:        2
>> len:        14
>> mode:        -rw-r--r--
>> nlink:        1
>> size:        16384
>> plugin:        sdext_unix
>> offset:        16
>> len:        28
>> uid:        0
>> gid:        0
>> atime:        Sat Jan  7 18:06:31 2017
>> mtime:        Sat Jan  7 18:06:31 2017
>> ctime:        Sat Jan  7 18:06:31 2017
>> rdev:        16384
>> bytes:        16384
>> plugin:        sdext_lt
>> offset:        44
>> len:        12
>> atime:        782212955
>> mtime:        782212955
>> ctime:        782212955
>>
>> ------------------------------------------------------------------------------
>> #6  TAIL (plain40): [2a:4(FB):312e6100000000:10000:0] OFF=648,
>> LEN=3182, flags=0x0
>>
>> ==============================================================================
>> NODE (26) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):312e6100000000:10000:c6e] OFF=28,
>> LEN=914, flags=0x0
>>
>> ------------------------------------------------------------------------------
>> #1  TAIL (plain40): [2a:4(FB):322e6200000000:10001:0] OFF=942,
>> LEN=3078, flags=0x0
>>
>> ==============================================================================
>> NODE (27) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):322e6200000000:10001:c06] OFF=28,
>> LEN=4030, flags=0x0
>>
>> ==============================================================================
>> NODE (28) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):322e6200000000:10001:1bc4] OFF=28,
>> LEN=1084, flags=0x0
>>
>> ------------------------------------------------------------------------------
>> #1  TAIL (plain40): [2a:4(FB):332e6100000000:10002:0] OFF=1112,
>> LEN=2908, flags=0x0
>>
>> ==============================================================================
>> NODE (29) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:b5c] OFF=28,
>> LEN=4030, flags=0x0
>>
>> ==============================================================================
>> NODE (30) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:1b1a] OFF=28,
>> LEN=4030, flags=0xc
>>
>> ==============================================================================
>> NODE (31) LEVEL=1 ITEMS=2 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):332e6100000000:10002:2ad8] OFF=28,
>> LEN=1320, flags=0x1cf
>>
>> ------------------------------------------------------------------------------
>> #1  TAIL (plain40): [2a:4(FB):342e6200000000:10003:0] OFF=1348,
>> LEN=2672, flags=0x0
>>
>> ==============================================================================
>> NODE (32) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:a70] OFF=28,
>> LEN=4030, flags=0x0
>>
>> ==============================================================================
>> NODE (33) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:1a2e] OFF=28,
>> LEN=4030, flags=0x21
>>
>> ==============================================================================
>> NODE (34) LEVEL=1 ITEMS=1 SPACE=0 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:29ec] OFF=28,
>> LEN=4030, flags=0x1cf
>>
>> ==============================================================================
>> NODE (35) LEVEL=1 ITEMS=1 SPACE=2408 MKFS ID=0x1bb9bd27 FLUSH=0x0
>> #0  TAIL (plain40): [2a:4(FB):342e6200000000:10003:39aa] OFF=28,
>> LEN=1622, flags=0x16
>>
>> ==============================================================================
>>
>>
>> As we can see with ext_1_fibre files were sorted:
>> 1.a
>> 3.a
>> 2.b
>> 4.b
>>
>> With dot_o_fibre:
>>
>> 1.a
>> 2.b
>> 3.a
>> 4.b
>>
>
> Thus, respect for fibration groups is also under a big question.
> TBH, I prefer to not kill time for projects basing on questionable
> assumptions and deductions. Just because there is a lot of
> interesting TODOs with tangible results.
>
OK
I'll get back to automatic R4 git tree lkp-tests project.

I planned to make two testing steps:
1. Build, functional and stress testing under VM with R4 partition on RAM disk
2. If it passes step 1 then some performance testing under VM with R4
on real partition

I'll probably use Ubuntu for VM, although I'm not familiar with Debian
clones, but automatic testing suite supports it.
I'm more for Gentoo but lkp-tests doesn't support it.

I'll probably run 4.5 to 4.9 patches first to see what I get.

> Thanks,
> Edward.